COSIG - Collection of Open Science Integrity Guides

[forestglip]

Copied from COSIG homepage:

---------

Anyone can do post-publication peer review.

Anyone can be a steward of the scientific literature.

Anyone can do forensic metascience.

Anyone can sleuth.

However, investigating the integrity of the published scientific literature often requires domain-specific knowledge that not everyone will have. This open source project is a collection of guides written and maintained by publication integrity experts to distribute this domain-specific knowledge so that others can participate in post-publication peer review.

COSIG currently hosts 27 guides and was last updated on 4 June 2025. Guides can be downloaded as individual PDFs. A combined PDF with all guides included can be downloaded here.

Suggestions to improve COSIG can be submitted by opening an issue on COSIG's GitHub repo or by emailing admin@cosig.net. Before contributing, read COSIG's Contributing and Code of Conduct pages.

Table of Contents​

General Guides

• PubPeer commenting best practices
• Extracting vector graphics from a PDF
• The vertical line test
• Image duplication
• Image compression artifacts
• Software for image forensics
• Reporting publication integrity issues to publishers
• Citations
• Citations to retracted publications
• Formulaic research
• Plagiarism of text
• Common dismissive responses to integrity concerns
• Suspicious venues
• Ethical approval of human subjects research

Biology and medicine

• Antibody validation
• Misidentified and non-verifiable cell lines
• Nucleotide sequence reagents
• Tumor burden

Materials sciences and engineering

• X-ray diffraction patterns - Scherrer's equation
• X-ray diffraction patterns - data duplication
• Energy dispersive X-ray spectroscopy
• Elemental composition
• Tauc plots

Mathematics, statistics and computer science

• Evaluating the performance of binary classifiers
• Evaluating the performance of multiclass classifiers
• Standard deviation versus standard error
• Multiple hypothesis correction

 

[forestglip]

Nature: 'How to spot suspicious papers: a sleuthing guide for scientists'
An open collection of tips and tools could help researchers and publishers to pick up on problematic research.

By Miryam Naddaf

----

'A group of research-integrity experts has launched a toolkit for researchers that outlines how to spot suspicious scientific papers.

The guide’s creators hope that it will be a useful resource for sleuths who review published work for signs of sloppy, fake or fraudulent science, and will perhaps inspire others to get started. “The sleuthing community is growing. And this will be the essential handbook,” says Jana Christopher, an image-integrity analyst at FEBS Press in Heidelberg, Germany, who contributed to the project.'

----

'“COSIG is really a compendium of all the tips and tricks that various sleuths have acquired over the years,” says Reese Richardson, a metascientist at Northwestern University in Evanston, Illinois, who led the project and authored a preprint on COSIG posted to the repository Zenodo on 4 June. “We want to arm scientists with the tools to uphold the integrity of the literature.”'

----

'The guides describe how to spot issues that are common across all disciplines, such as text that has been generated using artificial intelligence (AI), and publications from hijacked journals — scam websites that impersonate legitimate journals. They include instructions for how to use online tools such as the Feet of Clay Detector, which flags papers citing retracted or problematic research, and Imagetwin and Proofig, which use AI to detect integrity issues in scientific figures.'

 

[forestglip]

This seems like a good resource.

I read one guide so far, 'Standard deviation versus standard error'. It's a fairly easy to follow explanation of the consequence of studies accidentally using standard error instead of standard deviation in meta-analyses. And it gives three explicit examples of the mistake in existing studies to show how to spot it.