ME/CFS Skeptic - How many scientific papers are fake?

New blog post about fraud and research misconduct in scientific research.
https://mecfsskeptic.com/how-many-scientific-papers-are-fake/

Twitter summary:
https://twitter.com/user/status/1901672256738095441


1) How many scientific papers are fake?

A new review argues that fabrication and falsification of scientific results may be more common than previously thought. It estimates that approximately one in seven papers are fake.

2) In the past few people were actively looking for fraud but in the past 15 years, a group of data sleuths have developed various techniques to screen for research misconduct.

3) This has resulted in a flood of retractions: from highly cited papers on Alzheimer’s, controversial COVID-19 treatments to publications by the Dana-Farber Cancer Institute. Even the president of Stanford University had to resign.

4) In our blog post we look at some high profile examples of research misconduct, the smart tools data sleuths use to detect problematic papers, and how this is slowly changing the scientific literature.

5) Although still at a small scale and driven mostly by volunteers, the valuable work of research integrity advocates signals a hopeful change in the psychological and biomedical sciences.

 

I'm not sure whether "scientific papers" is the right term here, even though it is the term used by Heathers. Perhaps "psychological and biomedical papers" is more appropriate? I suspect that the differences between subjects will be massive and there are many disciplines where this argument doesn't apply, since there is essentially no data that can be fabricated (certain topics in philosophy, pure mathematics, theoretical physics etc), but where the other substantial problems in academic publishing still exist.

I think it might be meaningful to point out that for now, apart from a few positive signs, there are many signs of worsening, especially with how people are using ChatGPT (which is fraud by definiton of almost every journal, but still gets published almost everywhere) and the like to write all sorts of nonsensical papers in their academic h-index competitions.

 

I'm 'surprised' at the '1 in 7 are fake', i'd be less surprised at '1 in 7 aren't fake', because in many areas that i have seen virtually all papers are clearly fake, whole swathes of the internet returns thousands of fakes, or at least highly dubious papers.

But this is just my impression, maye there are loads of papers, accurate papers, on subjects I have no interest in, like tire rubber eating bacteria studies etc.

 

This might be of interest:
https://www.nature.com/articles/d41586-025-00455-y

Exclusive: These universities have the most retracted scientific articles
A first-of-its-kind analysis by Nature reveals which institutions are retraction hotspots.

 

The one in 7 is a reference to fabrication and falsification of data, not papers that are dubious, full of errors, nonsensical or simply wrong. That would be a very high number given that all the other problematic things don't fall into this category. I think the Alzheimers example is a pretty daunting example of how bad the situation is in many fields, but fabrication and falsification of data is only the end of the tip of the iceberg.

 

Yep. That word "data" does a lot of work too.

The bloke at the former pizza place insisted he got a 10% response to leafletting littering our road. This was because somebody wrote something about 10% response rates, in a context loosely connected with marketing, at some point in the 1980s. Data!

Sadly doesn't help if you've no idea how to make edible pizzas.

 

The US retracts papers at a lower rate than most. I doubt it’s because they are much more honest in their research.

 

The main point of posting this article here was the difference between the retraction rates and the suspected rate of falsified papers. 0.2 % suggests that less than 1 in 10 of falsified papers gets retracted.

 

I enjoyed the blog post. I continually wondered about survivorship bias when reading your blog. What papers aren’t getting flagged. I mean, you have to be reasonably knowledgeable to get to a place where you can publish academic papers. I’ll assume many of those falsifying their data are “competent” at falsification.

 

True, this is specified later in the blog and Twitter thread but was difficult to include in the title.

Yes, good point. I remember people posting studies where the text included sentences of the prompting or an introduction such as: 'Certainly, here is a possible introduction for your topic'. EDIT: I now see that it's Cabanac who posted this, the researchers who highlighted the problem of tortured phrases.

To me the 1/7 estimate seems quite high. I mostly read ME/CFS literature and there there are almost always big methodological flaws that produce 'significant' results so that probably means less need to fabricate data. Often researchers present their findings as quite impressive when the actual data is very weak or unconvincing. That seems like a sign that the data is likely not totally fabricated, otherwise they would have probably come up with something better!

Yes or in the case of Alzheimer's, fraudulent studies might have wasted millions of dollars and hold back research to an effective drug, so the detrimental effects for society can be enormous. Begs the question if it should be dealt with like a crime. Now it seems that the worst thing that happens is that researchers get fired and can no longer work in academia.

 

The burden of proof to convict would probably have to be quite high, so you’d end up with many ‘false, but legal’ papers. I’m not sure if that’s a problem.

 

Off topic but ME/CFS Data Sleuth would be a great name!

 

Yes, it seems too high for me as well. It's hard to believe that in 1/7 situations people would feel the need to fake data when in fact they can just get away with "faking a story" without that having any repercussions or even being frowned upon.

 

For some reason I’ve always connected the word ‘sleuth’ with ‘sloth’. Not being a native speaker probably contributes to the confusion!

 

Such low retraction rates pretty much make a mockery of the idea of science being self-correcting. Corrections and retractions should be common, because the process is so abysmal that it lets nonsense pass through.

It really looks like one of the key features of the system has not been functioning in a long time, and no one noticed. Like an alarm that hasn't gone off, because it was taped over to prevent it from going off.

What's most annoying is how this plays perfectly into the anti-science movement, which is looking at trashing the entire concept of science. Even though, somehow, the discipline with by far the highest rate of bad research has been entirely ignored. Because it pleases a lot of people.

Fake news. Fake research. Fake people. Damn the future is bleak.

 

Side-thought: it must be really frustrating to be a true scientist amongst this mess of fraud and incompetence. Imagine funding going to 'exciting' but totally untrustworthy or incredible studies over and over again, while valuable work gets overlooked. Promotions going to people with 50 publications per year, etc.

I assume true scientists try to overlook most studies and only follow researchers in their field that they know are reliable. But I suspect it must be very frustrating.

 

same!