ME/CFS Skeptic - How many scientific papers are fake?

rvallee said:
Fake news. Fake research. Fake people. Damn the future is bleak.
Click to expand...
Yes, but the future has probably always been bleak, some people have always been corrupt, and yet humans are still here. The same tools used to produce and magnify all that fakery can also be used to increase transparency and increase knowledge, to allow people to inform themselves, better than ever before. There are good people trying to make things better, as ME/CFS Skeptic's blog illustrates.
 
Excellent blog, as usual.

I suspect there isn't a lot of outright fraud in the ME/CFS field, just incompetence, vested interests, and third-rate researchers with niche ideas. Academic fraudsters aren't going to choose a field where it's difficult to attract funding, difficult to publish in high-impact journals and where research attracts significant critical scrutiny.

Having said that, last year I came across a paper with "tortured phrases", obviously the product of some automated tool, although it wasn't a biomedical paper.
 
From an interesting letter in the BMJ:
We investigated 172 clinical trials from one group and notified serious integrity concerns to all journals and publishers in July 2019.[3] The concerns were similar for all trials, including evidence that random allocation of participants could not have produced the treatment groups reported, evidence that the distribution of numbers of participants withdrawing from the trials was implausible, frequently contradictory reporting of the size of participant populations, implausibly prolific research activity, unethical conduct, and very frequent discrepancies between trial registration documents and journal publications for study conduct, study location, participant age, and participant number (including wholesale updating of trial registration documents after we notified these concerns to journals and publishers).
Click to expand...
More than five years later, only 22 of the 157 trials covered by Web of Science have been retracted, with to date 289 citations in systematic reviews, clinical guidelines, and consensus statements. The 135 unretracted trials have 1989 citations in systematic reviews, clinical guidelines, and consensus statements. Presumably, some of these currently unretracted trials will eventually be retracted, further compounding the problem.
Click to expand...
Link
 
...evidence that the distribution of numbers of participants withdrawing from the trials was implausible, frequently contradictory reporting of the size of participant populations, implausibly prolific research activity, unethical conduct, and very frequent discrepancies between trial registration documents and journal publications for study conduct, study location, participant age, and participant number (including wholesale updating of trial registration documents after we notified these concerns to journals and publishers).

Sounds all too familiar.

implausibly prolific research activity,

Some peoples' primary expertise is getting their names on papers.
 
Good article. Well done as usual.

It is shocking and disturbing. I also notice that many people trust research papers without thinking about the possibility that the information might be false.

The other day I met a psychology student who told me a positive attitude improves survival of cancer patients. When I told her this was an outdated view, she responded that it had been proven by science. People are still being taught this stuff.

In general my feeling is that education is too focused on the process of students absorbing information, and not enough on developing critical thinking and often also not enough practical application of that information. The practical application is where theory is tested by reality.

I don' think that "generating truly random data is rather difficult". It is very easy, and I suspect what is meant is generating data that looks real in every way. It would be easy to create profile of fake persons by randomly generating all the personal information... but then the dataset would contain implausible entries, like people who are very old or very young and allegedly doing things that are impossible or hard to believe at that age. Generating data that looks real is indeed very difficult, as you need a very good understanding of what real data looks like.
 
Last edited:
ME/CFS Skeptic said:
It still find it weird that standards are so low in (medical) science. So how do real scientists deal with this: do they simply ignore 90% of the literature and only focus on results from labs/teams they know and trust?
Click to expand...
My impression as a patient is that they don’t care about it and take anything official as «true».

From a risk perspective, it makes sense. You won’t suffer consequences from the government by doing what the government told you to do.
 
EndME said:
This is a funny one: Book on AI looks to be written by LLM full of made-up stuff https://retractionwatch.com/2025/06...achine-learning-is-full-of-made-up-citations/
Click to expand...
The same day Behrendt replied to our query, Springer Nature published a post on its blog titled, “Research integrity in books: Prevention by balancing human oversight and AI tools.”

“All book manuscripts are initially assessed by an in-house editor who decides whether to forward the submission to further review,” Deidre Hudson Reuss, senior content marketing manager at the company, wrote. “The reviewers – subject matter experts – evaluate the manuscript’s quality and originality, to ensure its validity and that it meets the highest integrity and ethics standards.”
Click to expand...
At least AI is making it glaringly obvious that the quality control at most journals is terrible..
 
Utsikt said:
My impression as a patient is that they don’t care about it and take anything official as «true».

From a risk perspective, it makes sense. You won’t suffer consequences from the government by doing what the government told you to do.
Click to expand...
I meant the true scientists, the people who really want to figure things out, discover truth etc, rather than academics making a career.
The people who will win future Nobel prizes. I suspect that they simply ignore most of what gets published.
 
ME/CFS Skeptic said:
I meant the true scientists, the people who really want to figure things out, discover truth etc, rather than academics making a career.
The people who will win future Nobel prizes. I suspect that they simply ignore most of what gets published.
Click to expand...
Kahneman comes to mind as a Nobel prize winner that failed to do exactly that (and has admitted to doing so):
retractionwatch.com

“I placed too much faith in underpowered studies:” Nobel Prize winner admits mistakes

Although it’s the right thing to do, it’s never easy to admit error — particularly when you’re an extremely high-profile scientist whose work is being dissected publicly. So…
retractionwatch.com retractionwatch.com

But I would think «good» scientists in general ignore most of the noise. I don’t see how they could get any work done if they didn’t.
 
ME/CFS Skeptic said:
So how do real scientists deal with this: do they simply ignore 90% of the literature and only focus on results from labs/teams they know and trust?
Click to expand...
On one of my first interactions with an ME/CFS researcher they explained to me 95% of research is not replicable. I think the biggest issue they explained was incorrectly controlled methods and sample handling. At the time I think I asked about metabolomics and Seahorse experiments. I was shocked but then discovered from others that the 90-95% was true.

I asked another scientist how they tell if a paper is worth reading or not. They said they go straight to the data and plots and you can tell very quickly. They also said they will always read a paper and trust the data from labs/teams they know and trust. I think they mentioned Dr Hanson as an example.
 
You must log in or register to reply here.
Back
Top Bottom