Have there been any high-profile critiques of open-label, subjective-measures BPS randomised trials? If not, why not?

Sasha

Senior Member (Voting Rights)
I can't understand how this s**tshow continues.

Academia is supposed to be a place of the cut and thrust of ideas. Rubbish gets spouted, rubbish gets ripped apart, the field moves forward. That's what academic science is supposed to be.

So where are all the high-impact critique papers burning down the BPS studies? Is no one writing them? Or do the high-impact journals refuse to publish them?

Would any heavyweights here like to write such a critique? Or can we inspire such a critique to be written? Maybe by a team? To get into a high-impact journal or be delivered at a major conference (Cochrane! :woot:)
 
If I remember its content correctly, I think Jonathan Edwards' Expert Testimony to the NICE ME/CFS review covers this well:
https://www.nice.org.uk/guidance/ng206/documents/supporting-documentation-3
I agree that does cover it well but in order to stop people churning out these terrible studies, or believing their results, these arguments need to appear where a lot of scientists and medics are going to see it and be suitably horrified by the appalling breach of basic scientific principles. I think that's a high-impact journal or a major conference (and preferably lots of those).
 
If I remember its content correctly, I think Jonathan Edwards' Expert Testimony to the NICE ME/CFS review covers this well:
https://www.nice.org.uk/guidance/ng206/documents/supporting-documentation-3
I've changed the title of my thread from 'major critiques' to 'high-profile' critiques, just to clarify that it's the latter that we need. That NICE critique by @Jonathan Edwards was vital but I would imagine that 99% of BACME members, for instance, never read it.

The perpetrators of this nonsense need to be confronted with these arguments, and so do the colleagues who surround them and aren't aware of what they're up to. We need to make doing such bad science as socially unacceptable as drink-driving.

How do we grab the megaphone?

@dave30th
 
If the problem of unblinded trials with subjective outcome measures were really examined and eliminated by all medical and psychological research bodies, then I suspect a large part of psychological medicine would have to be scrapped as having no evidence base.

I think the problem is partly that those who run and rely on the results of clinical trials of drugs and devices where blinding is possible have no idea that whole swathes of psychological and behavioural trials are based on unreliable evidence, and probably aren't too bothered that it is so, since it's outside their field.
 
You’re quite right there is a severe lack of this.

Bruce Levine is high profileish and critiqued PACE. But where is the rest of medicine?

There are some excellent blogs, for example science based medicine that critique all sorts of “BioBS”, but are silent or even accepting of the psychosocial BS.

It really feels like psychology in general, or atleast large parts of it, are subject to a much lower standard for empirical evidence than the rest of science.

Possible why: Psychology evolved as a subfield of philosophy focused on introspection more than scientific evidence. Throughout it’s existence, it has often been used as a handy tool to rationalise all sorts of oppressive tools and discriminations by governments. The social construct of the “psychiatrist” who represents rationality and his subjects irrationality, allow all sorts of unethical things to happen under the guise of scientific authority (see Foucault’s the Birth of the Clinic).
 
Last edited:
If methodological standards equivalent to the highest quality pharmaceutical trials were held to be necessary everywhere it wouldn't just be the MUS empire that would collapse: there would probably have to be large-scale layoffs. I doubt there would be sufficient evidence for any psychotherapy (including things as common as CBT for depression) & the effects would ricochet throughout medicine, physiotherapy, nursing, all the other "allied health professions" as well as into social work/social sciences and beyond. Whole academic and professional empires have been built up.

The concept of a financial conflict of interest (especially where it relates to the pharmaceutical industry or tobacco companies) is now taken seriously but the concept of a career or position-based conflict of interest seems currently beyond the pale. There is a cliche that it is difficult to get a man to understand something when his salary depends on his not seeing it and that is very true for academics.

One of the other problems, which we have not discussed here much, is that much of this intersects with serious structural problems in academia. ME/CFS papers are worse than average for medicine but the bar is far lower than it has ever been before. Quantity over quality, "publish or perish", and that is just the tip of the iceberg: we're now also seeing indications of far deeper problems, such as just how widespread research fraud has become. The incentives work against robustness and the structural problems in academia are very deep indeed.
 
If the problem of unblinded trials with subjective outcome measures were really examined and eliminated by all medical and psychological research bodies, then I suspect a large part of psychological medicine would have to be scrapped as having no evidence base.

Yes, ideal...

I think the problem is partly that those who run and rely on the results of clinical trials of drugs and devices where blinding is possible have no idea that whole swathes of psychological and behavioural trials are based on unreliable evidence

So why is no one going for the easy win and making a big name for themselves by mass-debunking this stuff?
 
But what about the senior people who are themselves grant proposal reviewers? There are people so senior that that are surely safe. And people who are retired, and are completely safe. Why aren't they writing and presenting this stuff?

Senior academics are judged on papers published and grant money brought in. Metrics that doesn't encourage mistakes to be challenged. I get the impression that the medical profession in particular don't like to challenge each other.

Of course if outsiders who expose flaws were taken seriously or there is a bounty for doing so that would help.
 
Senior academics are judged on papers published and grant money brought in. Metrics that doesn't encourage mistakes to be challenged.

But surely such a paper would be highly cited, enhancing the writer's career massively? And if writing from within a BPS-free/BPS-nondominant discipline, BPS grant reviewers would be unlikely, especially if you're putting in for purely biomedical grants, surely?

I get the impression that the medical profession in particular don't like to challenge each other.
So do I, and it's not like other disciplines in this respect, IMO. I wonder why not (genuinely, I wonder).
 
Just tried to read this, as I hadn't seen it before - the link to the PDF of his presentation from Virology Blog is broken and there is no copy on archive.org or archive.ph.

@Sean provided a working link to the document in this post from 2019.

It seems like it was removed from the original link quite soon after Bruce Levin gave his presentation on the subject.
 
Yes, blogs! But where's the stuff in high-impact journals and big conferences? Blogs are easy to ignore.
To be clear. I was making a point that the science based medicine blog doesn’t critique psychsomatic BS while being very critical of Bio BS. And this is a sort of stance we see a lot with general “critical thinking/sceptic” people.

I think you misread me because it sounds like you assumed I was saying the blog did critique it.
There are some excellent blogs, for example science based medicine that critique all sorts of “BioBS”, but are silent or even accepting of the psychosocial BS.
 
Back
Top Bottom