Bias due to a lack of blinding: a discussion

https://twitter.com/user/status/1759585340631765057

 

https://twitter.com/user/status/1759586163646460058

 

Interesting stuff given that for something like insomnia there might be legal implications surely if someone gave out incorrect advice to eg truck drivers , or worse said therapy made someone only think they had slept better than they objectively had who then was in a road traffic accident or injured themselves etc.

Surely the risk is that is what this study might be 'proving' or at least warning of could be the effect of said therapy - that it reduces accuracy in self-assessing sleep?

 

The FDA disapproved MDMA-therapy for PTSD.
https://www.bbc.com/news/articles/cl4465dpmrro

Although randomized trials showed a clinically significant improvement, there were concerns about unblinding as most participants were able to guess which trial arm they were in. So lack of blinding was one of the main reasons (in addition to safety concerns) why the FDA panel voted against approval.

The FDA document reviewing the evidence is available here and reads:
https://www.fda.gov/media/178984/download

Some comments about the decision:
https://twitter.com/user/status/1822391311321755712

https://twitter.com/user/status/1822576212679500276

 

What a shame I was looking forward to being encouraged to participate in free good quality MDMA recovery programs to help me beat my ME.

 

Everyone in the psych-skeptic community accepts these points as given when applied to various scammy clinical trials of biological interventions. Yet none of them have ever spoken against the exact same biases affecting BPS/functional disorders trials.

 

IIRC, the FDA has approved a number of apps like the CBT for IBS one from Mahana. On the basis of the very format of evidence they reject here.

I don't know on what basis the burden of evidence must be different. It truly makes zero sense. If anything, it should be even higher given that there is far more bias involved in those. But instead they are enabling grift and frauds in some places, while essentially making it impossible for some treatments to be approved.

At the very least the burden should be the same. There is nothing special about drugs other than the fact that they can, in fact, be blinded for testing. Even though it's usually done relatively poorly, because there are so many other factors that can bias outcome, and this is exactly why the double-blinded controlled requirement exists. And it still fails very often.

Given that it's becoming harder and more expensive to develop new drugs, this pretty much sets up a future where most treatments approved are pure grift that would be systematically rejected if it were a drug. What an absurd society we live in.

 

Trial participant in ecstasy for PTSD makes serious allegations regarding reporting of serious adverse events:

https://twitter.com/user/status/1822887395165192620

 

Interesting insight

I can't help but think so this is what happens when you have lax methodological regulations for therapy-based treatments being trialled, and then mix them with drugs (which would normally be measured with objective measures and have yellow-card reporting)....

basically it depends on who is running it, and if it's a therapy-based then they are used to and don't see the errors in what they see as their norms.

Some of the things flagged sound rather familiar to eg the newsletter point brought up about the PACE trial protocol where people were being sent newsletters suggesting it was working for lots of other people with testimonials


Just because you have some people that go above and beyond or don't even think of running a trial in the ways some do, doesn't mean there isn't an issue when regulations leave the door open for that behaviour. Worse of course those people don't keep on doing it if it isn't benefitting them, and so it has to become the culture because of the competitive advantage it gives so even those who are 'good' have no choice but to play the game.

 

Odd how all it takes to go off from biopsychosocial standards is to introduce anything biological. Because the descriptions above are pretty standard biopsychosocial methodology for the most part.

 

Here's a tricky one -- does that trial show that both modafinil and CBT are effective in relieving MS related fatigue, and that each treatment alone as well as their combination are equally effective?

Some very quick thoughts:

For what seems the current mainstream view on evidence on modafinil for Multiple sclerosis-related fatigue --

From Wikipedia:

Any thoughts / discussion appreciated.

Thread on the trial:
Comparative effectiveness of [CBT], modafinil, and their combination for treating fatigue in multiple sclerosis (COMBO-MS), 2024, Braley et al

 

Only had a quick look but I thin it pretty certain that the trial tells us nothing other than nothing probably works much, otherwise the results would have differed for different arms.

I am surprised that NICE endorses off label usage here. Presumably the committee was composed of believers.

If I had MS I am pretty sure I would not want to be taking all sorts of extra drugs because some physician thought he/she was clever to prescribe something for my fatigue. In my experience any drug that alters brain function is likely to produce more dysphoria than anything useful.

 

Real improvements aren't always the same as meaningful improvements, either. You shouldn't have to rely on statistical fudging to show the latter if you've set the outcomes properly, because they'll light up in that part of the cohort.

 

Intriguing! My initial response is that, if a person's symptoms are not rooted in disordered thinking, CBT by definition cannot work.

But there are other difficult-to-blind therapies that might work, such as pacing. Maybe it would be a useful approach there?

 

Also curious that in the ecological momentary assessments both treatments resulted in improvements in fatigue intensity & interference but no treatment resulted in significant improvements in perceived fatiguability.

The watch-like device ("PRO-Diary") that they used for the EMAs has both actimetry functionality and can ask the user self-report questions; something like that may be quite useful for ME/CFS trials.

[Edited to remove a potentially misleading sentence which needs much more thorough explanation. I shouldn't post at 2AM...]