David Tuller: Trial By Error: The Cochrane Controversy

The Cochrane Controversy

Cochrane–formerly called the Cochrane Collaboration–is respected worldwide for its systematic reviews of medical treatments. These reviews are often cited as the definitive source of information about treatment efficacy and safety. In taking on the thankless task of assessing the data on commonly used interventions, Cochrane performs an invaluable public health service and has advanced the cause of evidence-based decision-making in medicine.

But like any organization, Cochrane can get things wrong—as it has in the case of chronic fatigue syndrome.

 

The BPS'ers in both The Netherlands and Denmark seem to have banked heavily on the Cochrane reviews to justify CBT and GET. So this will be an incredibly interesting development. I do hope Cochrane does the right thing, although I do worry about the lethal amount of schadenfreude I will be filled with if the Danish Health Authority manages to die upon their own sword.

 

I have been looking much forward to Tuller's take on Cochrane. One of the reasons because the Norwegian Institute of Public Health has played a role, and it doesn't reflect too good on them.

It might be of interest that the science director, Per Magnus, wanted research into ME and Lightning Process.
I do hope he has changed his mind by now, and for the record he has shown interest also for biomedical research into ME, but for those who would like to read more:
Journal for the Norwegian Medical Association (2016) - Chronic Fatigue Syndrome and experience with the Lightning Process
(one of the co-authors is an LP-coach)

 

"That will continue as long as CBT/GET promotors can hide behind Cochrane’s skirts."

I think that this is a key point. Cochrane's influence means that a poorly thought through and misleading review can be used as an excuse by anyone who wants to avoid discussing problems with the research in this area. That new Watt comment illustrates that harm Cochrane have been doing, and continue to do.

PS: @dave30th - looks like you've got an extra 'o' in promotors.

 

Also, just for ease of access for everyone, the Robert Courtney comments pointing out some of the serious problems with Cochrane's exercise review can all be found here: https://sites.google.com/site/mecfs...exercise-therapy-for-chronic-fatigue-syndrome

 

Thank you, @dave30th - an excellent and clear summary of the situation.

Have you alerted anyone in Cochrane to it?

 

Me, too

 

yes, sending it to them today.

 

Thanks, @Esther12--I've fixed the typo and linked to the Courtney comments

 

"Muh Cochrane" is basically all I see in the wild. Whenever medical professionals are involved in discussion over this disease, this is the source I see the most, seemingly with a reputation above the NIH, NAM and CDC combined. Given that, they are betting right to always point to them.

It's hard to overstate how massively Cochrane screwed up in arbitrarily miscategorizing an entire disease and "reviewing" the body of evidence by not so much cherry-picking but actually field-picking, where nearly all of the published literature was excluded because they placed a medical condition outside of medicine.

I think that even above PACE, Cochrane is what's holding all of this together. I wonder if things will change with the new classification in the SNOMED (sp?) system. Will they be able to maintain locking away this disease completely outside of the ground truth?

 

To be fair I don't think this is the case. They were ONLY reviewing one therapeutic modality. That is an inherent limitation of such an approach, other things are ignored. The only large scale evidence based reviews are from the US. Another issue with other reviews, including the US ones, is there is not sufficient high grade clinical trials to claim anything is particularly fantastic as treatment. Ampligen probably comes closest. In time there will be, but as I have said elsewhere, the ME research is really a research wasteland, and evidence based approaches work best when there is abundant high quality evidence.

We were hoping Rituximab worked out, but right now its in a holding pattern, probably until they figure out who are likely to be responders and non-responders. Cyclophosphamide, antivirals, and immune and metabolic modulation are all in the treatment study phase. They might work out, as early as next year, or it might be a long time. When we have many different successful treatments then it will be time to use evidence based methods to try to prioritise which ones to try first.

The downside of GET is that there is also abundant evidence of it doing immense measurable harm. That is something we need to focus on more. People with a broken aerobic metabolism do not benefit from aerobic conditioning.

The last paragraph of this post has been copied to a new thread and subsequent posts responding to it and discussing 2 day CPET have been either copied across or moved.
2 Day CPET discussion - is it evidence that GET is harmful, and is it a biomarker?

 

I think that the evidence of harm from GET is still pretty weak, so reckon we're best off pointing to the lack of any good evidence for efficacy. The concern about harms made more important, even if the evidence there is still weak, when it's balanced up against poor evidence of benefit from research with serious flaws.

 

No, it isn't. The psych evidence for harm is weak, the exercise physiology evidence is quite robust. Its just that the psych papers totally ignore it.

 

But GET proponents can always argue that studies not specifically designed to test GET cannot be used to claim that GET causes harm.

 

Yet studies that show aerobic exercise always causes harm trump that.

 

Do we have studies showing that aerobic exercise always causes harm?

 

We have nearly 100% evidence of harm from aerobic exercise. What we do not have is proof of long term harm in formal studies. Its obvious, but not done in studies in part because its considered highly unethical. We don't have to prove many poisons are harmful by subjecting people to them over a long time to gather evidence.

 

Which do you think that strongest studies to show GET is harmful are? I've not kept up with recent ones, but in the past I thought that it was a bit of a reach to try to extrapolate from studies which indicate ME/CFS patients have abnormal responses to certain forms of exercise to claiming that they showed GET was harmful.