Cochrane Review: 'Exercise therapy for chronic fatigue syndrome', Larun et al. - New version October 2019

I don't know about the old review but the new review specifies the following selection criteria:

PACE was not a RCT. It also plainly concluded no difference in outcome between treatment arms. As the biggest piece of evidence, its own conclusions do not match the review's conclusions.

It seems the material is locked for now so I can't check the other trials but from Vink's review most of the other trials were not controlled as well.

Not the first time I have seen this, using the RCT label then talking only of randomized trials. So it seems to be a new thing, I guess, that RCT also applies to randomized trials. Drop a word here. Add a word there. Change another word's meaning. Ridiculous.

 

#MEAction are right that the review authors still claim they "included eight RCTs with data from 1518 participants."

Yet the biggest trial, the PACE trial, wasn't a Randomised Controlled Trial, and even the PIs called it only a "Randomised Trial".
Didn't ceck the other trials.

Edited to add: @rvallee beat me to it.

 

Here's the What's New panel:

I'm going to do a side-by-side comparison with the abstracts in due course. There are some "interesting" changes in there.

 

I'll add: I'm not impressed by no conflicts declared from the authors. Laughable. Either Cochrane's entire process is irreparably broken, or it allows arbitrary exemptions that make having a process irrelevant.

Then again, the protocol deviation in PACE is "addressed" by stating it was noted in private steering notes and approved. Approving of a conflict of interest and recording protocol changes does not excuse either, it merely describes what happened, and even at that very reluctantly.

The massive conflicts of interest from PACE authors were not declared either, so this is second degree conflict laundering. What a mess. The promise of a new review in 2020 with this in place is a mockery of clinical evidence and the scientific method.

 

Page 56:



We know this is not a low risk of bias for fully unblinded studies, where the date of first seeing the data is by no means the first time trending of data will be evident.




Must be something else.

 

Their section on selective reporting bias is BS. Truly shameful that Cochrane let them publish this.

 

The main thing that strikes me about the differences in this version is the number of times they state "We are uncertain..." (lots and lots).

In the 2017 version, the only time "uncertainty" is mentioned is here:

The "interesting" part for me was the substantial change to the Background section of the abstract. I'm not sure it helps though, because it gives the impression that the review adequately considered some things that it clearly didn't. Compare 2017 with 2019:

To me, this indicates that they equate "persistent, medically unexplained fatigue" with "persistent postexertional fatigue" (whatever that is), that the included studies adequately screened for ME (when they didn't).

To try to be positive, they have made a big deal of the chasm in evidence between findings at the end of therapy and long-term, where uncertainty abounds, and the uncertainty over the risk of serious side-effects.

I think they've tried, but all it does is show up that they really don't know what they are doing.

 

Haven't read the new review yet, but I'm rather optimistic by this statement from the new editor in Chief:

This might be more important than what's in the updated review.

It seems like they have decided to start all over again with this review. It's not made explicit but I suspect that this will involve new authors (would be weird otherwise).

So going back to where the Cochrane email correspondence stops. It seems that the new Editor in chief Soares-Weiser decided that Larun et al. should go ahead and publish the updates they had already worked on, but that this doesn't address all the problems and that Cochrane should simply start over.

I personally would have preferred to have the review retracted as it could be misleading therapists and policymakers. I suspect the current option was the preferred compromise from a political perspective. But it's good news that Cochrane accepts that this updated review doesn't address all the issues and that they want a full update of the review, starting with the protocol (I suspect that means that objective outcomes need to be included).

 

You could be right. Just so long as they don't sit on it too long.

 

I do not see how an editor could have approved ‘moderate certainty evidence that probably...’ This is meaningless. Moderate certainty cannot then be qualified by probably.

The text appears to be an appalling case of confused English. I had thought that Cochrane’s GRADE system gauged evidence of ‘moderate quality’ not moderate certainty. Moderate quality evidence of probably would make sense but moderate certainty of probably simply indicates that the authors have no understanding of what they are about.

I am in Peru at present but likely will Communicate with Cochrane about this when I am back.

How can we have a published review with a key conclusion making no sense at all?

 

From their correspondence, it looks like 'probably' is standard for moderate quality evidence in Cochrane reviews:

"Some practical issues will arise, e.g. which term shall we use to reflect the grading? Usually
“probably” is used for moderate level evidence, while “may” is used for low level. We are willing to
accept “may”."

https://www.s4me.info/threads/cochr...evelopments-2018-19.10030/page-16#post-205744

 

The statement frankly hits most of the notes and is committal. But the publication of this revised version of the review completely undercuts it. It's very hard to reconcile the two other than this future update being more of the same whitewash. Meanwhile the same recommendations stand as far as health professionals are concerned, despite lacking any credible evidence. After a whole year, there is still nothing but confusion and misleading claims. It still reeks of politics and conflicts.

 

I hope some people submit comments. The authors are expected to reply. Also the comments and their reply(s) are available indefinitely in the review document.

 

I get that probably is what moderate quality is supposed to mean but you cannot use the same probabilistic statement twice in a sentence. It is a bit like a double negative, it should combine to produce something else.

I think there is a Freudian slip at work here. They wanted to say there was moderate quality evidence... But that would be an outright lie. So they side step using the official moderate quality and use moderate certainty and because that would still be a lie they qualify it with probably. So they end up with a sentence that if it means anything means ‘God knows whether’ but which sounds as if it says moderate quality evidence.

 

I think to be scientific a word just needs to have an adequately defined meaning. Probably has a very precise meaning in ordinary English: a more than 50% chance. So it is fine - if based on adequate evidence.