Cochrane Review: 'Exercise therapy for chronic fatigue syndrome' 2017, Larun et al. - Recent developments, 2018-19

To win at tactical leaking timing is everything.

 

Thanks @Sly Saint .

 

Somewhere in this correspondence ( scanned quickly), Tovey refers to discussing this issue with ‘ Carla’, the new editor in chief. Since nothing has happened, what conclusions can be drawn from this?
Agree with @chrisb that it is very good to have this correspondence from Tovey’s last week in office in the public domain.

From a very quick scan, it read to me that Tovey was happy to disagree with the authors, withdraw the review and leave them free to publish elsewhere, while they were fighting tooth and nail to have this retained as the review, and defending the word ‘ moderate’ for quality of the evidence.

With the Nice guideline review in progress, this is hugely important.

 

Maybe in his last week in the job he was finally happy to drop a big pile of poo in his office, knowing it would not be him coming in to face it next week.

 

To me this looks like Cochrane head office are fighting to get the authors to do the bare minimum, just to avoid embarrassing Cochrane. But letting the authors get away with what is remotely 'defensible' is still going to be really harmful. This whole process looks completely broken to me.

 

There's a nuance here that I don't understand. What's the difference between a withdrawal and a retraction? Other than the former being a polite way of allowing the authors to save some face, I mean.

 

That is pretty much the difference as far as I am aware.

 

Cowardice.

 

Thanks to @Marit @memhj for providing these documents. Are these all the emails of that exchange that are publicly available? I got the impression we only got snapshots. The website linked, eInnsyn, also has documents on this from 2018 but you have to file a request to see them, apparently. Does anyone have them?

I agree. Tovey constantly argues that he wants to come to something that's defensible, instead of something that is right and accurate.

It's distressing to see how Atle repeatedly uses the argument that they only want to focus on the issues Tovey originally raised because that's what they agreed on. That came across as rather childish behavior to me, given the stakes involved. If there are additional issues with the review, then surely these should be addressed by the authors' regardless of whether this was part of an agreement or not. it's their responsibility as clinicians all over the world use these reviews to treat patients.

The main issue, the lack of blinding + the focus on subjective instead of objective outcomes, is only mentioned briefly and Tovey seems to argue that simply downgrading the reliability of the evidence would do to address this issue. [EDIT: If patients report having recovered from taking homeopathy, we don't say that 'homeopathy may reduce fatigue spectacularly but with a low level of evidence'. That would be misleading.]

Most of the discussion revolved around the Powell et al. 2001 study that reported much bigger improvements than the other studies and so created heterogeneity. Excluding it halved the SMD for fatigue at follow-up but apparently, the authors had only mentioned this on page 134 of 142.

Another issue discussed is that the improvement on the Chalder Fatigue Scale is really small. Tovey argues that the evidence that this points towards a clinically meaningful effect is low. He wrote:

The problem that the results of the GET-studies might not apply to patients who satisfy more recent diagnostic criteria was briefly mentioned. It seems that the authors will have to add a statement like "patients diagnosed using other criteria may experience different effects.”

Another important point is that GET-trials used many different outcomes and that only fatigue pointed towards significant improvement. Tovey seemed to be aware of this problem, as he wrote:

 

@Caroline Struthers, these emails may be of interest to you.

 

I'm still trying to work out if Tovey has withdrawn or retracted testicles in this case.

 

The people involved in this at Cochrane seem to have difficulty understanding that modest self-reported positive effects is exactly what you would expect if the treatment did not work but was tested without adequately controlling for bias while introducing positive expectations.

They might even be more than modest in some cases, while still not being real.

Such positive effect should not be seen as little bits of evidence of efficacy as they are no more consistent with this than a lack of efficacy.

 

The flip side to that is that with the correspondence out in the open I can ask anyone on the NICE committee who wishes to defend these trials to explain how any rational human being can conclude other than that the whole process is completely broken.

 

Actually the effect is so small it looks as if the real effect of the treatment is that it makes people worse!

 

Transparency is a beautiful thing. There should be more of it. In fact, there should be only it. Secrecy is a pillar of mediocrity.

 

Too bad he maybe did the very least in the circumstances and won't be deserving of any credit while being subjected to harsh criticism and howling by the ideological brigade.

It's pretty obvious he would be vindicated with time if he took meaningful steps. Or maybe he did and we'll know later but meanwhile people keep getting hurt as millions did during his tenure while the review stood for years so zero credit due.

 

I guess that's what the "collaboration" part is. Cochrane makes editorial decisions but not unilaterally.

Unfortunately the execution is completely broken.

 

That last sentence by Tovey is clear-headed and shows he understands that not only this is indefensible but that it always was and should never have been published, that the recommendations made in the review do not stem from reliable evidence but rather what its authors want to be true despite evidence to the contrary.

And yet it was and remained published through legitimate criticism that is so precise and thorough that it warrants immediate action. So what gives? Cochrane publishes reviews it knows to be junk and simply does not have a process in place to deal with this? This is literally their entire purpose and mission.

 

[my bold]

So science is about ignoring issues that are further discovered to need addressing? "We agreed that no additional issues would be brought into the process" ... are they scientists or babies?