Cochrane Review: 'Exercise therapy for chronic fatigue syndrome', Larun et al. - New version October 2019

Thank you @Barry; precisely - "Perhaps one day in a court room they may begin to realise it."

This "accident" just keeps rolling on - I think the only way to stop it is legal action....but legal action ain't got no traction at the moment.

ETA: None of these people will wholeheartedly give themselves over to logic.

 

https://twitter.com/user/status/1179841337589403656

 

And they can't claim they weren't aware of the paper as they mention it:

 

From one of Robert Courtney's previous comments with regard to the PACE Trial and risk of bias:

 

It's debatable whether 24 weeks should be considered the point at which therapy ended in the PACE Trial and the results at 52 weeks the follow-up results (with the follow-up results at approximately 2.5 years ignored) given that there was a "booster" therapy session at 36 weeks.

The PACE Trial authors seemed to consider the results of 52 weeks the main results.

 

Still, haven't been able to read the full study, but I suspect that this will be one of the main points that should be raised. I've tried to mention it here: https://www.s4me.info/threads/david...port-on-courtneys-complaint.8555/#post-150341

Most of the data come from three large studies: PACE, FINE and Powell et al. 2001. For all of these trials, the main outcome was specified as a measurement several months after the treatment ended. This is what the Cochrane review calls the long-term follow up.

So what the large studies reported as their main outcome was deemed too uncertain for Larun et al. to make any conclusion. But the measurement directly after treatment (when respons bias and placebo effects can be expected to be higher) was turned into the main outcome, simply because they had (a little) more data on this coming from the small studies.

So the distinction between (1) the results directly after treatment which are deemed certain enough to mention and (2) the results for the long-term follow up which are viewed as uncertain, seems rather arbitrary and it happens to coincide with results changing to no longer being statistically significant.

 

AKA cherry-picking.

The lack of concern over blatant cherry-picking is very disappointing, especially after written admission of making choices based explicitly on results they prefer.

 

Just to be clear - this was prespecified as the time for their primary outcome, but there were changes to the primary outcome at that point. If there's cherry-picking on the time, it's the way the Cochrane review fails to look at more long-term follow up data, emphasising instead the data at end of treatment. That could be what you meant, I just wasn't really sure.

If we could be confident that their 'fatigue' measures were measuring fatigue then combining their fatigue measure would presumably be okay. That we can't is such a big problem with the review that it makes the problem of combining their fatigue measures seem a bit trivial!

Does the protocol say that? - edit: Adam was just speculating about what the protocol may say about end pints.

Could they argue that PACE would no longer count as a randomized trial at that point?

 

We probably need to be careful with phrasing on that, as it's disputable to what extent PACE can/can't be considered an RCT. It fails to control for many of the things likely to bias outcomes in favour of CBT/GET, but it has features that would lead some people to class it as an RCT.

I'm suddenly getting scared that any exaggerated criticism of this review is going to lead to more spin about abusive patients: "How dare these terrorists dispute our RCT classification!"

 

Just to be clear, I don't know what their (Cochrane's) protocol says. But they probably do have a rule about which data point counts as long-term follow-up. That's why I was asking.

 

AfMEs statement on their website re the new review:

full post here:
https://www.actionforme.org.uk/news/​cochrane-review-of-get-our-concerns/

 

Hm. Do you think there are reasonable arguments to classify the PACE trial as a controlled trial? What arguments are there?

Is the knowledge I acquired on S4ME (that it needs adequately controlled groups to classify a trial as an RCT) disputable? Or is it disputable whether the groups were adequately controlled?

See: https://www.s4me.info/threads/a-general-thread-on-the-pace-trial.807/page-11#post-90488

 

It is notable that the main conclusion which was a sticking point for David Tovey, namely downgrading the evidence from "probably" to "may" and from "moderate" to "low-moderate" has not made it into the revised article. (See the FOI correspondence on 29th of May)

I suggest this is a point of contention that we can leverage.

 

Given the way Cochrane has acted, why would they assert Cochrane Reviews are "the gold-standard of systematic healthcare research reviews"?

Bringing up AfME's survey data to dispute this review is not a great move either.

"We are keen to see Cochrane progress this as soon as possible, with children and adults with M.E. at the very heart of it."

At this point, I don't see much reason to be keen on this. If Action for ME are the patient group Cochrane is working with then I feel a deep concern.

Rather than reasonable arguments, I'm more concerned about the cultural assumptions within the research community, and the way we phrase our valid concerns. eg: Talking about PACE not being adequately controlled to account for the biases likely to afflict their primary outcomes, and it therefore being questionable whether it should be classed as an RCT, is one thing. But just saying it's not an RCT, even though participants were randomized to four groups, is something that goes against the assumptions of many researchers and so could be interpreted as showing criticism of PACE is unreasonable or ill-informed. When there are so many researchers who view poor quality work as acceptable and assume ME/CFS patient criticism of trials like PACE is driven by our ideological opposition to psychologically informed treatments, it's worth trying to avoid any potential misunderstandings.

edit: The comment I made about 'exaggerated criticism' was not about your post, but just my fears.

 

I do think this matters. Researchers can be very lax when it comes to describing trials. The randomised controlled trial (RCT), and by that, I would mean a properly randomised, double (or even triple)-blind, placebo-controlled trial is considered the "gold standard" in epidemiology because of the steps it takes to reduce bias. However, what most researchers seem to describe as RCTs are really just randomised comparative trials or randomised clinical trials or sometimes even just random trials done in a clinic setting.

Not being able to properly blind, or adequately randomise, or compare against a valid placebo is not an excuse for still being able to call a trial an RCT when it isn't.

Unblinded trials with complex, composite, subjective endpoints and no placebo intervention ARE NOT RCTs!

 

No matter what our criticisms are or how valid they may be the BPSers will call it harassment. The scientists who back us are the ones we do not want to alienate and I do not see how calling out the sloppy use of terminology would do that.

 

Who gets to define what counts as an RCT as in what organisation has the ultimate authority to say if a trial does or does not meet the criteria ?

Looking at definitions such as on Wikipedia for example I do worry the bar is set quite low.

Another example from cochrane

Randomised controlled trial
An experiment in which two or more interventions, possibly including a control intervention or no intervention, are compared by being randomly allocated to participants. In most trials one intervention is assigned to each individual but sometimes assignment is to defined groups of individuals (for example, in a household) or interventions are assigned within individuals (for example, in different orders or to different parts of the body).

 

Considering that we know:

1. Participants in sham control arms also tried the active treatments
2. Trial leaders took no effort to account for what patients did (that we know of anyway) and so themselves were not necessarily aware of which patients tried which treatments
3. Trial leaders promised sham control participants they could try the active treatments after the initial run but before any follow-up

There's a reasonable claim to make that PACE isn't properly randomized. Or maybe that's arm contamination. Whatever, basically it's uninterpretable anyway since participants did not stick to their arm. Despite all the money wasted, it was a very badly run trial.