Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

When I read this, I remembered that Karl Morten has collected metabolomics data on patients doing GET and he said there was a decline in energy metabolism. He showed some of this data at a conference and I understand it will eventually be published. Maybe @Andy knows more.

I remember this was not an adequately controlled study so it wouldn't be cast iron evidence but it completely aligns with the reports of patients deteriorating after GET. It would also be unethical to design a study only for the purpose of observing the harm caused by a treatment.

Edit: these were long term effects, not something short lived.

Edit 2:

From the CMRC 2017 conference.

 

Which is ironic, because these reviews of theirs are in fact oh-so empty ... they just have a lot of words in.

 

"
How have selection bias and disease misclassification undermined the validity of myalgic encephalomyelitis/chronic fatigue syndrome studies? is a paper that compares the selection criteria of the Oxford criteria (OC) and the Canadian Consensus Criteria (CCC) noting that for every 15 patients selected under the Oxford criteria there are 14 false positives when compared to CCC.[1]"


""When studies using the broad Oxford criteria (Sharpe et al., 1991) were excluded, a virtual disappearance of effect for graded exercise therapy (GET), cognitive behaviour therapy (CBT) and other psychological therapies recommended by the NICE guidelines (National Institute for Health and Care Excellence (NICE), 2007) was revealed. ""

 

There weren't any studies in the version of the review I refereed that credibly passed as eligible by the criterion of 'controlled trial'.

 

yes there definitely was some substitution of activities in PACE, so people might have walked but didn't do their laundry.

 

I think it's more than an either/or choice about how high or low the bar is set for entry. When a wider group of studies is included, then choices follow about how they will be critiqued and analyzed, including potentially pre-specifying sensitivity analyses - which examine the impact on results of studies with particular characteristics. Sly Saint quoted an example of that from the AHRQ review:

The inclusion criteria for this particular review will be the subject of really critical and important discussion in the coming months. It's complicated, and it's critical for credibility, too. Thought experiment: imagine there was a trial that you thought was the pivotal and most important trial on a new drug for ME/CFS, and it showed important benefit. And a systematic review had elaborate highly specific criteria that excluded it from their systematic review, and concluded there was "no evidence" the drug benefited people with ME/CFS. What would you think?

We don't know how often Cochrane reviews have no included studies because of inclusion criteria that are stricter than, say, including only studies with control groups, or randomized control groups - or when it's just because a study question hasn't been addressed by any controlled study at all. Those "empty" reviews are common - nearly 9% in 2010. We don't know how common "empty" reviews in other journals, or in the review portfolios of other review organizations like health technology assessment agencies. One study of a sample of reviews from journals from February 2014, for example, found it was 7% of Cochrane and 1% of non-Cochrane reviews. I don't off the top of my head know of a study on this in health technology assessment agencies. Or of how often particular questions within reviews - even the primary ones - come up empty: that would come up very high.

I don't agree that "empty" reviews are seen as an editorial embarrassment in Cochrane: since the vast majority of editorial groups have published them, that's unlikely. Many people feel strongly that it's important to ask questions, whether or not the studies currently exist to answer them, and hope that it might stimulate good studies. On the other hand, given constrained resources, it's now common to give more priority to questions where there are known to be studies needing review. So presumably they will become less common because of that.

 

If the "pivotal" study was actually excluded because it was of low quality, eg unblinded comparison groups, did not use objective outcome measures and the primary outcome measures had never been tested for relevance to patients (by asking them!), then I'd suggest that the review was correct in excluding the study due to poor quality.

 

Say it did all that very well, but the reason it was excluded was because it had 498 people in it and the inclusion criteria stipulated a minimum of 500 participants.

 

Isn't that kind of arbitrary? I mean any highly specific criteria is going to need sufficient scientific justification and I cannot think of any reasonable justification for excluding a study with 450 participants, instead requiring 500+

 

You can make a case for all sorts of things, and it's done all the time for individuals in trials and studies in systematic reviews. People can disagree about what's reasonable justification about almost anything (indeed, could probably quite safely delete the "almost" there). There's often nowhere near as much consensus about many methodological issues as people might expect once they are themselves convinced about something. The point I was trying to make was not about the evidence on this particular question, or what this particular review should do. It was that one of the issues to consider here is that predetermining a review's outcome through a long list of inclusion/exclusion criteria can be a shortcut to a review that is easy to discredit as irretrievably biased.

 

Others argue that a range of other studies (eg without control groups) about the potential harm of exercise for people with ME/CFS and people with ME/CFS' views about it should carry weight in addressing questions of the effects - including people's self-reports about their wellbeing, and the impact on their lives. Many people think these questions have simple answers; many don't.

 

Widespread reports of harms need to be taken seriously even if they aren't obtained from a clinical trial.

Clinical trials cannot be assumed to accurately replicate what happens in clinical practice. The investigators may also be hiding the harm in ways that cannot be discovered with published information.

People are generally also highly biased towards positive treatment effects. When they report an unwanted and unexpected effect, it is more credible than the expected and desired effect.

 

I agree that detailed discussion is useful. On this forum and a predecessor such discussion has kept people engaged for five years and new issues are constantly arising. But the bottom line is not complicated. Open label trials with subjective primary endpoints (or switched endpoints) are valueless. Every trainee physician learns that and understands why it is so. We know there is a huge potential placebo effect on subjective outcomes from the phase 2 rituximab follows study and it could have been predicted. So 'control' has tome 'placebo control' and 'treatment as usual cannot be that. A comparator with the same level of 'new treatment' credibility for both patient and therapist is minimum requirement.

So on two grounds all studies are unable to provide usable evidence of efficacy. This is not a long list of inclusion/exclusion criteria. It does not include anything arbitrary like needing 500 patients. It is just what every competent physician knows to be the basics of trial design. I have asked hundreds of people about this now and it remains a simple truth that only people with vested interests in particular treatments, professional status, methodological research or whatever show any signs of disagreement.

 

Yes!

"...give priority to questions where there are known to be studies needing review"
This is exactly my point. Who are these people who "know" there are studies needing review? I thought the the review question, prioritised by the needs of patients, was supposed to drive the search for studies, not the knowledge that there are "studies out there" which need reviewing. Reviews can be empty (or nearly empty) because included trials which aim to answer an important question don't have much, or any valuable data in them - as Jonathan Edwards has said more than once...." the bottom line is not complicated. Open label trials with subjective primary endpoints (or switched endpoints) are valueless". I think there is some data from objective measures squirrelled away in PACE, and that may have some value.