Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Have written again to the Cochrane Editor in Chief this morning re-submitting my notification of a serious error which should lead to withdrawal of the reviews. I also forwarded it to Hilda Bastian so she can add it to the correspondence about the Exercise review update.


Subject: Notification of a serious error in reviews Exercise therapy for chronic fatigue syndrome and CBT for chronic fatigue syndrome: resubmission

Dear Karla (cc Rachel Marshall)

According to Cochrane’s policy on withdrawing published reviews (1), and the procedure which states that The Editor in Chief must be notified of all suspected or confirmed serious errors (2) I am resubmitting my notification to you of a serious error in the Cochrane reviews Exercise therapy for chronic fatigue syndrome (3) and CBT for chronic fatigue syndrome (4) which should lead to them being withdrawn.

For the avoidance of doubt, the criteria met for withdrawal in this case is one of the examples of a serious error specified in Cochrane’s own policy documents (1, 2)

“Following the conclusions of the review could result in harm to patients”

As I informed you in my original notification on 19 November 2020, the NICE draft guidelines (5) have reversed the recommendations for Exercise therapy and CBT to treat ME/CFS because of lack of good quality evidence of benefit and recent evidence of harm (6, 7). Therefore, following the conclusions of the Cochrane reviews which state that Exercise therapy and CBT may be helpful for ME/CFS is continuing to put patients in harm’s way.

Please will you acknowledge receipt of this notification and let me know when you anticipate concluding your investigation and deciding a course of action.

With best wishes

Caroline

1. Withdrawing published Cochrane Reviews last modified on Jul 30, 2019 [cited 5 Feb 2021] https://documentation.cochrane.org/display/EPPR/Withdrawing+published+Cochrane+Reviews
2. Serious errors in published Cochrane Reviews last modified on Jul 05, 2019 [cited 5 Feb 2021 https://documentation.cochrane.org/display/EPPR/Serious+errors+in+published+Cochrane+Reviews
3. Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database of Systematic Reviews 2019, Issue 10. Art. No.: CD003200. DOI: 10.1002/14651858.CD003200.pub8.
4. Price JR, Mitchell E, Tidy E, Hunot V. Cognitive behaviour therapy for chronic fatigue syndrome in adults. Cochrane Database of Systematic Reviews 2008, Issue 3. Art. No.: CD001027. DOI: 10.1002/14651858.CD001027.pub2.
5. Myalgic encephalomyelitis (or encephalopathy)/chronic fatigue syndrome: diagnosis and management In development [GID-NG10091] Expected publication date: 21 April 2021 https://www.nice.org.uk/guidance/indevelopment/gid-ng10091/documents
6. Geraghty K, Hann M, Kurtev S. Myalgic encephalomyelitis/chronic fatigue syndrome patients’ reports of symptom changes following cognitive behavioural therapy, graded exercise therapy and pacing treatments: Analysis of a primary survey compared with secondary surveys. Journal of Health Psychology. 2019;24(10):1318-1333. doi:10.1177/1359105317726152
7. ME Association. Consolidated report: Evaluation of a survey exploring the experiences of adults and children with ME/CFS who have participated in CBT and GET interventional programmes, April 2019: https://meassociation.org.uk/wp-con...nd-GET-Final-Consolidated-Report-03.04.19.pdf

 

It's striking the difference in attitude re patients well being when you compare with this researcher

https://www.s4me.info/threads/possi...ovid-19-long-covid.14074/page-172#post-322138

who actually stopped his trial from starting, reviewed all the evidence and subsequently redesigned the trial with safety as its top priority. As he pointed out, how can you go ahead with a therapy when patient survey after patient survey reported harms in over 50% of participants?

 

Another great letter.

For info, NICE finally added a warning to the current Guidelines around !0/11 Dec 2020, following much communication from charities etc
https://www.nice.org.uk/guidance/CG53

"This guideline covers diagnosing and managing chronic fatigue syndrome (CFS) which is also known as myalgic encephalomyelitis (ME) (or encephalopathy). It aims to improve the quality of life for people with CFS/ME by setting out the care and treatment options that should be available for them.

NICE is aware of concerns about graded exercise therapy (GET) and is updating the current recommendations. Please see the guideline in development page for information on our update (including draft recommendations on GET) which we expect to publish in April 2021. We are also developing guidance on the management of the long-term effects of COVID-19."

https://www.meaction.net/2020/12/11/new-graded-exercise-warning-on-current-me-cfs-guideline/

Also Scottish Health Depts issued a warning about GET in Aug 2020
See: https://meassociation.org.uk/2020/1...d-exercise-therapy-in-me-cfs-post-long-covid/

“Please also note that in August 2020 the Scottish Health Technologies Group (SHTG) published a rapid review into the use of Graded Exercise Therapy (GET) as a treatment for people with ME-CFS. This concluded that:

• In light of a Cochrane systematic review and the NICE guideline revision, caution should be noted on the use of GET for ME/CFS until the updates are published, and,
• No published evidence on the use of GET for people experiencing persisting fatigue following Covid-19 was identified. NICE has made a statement cautioning against assumption that ME/CFS recommendations apply to people with fatigue following Covid-19.”

 

Apologies, Caroline and @Robert 1973. I've only just seen this.

I will respond shortly. Many thanks.

 

@Caroline Struthers and @Robert 1973

Sorry for the delay. I only check in with this forum occasionally and I haven't been following this thread.

There's a lot to unravel here and there isn't one simple answer (other than the usual fallback of "it depends", of course...!).

In answer to the question "would litigation be a suitable next step in this process?" - the short answer is "probably not, certainly not at this stage". The long answer is below. Other views will be available.

As always, I'm taking a realistic view here, not just being negative for the sake of it. I would love to be able to say "yes, we should do this" - but I don't do false positives.

**********

A bit of background stuff: this is from a blog post I wrote two years ago:

"[M]ost public/private institutions are required to have a formal complaints and/or appeals procedure. Unfortunately, these are frequently (and, some might argue, deliberately) very difficult to access and inherently opaque, laborious and time/energy-consuming. However, unless the full internal procedure is followed closely – often involving many stages of escalation – it is inevitable that the complaint/appeal will be rejected."

So the first rule is to follow the internal process (which will always be cumbersome and frustrating by design) to its conclusion. The next stage after that would be the Charity Commission (and I see you've already been in touch with them, Caroline). The requirement there seems to be for the charity itself - not the original complainant - to report a "serious incident" and this is unlikely to meet the threshold for Cochrane to do that, from what I can see.

**********

I'm struggling a bit with where the two reviews are actually at. Have I got this right?

Exercise review: from what I can see, the amended version of October 2019 is being used in the high-profile reviews pilot with the IAG led by Hilda Bastian. As I understand it, Caroline, your argument is that, with the recent publication of the NICE draft guideline, this version should be withdrawn completely on the basis of potential for serious harm to patients. The dispute at this point turns on what amounts to "serious harm". Given that this pilot process is ongoing, it is quite a challenge to argue the case for withdrawal although the final publication of the NICE guideline might help.

CBT review: the review still exists but states at the beginning of the abstract that: "This review was last updated in 2008 and is no longer current. The author team is no longer available to maintain the review". As it stands, I can't see any mechanism for challenging that further - however much we may object to the review remaining extant. On any interpretation, it is long out of date.

As with all litigation now, this is a specialist area and it's possible that a lawyer who is an expert in this specific field could devise a strategy, even for this stage in the process. However, it's important to bear in mind that any litigation is a prolonged, expensive and traumatic process for the parties involved.

**********

Once all the internal procedures have been exhausted, the position changes and it is appropriate to give consideration to litigation. Even then, I think that legal action targeted at Cochrane is a very long shot. Hypothetically, this is the next logical step but I have nothing left with which to write more now.

If anyone wants more on the hypotheticals of legal action further down the line, I'll do a second part to this but I've run out of steam for the moment.

**********

I always take a very conservative approach to considering next steps. The reason for that is simple: most of us live with ME and have very limited resources. I prefer to focus whatever I have on actions which have a realistic chance of success - exercising a kind of triage system on all the possibilities - before deciding how best to direct precious energies.

This is not as clear as I would like as my brain-fog is really bad now. However, I decided it was better to get something out, even though it's not at a standard that I am happy with. Apologies for whatever confusion arises!

NB. I would caution against getting anyone like George Monbiot involved. He may have written one sympathetic article but he has his own agenda and he is not a lawyer.

 

I certainly agree about all of us having little bandwidth for the labours that would be involved in legally pursuing things as they are.

In law if I understand correctly the system can correct itself when tested by changing the law (in this case it would be how and under what circumstances charities / not-for-profits must respond to complaints) which in this instance is apparently quite weak / ineffectual.

There is also the option of change through political process -- also frustrating and labour intensive. But an option.

And as far as including others who may be able to help I would consider anyone who has expertise in human rights issues might be a place to look.

 

Not sure what you mean by this. As I understand it, at this stage, we are not yet in a legal process. That would come later, once the complaints procedure is exhausted. And systems do not correct themselves, unfortunately.

Human rights issues sit alongside every aspect of public and private life and can always be invoked. They become more formalised if/when a legal process is commenced. Article 8 of the European Convention (right to private and family life) is generally the relevant right when dealing with health-related matters although others can be considered.

As far as I can tell, @Caroline Struthers is doing everything possible so far within the limitations of the complaints procedure. Efforts to jump ahead will only be knocked back. When the Charity Commission suggested seeking legal advice, I'm not sure what they would have had in mind.

It's almost impossible to offer a useful opinion without a full case history and timeline.

[ETA: I have put together what I can from going through the latter part of this thread but it's not enough to compile anything more than some initial thoughts and observations.]

I may be making matters more rather than less confused here so probably won't comment further for the time being.

 

Poor wording on my part. I didn't mean to imply that there was a present ongoing legal process. I meant precedent when a judge decides in favour. But as you said it is not likely that a judge would rule in favour. Now that I'm totally confusing the issue I guess I meant that getting a judge to see the law is not up to task.

Apologies if this is still not clear. Way out of my area.

 

Thank you very much for this great response. I have not had a chance to fully digest it, but one key thing jumped out at me. The Cochrane policy on withdrawal does not talk about "serious harm" but "harm". So there is no argument to be had about what constitutes serious harm, so the dispute does not turn on that. An example of a "serious error" according to Cochrane's own definition is that "following the conclusions of the review could result in harm to patients" [my emphasis]. With the new NICE draft guidelines, I don't think they can argue that this is not the case for either review.

Withdrawal policy: https://documentation.cochrane.org/display/EPPR/Withdrawing+published+Cochrane+Reviews
Serious errors: https://documentation.cochrane.org/display/EPPR/Serious+errors+in+published+Cochrane+Reviews

 

Thanks, Caroline. Yes, you're quite right; I transposed the word "serious" from preceding "error" to preceding "harm".

The same sticking-point remains, however, that Cochrane does not accept that the review could result in harm to patients - bizarrely - and therefore there is no serious error to report.

I suspect that the get-out clause they may rely on is this: "Following the conclusions of the review could result in harm to patients or populations of interest (other than known adverse effects)."

This is quite ridiculous, of course, but they're clearly not backing down.

Again, I think that the case for withdrawal will become stronger after the final publication of the NICE guideline in April (assuming there is nothing particularly unexpected in it).

 

Thank you @Caroline Struthers and @Valerie Eliot Smith for your work and thoughts on this. Much appreciated.

I wonder whether we could convince [?] Cochrane at least to follow up erroneous references to their reviews made in recent 'high impact' publications, reviews, ME guidelines in diverse countries etc?

Lost track on the references -- forum thread collecting these is here (but I think only covering the GET review, not the CBT one, and not up to date? -- Anyway, it's their responsibility, so should be their task, not ours.) @Michiel Tack

 

To be fair to Cochrane, they were not specifically asked in my first notification of a serious error in November whether or not they accept that the reviews could result in harm. I quoted this example of what constitutes a serious error, thinking, naively, that they would get my point which they of course, didn't. So they didn't even mention harm in their response to me on 1 February. Then I re-submitted the notification spelling out more clearly the nature of the serious error on 5 February. I have attached the email thread. Do you think they will use the known adverse effects card?!

 

But looking at the Cochrane reviews, can the panel that examines @Caroline Struthers's notification justify that there are (known) adverse effects at all?

The review on CBT does not mention adverse effects, at least not in the full abstract.

The review on exercise therapy says "The evidence regarding adverse effects is uncertain". Specifically, in the "Main results" part of the full abstract:

The evidence of harm wasn't obtained from RCTs, most of which did not record it or not adequately, but independently through other sources. So of course it couldn't have appeared in the Cochrane review.

Now, does the deterioration from GET and/or CBT qualifies that is reported by ME patients qualifies as harm for Cochrane (how do they define "harm")?

 

One can also argue that interventions prescribed with no evidence of benefit are harmful even if they don't cause a patient's condition to get worse. They cost time and money to deliver and to adhere to, and divert resources from other activities which could be beneficial, including good quality research.

 

Cost-effectiveness and diverting funds from research are very valid points. To me, most importantly, they delay the delivery of adequate care to people with ME, especially the severe patients who need palliative care but whose GP or fatigue clinics don't have anything to offer to besides GET or a few sessions of CBT. And that endangers patients.

It's the same thing as quacks telling people who have a serious progressive condition (e.g. cancer, HIV/AIDS) that they don't need medical treatments because their miracle cures (diet, pills, therapies) work wonders. Matthias Rath, anyone? The only difference is that ME is not seen as a serious disease, so instead of stepping up and defending us from harm, the medical field doesn't really care about what kind of treatment we're offered.

 

I've been thinking about this whilst watching the cricket this morning. Is it possible that leaving this review on file, for all to see, could prove more useful to us in the longer term?

Bear with me.

It's easy for ME patients to assume that because their GP has suggested some form of exercise therapy, the doctor is committed to BPS theory. This isn't necessarily true; some – perhaps many – GPs will have suggested referrals because they are trying to support their patients, don't know much about ME, and in any case are expected to follow NICE guidance.

It could even be true of some of the clinics delivering the therapies. If you're trained how to deal with a condition, that's what you do. Even when faced with patients saying the interventions aren't working, it's hard for lowly practitioners to assemble compelling evidence, specially if a lot of the patients just withdraw and are never seen again. If you lose faith, it's easier to look for another job.

No, the real BPS lobby is a powerful but small minority, and the tide is gradually turning against them. Once the guidelines change, many of the people who actually do the work of supporting patients will probably come to accept them without much resistance – after all, it's exactly what PWME have been telling them all along.

Hopefully the old advice will start to look antiquated and absurd before too long, and some doctors will feel a level of regret for not seeing it sooner. And perhaps, the respect they used to have for institutions like Cochrane will be dented a little.

That's when we can ram home the point that we've been the victims of systematic prejudice ("Here's the evidence – look how hard we tried to get it changed"), and we're due reparations. It's not enough just to stop harming us. You were complicit, and you owe it to us to help demand progress.

Of course I understand that leaving reviews like this in place presents a real risk of harm, it's quite right to keep challenging it, and I'm grateful to the people who are. But maybe we also need to plan for what happens a bit further down the line?

 

This might be true in the UK where things seem to be changing, but keep in mind that it won't work for the many countries that do not have guidelines for ME/CFS.

When doctors from these countries want to know what kind of treatment are available, they look up the Holy Grail that is Cochrane, and 99% of the time they skip to the conclusion without reading the review. They're already overworked and won't spend time reading up on a condition that they never or rarely encounter in practice. This has happened multiple times to me with French doctors (both reviews have been translated into French). I should also add that the BPS and MUS psychiatrist lobby is strong here.

These reviews endanger ME patients across the world and need to be gone as soon as possible.

ETA: Though even in the UK, most BPS proponents are influential and often respected by their peers across various fields of medicine (at least those who don't know better). If you look at the number of 'likes' from MDs on Clare Gerada's recent tweet, I don't think we can say that the tide is gradually turning against them. Again, MDs who don't care for patients with ME (essentially any doctor besides a GP or an internist) won't take the time to read about ME and will defer their judgment to those of Cochrane and their BPS peers, who they think have expertise in the field of ME.
https://twitter.com/user/status/1355269381861871622

 

Interesting that in this systematic review of Homeopathy for IBS (in the Cochrane database of systematic reviews) , it is stressed throughout that

https://eprints.whiterose.ac.uk/150...9_Cochrane_Database_of_Systematic_Reviews.pdf

and yet in the Larun et al Cochrane review there is just this:

'safety' seemed to be of little concern in these trials (eg PACE) and Cochrane doesn't appear to have had any further concern on the subject.