Cochrane Review: 'Exercise therapy for chronic fatigue syndrome' 2017, Larun et al. - Recent developments, 2018-19

Thats right, although sometimes there is so much information of personal/confidential character that the whole document can be withheld. This has happened a couple of times when i have applied for a document, but is a clear exception to the rule and u have the right to appeal

 

It's from page 48 of the Cochrane review:

 

There is an interesting trial of multidisciplinary therapist treatment for hyper mobility called the BENDY trial. It showed that multidisciplinary care made absolutely no difference. But I think the people who ran it may have been sceptics!

 

I forgot they even explicitly claimed that the primary outcome was reported in accordance with the protocol. Why were these people ever allowed to publish their review?

 

Is this at the heart of the problem? Are Cochrane terrified they will open the floodgates to many other of their reviews being validly challenged? Have the authors (and others?) possibly pointed this out to them and how if their review gets pulled then "moves could be taken" to expose others? Purely conjecture on my part I have to emphasise, but human nature being what it is, and given the nature of these BPS humans, it would really not amaze me.

 

Thanks to @Kalliope I was able to read the older correspondence from back in 2018. I thought it was interesting, especially since it gives a whole different impression than what was reported in the Reuters article.

I've tried to summarize it in a blog post, so that those outside the ME/CFS community or those who do not follow these developments closely, can also easily read it. I've posted it on this WordPress page so that it's easier to link and read. See: https://mecfsskeptic.wordpress.com/2019/07/03/the-cochrane-correspondence/

The actual documents are too big to post on the forum, but I've added them in Dropbox. The links are available in the Blog post, so you can check them yourself. Please let me know if you notice any mistakes (regarding both grammar and content).

 

The Cochrane correspondence

The Cochrane review on exercise therapy for patients suffering from chronic fatigue syndrome has been the subject of ongoing controversy. In October 2018, a Reuters news article suggested that Cochrane would temporarily withdraw the review due to pressure and complaints from online activists. Publicly released correspondence between Cochrane’s former Editor in Chief and senior employees of the Norwegian Institute of Public Health offers a different perspective to the dispute.

A contested review
Cochrane is an international organization renowned for its systematic reviews of medical research, commonly regarded as the gold standard in evidence-based healthcare. [1] Its review on graded exercise therapy (GET) for patients suffering from chronic fatigue syndrome (CFS) however, is controversial and contested. [2]

GET instructs patients to gradually increase the duration and intensity of exercise beyond their perceived limit to reverse deconditioning. [3] CFS patients, however, report a marked deterioration in health when they exceed a certain activity level, a phenomenon commonly referred to as post-exertional malaise. [4] The Cochrane review by Larun et al. suggests that GET is a safe and effective treatment for CFS [5], although its methodology and conclusions have been widely criticized. [6]

Following a formal complaint about the review to the Editorial Board, Cochrane decided to conduct an internal audit to assess the validity of the critique. In October 2018, a Reuters news article reported that Cochrane would temporarily withdraw the contested review, a decision that was deemed “disproportionate and poorly justified” by the authors. [7] Publicly released correspondence between Cochrane’s former Editor in Chief David Tovey and senior employees at the Norwegian Institute of Public Health (NIPH), provides a different perspective to the controversy. NIPH hosts Cochrane Norway and employs two of the authors of the Cochrane review in question.

Justified criticism
The email exchange confirms that the review suffered from multiple methodological faults. Cochrane has added a notification to the review, claiming it is “substantially out of date and in need of updating.” [8] In the correspondence, Tovey stressed that the authors failed to address the criticism of their review. An independent assessment of submitted comments indicated that these were cogent and that “there was a case to answer.” [9] The authors were given the opportunity to address the errors, but their resubmission was considered inadequate by multiple reviewers. As Tovey explained:

“I asked my internal team to review the submission, and also three senior colleagues who I believe have the expertise and detachment required to provide an objective evaluation. As the attached documents show, all of these assessors continue to have important concerns […].” [10]​

Larun et al. previously faced similar problems with a related Cochrane review that intended to use individual patient data (IPD) instead of aggregate outcomes on GET for CFS. The IPD manuscript also received negative reviews and had to be resubmitted. Again, multiple experts indicated that Larun et al. failed to grasp the methodological issues raised. Tovey announced Cochrane’s decision to cancel the IPD review as follows:

“[…] we were disappointed when the peer reviewers of the re-submitted review all agreed that the authors had not adequately addressed the methodological issues. We therefore decided that we had no option but to reject the IPD review.” [9]​

Bias and activism
In total, Larun et al. erred on 5 different occasions while writing these two Cochrane reviews on GET for CFS. The email correspondence suggests the authors might have a bias in working towards a favorable conclusion. In an email of 25 October, Tovey wrote that “the overall tone of the conclusions appears to us to be overly optimistic.” [11] In another, he described the authors’ intention as “stretching the truth beyond its limit.” [12] The main author of the review, Lillebeth Larun has researched and promoted the use of exercise therapy for CFS before working on the Cochrane review. [13, 14] Another author, Kjetil Gundro Brurberg has since its publication been an outspoken defender of the treatment, claiming that “in the absence of evidence of efficacy for other interventions, health professionals should offer patients with CFS/ME the opportunity to try […] GET.” [15] Brurgberg wrote that some of the criticism towards the evidence for GET may be “counterproductive” and “discourage further research.” [15] Similar sentiments were expressed by Larun. In the email exchange, Tovey noted “that Lillebeth, and the trialists of the included reviews, tend to refer to all criticism of the trials and the review as being the result of ‘activism’.“ [9] Cochrane disagreed and emphasized that the criticism was “thoughtful and reasonable.“ [9] In an interview with Reuters, however, Larun expressed concern that the activists who have plagued her for years have now got to her editors. [16]

Leaked emails
In an email dated November 2, 2018, the Acting Director General of NIPH, Trygve Ottersen, advised Cochrane not to give in to the criticism:

“[…] on a controversial topic as CFS/ME, a continuous stream of criticism is to be expected. We hope however that the Cochrane editorial office will stay committed to Cochrane’s standard procedures also in this case […].” [17]​

Shortly before that, internal emails were leaked to the press. On October 17, Reuters reported that Cochrane would temporarily withdraw the review “amid fierce criticism and pressure from activists and patients.”[7] It is unclear who leaked the information but the tone of the article aligns with the authors’ suggestion that Cochrane is conceding to unfounded but fierce criticism from the CFS community. Despite Tovey’s caution that “this not about patient pressure”, the article strongly suggests otherwise. Colin Blakemore, a former chief executive of Britain’s Medical Research Council, is quoted saying that the decision “set a worrying precedent for scientific evidence being over-ridden by the opinions of activists.” The Reuters article was taken over by multiple media outlets [18] and provoked anger and concern on social media. Clare Gerada, former chairperson of the Council of the Royal College of General Practitioners, commented on Twitter that “Cochrane has lost its way” [19] and that “we are entering an era of bullying scientists when lobby groups disagree with the results.” [20]

A second resubmission
The publicly released email correspondence indicates such criticism to be unfounded. Concerns about the review were raised by multiple experts. “When a collection of experienced and dispassionate colleagues are all making, in effect, the same criticisms”, Tovey wrote, “it is hard to ignore this.” [21] It is unclear if the leaked emails and public criticism impacted Cochrane’s handling of the matter. The review on GET for CFS, however, has not been withdrawn. On June 17, 2019, Cochrane announced that it has received the second resubmission of Larun et al. The amended review will be published during the next 2 months. [8]

[EDIT: This post has been updated to correct spelling mistakes.]

 

References

[1] Cochrane Library: an improved online platform to guide health decision-making across the world. August 8, 2018. https://www.cochrane.org/news/cochr...orm-guide-health-decision-making-across-world

[2] Tuller D. Trial By Error: The Cochrane Controversy. Virology Blog. September 3, 2018. http://www.virology.ws/2018/09/03/trial-by-error-the-cochrane-controversy/

[3] Clark LV, White PD. The role of deconditioning and therapeutic exercise in chronic fatigue syndrome (CFS). J. Ment. Health. 2005 June; 14(3):237-252. https://doi.org/10.1080/09638230500136308

[4] Institute of Medicine. Beyond Myalgic Encephalomyelitis/Chronic Fatigue Syndrome: Redefining an Illness. The National Academies Press, Washington, D.C., 2015. https://www.ncbi.nlm.nih.gov/books/NBK274235/

[5] Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database Syst Rev. 2017 Apr 25;4:CD003200. https://doi.org/10.1002/14651858.CD003200.pub7

[6] Struthers S. Reply to David Tovey’s letter about the Exercise for CFS review. February 18, 2019. https://healthycontrolblog.wordpress.com/2019/02/18/response-to-cochrane/

[7] Kelland K. Exclusive: Science journal to withdraw chronic fatigue review amid patient activist complaints. Reuters. October 17, 2018. https://www.reuters.com/article/us-...mid-patient-activist-complaints-idUSKCN1MR2PI

[8] Cochrane Database of Systematic Reviews. Exercise therapy for chronic fatigue syndrome, see what’s new. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub7/information#whatsNew

[9] Email from David Tovey to Trygve Ottersen, dated October 4, 2018. Document: 36085(Ekspeder dokument for Innsyn sak 17/10566 dok 2-10). https://www.dropbox.com/s/0pdjqjd15...ndence from 02-10-2018 to 02-11-2018.pdf?dl=0

[10] Email from David Tovey to Atle Fretheim, Lillebeth Larun and Kjetil Gundro Brurberg, May 24, 2019. https://www.dropbox.com/s/8jynkfcrtpliuyy/Cochrane correspondence 24-05-2019.pdf?dl=0

[11] Email from David Tovey to Gunhild Wøien, Lillebeth Larun and Kjetil Gundro Brurberg, October 25, 2018. Document: 36085(Ekspeder dokument for Innsyn sak 17/10566 dok 2-10). https://www.dropbox.com/s/0pdjqjd15...ndence from 02-10-2018 to 02-11-2018.pdf?dl=0

[12] Email from David Tovey to Atle Fretheim, date unknown. https://www.dropbox.com/s/8jynkfcrtpliuyy/Cochrane correspondence 24-05-2019.pdf?dl=0

[13] Larun L, Malterud K. Treningsbehandling ved kronisk utmattelsessyndrom. Tidsskr Nor Laegeforen. 2011 Feb 4;131(3):231-6. doi: 10.4045/tidsskr.09.1475. https://tidsskriftet.no/2011/02/originalartikkel/treningsbehandling-ved-kronisk-utmattelsessyndrom

[14] Larun L, Malterud K. Finding the right balance of physical activity: a focus group study about experiences among patients with chronic fatigue syndrome. Patient Educ Couns. 2011 May;83(2):222-6. https://doi.org/10.1016/j.pec.2010.05.027

[15] Brurberg KG. A PACE-gate or an editorial without perspectives? The Mental Elf. July 19, 2017. https://www.nationalelfservice.net/...ce-gate-or-an-editorial-without-perspectives/

[16] Kelland K. Sick and tired: Online activists are silencing us, scientists say. Reuters. March 13, 2019. https://www.reuters.com/investigates/special-report/science-socialmedia/

[17] Email from Trygve Ottersen to David Tovey, November 2, 2018. Document: 36085(Ekspeder dokument for Innsyn sak 17/10566 dok 2-10). https://www.dropbox.com/s/0pdjqjd15...ndence from 02-10-2018 to 02-11-2018.pdf?dl=0

[18] Kelland K. Journal to Withdraw Chronic Fatigue Review Amid Patient Activist Complaints. Medscape. October 18, 2018. https://www.medscape.com/viewarticle/903621

[19] Clare Gerada #FBPE (@ClareGerada). “I think Cochrane has lost its way. Sad. As used to be most trusted source.” Twitter. October 18, 2018. https://twitter.com/user/status/1052837095247613952


[20] Clare Gerada #FBPE (@ClareGerada).“withdrawal decision sets a worrying precedent for scientific evidence being over-ridden by the opinions of activists. (Colin Blakemore). > sad we are entering an era of bullying scientists when lobby groups disagree with the results.” Twitter. October 17, 2018. https://twitter.com/user/status/1052636273180139520


[21] Email David Tovey to Atle Fretheim, May 27, 2019. https://www.dropbox.com/s/9w00sm7xkgyh5s2/Cochrane correspondence 27-05-2019.pdf?dl=0

 

At bottom of the post by Kalliope are images of correspondence:

I only skimmed through but noticed a few interesting things.

 

Thank you for a wonderful clear and concise overview Michiel! This was really good.

I might send this to some journalists if its ok, or something like that

 

Really awful answers by Gerada. Asked about the reliability of unblinded trials with self-reported outcomes (in pharma trials no less, to remove the ideological blinders) and the feigned outrage is directed at Cochrane for obviously finding such evidence to be weak (despite actually publishing it). Meaning she technically approves of such trials as long as they show what believes to be true.

And to speak of bullying when it's hard to conclude otherwise that there is ample bullying behind the scenes, as it's otherwise very hard to reconcile the horrible peer reviews and the review being published anyway.

Some people truly have no principles beyond naked self-interest.

 

I just noticed that the documents I provided in Dropbox, are the ones with my notes in it. Will try to correct this asap, to provide a clean version.

[Edit:it should be alright now]

 

In that case they cant be trusted with an ME review can they?

Or did they only lose their way the day and minute they said something she didn't agree with whilst being unable to justify her own argument in support of Larun anyway?

 

Thanks, That's ok.

But I would advise waiting just a bit until more people on S4ME have read it, so they can check if I didn't make any major mistake - that's always possible with a ME/CFS brain, no matter how careful I try to be.

 

Only when they agree with them, of course. That's the only grading allowed.

"Fake news" isn't "fake" because it's false, it's "fake" because it's bad.

 

"worrying precedent for scientific evidence being over-ridden by the opinions of activists" ... no, flawed, low quality evidence being justifiably challenged by good scientific reviewers.

"entering an era of bullying scientists when lobby groups disagree with the results" ... no, challenging scientists because the results were flawed, produced from low quality science.

 

Not just Cochrane, but a lot of people in the field of mental health research/rehabilitation have got away with low standards for a long time, and have come to be viewed as 'independent experts' on what standards are appropriate.

The idea that we should not be trying to raise standards is pretty nauseating.

 

My understanding from the correspondence I have had as a referee is that this is not a question of 'Cochrane' having a viewpoint but rather of there being strongly conflicting views amongst people who are part of the Cochrane collaboration. Some people in the administration were clearly concerned that peer review was substandard in the mental health section. Others presumably wanted to keep things as they were.

If anything, my impression of the mental health section was that it was run by rather fey people who thought that a pretty logo and fancy fonts for subtitles on the referee report form were more important than getting a valid opinion. It just seemed touchy feely amateur.

 

This is a stunningly good summary!

 

Did you know Clare Gerada is married to Simon Wessely?