2025: The 2019/24 Cochrane Larun review Exercise Therapy for CFS - including IAG, campaign, petition, comments and articles

Discussion in '2021 Cochrane Exercise Therapy Review' started by S4ME News, Dec 22, 2024.

  1. Peter Trewhitt

    Peter Trewhitt Senior Member (Voting Rights)

    Messages:
    4,676
    It is impressive that Mary Dimmock and Todd Davenport are being so outspoken and are still seeking to progress the matter by inviting comments on the draft protocol.

    Should we have a separate thread on the draft protocol and the survey to collect comments on it.
     
    janice, Keela Too, Hutan and 7 others like this.
  2. bobbler

    bobbler Senior Member (Voting Rights)

    Messages:
    4,518
    Brilliant

    it’s the first time I think that I’ve seen more info on what the draft protocol mentioned in feb 2023

    others outside me/cfs shouldn’t think this is a revelation because I assume these (eg having both objective and subjective measures if outcomes) are just basics for most areas to the point you’d assume they were norms. Hence why they branded cfs mental health to slide it under the radar.

    anyway I can’t help but think that para on what the new protocol would finally have brought in re:basic standards is /was the sticking point

    and when combined with this new protocol defining what standards would be applied ‘what they didn’t want it replaced with’

    combined with the mass and names of those who publically launched a campaign in 2018 when the old one was withdrawn temporarily (as it should have been) being put out there as a reminder of who doesn’t want it taken down eg the tweet from claire Gerada that is on an s4me thread somewhere

    then in the middle is the FOi email trail where the independent arbiters suggestion weee ignored re what could be claimed as the effect when it was published in 2019.

    then of course, in the middle of all this, similar contacts with faux outrage (faux in it being fir either the sake of patients or science) in the FOI of emails/text sent to those who were running the new Nice guideline - as some of these felt like a continuation of the same ie the timeline feels continuous to me
     
    janice, alktipping, Deanne NZ and 2 others like this.
  3. bobbler

    bobbler Senior Member (Voting Rights)

    Messages:
    4,518
    Yes I think having it clearly titled as such would be worthwhile

    important it is easily found by external search?
     
    janice, alktipping, Deanne NZ and 2 others like this.
  4. Peter Trewhitt

    Peter Trewhitt Senior Member (Voting Rights)

    Messages:
    4,676
    I have not yet read everything, but it does feel that the psycho behavioural intervention lobby decided that any new exercise review would undermine their already feeble evidence base so put all their effort into blocking any progress, eventually resulting in the Cochrane Board caving in with the December announcement and their deceptive redating of Larun et al.

    Certainly these two letters reinforce the likelihood that this was a decision based on internal Cochrane politics with perhaps some additional external off the books lobbying, without any appraisal of scientific evidence.
     
    janice, alktipping, Deanne NZ and 8 others like this.
  5. Trish

    Trish Moderator Staff Member

    Messages:
    58,470
    Location:
    UK
    I haven't looked at the protocol yet, but the letters are outstandingly good. Huge thanks to Mary Dimmock and Todd Davenport for sticking with this.
     
    janice, Keela Too, Amw66 and 14 others like this.
  6. Medfeb

    Medfeb Senior Member (Voting Rights)

    Messages:
    613
    For ease of access, the direct links to the open letters are

    Letter to Cochrane:

    Letter to the Community:
     
    Joh, janice, Arvo and 12 others like this.
  7. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    14,369
    Location:
    Canada
    I'm confused about that. What right does Cochrane have to work they had nothing to do with producing? Yeah, the editorial back-and-forth, or ooooh a "search stragegy", which I'm sure is soooo complicated, whatever it was entirely to please their agenda and it's not as if it added any quality to it.

    The authors did all the work, and Cochrane reneged on their end, thus breaking any agreement entered by any party. I don't see how that still leaves them any ownership, and it's not as if they have a legal claim for anything, certainly can't enforce squat. When you break a contract, especially on malicious grounds, you aren't entitled to a damn thing out of that agreement. And they even broke it unilaterally and without a valid reason.

    I hope the authors ignore this because it truly has all the legal weight of a squeaky fart.

    I also can't understand the need for this much secrecy. We were told absolutely nothing about what was going on during all this time, and there was clearly no reason for it. It all seems completely self-serving and corrupt from Cochrane, and that certainly fits their usual pattern.
     
    Sean, Amw66, alktipping and 5 others like this.
  8. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    14,369
    Location:
    Canada
    Which is truly absurd since no one produces more content disproving their junk than themselves. Lately there have been many studies and reviews from psychosomatic ideologues basically reporting sad trombone noises about most of their pet models being found invalid, their treatments useless and their reputation with patients completely rotten. And none of it makes any difference, because they operate in a completely insulated bubble of fantasy within systems that are unable to show the tiniest bit of spine.

    Hell, many of them produce 'systematic' reviews, which are actually very selective and therefore not systematic at all, that exclude most of the studies on account of being total garbage. Even one of their preferred positive spins, cancer-related fatigue, has had reviews published lately showing how it's a complete bust, but even generally speaking for the most generic definitions of 'functional' illnesses with the most generic loose attempts at treatments, they are all having to admit that it's a bust, they just get around it by just sticking with the script about how it's all promising and they know the money won't stop being thrown their way any time soon. Basically they go on only because the systems that enable them are broken and corrupt, unwilling and unable to admit they have been making disastrous mistakes for decades.
     
    Sean, Amw66, alktipping and 3 others like this.
  9. Trish

    Trish Moderator Staff Member

    Messages:
    58,470
    Location:
    UK
    I read 'they retained the rights' as referring to the authors. It's ambiguous.
     
    rvallee, Sean, alktipping and 8 others like this.
  10. Trish

    Trish Moderator Staff Member

    Messages:
    58,470
    Location:
    UK
    This part of the letter to pwME seems to me to leave open a new possibility.

    That says to me that sometime soon after the last version of the protocol was submitted, some BPS people were allowed to see it, panicked, and decided to set up a competing review group to do an update and insist Cochrane use their version.

    I wouldn't be surprised if we hear quite soon that a rival protocol has been accepted by Cochrane and BPS writing group given the go ahead, using the stuff Cochrane retained the rights to for their search strategy. My bet on lead author would be Paul Garner.

    That would also make sense of the 2024 date added to the 2019 review, which could be intended to discourage anyone else, including a group led by Mary Dimmock and Todd Davenport, from submitting another attempt. It would also make sense of the clearly bogus excuses Cochrane published for cancelling the process and their insultingly feeble back-of-the-envelope response to our complaints.

    Am I paranoid are or they out to get us?
     
    janice, Hutan, rvallee and 15 others like this.
  11. Peter Trewhitt

    Peter Trewhitt Senior Member (Voting Rights)

    Messages:
    4,676
    I agree it seems that once a draft protocol was completed things seem to have started happening behind the scenes. That the psycho behaviour intervention caucus will have seen it and kicked their resistance up a gear.

    However my reading last December was that Cochrane was following a standard process to retain rights for the work so far on the draft protocol, but had no intention of doing anything for the foreseeable future, rather were aiming to have Larun et al appear current for as long as possible into the future.

    Both my and Trish’s very different interpretations fit the evidence. Though if Larun et al can be kept as current, then there is no need for a second BPS new review process.
     
    Last edited: Mar 12, 2025
    janice, Hutan, Sean and 4 others like this.
  12. Medfeb

    Medfeb Senior Member (Voting Rights)

    Messages:
    613
    To clarify - it's the authors that retain the rights to use the draft protocol elsewhere
     
    Hutan, Sean, Binkie4 and 5 others like this.
  13. Peter Trewhitt

    Peter Trewhitt Senior Member (Voting Rights)

    Messages:
    4,676
    If I remember correctly from Cochrane’s letter to the writing group last December, certain rights to act in relation to the draft protocol were retained by Cochrane but the actual copyright of the draft itself remained with its authors.
     
    janice, Sean, alktipping and 2 others like this.
  14. Utsikt

    Utsikt Senior Member (Voting Rights)

    Messages:
    1,718
    Location:
    Norway
    They are the ones making up threats towards them..
     
  15. Hutan

    Hutan Moderator Staff Member

    Messages:
    31,728
    Location:
    Aotearoa New Zealand
    Just out from Hilda's Talk page - a submission from the IAG. It's welcome, although covering pretty much the same ground as the S4ME complaints that were waved away with barely the blink of an eye by Cochrane.

    Project Talk Page
    Independent Advisory Group: Submission to the Cochrane Editorial Board
    [​IMG]
    By Hilda Bastian on March 13, 2025

    Today, the Independent Advisory Group (IAG) sent the submission below to the Cochrane Editorial Board, copied to the Governing Board.

    This is the second of three recent actions by the IAG:

    1. January 2025: Open Letter to Cochrane;
    2. This submission; and
    3. March 2025: Comment on the Cochrane exercise review.
    Hilda Bastian on behalf of the IAG


    Submission to The Cochrane Library Editorial Board on Policies on Outdated and Potentially Misleading Reviews

    By the Independent Advisory Group on the Cochrane review on exercise and ME/CFS

    Dear Dr Soares-Weiser and members of the Editorial Board,

    We are the members of the Independent Advisory Group (IAG), a stakeholder engagement initiative for the Cochrane review on exercise and ME/CFS, appointed by the Cochrane Collaboration. Our group includes advocates, researchers, and clinicians with an interest in ME/CFS, as well as systematic review experts whose senior Cochrane leadership roles span the Collaboration’s history, with methodologist, editorial policy, and Cochrane and PubMed publishing policy experience. [1]

    We have recently expressed our dismay and concern to the Chair of the Cochrane Collaboration’s Governing Board at the decision to reject our formal advice to append an editorial note to the review stating that it is out of date and should not be used for clinical decision-making. [2] Instead, the review was re-issued as an apparently updated review, thus ostensibly affirming its contents as being of current relevance to people making healthcare decisions. We understand that this was on the advice of the Editorial Board.

    This submission addresses three policy issues that relate to these actions:

    1. Retiring outdated and misleading reviews;
    2. Re-issuing an unchanged review as an apparently updated one; and
    3. Unaddressed criticisms.
    We urge the Editorial Board to re-consider their recent policy direction and address the implications for the review’s editorial note. Our reasons are detailed below, and we would welcome discussion on the issues we raise. We look forward to your response.

    Yours sincerely,

    Hilda Bastian, Katharine Cheston, Lily Chu, Mike Clarke, Miranda Cumpston, George Faulkner, Peter Gladwell, Penelope McMillan (with Penelope Del Fante), Jaime Seltzer, and Samuli Tani.

    Part 1: Retiring outdated and misleading reviews

    In response to our recent communication with the Chair of the Cochrane Collaboration’s Governing Board, [2] Cochrane reported that:

    “Cochrane’s editorial policies have evolved since 2019 to align with standard publishing practices for academic journals. Cochrane no longer uses post-publication notices to express a judgement about the status or currency of a review in terms of its question or findings.” [3]

    No longer publishing notices about their status or currency would change a fundamental feature of Cochrane reviews. The Cochrane Database of Systematic Reviews is not a standard academic journal. It is a database of reviews, marketed as “the leading database for systematic reviews in health care,” with “reliable findings to inform decision making.” [4] Being adequately up-to-date is critical for reliable findings on questions that have not yet been settled. Cochrane reviews are promoted as being regularly updated so that people “can base treatment decisions on the most up-to-date and reliable evidence.” [5] The Cochrane Library, which contains the Cochrane Database, carries a slogan reflecting this aim: “Trusted evidence. Informed decisions. Better health.” [6]

    Current editorial policies provide for updating of reviews in the Cochrane Database, or withdrawing them in exceptional circumstances. [7] Although developments since a review was published can compromise its suitability for clinical decision-making, withdrawal may not always be applicable to these reviews, and updates may not be planned, or may take considerable time to be completed. Users should be made aware that such reviews, as published, are no longer fit for purpose.

    Editorial notes have been used to make users aware of reviews in these circumstances, as was done in 2021, for example, with the Cochrane review on cognitive behaviour therapy in chronic fatigue syndrome. [8] That editorial note includes: “…the review is no longer current. It should not be used for clinical decision-making.” [9] Editorial notes were used in this way into the second half of 2024. [10]

    In 2024, when using post-publication notes this way was still current practice, we advised adding a similar editorial note to the exercise review, because it is outdated and contains misleading content. Here, we outline the policy basis for our advice, and why this particular review does not meet Cochrane’s standards for being a reliable source of evidence on a controversial topic.

    The chapter on updating reviews in the Cochrane Handbookspecifies that updating may be needed because of further studies for inclusion or updated methods. [9] The Handbookacknowledges that maintaining the credibility of a review where findings are not of very high certainty may require incorporating information and evidence that “may shed light on more nuanced effects on the intervention,” whether or not it would substantially alter a review’s conclusions. This could include, for example, new information on additional outcomes, or information that makes estimates of effect more precise.

    In 2019, the Editor in Chief stated that the exercise review is based on “a research question and set of methods from 2002,” requiring revision of its protocol. [11] The original protocol (published in 2001) predates fundamental changes in diagnostic criteria for this condition. [12] The 2019 statement remains attached to the 2024 editorial note, and stresses the need for an update to enable people “to make well-informed decisions.” This stands in contradiction to the apparent implication of the 2024 editorial note that the review remains of current relevance for decision-making. The 2019 statement indicates, rather, that this review falls into the category described in the Handbook as one where it is “more appropriate to conduct a new review from scratch meeting current standards.” We agree with that assessment by the Editor in Chief, as published in 2019.

    This review meets the Handbook criteria for being outdated and requiring an update for its credibility because even on its current research question and methods, the review:

    • Has findings without a very high level of certainty (from very low to at most moderate certainty on primary outcomes);
    • Is based on a protocol with methods from more than 20 years ago;
    • Did its most recent search for eligible studies more than 10 years ago;
    • Is missing at least two recent trials that are eligible based on its current inclusion criteria; [13, 14] and
    • Lacks information that would be crucial for many decision-makers, including substantive additional data on harms from new and already-included trials, [13, 15] with the evidence in the current version of the review for this critical primary outcome being of very low certainty.
    These problems have contributed to the large quantity of criticisms of this review, many of which have remained unaddressed for years. This will require updating of the review if these valid concerns are to be addressed. We discuss this in more detail in part 3 below.

    In addition to the underlying problems in the current review, some of its other content has become misleading since it was last amended in 2019. For example, the review states that exercise therapy for people with this condition is recommended in treatment guidelines, citing a 2007 NICE guideline for people with ME/CFS. [16] This is reflected in a statement in the Plain Language Summary that apparently encourages use of the intervention: “Exercise therapy is recommended by treatment guidelines and often used as treatment for people with chronic fatigue syndrome.”

    This is misleading for modern-day users of the review because the NICE guideline was updated in 2021. [17] It no longer recommends exercise as a therapy, and the guideline stresses the possibility of harm. Recommendations in other major treatment guidelines are consistent with this, including national guidelines for the US and Germany. [18, 19]

    Keeping a review reliably up-to-date may not be necessary when there is general consensus on a question, and high certainty of evidence to address it, making the review “stable.” However, this version of the review has been contentious for over 10 years and is still being cited in clinical guidance. [20] Discordant conclusions among more recent systematic reviews underscore the need for up-to-date, reliable, and credible evidence synthesis on this question. [17, 21, 22] In failing to retire this review, Cochrane is actively participating in misinforming people on a controversial topic, with a review shown to be outdated and misleading.

    Part 2: Re-issuing an unchanged review as an apparently updated one

    Issuing a new citation for an unchanged review is not usual practice for the Cochrane Database. Editorial guidance provides for adding an editorial note to an unchanged review as a “What’s new” event which does not require a new citation, [23] as was done for this review when the 2019 editorial note was added.

    When a current Cochrane review is re-issued with a new citation date, it is indicated in PubMed, for example, as an updated review. [24] This is in contrast to Cochrane editorial policies [7] on updates, which are unequivocal:

    “An update of a Cochrane review must involve a search for new studies, with those studies incorporated into the Cochrane review accordingly. Any other change to a Cochrane review, and any change to a protocol for a Cochrane review, is classified as an amendment.”

    Issuing this review with a 2024 date gives the impression that the review is up-to-date. Indeed, the Altmetrics page for this review [25] leads to a Wikipedia entry that recently described the review as “updated in 2024.” [26]

    On 31 January 2025, the editorial note for this review was amended to include: “…which includes studies from searches up to 9 May 2014.” The May 2014 search date was already noted in the abstract. The specification of search dates does not clearly convey to readers who are not systematic review experts that this review may be out of date. The date of the most recent search alone is not a reliable indicator of currency. Reviews may already be out of date when they are published, or have findings that remain reliable for many years when there is a high level of certainty and/or the question is no longer studied.

    Cochrane’s response to our letter of concern included the following rationale for re-issuing the review and assigning it a new publication date:

    “The recent editorial note from December 2024 was published with a new citation version of the review to ensure it was discoverable and indexed by third-party services, such as PubMed.” [3]

    If Cochrane wishes to publish notes that are discoverable and indexed, then they could be published with their own citations, using the publication type “comment.” [27] The note could still be included within the review, but the date of the review would not change.

    As Cochrane’s editorial guidance makes clear, an editorial note can be added manually to a PubMed record. [23] Publishers have had direct access to editing citations in PubMed for many years. [28] Indeed, another review, which had an editorial note attached in the same week as the exercise review, was done in the usual way: The note was added in PubMed without changing the date of the review. [29] Further, the 31 January 2025 amendment to the editorial note on the exercise review was added to PubMed without needing a new citation.

    Meanwhile, people searching for systematic reviews are likely to mistakenly believe that this is the most recent systematic review on this subject. Citations of the review will also give people the false impression that this review is recent. The misleading 2024 date increases the urgency of amending the editorial note to make it clear that this review is outdated and unsuitable for clinical decision-making.

    Part 3: Unaddressed criticisms

    One of the first projects started by the IAG was gathering, and analysing the content of, criticisms made of the exercise review up to August 2020. [30] Some of the criticisms had not been published, but most were publicly available. The collection included 12 comments published through the Cochrane Database commenting system.

    Our as yet unpublished content analysis identified over 100 unique points of criticism about the contents of the review, fewer than 10% of which have been made redundant by updates or amendments to the review, or by changed arrangements for it, such as moving responsibility for the review from the Common Mental Disorders editorial group. The implications of some other points of criticisms have been only partially addressed, for example, the change in diagnostic criteria for ME/CFS recognising post-exertional malaise (PEM) as the hallmark symptom of the disease. [12]

    Since August 2020, a further nine comments have been published in the Cochrane Library. In addition, we recently submitted another ourselves. [31] Further, complaints of serious error requiring investigation have been submitted directly to Cochrane. [32]

    Our initial content analysis of critiques was emailed to Cochrane editors by the IAG lead (Hilda Bastian) in May 2021 for circulation to the authors, and again to the Cochrane editorial unit in 2023. Cochrane editors have replied to some comments on the Cochrane Database indicating that the criticisms raised would be addressed in the planned update, for example in a response published in August 2020. [33]

    The Cochrane Handbook points to the Cochrane Collaboration principle of “being open and responsive to criticism” as part of ensuring the quality of Cochrane reviews. [34] This was not done for the criticisms of this review. The inability to adequately address the accumulated criticisms of this review without a full update increases the urgency of amending the editorial note to make it clear that this review is outdated and unsuitable for clinical decision-making.

    References in a post below
     
    Last edited: Mar 13, 2025
    janice, alktipping, rvallee and 10 others like this.
  16. forestglip

    forestglip Senior Member (Voting Rights)

    Messages:
    1,917
    I didn't know they cited the old NICE, and it's interesting in that the statement is blatantly wrong now and pretty obviously potentially harmful.

    If anyone has the full paper, could you quote that specific part about treatment guidelines? Is NICE 2007 the only citation there?
     
    janice, alktipping, bobbler and 3 others like this.
  17. forestglip

    forestglip Senior Member (Voting Rights)

    Messages:
    1,917
    Here is the comment which Hilda also submitted:

    Project Talk Page
    3/13/2025
    Independent Advisory Group: Comment on the Cochrane exercise review


    Today, the Independent Advisory Group (IAG) submitted the comment below on the Cochrane exercise and CFS review to The Cochrane Library. If/when it is published, we will add the link to this post.

    This is the third of three recent actions by the IAG:

    1. January 2025: Open Letter to Cochrane;
    2. March 2025: Submission to the Cochrane Editorial Board; and
    3. This comment on the Cochrane exercise review.
    Hilda Bastian on behalf of the IAG


    Comment submitted on the Cochrane review on exercise and chronic fatigue syndrome by Larun et al (2024)

    TITLE: Review contains outdated and misleading content, and should not be used for clinical decision-making

    This review states that treatment guidelines recommend exercise therapy for people with CFS. This is reflected in a statement in the Plain Language Summary that implicitly encourages use of the intervention: “Exercise therapy is recommended by treatment guidelines and often used as treatment for people with chronic fatigue syndrome.”

    This is outdated and misleading.

    In support of these statements, the authors cite a NICE guideline for people with ME/CFS that was published in 2007. [1] The NICE guideline was updated in 2021. [2] It no longer recommends exercise as a therapy, and stresses the possibility of harm. Recommendations in other major treatment guidelines are consistent with this, including national guidelines for the US and Germany. [3,4] A systematic review with a search date in October 2022 concluded that data collected in trials “are insufficiently informative to exclude relevant harm due to serious side effects.” [5]

    This Cochrane review’s search for studies is now over 10 years out-of-date, and it does not include all eligible recent trials or recently published data from included trials, particularly on harms. [6-8] Up-to-date information on harms is critical for decision-makers, as the evidence on harms in the review is minimal and rated as of very low certainty. Furthermore, the participants included in this review’s included trials do not reflect the current population of people diagnosed with ME/CFS, as following the publication of a report by the US Institute of Medicine in 2015, post-exertional malaise has been recognised as the hallmark symptom of the condition. [9]

    As Cochrane’s Editor in Chief stated at the time of its 2019 amendment, the review “is based on a research question and a set of methods from 2002, and reflects evidence from studies that applied definitions of ME/CFS [sic] from the 1990s.” [10]

    A new review undertaken with current methods on this topic should therefore be very different. Yet, the content of the December 2024 editorial note, together with a new date for the review, give the misleading impression that the review is current.

    Other Cochrane reviews have editorial notes stating that the review is outdated and should not be used for clinical decision-making, including another review on this condition. [11] As the outdated content of this review is misleading about critical clinical advice and key outcomes that may influence decision-making, the editorial note should carry that same statement.

    We discuss these issues in further detail in a submission to the Editorial Board. [12]

    Hilda Bastian, on behalf of the Independent Advisory Group for stakeholder engagement for this review, including Katharine Cheston, Lily Chu, Mike Clarke, Miranda Cumpston, George Faulkner, Peter Gladwell, Penelope McMillan, Jaime Seltzer, Samuli Tani.


     
    Last edited: Mar 13, 2025
    janice, alktipping, rvallee and 6 others like this.
  18. Hutan

    Hutan Moderator Staff Member

    Messages:
    31,728
    Location:
    Aotearoa New Zealand
    Here are the references for the submission on Policies on Outdated and Potentially Misleading Reviews
     
  19. Hutan

    Hutan Moderator Staff Member

    Messages:
    31,728
    Location:
    Aotearoa New Zealand
    https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub9/epdf/full

    We made that point in the S4ME complaints. From memory, Cochrane did not even bother to acknowledge the point.
     
    alktipping, rvallee, Binkie4 and 9 others like this.
  20. Peter Trewhitt

    Peter Trewhitt Senior Member (Voting Rights)

    Messages:
    4,676
    This is worth repeating especially as it comes from the IAG including Hilda Bastian a founding member of Cochrane.

    How do we get wider cover of these serious critiques of what increasingly looks like deliberate obfuscation and potentially scientific misconduct on the part of Cochrane?
     

Share This Page