2025: The 2019/24 Cochrane Larun review Exercise Therapy for CFS - including IAG, campaign, petition, comments and articles

[Utsikt]

Our as yet unpublished content analysis identified over 100 unique points of criticism about the contents of the review, fewer than 10% of which have been made redundant by updates or amendments to the review, or by changed arrangements for it, such as moving responsibility for the review from the Common Mental Disorders editorial group.

That an impressive amount of criticism. To get to that amount, surely you either have to do criticisable things on purpose, or you are terrible at your job. I seems like both are relevant for Cochrane.

 

[Sean]

As Cochrane’s editorial guidance makes clear, an editorial note can be added manually to a PubMed record. [23] Publishers have had direct access to editing citations in PubMed for many years. [28] Indeed, another review, which had an editorial note attached in the same week as the exercise review, was done in the usual way: The note was added in PubMed without changing the date of the review. [29] Further, the 31 January 2025 amendment to the editorial note on the exercise review was added to PubMed without needing a new citation.

In support of these statements, the authors cite a NICE guideline for people with ME/CFS that was published in 2007. [1] The NICE guideline was updated in 2021. [2]”

Cochrane are barely even pretending, are they.

How do we get wider cover of these serious critiques of what increasingly looks like deliberate obfuscation and potentially scientific misconduct on the part of Cochrane?

Can Cochrane be sued over this (for negligence and harm, maybe misleading advertising as the definitive authoritative source of clinical reviews)? Particularly in the UK.

 

[Trish]

I am very pleased to see the IAG writing such comprehensive criticisms. Most of the points they make we have already made and been brushed away with scrappy little unsigned rejections. I hope Cochrane take more notice of the IAG.

 

[bobbler]

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub9/epdf/full

We made that point in the S4ME complaints. From memory, Cochrane did not even bother to acknowledge the point.

This is important point @forestglip noted

when you also think about what I think has probably been a continual and consistent lobbying from certain quarters and interested parties on a timeline from eg the twittter rhetorical outrage when Cochrane was initially withdrawn (was it 2017 or 2018?) to get it unwithdrawn then to stop new Nice guideline happening, with the Cochrane Larun review FOI in 2019 on signing off the conclusions vs the independent arbiter recommendation of what could be accurate, then lobbying to stop final Nice guideline from being published, round table and FOi showing correspondence suggesting the evidence etc be changed in those ,

the fact that keeping this Cochrane review (the latest of which is larun 2019) being now dated 2024 pointing to and suggesting ‘it is in line with the latest Nice guidelines (2007)’ is no mistake and highly significant in trying to print its audience away from those new 2021 guidelines that we know they lobbied against.

it’s quite horrifying when you put all this together and think of someone gaming out the picture on this eg to put a new date on the oldest review and then use that to infer the latest Nice was the 2007 version - all without methodology flags. Whilst the Nice (2021) evidence review and the FOI on discussion of the conclusion for Larun and what could be accurately claimed vs what was used in this picture get carefully hidden away.

then the years after Nice (2021) they all focus on augmenting Recovery Norway as propaganda and developing their Oslo group and coffi (which seems to be the same people? Just with Oslo being a new name - is it a slightly different purpose they discuss in that? Or a branding thing?).

And of course they now just focus on pushing ‘functional’ and psychosomatic for all in ‘the general’ by fishing people out whichever department they walk thru in a hospital for example. Rather than using the illness name - they just wanted that old stuff as an example ie the old research to stay in the archives as what they built this new more ‘transdiagnostic’ mus functional bigger belief system and section of the health services on?

 

[Robert 1973]

How do we get wider cover of these serious critiques of what increasingly looks like deliberate obfuscation and potentially scientific misconduct on the part of Cochrane?

I’ve brought it to George Monbiot’s attention.

I assume that Sean O’Neill will be aware of it too, so hopefully The Times may cover it.

 

[rvallee]

This part of the letter to pwME seems to me to leave open a new possibility.



That says to me that sometime soon after the last version of the protocol was submitted, some BPS people were allowed to see it, panicked, and decided to set up a competing review group to do an update and insist Cochrane use their version.

I wouldn't be surprised if we hear quite soon that a rival protocol has been accepted by Cochrane and BPS writing group given the go ahead, using the stuff Cochrane retained the rights to for their search strategy. My bet on lead author would be Paul Garner.

That would also make sense of the 2024 date added to the 2019 review, which could be intended to discourage anyone else, including a group led by Mary Dimmock and Todd Davenport, from submitting another attempt. It would also make sense of the clearly bogus excuses Cochrane published for cancelling the process and their insultingly feeble back-of-the-envelope response to our complaints.

Am I paranoid are or they out to get us?

It certainly fits with their usual behavior, and that of the ideologues. Assuming the worst of these people is basically a safe bet. But I think you're right about the rights, they just wrote it in a very tortured way.

 

[rvallee]

Just out from Hilda's Talk page - a submission from the IAG. It's welcome, although covering pretty much the same ground as the S4ME complaints that were waved away with barely the blink of an eye by Cochrane.

It's an excellent letter. To me, it reads more like a legal document than an academic one. Which is appropriate. This is about rules and misbehavior more than about substance, which they also ignore but it makes it a secondary issue. And it is presented with the detail and attention of a very good legal argument.

It's hard to tell whether they read us here, or just happened to arrive at the same conclusions. But it's presented in a very devastating and unimpeachable way. It's hard to imagine any neutral observer reading this and not being disgusted at Cochrane's behavior. And yet I assume the opposite will happen, because we have been successfully cast as enemies of the medical profession for so long that factual arguments don't make any difference. Us being right about everything makes no difference. Cochrane behaving like corrupt bullies doesn't even move the needle, in fact it's more likely to be applauded. Because we 'deserve' it, or whatever.

Still, this is as solid as it gets. And it places the blame squarely on Cochrane, especially for the added veil of secrecy that made this all happen in secret. It's clear that those involved in this process did their best, and Cochrane their very worst. This is an entirely one-sided problem, where Cochrane is 100% to blame and guilty of blatant misbehavior, of ignoring their own 'rules' and 'principles', which are clearly a mere formality and flexible as it suits their fancies.

Turns out that for however strong we came out and made Bastian run for the hills about Cochrane's and the ideologues' horrible behavior, we actually understated it.

 

[Trish]

I am so impressed and grateful to Hilda and the IAG members for their letter and comment. I hope the people at Cochane who made the decisions over the last 2 years are feeling ashamed. Though I suspect they are are still in self righteous mode, given their rubbish and unsigned responses to complaints.

 

[Peter T]

I am so impressed and grateful to Hilda and the IAG members for their letter and comment. I hope the people at Cochane who made the decisions over the last 2 years are feeling ashamed. Though I suspect they are are still in self righteous mode, given their rubbish and unsigned responses to complaints.

I was impressed that Hilda and the IAG actually understood the issues much better than I had previously given them credit for.

 

[SNT Gatchaman]

It's an excellent letter. To me, it reads more like a legal document than an academic one. Which is appropriate. […] It's hard to imagine any neutral observer reading this and not being disgusted at Cochrane's behavior. And yet I assume the opposite will happen, because we have been successfully cast as enemies of the medical profession for so long that factual arguments don't make any difference. Us being right about everything makes no difference. Cochrane behaving like corrupt bullies doesn't even move the needle, in fact it's more likely to be applauded. Because we 'deserve' it, or whatever.

“Mere factual innocence is no reason not to carry out a death sentence properly reached.”

— US Supreme Court Justice Antonin Scalia

 

[Midnattsol]

A new thread on the draft protocol comment survey has been created in the advocacy subforum:
Survey on the draft protocol for the cancelled update of the 2019 Cochrane ME/CFS Exercise Review

 

[Hutan]

Regarding the writing team. I haven't got through all of the protocol, but, although I have some reservations, its looking pretty good so far. I think it's interesting though to see who is left standing for moving things forward.

I'm not surprised that the Cochrane staffers aren't there, as I guess it was just a job for them and they have moved on to other things.

One of the patient reps who some of us had reservations about is no longer involved - perhaps her health has worsened, of course I don't know her reasons for her not continuing to be involved. Julia Newton (who some of us had reservations about) is also no longer involved.

So, anyway, thanks to Mary and Todd for choosing to put their heads above the parapet and trying to move things forward.

 

[Peter T]

So, anyway, thanks to Mary and Todd for choosing to put their heads above the parapet and trying to move things forward.

Also a lesser thank you to the other former members of the writing team, presumably including the Cochrane staffers, for ceding their rights to Mary and Todd, allowing them to take the action they have.

Do we know if the Cochrane staffers on the IAG have also withdrawn, or are they still participating in preparing the IAG’s output?

[added - the latest letter to Cochrane and the comment to Larun et al (falsely dated 2025) just say Hilda Bastian on behalf of the IAG as per her Blog posts doesn’t list the IAG’s current membership]

 

[Hutan]

Do we know if the Cochrane staffers on the IAG have also withdrawn, or are they still participating in preparing the IAG’s output?

The recent letter from the IAG does say 'including....' and mentions 9 people. I think that must be most of the IAG members. I lost track of the members, maybe the list is missing a couple of people?

Hilda Bastian, on behalf of the Independent Advisory Group for stakeholder engagement for this review, including Katharine Cheston, Lily Chu, Mike Clarke, Miranda Cumpston, George Faulkner, Peter Gladwell, Penelope McMillan, Jaime Seltzer, Samuli Tani.

 

[Peter T]

The recent letter from the IAG does say 'including....' and mentions 9 people. I think that must be most of the IAG members. I lost track of the members, maybe the list is missing a couple of people?

Thank you, I had previously read the list then completely forgot that and failed to find it when looking.

 

[ME/CFS Skeptic]

There is a new comment by Lillebeth Larun, the primary author:

Reply to Tack: Imprecision and inconsistency in post-treatment fatigue
Published : 19 March 2025

• Lillebeth Larun
• Norwegian Institute of Public Health
  
• Author

Thank you for your interest in our systematic review. We have with interest read your suggestion that the certainty of evidence could have been rated as 'low' or 'very low' due to imprecision and inconsistency (https://www.cochranelibrary.com/cds....pub9/detailed-comment/en?messageId=447929916). We recognize the points raised, and that it is possible to argue for a further downgrading. However, we disagree, and do not consider it appropriate to downgrade just because a confidence interval marginally crosses a value that is set with some degree of uncertainty. These points were taken into consideration and discussed in the GRADE process, a process that also involved consultation with GRADE experts outside the author team. Simultaneously, GRADE caution against a mechanistic approach towards the application of the grading criteria (1). Although the 95% CI crossed the MID of 2.3 on the 33-point Chalder Fatigue Scale, the whole of the CI is on the side of benefit to the patients.

In conclusion, while we recognize the points raised about imprecision and inconsistency, we believe that our comprehensive evaluation of the evidence using the GRADE framework supports our rating of the certainty of evidence as 'moderate'.

1. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH, GRADE guidelines: 3. Rating the quality of evidence, Journal of Clinical Epidemiology. 2011; 64; 401-6

Declarations of interest:
No conflict of interest declared.

https://www.cochranelibrary.com/cds....pub9/detailed-comment/en?messageId=451624933

 

[Utsikt]

There is a new comment by Lillebeth Larun, the primary author:


https://www.cochranelibrary.com/cds....pub9/detailed-comment/en?messageId=451624933

I think it’s short enough to post here:

Thank you for your interest in our systematic review. We have with interest read your suggestion that the certainty of evidence could have been rated as 'low' or 'very low' due to imprecision and inconsistency (https://www.cochranelibrary.com/cds....pub9/detailed-comment/en?messageId=447929916). We recognize the points raised, and that it is possible to argue for a further downgrading. However, we disagree, and do not consider it appropriate to downgrade just because a confidence interval marginally crosses a value that is set with some degree of uncertainty. These points were taken into consideration and discussed in the GRADE process, a process that also involved consultation with GRADE experts outside the author team. Simultaneously, GRADE caution against a mechanistic approach towards the application of the grading criteria (1). Although the 95% CI crossed the MID of 2.3 on the 33-point Chalder Fatigue Scale, the whole of the CI is on the side of benefit to the patients.

In conclusion, while we recognize the points raised about imprecision and inconsistency, we believe that our comprehensive evaluation of the evidence using the GRADE framework supports our rating of the certainty of evidence as 'moderate'.

 

[bobbler]

I think it’s short enough to post here:

Thank you for your interest in our systematic review. We have with interest read your suggestion that the certainty of evidence could have been rated as 'low' or 'very low' due to imprecision and inconsistency (https://www.cochranelibrary.com/cds....pub9/detailed-comment/en?messageId=447929916). We recognize the points raised, and that it is possible to argue for a further downgrading. However, we disagree, and do not consider it appropriate to downgrade just because a confidence interval marginally crosses a value that is set with some degree of uncertainty. These points were taken into consideration and discussed in the GRADE process, a process that also involved consultation with GRADE experts outside the author team. Simultaneously, GRADE caution against a mechanistic approach towards the application of the grading criteria (1). Although the 95% CI crossed the MID of 2.3 on the 33-point Chalder Fatigue Scale, the whole of the CI is on the side of benefit to the patients.

In conclusion, while we recognize the points raised about imprecision and inconsistency, we believe that our comprehensive evaluation of the evidence using the GRADE framework supports our rating of the certainty of evidence as 'moderate'.

snortles at them saying that with a straight face - and remembers the FOI email trail of when they got an independent arbiter in, as the 'expert' that was 'from outside the team' (after the Chief Editor before had withdrawn the review and then left shortly after it had been reinstated, and a new one was in place) for the 2019 review - and then ignored the reccommended limitations on what phrase they could use re: conclusions that could be claimed (either low effect with moderate certainty or low certainty there might be a moderate effect - but not just leave off the certainty bit etc if I remember correctly)


Maybe they needed to go for best of 5 games/places to look if it doesn't go the right way, or is it best of 7?

 

[Trish]

On 5th February 2025 the S4ME committee submitted a detailed set of complaints to Cochrane related to their actions in December 2024.
We received the decision on our complaints on 3rd March 2025.

The committee have today submitted a formal appeal.

Spoiler: S4ME committee appeal submitted to Cochrane 24th March 2025

Formal appeal to decision regarding complaint #COMP00195461

This appeal is in two parts:

Appeal March 2025/1:
Failure to respond appropriately to the part of #COMP00195461 that is addressed in the response, namely the creation of a new citation.

Appeal March 2025/2:
Closing the complaint without any action resulting from investigation, and failure to inform us of the outcome of any of the other major parts of complaint #COMP00195461
_______________

Appeal March 2025/1:

Failure to respond appropriately to the part of #COMP00195461 that is addressed in the response, namely the creation of a new citation.

Complaint #COMP00195461 is divided into 5 parts. The only part addressed in the response letter is the second part of:

"Complaint 2025-1:
Creation of a new citation and publication date as a result of attaching an editorial note to an unchanged review"

Cochrane's response to this part says:

"Cochrane decided to publish the editorial note in question with a new citation to ensure discoverability of our decisions regarding this review.

This decision does not breach our editorial policy but uses operational discretion in applying the publication guidance regarding editorial notes. Neither the editorial note, nor the decision to publish it with a new citation, indicates that the 2019 version of the review has been updated in any way."

This response makes no sense to us logically, ethically or in policy or practice terms.

Previous announcements relating to the process to develop a new review set up by the Editor-in-Chief in October 2019 were made both on the Cochrane website and as published notes and amendments attached to the 2019 version, for example on 6th February 2020, shown in the version history. As far as we can see, this publicity by Cochrane achieved Cochrane's aim to: "ensure discoverability of our decisions regarding this review".

We can see no reason why the same procedure was not followed with the current editorial note, and dispute that this would not achieve the desired discoverability.

While it is true that the section we quoted in the complaint comes from editorial guidance on how to attach an editorial note, not from a policy statement on citations, it is also true, as we explained in our complaint, that it is not listed as one of the three specific reasons for creating a new citation in the editorial policy.

We note also:

"Reporting search dates in Cochrane reviews
https://training.cochrane.org/onlin...lines/reporting-search-dates-cochrane-reviews
This guidance covers the reporting of search dates in Cochrane Reviews. It is informed by guidance on re-running searches covered in MECIR conduct standard C37. This standard requires that searches for all relevant databases be run (or re-run) within 12 months before publication of the review or review update, and that the results are screened for potentially eligible studies.

For definitions of search types (full, top-up, scoping) see Table below.

1. Updates vs. amendments: a review is considered updated and receives a new citation in Cochrane Database of Systematic Reviews (CDSR) when a new search is conducted and the results of the search are fully incorporated. If a scoping search is conducted to determine if an update is required, then the date of this search will not change the 'Date of search' in the review or lead to a new citation version being created. This should be published as an amendment if necessary. See also Dates and events in RevMan Knowledge Base."

Requested actions:

Please investigate and inform us why Cochrane's policies and guidance on new citations were ignored in this instance.

Please provide us with links to any other reviews where a new publication date and citation has been created for an existing review with no change to any part of the review itself, with the only change being the addition of an editorial note [apart from a withdrawn review, as specified in the editorial policy].
__________________________________

Appeal March 2025/2:

Closing the complaint without any action resulting from investigation, and failure to inform us of the outcome of any of the other major parts of complaint #COMP00195461

Complaint #COMP00195461 was presented in 5 parts. Only the issue of the new citation is addressed. The remaining parts are not mentioned in the response. Instead the response says:

"The published review does not make any recommendations for or against exercise therapy. This is in line with our guidance which prevents Cochrane Reviews from making prescriptive recommendations about using an intervention."

This reads as a pre-prepared generic dismissal of all complaints about the review, and does not address directly any of our complaints. It sets up a straw man argument, since we never claimed that Cochrane reviews are prescriptive.

As we understand it, the purpose of Cochrane reviews is to provide the highest standard of up to date evidence synthesis for clinical interventions. Cochrane's tag line of "Trusted evidence. Informed decisions. Better health" clearly joins the dots between its reviews and their clinical use. This is illustrated by the use of the review as one of only 3 references in the RACGP, Clinical guidelines, HANDI (2024).
https://www.racgp.org.au/clinical-resources/clinical-guidelines/handi/handi-interventions/exercise/incremental-physical-activity-for-cfs-me
_________

It is clear that no action has been taken as a result of the investigation of complaint #COMP00195461. Specifically:

The 2024 version has not been withdrawn on the grounds of divergence from editorial policy on new citations and guidance on editorial notes.

The editorial note still says the publisher agreed to the editorial note being attached to the 2019 version of the review. There is no mention of the publisher agreeing to the new citation, nor to the editorial note being attached to the 2024 version.

The critical comments published by Cochrane attached to the review dated between the version publications in 2019 and 2024 that should normally be addressed before any new version was published have not been addressed.

The incorrect labels on the published comments stating that they apply to a previous version have not been removed. Nor have they been correctly labelled as unaddressed and applying equally to the 2024 version.

There has been no explanation of why it is acceptable for a 2024 review to not take into account recent relevant evidence, to not use current research methods, and to link to superseded guidelines as if they were current.

Cochrane has not published an accurate explanation of the reasons for abandoning the new protocol and updated review.

We note that the Cochrane Complaints team failed to follow its stated acknowledgement and response timeframe for our complaint, as we set out in ticket number #COMP00196686.

The response to #COMP00195461 is brief, incomplete and unsigned.

Requested actions:

Please provide responses to the individual complaints.

Please confirm that the publisher has agreed to republication with a new citation and that the publisher agreed to attach the note to that 2024 version, rather than to the 2019 version as the editorial note states.

Please investigate and inform us what happened to the 5th February submission of #COMP00195461, the 12th February query, and the lack of acknowledgement of the 20th February resubmission.

Please investigate and inform us what action was taken in response to #COMP00195461, whether the response is in fact a generic one intended for all complaints about these issues, and why it is unsigned.
_____________________

Signed Trish Davis and Maree Candish on behalf of the committee of the Science for ME forum.

 

[Dolphin]

Copied post



‪Hilda Bastian
Cochrane has amended the editorial note on the #MECFS exercise review for the 2nd time since issuing it. Some improvement, which is good to see. But they still haven't gone far enough

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub9/full

Our comment online https://www.cochranelibrary.com/cds....pub9/detailed-comment/en?messageId=451525047

#CochraneStuff

From: https://bsky.app/profile/hildabast.bsky.social/post/3llaowdevrc2y