Blog: Hilda Bastian "Six Months Later: What Their Response on ME/CFS Tells Us About the Cochrane Collaboration"

[SNT Gatchaman]

https://absolutelymaybe.plos.org/20...fs-tells-us-about-the-cochrane-collaboration/

In giving this review the status of a 2024 version, Cochrane is remaining active on one side of a major controversy, rather than retiring the review and leaving the field with a neutral posture. This is extremely problematic on a topic where systematic reviews are cherry-picked and weaponized, with claims that reviews unfavorable to one side or the other represent caving in to pressure. The status Cochrane has given this review encourages the perception that it yielded to vested interests rather than staying true to its mission of providing reliable evidence and health information.

There are many people who care about the harm this outdated review can do, and won’t let it go – myself included. And it’s only a matter of time till other influential reviews that are misleading by virtue of being out-of-date stir up the same kind of conflict. Cochrane’s response to the complaints about their exit from updating the ME/CFS review show they aren’t well-placed to deal with that – and reveal some other issues of concern.

Six months ago, I ended my post with this comment about the organization’s current trajectory: “It needs to change course. The new conflict it has created around the ME/CFS review presents it with a valuable opportunity to do so.” The organization’s leadership have, so far, chosen not to take that opportunity. Given the Cochrane Collaboration’s reliance on their exceptional status and members, that is untenable. Not retiring influential out-of-date reviews is a ticking time bomb.

 

[forestglip]

S4ME got a mention:

On the other hand, issuing a new citation for a review just because an editorial note has been edited is definitely not covered by policy: Indeed, this is the only time that has been done. That’s because a new citation is constructed to show as an update. See for example the entry for the previous version in PubMed, which automatically flags it is updated in the new version. On Cochrane’s website, it is automatically flagged with, “This is not the most recent version.”

We weren’t the only people complaining, of course. And there was a flash of openness about this in their reply to a complaint by Science For ME:

“This decision does not breach our editorial policy but uses operational discretion in applying the publication guidance regarding editorial notes.” [Cochrane Complaints, March 3, 2025]

Policy is not standing in the way of making it clear to readers that the ME/CFS review is out of date.

 

[bobbler]

before I've got on with reading this it has reminded me that this was released on the last day before recess for xmas.

And of course the implementation plan was the last day before parliamentary recess this summer.

It seems a pattern and I haven't looked any further.

1. is there co-ordination going on for this choice of dates being on the last day before a big parliament break happenning for ME/CFS

2. has this 'habit' ever happened for anything else? particularly any other medical things?

 

[Nightsong]

The "national licence" that the British Government pays for (for everyone in UK IP ranges to be able to freely access Cochrane's content) is clearly the largest source of its income. At present the universities and NHS bodies don't need to pay for access to the Cochrane Library - as they pay academic publishers for access to other journals - because of this arrangement, which is administered by NICE. Hilda also discussed this in her previous blog:

One of the pivotal national licences is for England, paid for by NICE. For its last licence, NICE had called on the higher education sector to help with the costs. NICE’s annual report shows that this yielded less than £50,000 per annum, though they don’t report how much they pay for the licence. (For perspective, Switzerland, a country a small fraction of England’s size, paid over US$200,000 for their 2024 licence.)

NICE is obviously an arms-length quango but this is clearly going to be something authorised by the Department for Health. And, as Hilda mentions, New Zealand are ending a similar arrangement with Cochrane very soon. Perhaps if some of the ME patient community's friends in Parliament could be persuaded to raise Cochrane's diminishing standards, its behaviour towards patients, and the other Cochrane scandals such as the masking controversy - with DHSC. . .

British taxpayers absolutely should not be paying for this.

 

[Sean]

Thank you, @Hilda Bastian, for not letting this go.

Previously, reviews had been the domain of editorial review groups, but now, reviews are edited centrally, and can be submitted directly too. However, in mid-2023, the centralized editorial process for responding to feedback was ended. It is now the authors’ responsibility whether or not they respond to criticisms. Yikes!

In abdicating this editorial responsibility, along with a new policy of not informing readers when they know a review is out-of-date, Cochrane has dismantled a core part of what defined Cochrane reviews.

Every time I start thinking the 'authorities' couldn't get any worse, there they go and prove me wrong. Again.

 

[hotblack]

2. has this 'habit' ever happened for anything else? particularly any other medical things?

It’s very common for all sorts to go out on the last day(s) before recess. Not a good practice, but a common one.

 

[Hutan]

Great to see this followup from Hilda.

In abdicating this editorial responsibility, along with a new policy of not informing readers when they know a review is out-of-date, Cochrane has dismantled a core part of what defined Cochrane reviews. Meanwhile, the reviews are getting progressively more out-of-date. With the loss of funding for so many editorial review groups recently, and the centralization of editing in a small group, the rate of reviews being out-of-date is presumably escalating, while key mechanisms for dealing with outdated or inaccurate reviews have been curtailed.

Meanwhile, the organization’s promotional rhetoric hasn’t been updated to reflect the changed nature of Cochrane reviews. The Cochrane website has had a major update in the last few weeks. They still have the central goal and claim of of providing the “best evidence to help patients and health organizations to make informed decisions.” A new section was added: “Why our evidence is trusted,” with the tagline, “We provide high-quality, unbiased health information.” And there is this claim:

“We are committed to regular reviews of every piece of our evidence. We assess whether findings are still relevant, and we identify aspects to investigate further or update. We believe this is critical to ensure the ongoing reliability and trust in our work as medicine progresses.”

They go on to say, “Governments in all continents of the world regard our evidence so highly that they give their entire populations access to our systematic reviews by funding national provision through the Cochrane Library.” Of course, governments don’t typically do this for medical journals: They do this because of the unique nature the Database used to have. If key pillars of reliability of the Database for users are no longer there, and Cochrane doesn’t take action on high profile unreliable reviews, how long can this special status last?

This is existential for an organization whose current long-term economic strategy involves publicly-funded subscription, especially national ones. I wrote about this earlier this year. Since then, one of only 10 countries outside the UK funding like this has announced it will end soon (New Zealand).

British taxpayers absolutely should not be paying for this.

Neither should the taxpayers in countries and states still paying Cochrane for access to the Library.

I've made a post on the Cochrane's Funders thread, listing out the funders of these regional access arrangements.

 

[Maat]

It seems a pattern and I haven't looked any further

Yes. Cochrane's republishing of the review with the 2024 date coincides with the (3 days before or after) coroner's 56 day deadline to respond the PFD Report on Maeve Boothby O'Neill. Although not a recipient of the PFD, this inexplicable event landed within the same timeframe.

This follows a now clearly identifiable, recurring pattern surrounding all significant milestones in ME/CFS - unique to this illness. It's the mechanism employed to support the political/medical agenda for the last 30 years.

The core reason no progress has been made up to and including the Final Delivery Plan which is an official government policy document.

The complaint I'm finalising lays this bear with supporting evidence to this assertion.

 

[rvallee]

https://absolutelymaybe.plos.org/20...fs-tells-us-about-the-cochrane-collaboration/

Rather strong words, given Bastian's generally very moderate, even generous, attitudes towards Cochrane. She really seems to have genuinely tried her best, but with an organization that never had any intentions of doing the right thing, it was an impossible task. I really doubt this is confined to this review, it's almost guaranteed that those problems are common, even a feature of evidence-based medicine, which seems to operate within a parallel worldview.

The status Cochrane has given this review encourages the perception that it yielded to vested interests rather than staying true to its mission of providing reliable evidence and health information.

I think it's more than fair to say that this was never Cochrane's true mission. To me it seems more like a high school clique, only exists for its own sake and interests.

The controversy is extending, as the literature on Long Covid grows

I really wonder if they get that. I don't think they do. They are really not thinking of the consequences of them being wrong here.

But it’s not that alone that gives it a special status: It’s because they are meant to be reliable by updating in response to new evidence and valid criticism.

I think that up-to-date also works this way? It's not any better, that's for sure. Not much point in having up-to-date reviews of evidence that aren't actually up-to-date and distort the evidence while featuring heavy bias.

There are many people who care about the harm this outdated review can do, and won’t let it go – myself included.

Because of the secrecy around the entire process, it was always impossible to say exactly who did their job, and who didn't, and it's clear that the fault is 100% with Cochrane here. It looks as if they assumed that Bastian would be a good team player, eventually side with them and do a proper cover-up, but she is better than this. Thank you for not giving up Hilda.

 

[Peter T]

It looks as if they assumed that Bastian would be a good team player, eventually side with them and do a proper cover-up, but she is better than this. Thank you for not giving up Hilda.

I must admit for several years, I interpreted Hilda’s apparent willingness to indulge or even seemingly cover for Cochrane’s delays and obfuscation as indicating that she regarded Cochrane’s game playing and internal politicking as more important than good science. However, her subsequent persistence and outspoken criticism makes it clear I had misjudged her.

 

[Utsikt]

I must admit for several years, I interpreted Hilda’s apparent willingness to indulge or even seemingly cover for Cochrane’s delays and obfuscation as indicating that she regarded Cochrane’s game playing and internal politicking as more important than good science. However, her subsequent persistence and outspoken criticism makes it clear I had misjudged her.

The road to hell is paved with good intentions. I’m glad she is putting science first now.