Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

I would have thought doing a review of the evidence base for exercise in ME/CFS was up there with the easiest review ever. If Carlsberg did ----

Most/all of the studies are unblinded and use subjective outcome criteria, so there is no evidence to review.

This comes to mind "If nobody is trying to prove anything it actually becomes quite easy to---*" These folks are trying to turn non-evidence into evidence. I have a bit of sympathy for them in that ---- old joke about "evidence based policy development" versus "policy based evidence development". These folks are engaged in the latter and they are making this whole thing way too complicated ---- there is no evidence.

*https://www.s4me.info/threads/if-and-how-to-research-pacing-discussion-thread.22860/

 

But if the IAG doesn't agree with the Cochrane reviewers, the final say goes to Karla Soares-Weiser. Editor in chief of Cochrane. She has already used Gordon Guyatt to ensure the current review still rates the quality of the evidence on one outcome as "moderate" for a non-zero effect. If the new reviewers also rate it as moderate, and the IAG disagree, who is to stop Karla overuling? But as you say, the new review should not include any trials as they are all un-blinded with subjective primary outcomes

[edited at 17:27 to expand the point]

 

Hi there
Yesterday I wrote to Rachel Marshall who is managing the Exercise Review update. I copied in Cochrane’s Consumer Engagement Officer Richard Morley who is also listed as supporting the review, Nicholas Henschke, lead author, Hilda via the IAG address, Toby Lasserson (Cochrane Deputy Editor in Chief, leading the collaboration between cochrane and NICE) and Samantha Roberts (new NICE chief exec)


From: Caroline Struthers
Sent: 19 March 2022 17:20
To: Rachel Marshall
Cc: cochrane.iag@gmail.com; tlasserson@cochrane.org; samantha.roberts@nice.org.uk;
nhenschke@cochraneresponse.com; rmorley@cochrane.org
Subject: Cochrane/NICE Collaboration and the update to the Exercise therapy for ME/CFS review
Attachments: 2021-08-11 NICE_Cochrane Collaboration Agreement and Licence Unpublished Data
SIGNED.pdf

Dear Rachel

(cc Toby Lasserson, Samantha Roberts, Hilda Bastian, Nicholas Henschke, Richard Morley)

I am writing in a personal capacity and not as a representative of my employer.

There has been no monthly progress report about the update of the Exercise review from Hilda since
July last year, and I understand the very sad reasons for that. Without knowing at what stage the review
is, I am writing to suggest something that you may already have decided to do.

In August last year Cochrane and NICE signed a collaboration agreement (attached) to facilitate the
sharing of data and analyses with each other.

The Parties agree to work together from time to time to facilitate the sharing of literature searches,
unpublished extracted data and analyses and assessments (“Unpublished Data”) to inform the
development of NICE guidelines and Cochrane Reviews as per the terms of this Agreement (the
“Purpose”).

As NICE have just published a new guideline for ME/CFS informed by a rigorous and comprehensive
evidence review, it would make sense for NICE to share the relevant data and analyses from their review
with Cochrane rather than Cochrane starting the whole process again.

Please could you let me know what stage the review is at, and whether you are already planning to ask
NICE to share the data and analyses from their evidence review. If you have not considered this idea,
please could you let me know your response to it?

With best wishes

Caroline


Caroline Struthers
Senior EQUATOR Research Fellow
UK EQUATOR Centre, University of Oxford, United Kingdom

 

I realise that NICE's data and analyses are published (in full, maybe not??), but to refuse to use them for that reason would be the epitome of research waste.

 

Yes. They did. But it made no difference even when the NICE evidence review highlighted its flaws. I honestly don't understand why they won't follow their own rules about withdrawing a review when following the findings could cause harm to patients. What sort of patient engagement is Richard Morley doing if he won't try and persuade Cochrane that the current review should be withdrawn for that reason alone? Perhaps i should write to him.

 

They will go in fear of your pen!

 

Right on cue, the statutory two weeks after I wrote, Cochrane have replied! It's not very exciting...

From: Cochrane <cochrane@cochrane.org>
Sent: 04 April 2022 07:14
To: Caroline Struthers
Cc: cochrane.iag@gmail.com; Toby Lasserson; samantha.roberts@nice.org.uk; Richard Morley;
Nicholas Henschke
Subject: Cochrane/NICE Collaboration and the update to the Exercise therapy for ME/CFS review

Dear Caroline,

Thank you for your email. Regarding your question about whether Cochrane plans to ask NICE to share
the data and analyses from their evidence review, this issue is currently under consideration, but we
have no further information to share at this time. With best regards,


Katie Abbotts
Communications
Development Directorate
Central Executive Team
Cochrane

 

It is like talking to a rock.

 

But a bit less fun

 

Can't help wondering if both have the same root cause - obfuscation.

 

Or I may not care at all ...

 

Nice one Caroline - I can feel the salt being rubbed when I read this:
"As NICE have just published a new guideline for ME/CFS informed by a rigorous and comprehensive
evidence review----".

Wonder how they are going to address that-
1) rubbish NICE's evaluation of the evidence - not attractive since they now have the NICE/Cochrane "collaborative agreement";
2) agree (through gritted teeth) to accept NICEs evaluation --- problematic. They'd have to face down the (Cochrane) internal dissenters;
3) run their own (Cochrane) re-evaluation and Voilá --- they come up with the same answer as NICE --- as per 2, it's internally problematic;
4) none of the above - cos the above require a decision and create conflict!

Folks on this site have said something like ---- just do the right thing ---- I'm not expecting Cochrane to do that any time soon.

 

Weren't we supposed to get an update about... having future updates? Some time? In this timeline? Before the damn Sun reaches Venus' orbit during its expansion phase? About the process that started 5. YEARS. AGO!

What a joke organization Cochrane is. Have they even produced anything about Long Covid? Is Cochrane aware that Long Covid exists at all? The gold standard in clinical evidence that can't tell its own ass from a hole in the ground? Because right now if a physician wonders what to do about Long Covid and looks at what Cochrane says, well, that's probably a very bad idea, frankly.

 

Yip ---- let it die --- do nothing ---- and to be fair, while irksome to us (which they'll enjoy), it's a strategy that works!

I'd just contrast this to Chris Ponting's approach. His friend Simon was ill and he was keen to do something to help. When the cost of GWAS studies dropped sufficiently he was able to get DecodeME up and running --- we also need to bear in mind MRCs role i.e. in convening the expert group (including Jonathan) which recommended funding a GWAS study and subsequently funding it.

If Cochrane were similarly motivated then they'd flag up that PACE etc. are flawed since they are unblinded and rely on subjective outcome criteria. Also, (Cochrane) run a lobbying campaign to require that publicly funded studies use objective outcome criteria (actimetry). Of course that would assume the motivation is promoting sound science ---- whatever the motives behind Cochrane are, they don't appear to be that simple.

I assume I'm being a bit biased about Cochrane i.e. there was an opportunity for it to "do the right thing" and some of those involved are/were motivated to do that. Seems that the motivation to do the right thing wasn't strong enough/widely enough held.