Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

This is what I was getting at.

 

To be honest the recent ME/CFS guideline seems to be a bit of an outlier as far as NICE is concerned re: their tendency to get BPS-friendly guidelines through. (Recent examples of chronic pain and long covid situation as well). Relying on this talking point too much just incentivizes people to try to get a different result and "update" the NICE ME/CFS guideline. I got the sense if different people were involved there would have been a different result.

 

Thanks for wording that better than I was able @Ariel.

 

We have so many good arguments why Cochrane should withdraw the current exercise review.

I don't think this argument adds anything useful -- on the contrary I think it's easy for Cochrane to rebut it.

They could say: "Of course we can use NICE reviews if we agree with their methods and conclusions, but we are also free to decide that we don't agree."

 

I had not realised NICE had said there was no good research so it wouldn't do a review at all. It will be interesting to see if Cochrane reply by saying that they still want to do a review using all the same studies NICE did (was Crawley the last one?) even though there have been no more since the NICE review was done. It's not about NICE's decisions, it's about Cochrane's

 

Yes, we do. But they haven't. If it's easy to rebut, let them go ahead. Surely it's better than not making the argument at all.

 

This is why they are fighting us like their whole career depends on it, because it does. We are the first to break through the wall they built for decades, in part using us as foundation. Like an empire that has crushed rebellions for decades and finds itself losing its first battle.

By delaying this long enough, the same BS cycle can begin all anew with Long Covid and the general shift towards even more generic claims, on the back of yet another cycle of garbage pseudoscientific "studies".

Although it's not as if delaying is some grand strategy, this is all purely reactive and only works because of the totalitarian power imbalance that simply allows them to force their will onto us.

 

Apologies for not being clearer:

From the bmj article I think it isn't clear what actually happened.

What I understood from the article is that

1) Cochrane wanted to do a review (meta-analysis) on puberty blockers in minors, the authors team wrote a protocol but this was rejected.

2) I think It isn't clear from the bmj article whether the protocol was rejected because of the NICE review or whether Cochrane had already rejected the protocol but only decided to not pursue the review further (which as I understood would have involved writing a new protocol) when the review commissioned by NICE was published.

In any case, NICE seemed to agree with the externally commissioned review that
"the available evidence was not deemed strong enough to form the basis of a policy position".

3) Cochrane justified not doing a review by referring to the review commissioned by NICE.


But I think the NICE review on the evidence on exercise for ME/CFS is much more complex. In the evidence assessment of treatments for ME/CFS, NICE apparently deemed the evidence base, both for benefits and harms, as much more useful than they did in the review of the evidence for benefits and harms for puberty blockers for gender affirmation in minors .

I'm afraid it's possible that Cochrane could find an easy way out in following the example of IQWiG and only pickout parts of the NICE evidence review on treatments for ME/CFS and arrive at a conclusion different from NICE.

For example, even though according to NICE the quality of evidence for a benefit from exercise is very low to low, Cochrane maybe could say that overall the results point in the same direction (towards a benefit) -- and if Cochrane at the same time discarded NICE's assessment of cost-effeciveness and harms, the new review could still conclude that, under certain conditions, it's worthwhile to try exercise for ME/CFS.

As we discussed elsewhere on the forum, GRADE is tricky -- see the threads linked below.

So given all that I think it's not helpful to refer to that one decision by Cochrane, not only because the point can be easily rebutted but also it could be harmful if Cochrane actually followed the suggestion [*] only in parts.


[*] edit to clarify: the suggestion to use the NICE evidence review for making a decision about what to do with the update process on the review on exercise for ME/CFS.


Discussions on GRADE:

https://www.s4me.info/threads/who-a...fied-in-theory-and-b-wrong-in-practice.19403/

https://www.s4me.info/threads/bmj-r...and-members-of-the-grade-working-group.19317/

https://www.s4me.info/threads/germa...-by-27-november-2022.21266/page-5#post-442956

In the 2nd thread you also find some GRADE and Cochrane VIPs attacking NICE's new guideline on ME/CFS saying NICE didn't properly apply GRADE.

 

What we are seeing in these examples is that both NICE and Cochrane are potentially at risk of being captured by academic cabals, which perhaps makes how the escape of the NICE ME/CFS review happen the most interesting question.

As far as we can tell Cochrane’s Exercise review delay represents some admitting the scientific failures in the old review but the devotees of exercise therapy fighting a rear guard action to keep poor research in the loop to justify their position.

 

My point has nothing to do with the complexities or otherwise of the review methods, use of GRADE etc etc. This has been covered already. I will be submitting a complaint about that shortly.

It is simply Cochrane taking notice of what NICE have concluded in the case of puberty blockers and not taking notice of what NICE have concluded in the case of Exercise for ME.

 

https://www.who.int/publications/i/item/WHO-2019-nCoV-Clinical-2022.2

(I haven't gone through this in detail but there are several other references to Cochrane reviews).[/QUOTE]

Can someone send the complaints about what happened with this review and the Cochrane FOI emails (where the authors pushed for 'moderate' but independent said 'only with caveats etc) to the WHO?

Timing seems of the essence when someone might have used it whilst not knowing such info - because if found out too long after they've quoted it for too long it ends up seeming a big climb-down, rather than a simple 'let's quietly stop mentioning this one whilst we aren't sure about it' type thing.

 

I will send the complaint to Cochrane today and forward it to WHO and Charity Commission

 

I understood that you were wanting to make a simple point but think a) it is a weak point b) it isn't that simple c) it could prove detrimental to our cause -- for various reasons I was trying to explain in my posts above, including that your only source is a bmj article that doesn't give a clear account of what actually happened.

We use to be very critical of this kind of bmj articles when they cover ME/CFS related topics, in particular critical of their spin with regard to the process of the NICE guideline update.

Don't think I have anything useful to add, but perhaps will try to word my points more succinctly later.

For now I just would like to repeat that I think we need to be very cautious with referring to material and decisions that seem to suit our cause without taking a closer look at the actual material and sources.

 

Have finally submitted the complaint about Karla Soares-Weiser to Cochrane. I was intending to keep it brief, but decided to err on the side of spelling it out in the end. Although I could have gone a lot further.

 

Thanks Caroline

 

It's quite exciting having a new person to write to (Catherine Spencer)

 

Thanks for this. Agree with others it's a very useful letter.

Maybe now redundant but one last post already written on the issue with the NICE review before you posted your complaint about the editor-in-chief.

[Edit: the follwing is not a critique of your most recent letter but about the point discussed above related to the NICE review on puberty blockers and that you made in the letter to Hilda posted here.]


I think it's wrong to imply that Cochrane didn't "take notice" of the NICE review on the evidence of exercise as a treatment for ME/CFS.

I agree it's good to refer to the new NICE guideline for a decision what to do about the Cochrane review, but only together with additional arguments why the current exercise review is harmful and needs to be withdrawn.

I think it's unlikely that Cochrane agrees with NICE's assessment of the evidence provided by the included RCTs and even if they did, they still can come to different conclusions -- see my posts above.

So for Cochrane using the NICE evidence review for a decision about the exercise review I think would not lead them to give up their own review, but rather to disagree with the NICE review in various relevant aspects.

I may be wrong but I doubt it would be useful to have a new Cochrane review with this outcome. Even if a new Cochrane review downgraded the certainty of evidence according to the NICE review, it still could deem exercise as a useful treatment for 'some' people with ME.

At this stage, such a review maybe would better than the current one, but it still would endorse the attacks on the new NICE guideline.

I think we really need to make different arguments why the current review needs to be withdrawn. I agree it's sensible to refer to the new NICE guidance but I think we need make additional points that endorse the conclusions of the guideline committee. These additional points must not be that easily to rebut.

Edit: By using additional endorsing arguments I don't mean to invent new ones, just that I think we have additional arguments and if we want to make people outside the field of ME aware of those arguments, it will be important to make this in a way that does not start with implying that they are stupid if they didn't see this earlier or even if they didn't agree

Maybe it could be helpful to incite a broader discussion on the issue with relying on patient reported outcomes in open label trails. I think how we / ME advocates discuss this topic is very 'special' and I couldn't find any other patient community and advocacy for other illnesses that share our concerns and suggestions.

I think we need to take this into account and engage in genuine discussions with non-ME patient reps /disability organizations -- I have searched a bit on that topic and contrary to ME advocacy found only patient reps/ disability organizations endorsing the need to give more weight to patient reported outcomes -- no matter if in unblinded or blinded trials.


Edited to clarify

 

Absolutely agree - the thing about Patient Reported outcomes always being good, even in unblinded trials drives me absolutely nuts! There are small signs the pain research community are beginning to get it...
In the letter I deliberately only tackled one thing, and it was still seven pages!

 

Thank you @Caroline Struthers for such a clear, well argued letter. Your work on this is very much appreciated.