BMJ: Rapid response to 'Updated NICE guidance on CFS', 2021, Jason Busse et al, Co-chair and members of the GRADE working group

Thanks. I had to shorten it a bit because it was originally too long.

 

@Michiel Tack Good Article. How did you get bmj to publish the response? Will many people see it?

 

My assumption is this is what's happening behind the scenes.

 

I doubt it. The opportunity has passed and nothing was put out as far as I know. The Colleges would look pretty stupid if they put out a complaint after the consultation process had ended. The tweet seems to refer to something that has already happened and as far as I know it hasn't.

 

https://twitter.com/user/status/1366793865928335368

 

I have a suspicion that he is talking through his hat. I think I have seen it written that GRADE is not about recommendation for service policy but about quality of evidence. Anyway if he is right then GRADE is absolutely unfit for purpose.

I have a feeling that Dr Busse may be digging a hole for himself.
I like the way Michiel keeps a spade handy for these people.

 

Thank you @Michiel Tack
Nonsense shown for the nonsense it is succinctly

 

I don't know how many people will see it but responses like Michiel's are submitted using the online form. Here is the link for responding to the original article in question, https://www.bmj.com/content/371/bmj.m4774/submit-a-rapid-response

 

https://gdt.gradepro.org/app/handbook/handbook.html#h.33qgws879zw

https://gdt.gradepro.org/app/handbook/handbook.html#h.qmsmbyw3yxsl

 

It's both, but assessing the quality of evidence and recommendation for service use are two separate things, even according to GRADE.

From the GRADE Handbook:

 

No, this was Cochrane. It seems that GRADE does deal with recommendations. But notably it regards low quality evidence as 'unconvincing' and also notes that since resources are always limited cost-effectiveness must be considered and recommendations should not be made where they might impact on more useful services.

I have not yet seen any suggestion that low quality evidence should lead to a weak recommendation to use the treatment.

Edit: it looks from the crossed posts above that this does not follow, although there may be more detail buried in the haystack of confusion.

 

I like this:
Grading systems that fail to separate these judgements create confusion, while it is the defining feature of GRADE.
Yes, the defining feature of GRADE is indeed confusion. (At least their grammar is confused.)

 

@Jonathan Edwards : crossposted.

From the GRADE handbook:

That seems to be a missing link?

But there's an example:

 

The link is broken in the handbook, but it's section 6.3.2.

 

Yes, it all comes back to me. I have read this in detail. I remember the example.
But I doubt there is anything that says that weak evidence should, as a rule, be followed by any recommendation. The only rational response to 'unconvincing' evidence is not to recommend.

All of this illustrates how despite the GRADE people thinking that they have invented some sort of robust objective system all they have really done is rather clunkily tried to explain how they think they make decisions themselves. The antibiotic example simply indicates that they realise that the system needs to be adjusted for common sense. So why not just leave people to use common sense?

 

It looks as if Dr Busse is clutching at straws for some reason. First he says NICE got it wrong. Then he says if they got it right they should still let him recommend GET (weakly). There is a strong impression that it matters to Dr Busse that GET is recommended.

 

I think it's interesting because if I understood properly this is mostly about weighing the evidence of an estimated desired effect against the evidence of an estimated undesired effect, i.e., mostly, harm.

Edit: On the other hand, it looks strange to me that very low quality evidence of benefit in the absence of harm generally should still lead to a recommendation in favor of the treatment, even if a weak one? That seems to be what GRADE advices though:

From section 6.1:

(Edited for clarity.)

 

There is pretty much low quality evidence for anything you can imagine. It's literally what separates things that are real from those that aren't. By creating a standard that is below ground, everything goes: all the pseudosciences and alternative approaches, no matter what the evidence, they all have some low-quality evidence to "support" them. Cochrane has plenty of reviews for stuff like acupuncture and other woo. If low-quality evidence is sufficient, all that it takes for anyone to claim something is effective is to get it published and that's it. Doesn't even need to relate to reality at all.

This is what happens when exemptions are created in a system: it encourages to work only with those exemptions as they are far easier to meet than the normal standards. Why bother researching anything if you can simply try a bunch of stuff that doesn't work and promote it saying the evidence isn't good but it's there and that means it is 100% useful, somehow?

So there is a fork in the road ahead: whether medicine sticks to science or abandons it entirely. This process isn't even alternative medicine, it's alternative to medicine, a cheap knock-off. Like a smaller version of the post-truth era in politics, medicine is engaging in its own post-science era. And it's happening seemingly without notice. And the people who value truth are frankly not seeing that there is a split going on.

Safe to say that without Covid all of our efforts would have been cancelled entirely. Even with it's not even safe to say politics won't win the day. That's a huge lesson for Long Covid, they still don't pay attention to this stuff, can't imagine medicine is so utterly broken. Problem is how to make them aware of it without it being so demoralizing they brush it off, hoping it will be different for them.

 

No I think this is simply a precaution to not make strong recommendations when things are unclear.

The full section reads (my bolding)

so it seems that the 'more cautious' refers to more cautious than making a strong recommendation. If the evidence was unclear or unconvincing the cautious option would simply not to make a recommendation for or against.

 

Very low quality is the lowest the GRADE system goes. That's the level of case reports etc. See for example:

So if very low quality evidence should lead to a recommendation, then NICE would have to recommend everything (carnitine, LDN, Ampligen) that has scientific studies in support, no matter how bad those studies are.