BMJ: Rapid response to 'Updated NICE guidance on CFS', 2021, Jason Busse et al, Co-chair and members of the GRADE working group

I imagine there is supposed to be a point somewhere in there but all I read is people saying if they had been tasked with it they would have done it differently because they don't like the outcome it gave without being there to change it to their preference. No substantial argument other than they don't like being shown wrong and simply want to continue with their bad and harmful pseudoscience. It's very meandering. Unclear if it's submitted to the NICE committee or trying to influence the outcome using external political pressure.

As usual it cites the Cochrane exercise (allegedly) under review. Cochrane's inexplicable behavior is continuing to impair progress and cause harm. The obsession these people have with snake oil is beyond absurd, it's deranged and completely self-serving, pays no attention whatsoever to their actual work, this is naked self-interest, how it affects them personally.

https://www.bmj.com/content/371/bmj.m4774/rr-7

It appears to be people from the GRADE working group, which explains a lot about why it produces such poor results:

 

I do not recognize most of these names, but it shouldn't surprise us to see Paul Garner and Flottorp, who I recall recently endorsing an RCT of the Lightning Process, on there. What a sham.

 

I know nothing about GRADE and how it works.
And I would not be able to read and sort out the details of it.
But here for anyone interested are some links to the GRADE system:

BMJ:

https://bestpractice.bmj.com/info/toolkit/learn-ebm/what-is-grade/

Cochrane:

https://training.cochrane.org/grade-approach

CDC (it seems via McMaster University in Hamilton ON presentation)?

https://www.cdc.gov/vaccines/acip/recs/grade/downloads/guide-dev-grade.pdf

CDC--rather long, though with some useful slides I think.

 

I think that this is a sign that there's going to be more of a push-back on NICE changes than some people expected. And that it's likely to go on even if NICE does not change course.

 

It seems a bit far-fetched to combine the two: Which tables the authors of the guideline show about the grading of the the evidence of intervention studies for the benefit from the studied interventions is one thing.

(That the authors don't show tables or summaries on specific outcomes idoesn't mean they didn't grade the studies properly.)

Report of harm from other sources is a different thing.

That the authors of the guideline took into account reports on harm from other sources was due to the fact that most treatmenat studies failed to properly report harm -- a "patient-important outcome " that in therapist delivered interventions appears to be underreported per se.

Not able to check and to look again into the draft guideline and the suppementary material ATM. So please correct me if I'm wrong.

 

It does look like that part of the RR misrepresented what was said:

They take the quote on the introduction on page 5 of this evidence review: https://www.nice.org.uk/guidance/gid-ng10091/documents/evidence-review-7

That was not saying why the "NICE evidence review associated with their guideline does not provide a GRADE evidence summary of findings table for fatigue related to exercise interventions".

 

Flottorp is also a close colleague of Lillebeth Larun (main author of the Cochrane review on ME and GET) at the National institute of public health in Norway.

The name Per Olav Vandvik didn't ring any bell, but here's from the presentation of him at Oslo University's webpage:

 

I think this was fully expected @Esther12 but it gives us an opportunity to push back on GRADE.I have in mind a formal publication about it once the dust has settled.

GRADE is a fiasco. The ME/CFS NICE process may be a useful way to demonstrate that. It looks as if maybe all the GRADE people can do to defend it is misrepresent things.

 

I have just read the comment from GRADE - it is wonderful - it reveals just how little they understand about anything.

 

Can you elaborate? And not having been much familiar with GRADE before the recent issues around Cochrane and PACE, can someone explain in what other ways it has been a disaster?

 

I'm curious, is there any reason for concern that the cited 2020 Moustgaard study could still end up having a negative effect on the evidence review process for the final version of the guidelines? Obviously, the lack of blinding was a major factor regarding the downgrade in quality of non-pharmacological interventions for ME, so would the results of this study not be a welcome tool in the hands of BPS'ers who still have an axe to grind?

 

GRADE works like this. You take a group of people who think that they have a reasonably good appreciation of the risks of methodology being faulty and giving unreliable results. You get them to invent a set of numbers that roughly reflect how they think they decide (although there are nice papers showing that we often do not decide the way we think we do). You give this set of numbers a name and suggest that people who may not quite know themselves how to decide on unreliability use the numbers to do it - not even their common sense. You then recommend that all studies are judged by these numbers, even by people who think they have a reasonably good appreciation of the risks and do not need numbers.

It is a bit like recommending that Michel Roux rather than judging whether a salmon en croute is perfectly cooked goes to the set of numbers in his book and works out whether it is cooked or not using the numbers. It is plain silly.

As an example, try applying a set of numbers to the likelihood of statements made by heads of state with odd-looking hairstyles being true, rather than listening to what the guy says. It's nuts.

And it was identified long ago by Ralph Waldo Emerson.
A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines.

 

The whole discussion is surreal. As I understand it, although I admit to being confused by the complexity:

Turner Stokes and Wade said that NICE was too harsh on PACW because CRADE is too harsh.
The GRADE people are saying no, GRADE is really very forgiving to bad studies (in effect) the problem was that the patients were too harsh on PACE.
The reality is that the patients and academics on the committee made a fair assessment.
So the GRADE people have merely come clean about how hopelessly lax their system is, but without realising it.

 

Thanks.

 

What I also found funny is how the statement that GRADE's emphasis is supposedly on "what is most important to patients", is juxtaposed by the argument that the NICE committee is overly influenced by the patient side, via "theoretical arguments and anecdote." The bitter irony aside, the complete disregard of the fact that the CBT/GET working hypothesis never had any theoretical foundation in reality sticks out. I suppose he thinks he's making an argument in favour of accepting the available randomized trial evidence in spite of patient complaints, while in actuality, he's making a much better point in favor of proper blinding+objective outcomes due to investigator allegiance. And I suppose his citation of the 2020 Moustgaard study is supposed to simply gloss over this, in some manner that is unknown to me.

 

But the question remains: was GRADE applied properly on the whitewash that is the "CFS" Cochrane reviews and similar work, giving high marks to unblinded studies with poor methodology and did not respect Cochrane's own rules and requirements, or was the correct application the one done by the NICE committee, which found that they are of such low quality as to be useless?

Or something in-between? Frankly what this looks like is that GRADE depends entirely on who applies it and for what purpose. Give it to someone who wants evidence to be true and it will find it to be above reproach, and vice versa. Unless I'm missing something, or many things?

Because even beyond GRADE, the Cochrane reviews on CBT and exercise should have been marked as worthless based on Cochrane's on guidelines, most trials should have been discarded at the first step yet were marked as high importance and low bias. So it's not even clear whatever people are doing in EBM respects any particular set of rules or process, it's all entirely down to whatever people want to be true and whether that thing happens to be popular enough to go through any process, regardless of substance.

Honestly this is no different than how a legislature functions, it's fully arbitrary. Evidence hardly even matters, it's entirely about what can be sold to the stakeholders who make the final decision.

 

Merged thread

New Response by an associate professor called Jason W. Busse McMaster University Canada.

 

This would seem to be Jason Busse. Busse, Jason (mcmaster.ca) I wonder whether there are any conflicts of interst to declare.

 

I never expected for a moment that there wouldn't be HUGE pushback, of immense resistance. It was depressingly predictable; too many careers and reputations, awards received, etc are at stake to either simply keep quiet or, miraculously, to accept they had got it wrong and that things need to move on. I suspect that NICE will cave in and revert the guidelines back in favour of the wonderful 'evidence based' CBT/GET, really hope that won't be the outcome.