Anomalies in the review process and interpretation of the evidence in the NICE guideline for (CFS & ME), 2023, White et al

Many of the arguments are the same as in the Lancet commentary. Here's a brief overview of my comments based on the arguments in the press release.

1. The 2021 guideline proposed a new definition of CFS/ME which required the presence of four symptoms. One of these was ‘post-exertional symptom exacerbation’ (post-exertional malaise), which had not been mandatory in previous Applying this new definition retrospectively, the NICE committee downgraded the large majority of trials of CBT and GET that had not specifically and explicitly required post-exertional symptom exacerbation to be present.
It was not the NICE committee that came up with a new definition. In the past 20 years, multiple case definitions have been published that require PEM as a core feature of ME/CFS such as the Canadian Consensus Criteria (CCC), International Consensus Criteria (ICC), and the National Academy of Medicine criteria. The NICE definition is based on the latter. These criteria are the ones used in research and clinical practice. It is logical that NICE evaluates the evidence for this patient group, for how ME/CFS is defined today, and not for an older case definition published 30 years ago. Other reviews such as the one by the German IQWIG have used a similar approach. Also: the previous NICE guidance from 2007 already highlighted PEM as a core feature of ME/CFS and studies that used this 2007 description were not downgraded.

2. The NICE committee considered outcomes for each trial at a time furthest away from when participants were allocated to their groups and received their treatments. This meant that some treatment outcomes were simply ignored. In other trials, participants in the comparison (control) groups had received another therapy by this longest follow up, making it impossible to accurately assess differences between groups, which is methodologically imperative in the context of a clinical trial.
I agree that NICE could have used the primary outcomes of the trials as well so that patients were still randomized to their treatment group. In the long-term follow-up, they might have received additional treatments.

Past reviews, however, have often overlooked the negative results of these long-term follow-ups, so I think it is good that NICE highlighted these. It would be strange to recommend treatments if the long-term follow-up showed no benefit at all.

In both GETSET and PACE, a sensitivity analysis showed that it was unlikely that additional treatment after randomization explained the null results. The authors of GETSET for example, reported that “there is no evidence that the improvements observed in the SMC group [the control group] were due to them having received more exposure to therapy than the GES group [the intervention group] after trial completion.” Similarly, in the PACE trial the control group caught up on the intervention group and “there was some evidence from an exploratory analysis that improvement after the 1 year trial final outcome was not associated with receipt of additional treatment with CBT or GET.” In other words, at long-term follow-up patients who received GET or CBT seem to do just as poorly as those who did not.

3. The assessment of harm from treatment in the NICE 2021 review prioritised evidence from qualitative studies and patient organised surveys. On this basis NICE concluded GET was unsafe. However, rigorously conducted systematic reviews, which included gold standard randomised controlled trials, found no evidence that GET caused harm.
This is not entirely true. The Cochrane review looked at the randomized trials and stated: “We are uncertain about the risk of serious adverse reactions because the certainty of the evidence is very low.” There was simply not enough data in the RCTs to make a conclusion.

On the other hand, there are lots of surveys that indicate GET has a negative impact on some ME/CFS patients so it is not unusual for NICE to express a concern about this treatment. In medicine, potential harms are evaluated differently (taken more seriously) than potential benefits – primum non nocere etc.

4. In their assessment NICE downgraded all outcomes related to fatigue, considering it to be too subjective to measure. But how else than asking patients how they are can one assess whether a treatment has worked?
I think this is probably the weakest point. NICE did not say this at all. Blinded trials that used fatigue as an outcome were not downgraded.

NICE downgraded quality of evidence when subjective outcomes (not just fatigue, but also pain, sleep, quality of life etc.) were used in non-blinded trials because this creates a high risk of bias. This is standard practice.

In the case of graded exercise and CBT, the treatments aimed to change how patients interpret and report their symptoms. Therapist were instructed to encourage optimism while patients were told to focus less on their fatigue. Symptoms were said to be the result of anxiety, stress, or deconditioning which could be overcome by graded exercise or CBT. This creates a high risk of bias where patients are stimulated to report their symptoms as less severe, even if these did actually not improve.

‘How else than asking patients how they are can one assess whether a treatment has worked?’ By using objective data such as activity levels, employment levels, disability payments, fitness data, etc. These showed available but showed no improvements.

5. Normal NICE guideline development involves the synthesis of research evidence by experts but for this guideline there was remarkably little aggregation and meta-analysis, making it hard to conclude what treatments worked. The analysis fell short of international standards.
They seem to suggest that the evidence review was not prepared by methodological experts as usual. As far as I can tell, however, this was the case: hundreds of pages with meta-analyses were prepared by the National Clinical Guideline Centre (NCGC) which is hosted by the Royal College of Physicians (RCP).

White et al. seem disappointed that NICE didn’t perform the data synthesis as they would like it, namely by combining as many GET and CBT trials as possible so that a strong conclusion can be made.

I suspect NCGC did not do this because it results in high heterogeneity. This signals that the trial design or interventions differed too much to be added to the same analysis. In case of high heterogeneity in meta-analyses, it is usually recommended to split up comparisons into studies and interventions that are more alike. This is what NICE did. A 2015 review on ME/CFS by a National Institutes of Health Working Group in the US used a similar approach.

6. In the 2021 guideline NICE described GET as incorporating fixed increments of exercise that are pursued irrespective of how the patient feels. However, this does not reflect the therapy reported in the research. Clinical trials, and the previous NICE guideline are clear that activity in GET is determined collaboratively with the patient and only increased as the patient feels able. NICE banned a treatment that no one provides.
Here they might have a point that the wording ‘fixed increments’ is not on point. But that is merely a language issue because the underlying argument remains the same.

Graded exercise therapy is not ‘symptom-contingent’ because symptoms are viewed as an unreliable signal that keeps patients in a vicious cycle of inactivity and disability. The goal of the therapist is therefore to break this cycle by encouraging patients to gradually increase their activity levels even if they do not feel up to it. Here’s how Peter White himself described the rationale behind GET: “Patients can be released from their self-perpetuating cycle of inactivity if the impairments that occur due to inactivity and their physiological deconditioning can be reversed. This can occur if they are willing to gradually exceed their perceived energy limits.” If a patient develops symptoms as a response to increased activity, they were told to keep exercising at that level, based on the assumption that the body would adapt. PEM is not taken into account.

7. In NICE guidelines on other relevant health conditions such as chronic unexplained pain, GET and CBT are still recommended. Since such pain is common in CFS/ME, how can a clinician choose which guideline to follow?
Since ME/CFS is a more specific diagnosis than pain, it is likely that a clinician will folow the ME/CFS guideline. This seems like another weak argument. It is not uncommon for patients to have multiple conditions where treatment advice conflicts. A patient with kidney or liver disease, for example, will have to take this into account when he/she takes drugs for other health conditions. Clinicians always need to balance the benefits and possible side effects of treatments based on their patient's individual case.

8. Having downgraded the evidence for trials of CBT and GET, NICE recommended the use of “energy management,” in which patients are encouraged to stay within the energy limits imposed by their illness, also known as pacing. But there is little or no evidence to support such an approach. The only substantial trial of pacing for CFS/ME published to date showed that such an approach was no more effective than specialist medical care alone and less effective than either CBT or GET.
The version of pacing used in the PACE trial does not correspond to how patients use and interpret it. It is not seen as a treatment or intervention, but a way of managing symptoms more efficiently much like MS patients would use a wheelchair for difficulty walking.

 

Here's an example: fatigue assessed in the Rituximab trial was not downgraded (page 36 of Appendix F). The quality of evidence was considered high.

 

I'm pretty sure NICE did not invent descriptions of GET as 10% increase in activity every 1-2 weeks. It must have been described that way in some context.

The PACE trial GET manual for therapists says on page 36



What is meant with "exercise for the following two weeks negotiated and planned" is therapist and patient agreeing on an exercise goal. Looking at the other things that were meant to be included in a session, this negotiation could be heavily biased in favor of the therapist's view of what is constitutes appropriate exercise goals.

Describing GET as negotiated increase in activity outside of this context and without details makes it sound more flexible than it really is.

 

I hope you will submit these comments to the journal @ME/CFS Skeptic. You know this stuff well.

 

Another bit from the manual


So in the PACe trial GET sessions took place every two weeks, and therapists were instructed to encourage patients to increase exercise duration for the next two weeks by 20% after a "negotiation" which was accompanied by various other things that would probably influence what patients would agree to do.

Later it says "GET plans for incremental increases in activity".

It seems credible to me that GET as delivered in many contexts was effectively a 10-20% increase in exercise duration every 2 weeks, with (from the patient point of view) inflexibility, pressure or coercion.

Sick people are more vulnerable to pressure because they're stress intolerant and insecure. And therapists who are told that patients have irrational beliefs about exercise being dangerous will be much more willing to apply pressure.

That said I've also had the impression that softer forms of GET existed that were called GET but emphasized reduction in activity in response to symptoms bringing it perhaps closer to pacing than rigid GET.

 

Blog from Brian Hughes.

The cries for help are getting louder. And that’s a good sign

"The JNNP article is little more than a self-serving letter to a friendly editor, infused with special pleading and groupthink. It has no standing other than as a work of academic contrarianship.

It is a cry for help.

Please, nobody help them."

https://thesciencebit.net/2023/07/12/the-cries-for-help-are-getting-louder-and-thats-a-good-sign/

 

GET as described in the main research studies, such as PACE is described as using fixed increments, so if the authors are saying that their GET is ‘symptom contingent’ they are then saying that they have introduced a new unevidenced therapeutic intervention, so the previous research, regardless of its quality, is irrelevant. Here the authors are guilty of doing what they accuse NICE, changing definitions to suit their own objectives.

Anecdotally GET as practiced by individual clinicians across the UK specialist services seems to be variable from those rigidly implementing fixed increments right through to something that might be more akin to pacing, which rather suggests many/some clinicians were already uncomfortable with GET as recommended by the replaced 2007 Guidelines. So what is being advocated here is not GET as previously recommended by NICE and is not an evidenced based treatment, but rather an approach based on clinical experience, ie anecdote, the very grounds that the authors use to reject the evidence of harm.

However we do have various surveys of that indicate the vast majority of people treated with something called GET do not believe that it was in the long term helpful, with only a small percentage reporting any improvement but a much larger minority reporting ongoing harm. Most relevant is the survey of people with ME in the UK (ie treated by services operating under the old NICE guidelines, including services several of the authors are/were linked to) commissioned by NICE as part of the new guidelines process that clearly replicated this pattern of results. So, however it is defined, GET as provided in the UK is not supported by the survey evidence.

A number of the authors elsewhere argue that it is unacceptable not to offer any treatment aimed at curing the condition, so GET should be offered because they know clinically some report it helpful. This argument was historically used by defenders of blood letting, not a good starting point. Further it seems to display a lack of understanding that the evidence thresholds for not providing a treatment because of harms ethically needs to be very different for the evidence threshold for giving everyone a controversial intervention. This is especially so as the authors here or elsewhere make no attempt to distinguish between those who will be harmed and those who might benefit. Essentially they are saying everyone should be offered GET, because we believe in it, though we are unclear quite what that is, on the off chance you might be one of the very few that benefit, though you are more likely to be harmed by the exercise (pun intended) and most will receive no long term benefit.

 

NICE guideline is robust and globally supported

"On 11th July, a new study was published in the Journal of Neurology, Neurosurgery and Psychiatry purporting to demonstrate flaws in the NICE ME/CFS guideline review process.

The World ME Alliance strongly rebuts these suggestions.

Our 24 member organisations are coordinating a global response to this study to demonstrate, on record, the unity of the ME community.

The study has been reported on in various media outlets, initially the Guardian. As a first step, Sonya Chowdhury, CEO of Action for M.E. and Co-Chair of the World ME Alliance, has sent a letter to the editor of the Guardian outlining our support for the NICE ME/CFS guideline. We also note that the arguments presented by these researchers have repeatedly found to be lacking, and the clear conflicts of interest that are not being reported in the press."

https://worldmealliance.org/2023/07/nice-guideline-is-robust-and-globally-supported/

 

This extract from Brian’s blog cited by @Andy is worth repeating:

My bolding.

 

I wish Stewart Lee (comedian) or Marina Hyde (political columnist), both regular Guardian contributors, had the inside track on this story.

I'd pay good money to read one of their scathing, hilarious commentaries on this crew and its antics.

 

I remember a reply from Crawley to reviewers of one of her papers querying the percentage increase and her saying that it was standard practice as outlined by NICE.

found it, it was in the MAGENTA protocol so it would have been in the old 2007 guidelines(?) if NICE did in fact cite it, I havent looked. (or maybe she made it up?)

https://www.s4me.info/threads/why-s...t-therapy-act-2022-crawley.28384/#post-426613

 

According to my memory, in the past pacing was sometimes described by patients as a treatment that could lead to an improvement in the illness. This wasn't a widely shared view. This conceptualization of pacing would sometimes appear on social media. After the PACE trial it seemed that belief in the therapeutic/curative power of pacing disappeared.

I'm just anticipating a possible criticism here because there might exist an older document somewhere by say Action for ME where pacing is described as treatment.

In the PACE trial APT was not intended to be curative so it's irrelevant anyway. I agree that in the PACE trial APT was not how patients generally envision it

 

Yes and no. Their "defence" of their crap studies is so bad it goes into the hilarious. On the other hand I wouldn't want to lend any credence to their bullshit by responding to it. If the stories aren't getting any traction they don't warrant a response imo. I was on twitter last night planning to respond to some of the authors but thought better of it.

There's the public fight which doesn't really work without publicity for their view. Then there's the academic fight which NICE or other scientists or science literate people should take on. I'm none of the above, but can see they haven't got a leg to stand on and are gonna get shredded once NICE or others get right of reply to their article for BMJ.

 

Good spot

 

This is excellent

 

Good to see