(No discussion) E-letters submitted to JNNP replying to White et al. "Anomalies in the review process & interpretation of the evidence in the NICE..."

Moderator note:
This thread is for collecting copies of letters submitted as rapid responses to the article discussed here:
Anomalies in the review process and interpretation of the evidence in the NICE guideline for (CFS & ME), 2023, White et al
To discuss these letters, please go to that thread. If you want to have your letter added here, but don't want to be identified as the writer, send a copy by private message to a mod and we wlll post it for you.

A thread for both published and unpublished letters.

I thought it would be good to collate them all in one place.

 

Published:
https://jnnp.bmj.com/content/early/...shortcomings-in-the-commentary-by-white-et-al

Shortcomings in the commentary by White et al.

• Dom Salisbury, Independent Patient Researcher None
• Other Contributors:
  • Robert H Saunders, None
  • Jonathan CW Edwards, Emeritus Professor of Medicine

There are several shortcomings in the commentary by White et al. For brevity, this response focuses on four main points.

1. New case definition
In the past 20 years, multiple case definitions have been published that require post-exertional malaise (PEM) as a core feature of ME/CFS, such as the Canadian Consensus Criteria (CCC), International Consensus Criteria (ICC), and the Institute of Medicine (IOM) criteria. NICE’s definition is based on the latter.

These case definitions are the ones used in research and clinical practice today. White et al. refer to the 1994 criteria developed by the Centers for Disease Control and Prevention (CDC) but the CDC no longer seems to use this case definition. Instead, they advise healthcare providers to diagnose ME/CFS using the IOM criteria where PEM is a required symptom.

NICE evaluated scientific evidence for ME/CFS as it is currently defined and not for a case definition that was published nearly 30 years ago. Other reviews on ME/CFS, such as the recent one by IQWIG in Germany, have used a similar approach. (1)

It is incorrect to state that NICE “downgraded nearly thirty years of research.” The previous NICE guidance from 2007 on ME/CFS already highlighted PEM as a core feature of ME/CFS, and studies that used this description were not downgraded in the evidence review. Neither were studies that used the CCC, ICC, or IOM criteria mentioned above.

2. Blinding and subjective outcomes
White et al. write: “The NICE committee decided to downgrade all fatigue outcomes based on the premise that it is a subjective measure.” This is incorrect. Fatigue outcomes in studies with a low risk of bias such as the double-blind Rituximab trial were not downgraded.

NICE did downgrade quality of evidence when subjective outcomes (not just fatigue, but also pain, sleep, quality of life, etc.) were used in trials where participants and therapists were not blinded as this combination creates a high risk of bias. This grading is in accordance with the Cochrane Handbook which states that “the potential for bias cannot be ignored even if the outcome assessor cannot be blinded." (2) In the case of patient-reported outcome measures such as fatigue, the Cochrane Handbook considers patients to be the outcome assessor.

Moreover, trials of GET and CBT also included objective measurements such as fitness tests, actigraphy, and employment data and these indicated no clinically relevant improvements. (3)

3. Long-term follow-up
The largest studies on GET and CBT — the PACE (4) and GETSET (5) trials — failed to find significant benefits of these interventions at their longest follow-up. This suggests that initial improvements might be due to response bias or a placebo effect. It is questionable to recommend treatments if the long-term follow-up shows no benefit.

In both GETSET and PACE, a sensitivity analysis indicated that additional treatment received after randomization was unlikely to explain the null results. The authors of GETSET reported that “there is no evidence that the improvements observed in the SMC group were due to them having received more exposure to therapy than the GES group after trial completion.” (5) Similarly, in the PACE trial the control group caught up on the intervention group and “there was some evidence from an exploratory analysis that improvement after the 1-year trial final outcome was not associated with receipt of additional treatment with CBT or GET.” (4)

4. Harms
The Cochrane review on GET included only randomized trials but concluded: “We are uncertain about the risk of serious adverse reactions because the certainty of the evidence is very low.” (6) There was too little data on adverse effects to form a conclusion.

Surveys and qualitative studies, on the other hand, have consistently found a negative impact of GET on some ME/CFS patients. This has been reported for several decades, in multiple countries, and with little indication that harms are due to improper implementation of GET. (7) Considering the adage ‘primum non nocere’, it is sensible to use a lower threshold for evaluating evidence on the potential harms of a medical intervention compared with its potential benefits.

At the NICE Roundtable Discussion prior to Guideline release (8), health profession stakeholders had ample opportunity to raise concerns. Following clear, succinct responses from a representative of the committee, consistent with our comments above, no further objections were raised.

References
1. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG). Current scientific knowledge on myalgic encephalomyelitis / chronic fatigue syndrome (ME/CFS). [N21-01]. 2023. Available from: https://www.iqwig.de/download/n21-01_me-cfs-aktueller-kenntnisstand_absc...

2. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions. 2nd Edition. Chichester (UK): John Wiley & Sons; 2019.

3. Vink M, Vink-Niese A. Graded exercise therapy for myalgic encephalomyelitis/chronic fatigue syndrome is not effective and unsafe. Re-analysis of a Cochrane review. Health Psychol Open. 2018 Dec;5(2):2055102918805187.

4. Sharpe M, Goldsmith KA, Johnson AL, Chalder T, Walker J, White PD. Rehabilitative treatments for chronic fatigue syndrome: long-term follow-up from the PACE trial. Lancet Psychiatry. 2015 Dec 1;2(12):1067–74.

5. Clark LV, McCrone P, Pesola F, Vergara-Williamson M, White PD. Guided graded exercise self-help for chronic fatigue syndrome: Long term follow up and cost-effectiveness following the GETSET trial. J Psychosom Res. 2021 Jul 1;146:110484.

6. Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database Syst Rev. 2019 02;10:CD003200.

7. Geraghty K, Hann M, Kurtev S. Myalgic encephalomyelitis/chronic fatigue syndrome patients’ reports of symptom changes following cognitive behavioural therapy, graded exercise therapy and pacing treatments: Analysis of a primary survey compared with secondary surveys. J Health Psychol. 2019 Sep;24(10):1318–33.

8. https://www.nice.org.uk/guidance/ng206/documents/minutes-31

Show Less
Conflict of Interest:
None declared.

 

[This was initially posted on the JNNP then deleted with the following comment posted https://jnnp.bmj.com/content/early/...63.responses#notice-regarding-rapid-responses ]


The full text of MEAction UK’s rapid response submitted to the JNNP article can be read here:

https://www.meaction.net/2023/07/12...apid-response-to-the-jnnp-in-support-of-nice/

 

Francis Ak'enamé

 

Charles Shepherd (medical adviser to the ME Association) had a letter published on the BMJ website in response to its report about the JNNP article: https://www.bmj.com/content/382/bmj.p1621/rr

Dr Shepherd posted the following on Facebook:

NB: The first part of my letter, which was concentrating on the conflicts of interest by some of the clinicians and researchers who authored this paper, has not been published.

People with ME/CFS welcome the recommendations in the NICE guideline

Dear Editor

The NICE guideline committee (of which I was a member) spent an enormous amount of time over three years reviewing all the evidence on CBT and GET from both clinical trials and from people with ME/CFS.

They concluded that the clinical trial evidence for the use of CBT and GET was of very poor quality and acknowledged that people with ME/CFS have consistently reported that CBT was ineffective and that GET made their condition worse - some ending up in wheelchairs as a result. off receiving harmful advice on activity mansagement.

The ME Association is therefore pleased to see that NICE is robustly defending the new recommendations regarding the use of CBT and the removal of GET and the way in which these recommendations were reached.

These recommendations have been widely welcomed by the ME/CFS patient community here and abroad, and by the vast majority of health professionals who are actively involved in managing people with ME/CFS in both primary and secondary care.

Competing interests: No competing interests

 

Well the delay has passed so here is mine.

____________________________________


I want to point out glaring omissions in the review by White and its 50 signatories. Following publication of the final draft of the guidelines for public consultation, the process was paused, the final publication delayed, so that a roundtable where those so-called "anomalies" could be voiced and heard. And they were.

After weeks of delay, the roundtable was held. Those so-called "anomalies" were presented. And it was concluded that there were no valid concerns warranting further delay or modification. Thus, the guidelines were published and NICE proceeded with their implementation.

There are few public references to this roundtable, as it was subject to privacy restrictions and its organization communicated in private. Snippets and a timeline have been documented on the Science4ME forum [1], and minutes have been published by NICE [2], alongside very extensive documentation of their process.

Since then, the German health body IQWIG has also concluded that much of the same evidence is unfit for purpose, excluding all but three trials for consideration [3], on the same evaluation that they are of low quality and subject to high levels of bias, with years of routine use in clinical practice having further made the case for their inadequacy. It is widely accepted that there are no effective treatments for ME/CFS. This remains the case, a failing that has been massively amplified by Long Covid, causing much the same illness in millions. The new guidelines merely reflect this fact.

The exclusion of the near totality of the evidence base by IQWIG, which many of the signatories, including two who are on the editorial board of JNNP, contributed to or frequently cite in their own work, is further evidence that the process NICE followed was adequate. It follows similar downgrades by the CDC in 2017 [4] and a thorough analysis by the National Academy of Medicine, then the Institute of Medicine, in 2015 [5].

It should also be noted that the much-publicized members of the committee who resigned in protest signed off on the guidelines before they left. This is documented in the aforementioned minutes. They may have had opinions about the substance of the guidelines, but they could not fault the process. And they did not. Their resignation may be over a difference of opinion, but it is immaterial to the process that NICE dutifully followed, as is alleged in the review by White.

And the most important detail in this sordid affair: NICE rejects all of those points [6]. NICE commented: "We reject entirely the conclusions drawn by the authors of this analysis, and in particular their conclusion that Nice has not followed international standards for guideline development which has led to guidance that could harm rather than help patients."

The evidence review was done by independent technical staff and followed NICE's standard procedure. Disagreeing with the outcome does not grant license to dismiss its process, especially from a position of long-held, and well-documented, biases and conflicts of interest.

[1] https://www.s4me.info/threads/nice-...rom-the-s4me-news-in-brief.27957/#post-421972

[2] https://www.nice.org.uk/guidance/ng206/documents/minutes-31

[3] https://www.iqwig.de/download/n21-01_me-cfs-aktueller-kenntnisstand_berichtsplan_v1-0.pdf?rev=209654

[4] https://virology.ws/2017/07/10/trial-by-error-the-cdc-drops-cbtget/

[5] https://pubmed.ncbi.nlm.nih.gov/25695122/

[6] https://www.theguardian.com/society...scouraging-exercise-is-flawed-say-researchers

 

I just found this one by Peter White (a different Peter White to the one who was lead author of the anomalies paper).

It starts:

The following points were submitted in a short Rapid Response but not published. The JNNP has stated that they took-down Rapid Reponses because of ‘inappropriate inflammatory language’. This reply includes more detail than the Rapid Response, but is similar in tone.

In Brief:
• White et al’s assertion that Post-Exertional Malaise is a new requirement in the 2021 NICE guideline is false. Post-Exertional Malaise was required in the previous 2007 NICE guideline.
• White et al’s assertion that, in trials, Graded Exercise Therapy was “only increased as the patient feels able, dependent on their symptomatic response”, is also false, see Moss-Morris (2005).
• There is a systemic under-reporting and dismissal of patient reports of harm across all medical fields. Dismissal of reports of harm repeats mistakes of other devastating medical scandals as evidenced by the UK Govt ‘Cumberledge Review’ published in 2020.


Continues at:

https://pubpeer.com/publications/95FD14B92656A4AD5498AD9928930C

 

And here is my rapid response submission:

Dear Editor,

Contrary to the claims made in the published review article, ‘Anomalies in the review process and interpretation of the evidence in the NICE guideline for chronic fatigue syndrome and myalgic encephalomyelitis’ [1], NICE have rejected these in their entirety [2]. The article makes multiple allegations. This includes the opinion that the NICE process deviated from standard process and was, as a result, unexpected and/or unacceptable to the authors in its findings. The authors, whose extraordinarily long list of conflicts of interests, include being the developers, testers of their own clinical trials/therapies that have been deemed to be low quality by the NICE evidence review [3] dislike robust and thorough scrutiny. Sadly, the authors have failed to provide objective, robust evidence to back up their assertions. Neither did the authors demonstrate how they have counteracted their bias regarding their own theories, hypotheses, and resulting low quality evidence due to the use of well-known, flawed, poor-quality methods resulting in a high risk of expectation bias by relying largely on subjective outcome measures in clinical trials. [3] Due to the very nature of the phenomenon being assessed, such trials cannot be blinded. The complexity and severity of the condition required consideration by NICE to strengthen the scientific process, which is evidence of their determination to produce a robust guideline following the failure of the 2007 guideline.[4]

The article failed to inform the editor, reviewers, and readers about the pause in publication by NICE due to the challenge by a group of professionals, including some of the authors of this paper, prior to the proposed release of guidance in August 2021. During this pause, a round table was convened whereby evidence could have been presented to NICE which, if this passed a quality threshold, would have resulted in a reconsideration of findings. For the challenging group, who include co-authors of this paper, the outcome was that no evidence was presented, and this absence of evidence was agreed upon by attendees. There was nothing of substance on offer, as there is not in the article by White et al. now either. The NICE guideline was then published unaltered in October 2021, [5] after an unnecessary two-month delay.

The authors’ focus suggests that they believe NICE are exceptional in their assessment of the published evidence or are out-of-sync with other international institutions. This is troubling as the first reference that they use is the Institute of Medicine 2015 report [6]. After extensive review evaluating the evidence for the diagnostic criteria for ME/CFS, concluding that, “Patient reports and symptom surveys as well as scientific evidence consistently showed that impaired function, PEM (an exacerbation of some or all of an individual's ME/CFS symptoms after physical or cognitive exertion, or orthostatic stress that leads to a reduction in functional ability), and unrefreshing sleep are characteristic symptoms almost universally present in ME/CFS; thus, the committee considered them to be core symptoms.” Thus, it would be remiss of NICE to not incorporate this into their review process. Demonstrating that NICE did not arbitrarily redefine ME/CFS as suggested. It should be noted that PEM was also required in the 2007 NICE guidelines (CG53 2007, p14) for diagnosis, “characterised by post-exertional malaise and/or fatigue (typically delayed for example by at least 24 hours, with slow recovering over several days).” Also, p18, “The diagnosis of CFS/ME should, be reconsidered if none of the following key features are present: post exertional fatigue or malaise.” [5] The authors must have known this.

Word count 576 (excluding references).

References
1. White P, et al. J Neurol Neurosurg Psychiatry 2023;0:1–8. doi:10.1136/jnnp-2022-330463
2. ME/CFS guidance that discourages exercise is flawed, say researchers Available: https://www.theguardian.com/society...scouraging-exercise-is-flawed-say-researchers The Guardian newspaper.
3. Edwards, J. (2021). Myalgic encephalomyelitis (or encephalopathy) / chronic fatigue syndrome: diagnosis and management. NICE guideline NG206. Appendix 3: Expert testimonies The difficulties of conducting intervention trials for the treatment of myalgic encephalomyelitis/chronic fatigue syndrome. Available: https://www.nice.org.uk/guidance/ng206/evidence/appendix-3-expert-testimonies-pdf-333546588760
4. National Institute for Health and Care Excellence. Myalgic Encephalomyelitis (or Encephalopathy)/Chronic fatigue syndrome: diagnosis and management. NICE guideline [NG206], 2021. Available: https://www.nice.org.uk/guidance/ng206
5. National Institute for Health and Care Excellence. Chronic fatigue syndrome/
myalgic encephalomyelitis (or encephalopathy) Chronic fatigue syndrome/myalgic encephalomyelitis (or encephalopathy): diagnosis and management of CFS/ME in adults and children NICE guideline [CG 53], 2007. Available: https://sacfs.asn.au/download/CG53NICEGuideline.pdf
6. Institute of Medicine. Beyond myalgic encephalomyelitis/chronic fatigue syndrome: redefining an illness. Washington, DC: The National Academies Press, 2015.

 

Ps., I know the journal office received my rapid response as I emailed them to check as I only received confirmation on the screen, not an email. This occurs if the rapid response is approved for publication on the website. So, mine like everyone else's rapid responses have fallen into a dark hole

 

A version of this has now been published:
https://jnnp.bmj.com/content/early/...atigue-syndrome-and-myalgic-encephalomyelitis