Germany: IQWIG Report to government on ME/CFS - report out now May 2023

Side note but relevant I saw on Fb someone talking about a slide from the Berlin conference that compared approaches where consideration to PEM was used and wasn’t in Long Covid. It looked pretty good but was linked to research yet to be published I think.
Does anyone know what I’m talking about and whether it can be linked to etc (I get sense that conference wasn’t one where people could just go pasting material around?)

 

There is also a longer page on treatments which can be viewed here:
https://www.gesundheitsinformation.de/behandlung-von-mecfs.html

It presents pacing, CBT, physical activation and sleep hygiëne as treatment options but all with little evidence.

on GET they write:

"A targeted, gradual increase in activities has been investigated in a number of comparative studies involving people with mild and moderate disease. The corresponding program is called "Graded Exercise Therapy" (GET). However, the advantages and disadvantages of activation cannot be judged well. There is a controversial debate about who can benefit from activation and for whom it is even harmful."
​

on CBT they write:

"Research suggests that a cognitive-behavioral therapy may also have benefits for people with ME/CFS. In studies, for example, she has been able to help alleviate some of the fatigue in ME/CFS and to attend everyday activities such as housework and meetings with other people a little more frequently."
​

The bolding of the last sentence is mine because that seems to be a problematic one.

 

Looking back at the report the main thing that stands out is that they only looked at studies that required 80% of patients to have PEM. This meant that they disregarded about 90% of all RCTs. That is a much more extreme position than the NICE committee which only downgraded the quality of evidence of those studies because of indirectness.

Because of this decision, IQWiG only had 3 trials on GET and CBT to work with: The PACE trial, the GETSET trial and a 2018 Dutch trial on CBT by the research team of Hans Knoop. IQWiG correctly notes that all 3 studies were at high risk of bias because of lack of blinding and the reliance on subjective outcomes. They even write (on page 124) that: "Since the risk of bias of all 3 studies included was already classified as high across outcomes, there was no outcome-specific assessment of the risk of bias." So no need to look closer, these trials were already high risk. But then they conclude that there is evidence for CBT, without taking the high risk of bias into consideration. So once again we see the problem of a 'bias ceiling': where the highest risk of bias taking into consideration is still good enough to make treatment recommendations. There is no option for a junk trial: one that is so high risk of bias that nothing useful can be concluded from it.

Even if one would to disregard the bias, the trials still do not show a convincing effect. Most of the outcomes were not statistically or clinically relevant and none were maintained at the long-term follow-up. The clearest estimate was arguably for CBT after 24 weeks or 6 months. The meta-analysis of 2 studies found an effect size of 2.95 points on the Chalder Fatigue Scale, corresponding to a SMD of -0.39 [-0.57, -0.21]. The confidence interval is 0.01 point above the insignificance threshold of 0.2 and some studies have suggested 3 points to be the minimally important difference on the Chalder Fatigue Scale. So this isn't a clear effect at all. The same story for 'social participation' using the Work and Social Adjustment Scale (WSAS).

For both outcomes, there was no longer a statistically significant effect at long-term follow-up. And for sleep quality, pain, activity levels, physical performance, physical function, cognitive function, and mental status there was no indication of an effect of CBT at any time point. So even if they speak of a 'hint' of an effect it still seems like an overstatement of the actual evidence.

 

There is an overview of the changes made on page 202 onwards (PDF).

Spoiler: Deepl translation: Preliminary report compared to report plan 1.0

In addition to editorial changes, the following specifications or changes resulted in the preliminary report:

• Specification of Chapter A2: After publication of General Methods 6.1, the report was prepared based on this current version [88].
• In the Background section of the report, the clinical details of the disease (e.g., symptomatology or etiology) were deleted. These are results of the report section of the current state of knowledge and presented in the preliminary report at the appropriate place (see Section 4.2).
• Specification of section A2.1.1.1: With regard to the inclusion criterion for the population, the following specifications were made:
  • The inclusion criteria E1a, E1b, E1c, and E1d for the inclusion of sources in the report were specified as follows: In order to assume the existence of ME/CFS, a diagnosis must have been reported according to criteria catalogs describing PEM as an obligatory symptom of the disease (e.g., in terms of the CCC or the IoM criteria). If a diagnosis is made according to a criteria catalog that does not describe PEM as an obligatory symptom, additional information on the proportion of the respective population with PEM symptoms must be provided for study inclusion - if no analyses of the subpopulation of the study with PEM are available - that suggests that at least 80% of the population has a disease that can be classified as ME/CFS.
  • In exceptional cases, studies with a violated inclusion criterion E1a were included and used in the report section of the current state of knowledge. On the one hand, study populations without a definite PEM diagnosis were included if it could be reasonably assumed that the basic statements of the studies could be transferred between populations with and without a definite PEM diagnosis (e.g., report section 4.2.2.2 on symptoms) or if the study results allowed a comparison of patients with and without PEM (see, for example, report section 4.2.2.5 on epidemiology). On the other hand, sources based on a healthy normal population were also included if the development of ME/CFS disease was investigated in this population. This concerned the report section 4.2.2.4 on etiology.
• Specification of section A2.1.2: The following specifications were made with regard to the joint information procurement on the current state of knowledge, evidence mapping of therapy options, and health information:
  • For the comprehensive bibliographic search for systematic reviews, a restriction was made to sources with a publication date of 2018 and later. In the case that there was no or insufficient information from current systematic reviews on certain topics, results from other sources with a publication date before 2018 were considered. This time restriction was made because the timeliness of evidence on ME/CFS is essential for the development of all parts of this report-particularly for the current state of knowledge.
  • To complement the comprehensive bibliographic search for systematic reviews, a focused search for primary studies was conducted for the topic area "Care situation in Germany" of the current state of knowledge. The selection of sources was done by 1 person, the quality assurance of the result was done by a 2nd person.
  • To complement the comprehensive bibliographic search for systematic reviews, an orienting search for current primary literature was conducted for the topic area "Etiology" of the current state of knowledge (see Section 4.2.2.4). This supplementary search was performed because, due to missing information in the systematic reviews, it cannot be assumed with certainty that this is the target population of the present report. Studies with a publication date of 2018 and later were predominantly included. Source selection was performed by 1 person, and quality assurance of the result was performed by a 2nd person.
  • For the focused systematic search for qualitative studies to identify lessons learned and information needs, only studies published in 2018 and later were included.
• Change to Section A2.1.3: The following changes were made regarding the quality assessment of sources:
  • A quality assessment of the included sources is performed for the report sections of the evidence mapping and the benefit assessments. A quality assessment of the included sources of the current state of knowledge was not considered necessary, as the results in this report part are descriptive in nature and no recommendations are derived from them.
  • In the evidence mapping, only a quality assessment of the systematic review that served as the evidence base was performed. The quality of the other sources included but not used was not assessed because no results were presented from these sources.
• Specification of section A2.1.3:
  • The assessment of the overall quality of systematic reviews was based on the validated Oxman and Guyatt quality index. The assessment includes, for example, checking whether information retrieval and study selection were performed appropriately. Furthermore, it was checked whether the instruments or criteria used for the assessment of the potential for bias at the study level appear to be sufficient and whether the results of this assessment are fully reported for all included studies. If this is not the case, the quality of the systematic reviews is downgraded due to methodological deficiencies.
  • The presentation of the results in the report section of the current state of knowledge is done as a narrative review by selecting the core results from the sources consulted on the topics of definition, complaints/symptoms, diagnostics, etiology/causes, and epidemiological and health care aspects.
• Change of section A2.2: The section "Evidence mapping of therapy options relevant to care" was renamed to "Evidence mapping of therapy options", because there is currently no established therapy for ME/CFS - apart from symptomatic treatment.
• Change to section A2.2.1.3: In the evidence mapping, all endpoints reported in the systematic review that served as the evidence base were extracted for completeness. Accordingly, no selection of endpoints and no own assessment regarding patient relevance (inclusion criterion E6b) was performed.
• Modification of sections A2.2.1.3 and A2.3.1.3: In the report sections of the evidence mapping of therapy options and the benefit assessments, harm-related endpoints such as serious adverse events are presented under the endpoint category name "adverse events" instead of "adverse drug reactions".
• Specification of section A2.3.1.2: Based on the results of the evidence mapping, 2 therapies were selected for benefit assessment using the following 2 criteria: (1) at least 2 RCTs on clinical effectiveness were identified and (2) at least one of these RCTs showed statistically significant effects in favor of the intervention.
• Specification of the section A2.3.1.5: The criteria for inclusion of studies in the benefit assessments were not restricted with regard to study duration.
• Specification of section A2.3.2.1: The assessment of the quality of information provision of the basic screening was based on AMSTAR, item 3 [156].
• Specification of section A2.3.3.3: The following passage on the procedure for meta-analyses was added [specification in square brackets]: If the heterogeneity test yields a statistically significant result (p < 0.05) [for a situation in which effect estimation is performed according to random-effects model], only the prediction interval is presented in the case of at least 5 studies.
• Specification of section A2.3.3.6: For the derivation of the evidence situation, the usable data were presented grouped by evaluation periods into short-, medium- and longer-term. Here, data with observation periods of up to 6 months were considered short-term results, of more than 6 and up to 15 months as medium-term results, and beyond as longer-term results.
• Change to section A2.4.4: No qualitative user testing of the health information was conducted prior to publication of the preliminary report. This was due to the decision made during the course of the project to conduct 2 benefit assessments. Since the results of the benefit assessments can only be incorporated into the information materials after their completion, user testing of the health information prior to publication of the preliminary report was not expedient. Furthermore, with regard to the involvement of those affected, it appeared to make more sense to conduct the user testing and the commenting procedure on the preliminary report at the same time. Consequently, the user testing of the health information took place after the publication of the preliminary report.

Spoiler: DeepL translation: Final report compared to the preliminary report

A1.2.2 Final report compared to the preliminary report
In addition to editorial changes, the following specifications or changes were made in the final report during the course of the project:

• In the follow-up search for systematic reviews, additional sources were included for the report sections current state of knowledge and health information.
• After publication of the current versions of the S1 guideline "Long-/Post-COVID" and the S3 guideline "Fatigue," the related information in the report was updated.
• Modification of section 4.2.2 Results on topics of current knowledge and chapter A4 Selection of studies from current research on the etiology of ME/CFS: The texts were supplemented with data from further sources from the bibliographic searches and further sources of information (see also section A3.1).
• Amendment to sec. 4.3 Evidence mapping: "(Serious) adverse events" have been renamed "treatment-related (serious) adverse events".
• Change to Chapter 5 Benefit Assessment:
  • An evaluation at 12 weeks was added to the results on sleep quality.
  • Outcomes on mental status were amended to include outcome data from the HADS subscales at 12 weeks.
  • The section on "Information on Replacement Procedures" (formerly Section 5.6) was moved to Section 5.3 on Characteristics of Studies Included in the Report.
• Change to Section A2.4.4 User Testing: No in-house user testing was conducted with experts. Technical expertise was already involved in the preparation of the information materials through the external experts. The comments procedure was used for broader comments on the drafts by experts.

Specifications and amendments resulting from the consultation on the preliminary report

• Specifications and changes to section 4.2 Current knowledge:
  • In the Complaints/Symptoms section, text on symptoms of PEM, pain, and fatigue was specified and added, separate descriptions of ME/CFS severity levels of "severe" and "very severe" were made, and additional comorbidities were added.
  • Literature references were added to the statements on the lack of a biomarker supporting the diagnosis.
  • The section on diagnosis was supplemented with information on differential diagnosis and with a literature reference from the bibliographic follow-up search.
  • The section on assessment of PEM was expanded to include mention of hand strength measurement as another example of an objective assessment tool and an explanation of the cardiopulmonary exercise test (CPET) in ME/CFS patients.
  • Estimates of the prevalence of ME/CFS have been supplemented in the chapter on epidemiological and health care aspects by the data on the Crawley 2011, Jason 1999, and Jason 2020 studies.
  • Regarding the association between ME/CFS and COVID-19, it was added that an increase in prevalence estimates of ME/CFS after SARS-CoV-2 infections seems plausible, but precise quantification on the magnitude of the increase is not possible.
  • In the report chapter on aspects relevant to care, it was added that patients often experience that many physicians do not know about the disease ME/CFS.The report section on aspects relevant to care in Germany was supplemented by the points on strengthening the care of ME/CFS patients (especially those with severe and very severe disease) such as the possibility of telemedical care or the specific case-based flat rates.

• Specifics and changes to section 4.3 Evidence mapping:
  • In the consultation on the preliminary report, it became clear that there were misunderstandings about the approach of both NICE to evidence reviews and IQWiG to evidence mapping. Therefore, the methodology of study inclusion by NICE and IQWiG and the assessment of study quality by NICE were described in more detail.
  • For the studies used in evidence mapping, it was added that NICE assessed the respective methods used to assess/diagnose the target condition as adequate.
• Specifications and changes to chapter 5 benefit assessment:
  • The contents of the CBT and GET interventions were described in more detail based on the information provided in the corresponding treatment manuals of the PACE and GETSET trials.
  • In the report sections on the results of patient-relevant outcomes, the terms benefit and harm were replaced by advantage and disadvantage.
  • Outcome data on the Self-paced Step Test at 12 weeks and 6MWT at 24 weeks have been added to the physical performance outcomes. In addition, the results of the Borg scale are mentioned.
  • A new section 5.7, "Evaluation of arguments raised at the hearing regarding GET," has been added.
  • In Section 5.8, the benefit-harm trade-off and the evidence map were revised.
• Specifications and changes to Chapter 6: The discussion/classification of the work product was extensively revised.
• Specifications and changes to Chapter 0: The conclusion was adapted to the revised report results.
• Specifications and amendments to the chapter
  A9 Health Information:
  • The uncertainty of the evidence base is described in several places. The importance of the individual situation in managing the disease is pointed out. In addition, the health information does not make treatment recommendations, but describes available treatment options.
  • The designation of ME/CFS as a "physical disease" has been adjusted.
  • It was made clearer that possible examinations by various specialties (including psychosomatic medicine) can serve differential diagnosis.
  • The section on "Rehabilitation" and the section on CBT and social participation were reworded.
  • The description of the symptoms was revised. More emphasis was placed on the core symptom of PEM.
  • The notes on sleep hygiene have been reworded. More emphasis is placed on individual recommendations being potentially inappropriate for (a proportion of) people with ME/CFS. The potential clarification of other sleep disorders was added.
  • The description of severity levels has been adapted according to the NICE guideline [5].
  • The description of activation therapy or GET has been adapted. The term Graded Exercise Therapy (GET) was added to more clearly differentiate this intervention from pacing. The description of CBT was also adjusted.
  • The statements on physical activation were adjusted according to the changes from the preliminary report to the final report.

Btw, the three external experts were:

• Erika Baum, Bieberta: Lead author of the German DEGAM Guideline on fatigue/tiredness
• Wolfgang Eich, Heidelberg: Section Head (Integrated Psychosomatics Section) at Heidelberg University Hospital
  • Focus: "Biopsychosocial aspects of chronic pain, fibromyalgia and somatoform disorders, shared decision making and communication research, medical anthropology"
• Carmen Scheibenbogen, Berlin

Edit: Added the parts

 

I have the impression that the final result is more the result of diplomacy and politics than scientific assessment of the evidence.

Take for example the description they use for CBT:

"This Therapy should help to better deal with the mental stress caused by the disease. In this way, strategies are to be learned that help in dealing with the symptoms and with consequences such as depressive thoughts and fears. This can also include better assessing the disease and learning to dose activities correctly."
​

This does not correspond to how CBT was used in the PACE trial and the trial by Janse et al which argued that ME/CFS is reversible by addressing deconditioning, catastrophizing and fear of activity. The PACE trial wrote about CBT: "The aim of treatment was to change the behavioural and cognitive factors assumed to be responsible for perpetuation of the participant's symptoms and disability". It aimed to cure ME/CFS not help with mental stress caused by the disease.

So if you think about it there are only 2 reasonable positions. Either you support this radical version of CBT or you admit that there is currently no evidence for CBT for ME/CFS at all. Unfortunately, NICE, IQWiG, the Dutch and Belgian health councils all have invented their own version of CBT for ME/CFS and pretend that the trials support this version.

I think it shows that it is very much taboo to question the usefulness of CBT.

 

Though I agree with your point @ME/CFS Skeptic, it could alternatively be argued that though there is no ME specific evidence relating to psychological support, people with ME should have the same access to psychological support as everyone else.

The question is then what evidence is there for any specific forms of psychological support benefiting people in generally difficult situations or more specifically helping people adapting to long term disabling conditions? It seems to me that here, as with the NICE guidelines, CBT is specifically being recommend without any discussion of whether there is evidence of it being relevant to the sort of circumstances people with ME find themselves it or whether other forms of psychological support may be more appropriate.

 

Of course it does. This is just a cop-out from the fact that most treatments have zero effect but they've been marketed and sold as having an effect but they can't show it. So basically like how Theranos managed to scam people for years because no one insisted on seeing the actual machine in operation. When someone demanded it and it couldn't be delivered, the whole scam fell apart.

There are always objective endpoints that can be used. Any subjective effect would have objective effects. Otherwise it's either known as a distinction without a difference, or so completely trivial that it's worthless. Somehow, though, the entire output of the BPS ideology checks both in full. And this is why they avoid objective endpoints: because they have none. Never had any. It was always a scam. And it still is.

 

Journalist Martin Rücker, who published snippets of the draft article spearheaded by Peter White about the “eight errors” made by NICE in the elaboration of their ME/CFS guideline:

“What I don't understand: IQWiG recommends investigating the potential benefit AND harm of GET at MECFS in further studies. How do you make that ethically justifiable when there have long been serious indications of serious damage?”

https://twitter.com/user/status/1658136239197888514

 

It's good that the report acknowledges that PEM is hallmark for the disease. And as @ME/CFS Skeptic points out above, they even originally used the Nacul 0.1% prevalence estimate because it is the only one that clearly used a case definition where PEM is required.

But if they really get that the hallmark of ME/CFS is the abnormal response to exercise and other exertions that is PEM, then it's just stunning that they'd then base their treatment recommendations on exercise studies that did not require participants to experience that abnormal response to exercise.

 

@Medfeb in the post above mine highlights that PEM is acknowledged as hallmark for the disease which should immediately make this sort of research unfeasible and immoral.

 

A comment and IQWiG's reply on that issue ( posts immediately above) on Mastodon:

https://wisskomm.social/@smu@digitalcourage.social/110373082280112004


Anyone up to provide a translation? (Sorry not able myself ATM)

 

@iqwig Your press release leaves me quite perplexed.

You write "In the context of appropriate studies, the potential benefits and harms of activating therapies such as GET should be further investigated."

Against the background of the many case reports of patients who have suffered massive damage in connection with GET and the fact that experts strongly advise against it: What would a good study design look like that is ethically justifiable, i.e. minimizes the risk of serious harm to patients? Shouldn't there rather be a risk-benefit assessment of whether such studies should be carried out at all and whether other options should be explored instead? This clearly comes up short in your comments. Simply demanding more data seems very technocratic at this point and does not exactly win the trust of patients.

---

@smu Thank you for your comment!

We can't shake a study design off the cuff here, but at least we can record it:

(1) According to the descriptions of numerous affected persons and experts, activating therapies are currently carried out in rehabilitation facilities in Germany. This widespread practice could be accompanied by studies without the need to treat even one additional person with ME/CFS with GET.

(2) There are other variants of cautious activation that could be compared to GET in such trials, as well as pacing. Thus, the number of affected individuals treated with GET might actually decrease - because many would end up in the comparison arms.

(3) If methodologically adequate, meaningful studies show clear disadvantages of activating therapies such as GET, this may help to rapidly change treatment practice.

(4) In surveys like the one by Kindlon 2011, a part of the affected persons report deteriorations of their condition after a GET, but a part also reports improvements (see p. 175 of our report). At the moment, it is unclear to what these extremely heterogeneous results are due. It would be important to find out, within the framework of appropriate studies, on which parameters this depends. In this way, in the future, those for whom the chances of improvement are high could be given a cautious activation offer without at the same time causing harm to the other persons concerned.

(5) No one can be forced to participate in a study. Consent is always preceded by an explanation of the aims of the study and of the therapies that will be used in it.

This would presumably also be an improvement compared to the current situation, in which, according to the descriptions of many patients, there is often no open information about the chances and risks (!) of an activation therapy.

Many greetings from the IQWiG - AK

 

The first comment is a woman who's asking legitimate discussions about how they're gonna devise a trial that doesn't cause severe harm to patients and is ethically feasible. This in light of the many patients that reported enormous harm from treatment with GET. A rehabilitation method strongly discouraged by professionals. Shouldn't there be far more of a cost-benefit analysis if these kind of studies should be done anyway. Isn't it better to pursue other options instead? This is a clear shortcoming in your(IQWIG's) statement. Simply gathering more data is only technocratic and doesn't help to build trust.

IQWIG reply: They can't produce a study-design on the spot at the moment. But

(1) By following descriptions* of many patients and professionals activating therapies got implemented in rehabilitation clinics. This far-spread practice was supported by many studies, not one patient with ME/CFS had to take the GET-treatment.

Spoiler

I'm not translating Schilderungen properly here, it's a bit more forceful than just descriptions, it implies some sort of action has been taken I think

(2) There are more variants of careful activation that one could compare to GET in studies like this, also to pacing. That way the number of patients being treated by GET could drop, a lot of them could land in the control group of the study.

(3) If methodically adequate, sufficiently powered studies show clear disadvantages of activating therapies, this could help to change the current/accepted methods of treatment quickly.

(4) In surveys like that of Kindlon(2011) part of the patients report getting worsened by GET, another part reports improvements though. At the moment it's unclear why responses differ so greatly. It is important as part of appropriate studies which parameters cause this. That way one can offer careful activation to patients who's chance at improvement is greatest without harming other patients.

(5) Nobody can be forced to partake in studies. There is a declaration about the goals of the study and the therapies that are studied when you give consent through the application-process.

That is also a presumed improvement to the situation where patients couldn't know about the possible opportunities and risks of activation-therapy.

 

Same ethics as Esther Crawley doing a LP trial on children after saying 'people have said it can be harmful and we need to find out if it is'.

 

@SNT Gatchaman has done a good job of translating. A few notes though. In (1) IQWiG is talking about practices already in place and studies already done. They're implying GET got introduced by will of patients and practitioners. At the time this treatment was also supported by studies which we now know to be worthless.

(2) is spot on as far as I can tell.

(3) Aussagekräftige comes out of google translate but I think it means sufficiently powered. Maybe a native German-speaker can help out on this specific point.

(4) spot on

(5) spot on

The first translation is better than mine because it uses much clearer language. (1) is not entirely accurate though. With regards to (3) I hope someone who is a native German can clarify what they mean by aussagekräftige.

 

Well that to me is the kicker. They talk like they understand ME but their actions show different. They give lipservice to patients fear of harms without actually taking it seriously. Even if people willfully consent to their studies, there are still a lot of desperate people that would join these studies in the hopes of getting better. Whether patient surveys are accurate when it comes to percentages of people harmed or not, I think it's fair to say that this will harm a lot of patients that actually have ME.

Thus it imo preys on people too weak and uninformed to make a better decision with regards to this. I think most people on this forum know better than to participate in a study like this, but Germany is a large country and a good number of desperate ME-patients that don't know better yet are easily found. If you're desperate enough you will try anything at first. I did at least.

Informed consent doesn't mean the study is suddenly ethical. These people can read surveys, have talked to patients etc. They know how much harm their studies can cause and are likely to cause. The onus is on them to prevent that, and that responsibility goes beyond giving a patient a leaflet shortly outlining there might be some drawbacks from participating. They also use very soft language to describe possible harms, while it's an absolute horrorshow to come out of a GET program for a great number of patients. This doesn't bode well for the information they will be giving out to possible participants to said studies.

 

Also once more: if PEM is recognized as a feature, using GET as a treatment is contradictory.

 

Can't claim credit for the translation, I just pushed the button kindly provided by my Mastodon instance. It uses DeepL which is based in Cologne, so I expected it would do a passable job.