Use of selection criteria in ME/CFS research

[Medfeb]

This post has been copied and following discussion moved from this thread:
UK NICE 2021 ME/CFS Guideline, published 29th October - post-publication discussion
_________

My theory on how they came to publish consistently flawed papers is that by repeatedly doing studies they got a sense of what design choices and methods were likely to give results that aligned with their theory. The design choices and methods were as poor as was necessary to get the desired result. The removal of actometers from PACE showed an example of this happening.

Could their choice of selection criteria have also contributed? By its nature, criteria that are focused on medically unexplained fatigue could overrepresent for people who could respond positively to talk therapy and exercise while underrepresenting people who have ME and thus would experience a negative effect to exercise? And the lack of harms evaluation or use of inadequate harms measures would mean studies could not detect the kind of harms relevant in ME on a day to day basis.

I can imagine continued use of Oxford is not really a viable option. But they could continue to use Fukuda, which can include people who do not have ME but have other conditions including depression. Are there other options? NICE 2007? Or some new one? It sounds as though each researcher gets to decide?

The circularity in the psych focus research is I think different from the problem of selecting a research population that is 'a priori' taken to be an exemplar of the whole patient population when the research logically has to allow that the whole patient population may have no significant exemplar cohorts. That problem is overcome by appropriate methodology using multiple diagnostic criteria, something which is already happening.

I think this (research funding) will be influenced by the outcome of the Priority Setting Partnership, and is also best tackled by organisations rather than individuals. Forward ME will have a lot more clout than any of us trying individually to write to them.

I haven't been following the PSP so this may be discussed elsewhere. But I'm wondering if in addition to priority areas of research to study, the PSP will also make priority recommendations on study design and instrumentation such as how patients are selected and how harms are assessed?

For instance, that Fukuda should not be used because it does not require any key features such as PEM and instead focused on medically unexplained fatigue and includes people with other conditions. Or as @CRG notes, that multiple criteria (including e.g. CCC, ME-ICC) should be used as some researchers have done?

Or on the use of appropriate harms measures that explicitly evaluate worsening of symptoms and function, a worsening of tolerance for exertion, etc. Would need to be designed but these are critical for a disease with sensitivity to medications and a negative reaction to all types of exertion.

 

[Jonathan Edwards]

By its nature, criteria that are focused on medically unexplained fatigue could overrepresent for people who could respond positively to talk therapy and exercise while underrepresenting people who have ME and thus would experience a negative effect to exercise?

I think this argument should be abandoned. If we look at the studies carefully we see no meaningful effect even though the criteria are wide. The longer people cling to an idea that these trials actually showed something for a broad group the longer before things move on. We need regulatory bodies to understand why these studies should be considered negative or at least worthless. Sadly, not even NICE made that explicit because they used GRADE.

For instance, that Fukuda should not be used because it does not require any key features such as PEM and instead focused on medically unexplained fatigue and includes people with other conditions.

It is wrong to tell scientists what they can and cannot study. They can study whatever they think may provide useful results as long as ethics committees approve. Studying a Fukuda group may provide useful information for Fukuda groups.

BUT

I see something interesting in the NICE 2021 guideline. That is the shift from CFS/ME to ME/CFS. Although nobody should tell researchers what they can do, researchers will want to do studies with a chance of impacting clinical practice. CFS does not exist any more in the UK and I think that may not have sunk in. Yes there is ME/CFS and yes there are garbage categories like MUS or even chronic fatigue but CFS has gone. And it is NICE that sets the agenda in this case - more by chance than plan.

As things stand any study of 'CFS' can be expected to be downgraded by NICE whenever it makes a new decision. So there may be a major disincentive to use Fukuda now.

 

[Trish]

I haven't been following the PSP so this may be discussed elsewhere. But I'm wondering if in addition to priority areas of research to study, the PSP will also make priority recommendations on study design and instrumentation such as how patients are selected and how harms are assessed?

I wish they would, but there didn't seem to be the option to do that in the survey, if I remember correctly. Best discussed on this thread:
https://www.s4me.info/threads/uk-priority-setting-partnership-for-me-cfs.13973/

 

[Ariel]

CFS does not exist any more in the UK and I think that may not have sunk in. Yes there is ME/CFS and yes there are garbage categories like MUS or even chronic fatigue but CFS has gone.

Is it remotely likely that practitioners will stop referring to and thinking of ME as "chronic fatigue" and the like anytime soon, however? Conceptually "CFS" is gone but the confusion and obfuscation lives on in practice currently. The NHS website changes for ME/CFS didn't even reflect what was in the NICE 2021 Guideline. Is the message going to get through that CFS is gone?

 

[Jonathan Edwards]

Is it remotely likely that practitioners will stop referring to and thinking of ME as "chronic fatigue" and the like anytime soon, however?

That is a slightly different issue from the political one about doing studies on a disease that no longer appears in the clinical guidelines that control the service pursestrings.

Nevertheless, I think it quite likely that GPs and multidisciplinary practitioners will switch to ME/CFS simply because that is what they will be putting on the forms that bring in the cash. UK medics are very quick to change to a new name if it is part of guidelines. They changed diabetes to type 1 and 2, osteoporosis to maybe osteopenia, indigestion to GERD, bronchitis to COPD.

And again, it may not matter to the point about criteria if some people stick to chronic fatigue or shift to MUS. Fukuda is for CFS and there isn't a home for that now. Oxford was never used much outside Oxford and it carries the penalty of a one point downgrade on GRADE.

 

[Suffolkres]

That is a slightly different issue from the political one about doing studies on a disease that no longer appears in the clinical guidelines that control the service pursestrings.

Nevertheless, I think it quite likely that GPs and multidisciplinary practitioners will switch to ME/CFS simply because that is what they will be putting on the forms that bring in the cash. UK medics are very quick to change to a new name if it is part of guidelines. They changed diabetes to type 1 and 2, osteoporosis to maybe osteopenia, indigestion to GERD, bronchitis to COPD.

And again, it may not matter to the point about criteria if some people stick to chronic fatigue or shift to MUS. Fukuda is for CFS and there isn't a home for that now. Oxford was never used much outside Oxford and it carries the penalty of a one point downgrade on GRADE.

I Google SNOWMED and NICE

Interestingly my husband ( ME diagnosed finally 2006) has a non ME related acute prtoblem and blip recently resulting in... 'meeting the criteria? for a 2 week fast track investigation under the NHS
His resume from GP actually referenced ME snowmed hurrah!! so they can do it!

https://www.nice.org.uk/about/what-...al-procedures-guidance/coding-recommendations
SnOWMED is there we need to remind and use it!

 

[Medfeb]

I think this argument should be abandoned. If we look at the studies carefully we see no meaningful effect even though the criteria are wide. The longer people cling to an idea that these trials actually showed something for a broad group the longer before things move on. We need regulatory bodies to understand why these studies should be considered negative or at least worthless. Sadly, not even NICE made that explicit because they used GRADE.

My point was purely about the criteria, not the quality of the methodology or findings . Even if the study is superbly designed and executed on every other count, if the criteria used are so non-specific as to require nothing more than medically unexplained fatigue, then to whom/what conditions do the conclusions apply.

It is wrong to tell scientists what they can and cannot study. They can study whatever they think may provide useful results as long as ethics committees approve. Studying a Fukuda group may provide useful information for Fukuda groups.

I wasn't suggesting telling scientists what to study. If they want to study a Fukuda group to provide useful information for "Fukuda groups", that's their choice although I question the utility of doing so. But I'd question whether that's the same condition or group of conditions described by the CCC, ME-ICC, IOM, and NICE 2021? And if not, should those Fukuda studies keep using the label "ME/CFS" for their cohort?

I see something interesting in the NICE 2021 guideline. That is the shift from CFS/ME to ME/CFS. Although nobody should tell researchers what they can do, researchers will want to do studies with a chance of impacting clinical practice. CFS does not exist any more in the UK and I think that may not have sunk in. Yes there is ME/CFS and yes there are garbage categories like MUS or even chronic fatigue but CFS has gone. And it is NICE that sets the agenda in this case - more by chance than plan.

As things stand any study of 'CFS' can be expected to be downgraded by NICE whenever it makes a new decision. So there may be a major disincentive to use Fukuda now.

That's an interesting point - and will be interesting to see if it plays out that way in the UK. In the US. CDC adopted the IOM criteria in 2017. But we don't have a body that produces national guidelines so guidance gets produced by many different bodies and they don't always agree. Perhaps because we lack a national guideline, we still have researchers using Fukuda and referring to them as ME/CFS. I don't know if they consider potential translation to clinical care as an incentive or not - I'd think it depends on how basic their research is.

 

[Jonathan Edwards]

My point was purely about the criteria, not the quality of the methodology or findings . Even if the study is superbly designed and executed on every other count, if the criteria used are so non-specific as to require nothing more than medically unexplained fatigue, then to whom/what conditions do the conclusions apply.

The wider criteria apply to those who fit the wider criteria. That's the whole point. If that means dilution with a significant number without PEM then illness without PEM is just as important as with. If it doesn't then there is nothing to argue about.

But I'd question whether that's the same condition or group of conditions described by the CCC, ME-ICC, IOM, and NICE 2021?

All those criteria will identify different groups. That may be useful (although like you I doubt it. And I think ICC is make-believe.) But I used to study ACR Probable, Definite and Classical Rheumatoid Arthritis, Seropositive Rheumatoid Arthritis, Chronic Synovitis, Inflammatory Arthritis, and all sorts of other cohorts according to the experiment I was doing. There was never a justification for just one pigeonhole.

And if not, should those Fukuda studies keep using the label "ME/CFS" for their cohort?

Do studies using Fukuda bill it as 'ME/CFS'? In the UK I cannot think of an example, although there may be. If researchers use ME/CFS it usually means they have bought in to the idea that CFS is now considered synonymous with the PEM-specific syndrome of ME.

 

[mango]

The wider criteria apply to those who fit the wider criteria. That's the whole point. If that means dilution with a significant number without PEM then illness without PEM is just as important as with. If it doesn't then there is nothing to argue about.

Could you please help me understand how one could ensure correct reporting and recognition of PEM related harms in such a scenario?

If the dilution means there are very few participants with PEM, isn't there a considerable risk that the hypothetical/very real harm suffered by those few participants is "disappeared" in the reporting of the results and the conclusion, simply ignored because it isn't "statistically significant"? Unless you do a thorough subgroup analysis based on PEM/not PEM..?

 

[livinglighter]

The NHS website changes for ME/CFS didn't even reflect what was in the NICE 2021 Guideline. Is the message going to get through that CFS is gone?

I’ve noticed the NHS is not relenting and the CKS is written in a way that casts pure confusion. The different diagnostic criteria have been discussed a few posts back and that is what I was told by the Autonomics lead is the first problem. People with CFS/ME have ended up being different subsets including those without neurological disorders. In clinical practice, it looks like the different diagnostic criteria have created a problem that neurologists don’t want to be involved with.

In reality ME/CFS patient care has to be provided for all subtypes until we are separated by diagnostic criteria. We are on a forum where typically everyone has the same experience. But it has been said most patients at fatigue clinics don’t really have ME. They have ME based on Oxford or Faduka symptoms.

 

[Jonathan Edwards]

Could you please help me understand how one could ensure correct reporting and recognition of PEM related harms in such a scenario?

Reporting of harms is fairly easy - even just one case of deterioration should be reported because evidence of harm does not require statistical significance. It is too important to require such certainty. 'Correct reporting of harms' is tricky though because it can be very difficult to prove causation - harm rather than deterioration.

But dilution by 50% or even 75% is not going to make a lot of difference to picking up adverse events. Typically adverse events are taken seriously if in 1% of cases or less.

Again, I think the key problem is sloppy methodology.

I am not encouraging people to study broader fatigue groups but there may be valid reasons to do so.

 

[Trish]

I think this is an example of the dangers of using the Fukuda criteria in clinical trials.
https://www.s4me.info/threads/diffe...igue-symptoms-in-adolescents-2022-zhao.24756/

As a result of selecting participants using Fukuda for a daily half hour run based exercise program for teenage boys with fatigue and things like sleep and concentration problems and depression, and calling it CFS, they manage to show that this exercise program works for boys with what they call CFS.

We can be sure this will be held up by the likes of Crawley as proof supporting GET, yet the trial specifically excluded anyone for whom:
"fatigue results in the inability to exercise as required".

 

[Jonathan Edwards]

As a result of selecting participants using Fukuda for a daily half hour run based exercise program for teenage boys with fatigue and things like sleep and concentration problems and depression, and calling it CFS, they manage to show that this exercise program works for boys with what they call CFS.

No they don't. The study shows nothing. Nobody involved in serious academic medicine or guideline production should regard this paper as anything other than meaningless.

If it is really true that the majority of people who fit Fukuda do not have PEM then if this study had been done well and found an improvement without adverse events then I don't really see how one can complain.

I am pretty sure PACE recruited quite a lot of people with ME and yet the results were painted as positive for GET. If studies are done as badly as they are then criteria really don't make much odds I suspect.

 

[Caroline Struthers]

CFS does not exist any more in the UK and I think that may not have sunk in. Yes there is ME/CFS and yes there are garbage categories like MUS or even chronic fatigue but CFS has gone

It hasn't gone though. I have asked NICE to change all references in their tiredness guidance "https://cks.nice.org.uk/topics/tiredness-fatigue-in-adults/diagnosis/diagnosis-of-cfs/" from "chronic fatigue syndrome" or CFS to "ME/CFS" and they have refused (so far)
This is their "reasoning" https://docs.google.com/document/d/1VklGgePjOcbBc6-kprBmrZCKJpjfbpJIZb0yDUyKIE4/edit?usp=sharing
I have written back to them twice but they have ignored me

 

[Hoopoe]

The reason PACE failed to show an improvement with exercise therapy is probably precisely because they recruited a large enough number of ME/CFS patients. If I'm not mistaken, a lot of their usual patients that they label as having CFS comes from psychiatry clinics. Or at least it used to be the case for a while. This helps explain why they believe they could treat ME/CFS.

Prof. Coyne also said that he thought he could treat ME/CFS until he started getting referrals from rheumatologists and the patients were different and got worse with more activity instead of getting better like his previous patients.

In that Chinese study they excluded people with exercise intolerance so they might have had not even a single person with ME/CFS.

 

[Medfeb]

The wider criteria apply to those who fit the wider criteria. That's the whole point. If that means dilution with a significant number without PEM then illness without PEM is just as important as with. If it doesn't then there is nothing to argue about.

Do studies using Fukuda bill it as 'ME/CFS'? In the UK I cannot think of an example, although there may be. If researchers use ME/CFS it usually means they have bought in to the idea that CFS is now considered synonymous with the PEM-specific syndrome of ME.

This is the crux of the issue. Studying those who meet Fukuda but do not have PEM may be important. But today, researchers are still using Fukuda and then labeling the cohort as an ME/CFS cohort when they publish. At what point should those use a different label for their cohort?

(Edited to add - although I question the importance of Fukuda/no PEM studies given that it would be hard to apply consistently as in the next item)

If it is really true that the majority of people who fit Fukuda do not have PEM then if this study had been done well and found an improvement without adverse events then I don't really see how one can complain.

It's not that the majority of people who fit Fukuda don't have PEM. Its that because Fukuda is polythetic, it varies widely from one study to the next. For instance, a pre-2015 review of 53 Fukuda studies reported that anywhere from 25-100% of patients had PEM. And that's just PEM. Unrefreshing sleep, also required for IOM and CCC, had a similar range.

[edited to add - another study noted that patients with depression and not ME could meet one of the combination of Fukuda' medically unexplained fatigue plus any 4 of 8 symptoms]

 

[Hoopoe]

The reason PACE failed to show an improvement with exercise therapy is probably precisely because they recruited a large enough number of ME/CFS patients. If I'm not mistaken, a lot of their usual patients that they label as having CFS comes from psychiatry clinics. Or at least it used to be the case for a while. This helps explain why they believe they could treat ME/CFS.

The released PACE dataset showed that dropout were much more likely in a group of entries at the beginning. The intepretation of this depends on the ordering of the entries. If they are ordered by date of entry into the trial, it could mean that therapy was silently modified mid-trial to reduce the otherwise unacceptably high dropouts rates. If there are ordered by treatment site it could mean that the patients or the treatment at this site were different from those at the other sites. Knowing why this anomaly in the data occurred could tell us much about adverse reactions to CBT and GET.

The PACE authors didn't comment on this and it has never been properly examined by skeptics either.

My memory is a bit hazy and I might have gotten something wrong, it might have been missing data for some outcome rather than dropouts. In any case I think it could have told us something important about th reasons patients cannot tolerate CBT/GET.

 

[FMMM1]

I only read a few of the comments above. I think I recall that one of the forum members, with extensive clinical experience, pointed out that they:

• monitored the outcome, of interventions, at a group level; and
• monitored the effect for each individual.

Therefore, if there was a sub-group who responded +ve to the intervention, you'd expect to see some individuals improving.
Of course, as per comments above(?), if the outcome is measured subjectively (questionnaires) and the study is unblinded, then the results are presumably useless.
So yes, you can cope with heterogeneity i.e. if you have a well designed trial - objective outcome criteria etc.

 

[FMMM1]

a lot of their usual patients that they label as having CFS comes from psychiatry clinics. Or at least it used to be the case for a while. This helps explain why they believe they could treat ME/CFS.

Prof. Coyne also said that he thought he could treat ME/CFS until he started getting referrals from rheumatologists and the patients were different and got worse with more activity instead of getting better like his previous patients.

With respect strategist some of the folks [I'm not thinking of Coyne here] who are involved in ME/CFS "believe"/claim they can treat much ---- CBT seems to be a way to treat any difficult to treat disease. Others, who worked with them and know how the system works, actually suggest that these "treatments" are simply convenient fob offs - can't treat em -- send them for oh CBT.
I'd like to see evidence that any of these treatments actually worked ---- increase in functionality, returning to education, work ----. OK they can produce some questionnaires probably filtered to remove those problem individuals who point out it didn't actually work.

I was a bit shocked to see that Crawley has licensed this non-treatment:
"Outside the submitted work, RMM reports personal fees from training in CBT for irritable bowel syndrome for Central and North West London NHS Foundation Trust, the University of East Anglia, and University of Southampton. She receives payment for consultancy to Mahana Therapeutics and is a beneficiary of a licence agreement with King's College London and Mahana therapeutics to bring Regul8 (to the NHS and other international markets)."
https://www.s4me.info/threads/funding-of-me-cfs-research-in-the-uk.2533/#post-407017
@CRG

We've been had by these charlatans [I'm not thinking of Coyne here] and those who are taxpayers are funding these non-treatments, via the NHS.

 

[CRG]

Poorly informed or biased researchers will, excepting serendipity, produce poor research. There isn't, and never could be a criteria set - whether diagnostic or research focused, that can stop ill informed, prejudiced or mendacious researchers from producing inadequate, misleading or dishonest research.

In addition to all the points Jonathon Edwards has made I'd argue it's especially important when considering ME/CFS research to set aside what among PwME has often been an overly optimistic view of what 'the right criteria' would achieve. Over the decades there's been understandable conflation of the need for the best available diagnostic criteria, with the notion of best research criteria, but these are not necessarily the same thing - though of course they could be depending on what the research was.

With NICE 2021 we have a reasonably sound diagnostic criteria for England - hopefully to be emulated in the devolved countries, while in the US the IOM2015 seems an adequate diagnostic set. As yet we have no idea what the capture rate of NICE2021 will be in practice, or what levels of inclusivity or exclusivity it will have on those previously included/excluded in/out from the PwME population; there also seems to be no research on the impact of IOM2015 and therefore exclusive reliance on either of these as research criteria may introduce unmeasured shifts in the patient population that is being investigated. We can't assume that greater specificity of symptomology produces a research cohort that has greater pathophysiological commonality - NICE2021 or IOM2015 may have the same variability as Fukuda1994 or each may have similar levels of variability but expressed via greatly differing characteristics, or variability maybe so chaotic in its spread through the patient population that for research purposes none is more helpful than another.

In terms of advocacy where PwME are able to influence research, or otherwise can only examine after publication, then at this time on criteria I think our expectation should be that, depending on the resources available to the researchers, selection for research cohorts should:

1. where trial participants are included on the basis of their historic diagnosis, and where the study population is large enough to provide statistical power based on sub cohorts defined by the criteria, then at least three out of: Fukuda, ICC, CCC, NICE2021 and IOM2015 should be applied.

or

2. where researchers are themselves assessing participants against criteria and where the study population is large enough to provide statistical power based on sub cohorts defined by the criteria, then at least Fukuda, NICE2021 and IOM2015 should be applied.

or

3.for smaller studies where it may be difficult to ensure statistical power with more than a single criteria, that single choice of criteria should always be clearly justified by the researchers and prior criticisms of that criteria made explicit - something notably absent in Zhao et al https://www.s4me.info/threads/diffe...igue-symptoms-in-adolescents-2022-zhao.24756/