Discriminating Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and comorbid conditions using metabolomics in UK Biobank, 2024, Huang et al

[Hutan]

The production of the model was using 7 disease populations.

That means that the model is definitely not useful for diagnosing ME/CFS then. All of those highly selected homogeneous disease populations exclude many people with diseases associated with dislipidemia; for example people who are obese. The ME/CFS group, as a heterogenous cohort, does not.

Of course, if you have data where the ME/CFS group is the only group allowed to have obese people, you will find that levels of blood lipids are different in the ME/CFS group compared to the healthy group.

From Table 2, the number of people on cholesterol-lowering medication in the ME/CFS group is 15.7%. This is probably normal for people of this age in the UK who are educated and concerned about health enough to be part of the UK Biobank. Indeed a study on the UK Biobank data reported statin use at baseline of 15.4% for its whole population.
(from Association of statin use with risk of depression and anxiety: A prospective large cohort study)

Here are the percentages of people on cholesterol-lowering medication in the other groups of people used as comparator groups in this paper (also Table 2)

Hypertension 9.7%
Depression 0.7%
Asthma 0.9%
IBS 0.5%
Hypothyroidism 0.6%
Migraine 1.7%
No health conditions (C2) 0.8%

Can you see how highly selected the comparison groups are? By not allowing people in the comparator groups to have any health condition other than the one the group is labelled with, you are badly skewing the comparison.

Taking hypothyroidism for example, there's a paper 'Prevalence of Hyperlipidaemia in Adult Patients with Hypothyroidism: A Systematic Review'. It lists out the findings of a whole lot of papers in hypothyroidism.
One study reports the prevalence of Hypercholesterolemia: 48.4%. Hypertriglyceridemia: 32.3%
Another notes that low HDL-C is present in 69.2% of people with hypothyroidism.
I haven't checked the details of those studies, but there is overwhelming evidence that people who are hypothyroid are very likely to have issues with weight control and issues with blood lipids. The selected people with Hypothyroidism control group are not at all normal for people with hypothyroidism.

It's like, I don't know.... having two jars of M&Ms filled straight from the packet, calling one ME/CFS and the other Hypothyroidism. And then deliberately taking nearly all of the yellow M&Ms out of the Hypothyroidism jar. And then saying that 'we can diagnose if a jar of M&Ms has ME/CFS with a reasonable degree of accuracy by looking at how many yellow M&Ms are in it'. This paper is like that.

 

[Jonathan Edwards]

Even 5 years ago our discussion with the business groups at the University had pointed to problems they'd had in translating AI tools to GPs, that largely fallen away on the past 5 years.

I am sceptical of that. GPs were taught to diagnose primary hyperparathyroidism by asking for a 'bone profile' of calcium, phosphate and alkaline phosphatase fifty years ago without most of them remembering why those tests were useful.

Maybe wariness of black box sets of results where none of the individual results is outside the normal range reflected a healthy caution about diagnostic tests that might not be all they seem and maybe everyone has got so obsessed with tech now they have lost that caution?

Comorbids we used were the common ones that mecfs patients in ukbiobank actually had and were enriched against a general population background in ME. We added hypertension to account for the lipid and lipoprotein elevations. Interesting thing is that mecfs group (25% hypertension) had lipoprotein profiles that were close to equal to a group where 100% of people had hypertension.

OK, so you are using comorbidity to mean something found in statistical association with ME/CFS that may confound attempts to identify markers that sort with ME/CFS because of ME/CFS biology specifically. If you can do that you have some biological explanatory data. That is what Beentjes and co hope they had done but I fear may have been due to confounders.

But I thin this is actually a different problem from discriminating diagnoses in an individual. It may translate to that if the biological link is robust but again, the question is whether anyone has ever found a set of markers with a pattern of values within normal ranges that is robust enough that does not make sense biologically. I don't know of a case. It seems you don't either, since none has been mentioned so far?

If we are allowing AI why not tell patients just to log on to Google and ask it 'Have I got ME/CFS?' No doubt Goggle has the sense to go through a history including questions that test for fitting CCC. You cannot do better than that if that is how ME/CFS is defined.

 

[Trish]

I am trying to catch up with this discussion. I have not been able to read the whole paper yet, but from what I have read, it seems to me that Hutan is raising relevant points. This post is a work in progress.

I have just reached this bit in the paper and it seems relevant

Addressing comorbidities within ME/CFS​

To thoroughly investigate the impact of comorbid conditions in ME/CFS requires stratifying the cohort into groups of isolated condition combinations, which can substantially reduce the sample size and the statistical power. For example, there were 211 ME/CFS individuals with a combination of depression and other comorbid conditions, and 24 individuals with depression only. We recognise that the other 265 comorbid conditions not analysed in this study may influence the biomarker associations. Therefore, we created another cohort with 354 ME/CFS individuals with or without hypertension, depression, asthma, IBS, hay fever, hypothyroidism, or migraine and performed association tests (Supplementary Fig. 7) and sensitivity analysis for this subset (Supplementary Data 9). Thirty-one of the initial 168 ME/CFS biomarker associations remained significant (P < 2.01×10−4). SFA% and omega-3 were the only significant associations that produced greater odds ratio in the subset than the full cohort. The lower odds ratios observed may be attributed to the reduced number of comorbid conditions reported by each individual, rather than the specific condition. The average number of comorbid conditions was 3.0 for the full cohort and 0.6 for the subset. This suggests that the burden of having several comorbid conditions might exacerbate ME/CFS symptoms (inclusive of symptoms from common comorbid conditions), reflecting a higher disease severity, leading to more pronounced biomarker signals in the full cohort.

If direct comparisons are to be made between pwME and healthy people, then the ME/CFS group needs to be a group with no co-morbidities. That is, the only diagnosed difference between the groups is ME/CFS. Once co-morbidities are added to the ME/CFS group, and not to the comparitor group, there is no way of knowing the source of the differences in tests.

The research attempts to get around this by comparing groups with single conditions against the healthy group. If each of these data sets is to be useful for identifying ME/CFS specific biomarkers, the direct comparison needs to be between people with, say, migraine alone, and people with ME/CFS and migraine and no other diagnosed condition.

Or for combined conditions, you need to compare, say, people with migraine and IBS and nothing else with people with ME/CFS, migraine and IBS and nothing else.

It seems from the section I quoted above, once you start subsetting like this, the significant markers fall away rapidly.

I'll try to read more of the paper later, and comment further. This is a work in progresss but I wanted to make my notes here now so I don't lose track.

 

[Jonathan Edwards]

Another thing that worries me is that there seems to be an assumption that there are definable number of 'diseases' each with a causal pathway that may include certain measurable chemical variables. I think it more likely that what we call diseases are the tips of icebergs with vast stretches of differences in variables invisible - sometimes separate, sometimes part of the same iceberg underneath with several bits visible. Maybe even some bergs being weighed down by other overlapping bergs. Which means that there is probably no clearly definable causal pathway for most of the variables that show differences to fit in to.

The assumption of a causal path is what worried me about the Beentjes study. It didn't look realistic to me.

There seem to be too many unknown unknowns here.

 

[Utsikt]

The problems were around AI tools could pick up patterns without explaining the why. Clinicians were weary of using tools if they themselves didn't understand the biological. I believe clinicians are taught this way. But as trust in AI is growing there is beginning to be trust in not needing to know the why, just that AI analysis has seen a complex pattern and the outcome is a result that is beneficial.

The trust in AI is declining, even thought the use of it is increasing. The distrust is usually higher among the people with the most knowledge and expertise.

The explainability issue isn’t just a case of trust either - it’s about ethics and law as well. If we can’t know why a decision was made, we can’t ensure it’s medically sound. That makes it impossible to ensure the patient’s rights are met. We need transparency and traceability.