Participation bias

[Simon M]

Split from The course of the illness for ME patients in Norway, 2021, Schei and Angelsen


The potential effect of sample biases on DecodeME
Summary: there is some evidence that this could be a problem, but is not clear how big a problem and it may be possible to adjust for possible biases.

IMPORTANT these are very much my own views and this is not an official DecodeME response.

Let us say that there is a gene allele A that for some irrelevant hum-drum reason that may be very indirect means that people with this allele are more likely to volunteer for GWAS projects. It might be that they have an unusual blood group and are sought out by blood banks so are used to contact with healthcare systems. Even if when they get ME they cannot give blood they might be readier to volunteer for research. Let us say that 1% of the population have this allele. It might be an allele that goes with choosing science subjects at school. It could be all sorts of things.

this is an important point and there is some evidence to back it up.

This new study in Nature Medicine finds genetically biased participation rates both for the UK biobank and the Avon mother and child longitudinal cohort. Genetic differences linked to higher intelligence and more time in education led to higher levels of up participation. High levels of body fat, and risk of Alzheimer's and schizophrenia, for instance, reduced participation.

The affects are generally quite modest, but could throw up spurious answers and the paper recommends trying to correct for these factors.

What isn't clear is how relevant this research is to DecodeME. In the case of both the biobank and the Avon cohort, people had already been recruited into large studies and the new study was ]measuring the effect of bias on people who either took part in additional activity, such as a mental health questionnaire or completed follow-up activity.

For DecodeME, we need to understand the risk of bias when recruiting from the public, primarily through social media and PR. And I'm not aware of any research that looks at this.

The keen volunteer issue.

Normally, you would expect a low response rate to adverts or PR stoires, and that could lead to a "keen volunteer" issue:

because as more people volunteer the bias from 'keen volunteers' will drop off.

but there is some evidence that we are going way beyond keen volunteers. Not least because we were offering people who have no treatments and who are often disbelieved about their illness, the chance to take part in a study that could find answers.

We have around 25,000 people already signing up to indicate their interest in the study, which I think goes way beyond any previous participation rate for an ME project in the UK. And response to a single new story with a link back to the decodeme website was extraordinary, again suggesting huge interest. More details in the quote box below.

In the case of both the UK biobank and the Avon mother and child study, people were primarily volunteering to help research in a general sense, and most people wete actually healthy. So the motivation is to help the greater good, rather than for personal benefit.

But our target audience are severely ill with a poorly understood illness, one with no good treatments and one where, due to the lack of understanding you suffered from prejudice from the health system and often society at large. We are offering people the chance to take part in the world's largest ME/CFS study that aims to help identify biological causes of the illness, which could lead in time to treatments as well as tackling stigma. Potentially, this provides a very strong incentive to participate, one that might go a long way to overcome the "keen volunteer" problem, but does it?

There is some evidence from our recruitment results to date. First, are the very high engagement levels. I think we are up to nearly 25,000 people in the UK who have a clinical diagnosis and have already signed up to signal their interest in the study, mostly recruited from social media posts through the ME online community. I don't think any other project has had such high level of participation from the community.

But what really struck me are the results from a single online article in a newspaper, a fairly basic piece based on our press release of the funding announcement. Crucially, this article included a link to the decodeme website. I would have thought a few hundred people signing up as a result of this article would have been a good result. We got over 5,000.

What proportion of patients might DecodeME recruit?
As @Jonathan Edwards points out, a lot depends on what proportion of all UK patients join the study, and he starts with an assumption of 120k patients.

My guesstimate (not an offical DecodeME one): 67k patients in the UK with a diagnosis and meeting our criteria. Allowing for inevitable losses due to some people, not consenting, not complete a questionnaire or not returning the spit kit, we would need half of these patients to start the process in order to end up with 20,000 participants.

Warning! There are a lot of assumptions in this. IDetails in the quote box.

We are only looking to recruit people in the UK who have been given a clinical diagnosis. Let's sing their 100,000 of those. Let's assume that two thirds of these meet our criteria (clinical diagnosis reported plus meeting either IOM or CC C criteria). That means the total available audience is 67,000.

However, not everyone will consent, not everyone will complete the signup questionnaire and not everyone will return the spit kit (that certainly the experience of every other GWAS). If we were to assume an optimistic 90% consent rates, 85% questionnaire completion and 80% spit kit return rate, we would still lose nearly 40% of the people who started the signup process. (Actually, we will also screen out people don't meet the criteria, but I factored that into the 67,000 figure above).

Which works out to needing about 33,000 people with a diagnosis meeting and our criteria in order to end up 20,000 participants. We don't know to what ex-what role biases might play in those losses.

So we might have a total available audience of 67,000 and needs to engage 33,000 of those, around half. That suggests we are going deep into the patient population, particularly as even if the PR goes well, we can't expect everyone with ME to hear about the study.

 

[Hutan]

but there is some evidence that we are going way beyond keen volunteers. Not least because we were offering people who have no treatments and who are often disbelieved about their illness, the chance to take part in a study that could find answers.

I would have thought the relatively low effort involved in spitting in a tube and sending it off would go a long way to work against participation bias. It's interesting to see the problem being identified with the UK biobank, which presumably also has a low effort requirement. It's interesting to think about what groups of people with ME/CFS might be excluded/exclude themselves from the DecodeME study, and how that might be prevented.

If it is a problem with a 'spit and send' sample, then how much more of a problem must participation bias be in ME/CFS studies where people have to risk PEM (especially in studies with an exercise component)?

There must surely be participation biases also where many of the target recruitment population might surmise (from the study description or the names of the researchers involved) that a study is aiming to show that faulty personality traits, behaviours or past traumas cause ME/CFS, and therefore either actively avoid or support it.

 

[Invisible Woman]

N=1 perspective but I rarely volunteer.

• Partly due to the effort involved
• because I am very cautious about anyone who might want to write to my GP and thus add something to my medical records - note the GWAS allows you to deny access to medical records.
• Partly deep suspicion and scepticism born of people who seem to be acting in good faith but the reality is quite different. I remember all pro PACE arguments.

I can see that participation bias could be a problem but we are also a fairly unique patient community in that much of the research done in the past few decades has done us harm. That may make much of the longer term ill less likely to participate without careful consideration.

There might also be an element of "voting with our feet" & demonstrating support for what is seen as honest, good quality research while avoiding poor quality studies or studies run by those with a history of outcome switching etc. For example the blocking of the proposed genetic study by Davies & Crawley but the endorsement of DecodeME.

As usual, when it comes to ME, nothing is straightforward!

 

[tralfamadorian97]

The discussion here made me think about participation bias. It seems to me there is a small but non-negligible risk that participation bias could have partially affected DecodeME. I would guess that participation bias would be more likely to affect the broad polygenic signal used in LD-score regression, rather than sharp single-locus signals used in coloc/fine-mapping studies. Interested if anyone has thoughts on this. I plan to read more on GWAS participation bias, including whether there are reliable statistical techniques to detect and mitigate it.

 

[Hutan]

I was just looking at the discussion about LRRIQ3, which is linked to IQ, as well as being somewhat close to some genetic variation in DecodeME.

Possibly the IQ in LRRIQ3 is for intelligence? Because the strongest specific association of LRRIQ3 reported is for mathematical ability. 'Participation in health studies' is in the top 5 associations. An association with neurodegenerative diseases is also in that top 5; an association in pain has been found.

If "IQ" is a reflection of some forms of intelligence + social advantage (supportive family, family wealth), then the steps involved in taking part in something like DecodeME probably bias to people with a higher than average IQ. It takes resources and support to get a diagnosis, and a valuing of science and education, and a belief that 'things can be made better'. And those things make it more likely that the person with ME/CFS will seek out and participate in research.

So, I think it's quite likely that people with ME/CFS who get a diagnosis, hear about research and take part in research are more likely to have the traits that result in a higher than average measured IQ than the average person, even the average person who takes part in research. I think the selection bias is probably higher than for some more easily diagnosed health conditions. And that might skew the genetic variation that seems to be characteristic of ME/CFS.

(Note, I don't know if the variation identified in DecodeME means anything for LRRIQ3, or, even if it does, if the variation suggests higher or lower mathematical ability. )

 

[BrightCandle]

sending it off

This is actually high effort for an ME patient. Either they have a post box relatively near to them and they can walk to it (excluding many moderate and worse patients) or they have to organise a pickup with the post office and answer the door and hand it over (removing severe patients and some moderates). This requirement is lower than most studies that require trips to a clinic that is many miles away and isn't even offering transport options and isn't accommodating all the patients special needs, but its also something that for severe patients requires help.

I have avoided doing things that require me to send things in the post as I find setting them up quite taxing and the timing of pickup being all over the day very problematic with a shifted and disrupted sleep pattern. I would have likely made that effort for DecodeME but generally I don't. We shouldn't underestimate that this requirement of sending something off is beyond plenty of patients and biases studies a bit.

There is also going to be a similar effect as to whether a patient even hears about the study, which is more likely the more mild you are. Those people in dark rooms with no screen use or minimal use are not going to be finding such studies through the ME grapevine, they simply aren't connected. I think is potentially also true of people who have had the disease longer, they are just less likely to be seeking news on the condition and searching to participate in studies, why bother when you have had it 30+ years and no one has turned up to help you yet.

In some areas the NHS is more abusive than others and in those places there is going to be more distrust of researchers than in others. These areas will also have less diagnosis and people with less support.

This is all beyond just genetics and intelligence aspects but the social aspects probably dominate in introducing bias.

I still think the most biasing thing is requiring a GP diagnosis, its likely done with the wrong criteria anyway and DecodeME already showed it excluded more men and BAME and biased the study significantly towards white women, that requirement is something that needs to be avoided especially since its insufficient and further assessment of CCC/IOM is required as the NHS is incapable of this diagnosis.

 

[Utsikt]

If "IQ" is a reflection of some forms of intelligence + social advantage (supportive family, family wealth), then the steps involved in taking part in something like DecodeME probably bias to people with a higher than average IQ. It takes resources and support to get a diagnosis, and a valuing of science and education, and a belief that 'things can be made better'. And those things make it more likely that the person with ME/CFS will seek out and participate in research.

From Norway:
High education is associated with an increased risk of receiving a diagnosis, and low income is associated with an increased risk of receiving a diagnosis.

To my mind, it’s likely that high education is associated with being able to navigate the healthcare system more efficiently, and that low income creates a need to get the diagnosis for benefit reasons.

 

[Hutan]

and that low income creates a need to get the diagnosis for benefit reasons.

Possibly low income (and low financial support from family) would increase the likelihood that any diagnosis that provides access to government benefits is accepted e.g. depression.

Lack of education, low self-confidence and a history of family trauma resulting from the stresses of poverty might also increase the likelihood that a person would be given and accept a diagnosis of FND or depression or some other mental health label instead of ME/CFS.

 

[EndME]

The discussion here made me think about participation bias. It seems to me there is a small but non-negligible risk that participation bias could have partially affected DecodeME. I would guess that participation bias would be more likely to affect the broad polygenic signal used in LD-score regression, rather than sharp single-locus signals used in coloc/fine-mapping studies. Interested if anyone has thoughts on this. I plan to read more on GWAS participation bias, including whether there are reliable statistical techniques to detect and mitigate it.

I think we discussed this quite a bit on the DecodeME thread after the study came out, where we for example also discussed a FND GWAS and also on some other threads (even a EDS GWAS one if I remember correctly). I also still have some questions and reservations. I suspect that with more replications and other genetic studies the picture will become clearer quite soon.

 

[tralfamadorian97]

I suspect that with more replications and other genetic studies the picture will become clearer quite soon

I agree that other genetic studies will be clarifying. To shed maximal light on participation bias, I suspect a new genetic study would need to a) use rigorous ME/CFS diagnostic criteria (like DecodeME) and b) use a very different patient recruitment strategy from DecodeME. Then, by "triangulating" between DecodeME and another study with a different recruitment strategy, we would distinguish participation bias from real signal.

 

[ME/CFS Science Blog]

I think the high genetic correlations with fibromyalgia, irritable bowel syndrome and self-reported CFS (rg > 0.7) compared to a much lower correlation for measures of intelligence and education (rg < 0.2) suggest that the majority of the signal reflects the diseases and not participation bias.I wrote more about this here:

Post in thread 'The genetic architecture of fibromyalgia across 2.5 million individuals, 2026, Kerrebijn et al'

Aug 11, 2026

Perhaps others can double-check but I calculated the genetic correlation with DecodeME and the European-only GWAS of fibromyalgia from this study using LDSC and I got a value of rg = 0.757. That's quite high and similar to the rg = 0.751 between DecodeME and self-reported irritable bowel syndrome in the UK Biobank.

So that confirms share features in the pathology of these conditions. I think it also confirms that the DecodeME results aren't due to some selection bias and reflect for example geographic or social differences. The fibromyalgia and IBS cases were selected very differently so...

• ME/CFS Science Blog

• 

Nonetheless, as Hutan and others explained it would make sense if there was some selection difference in the ME/CFS participants who ended up participating in decodeME compared to controls participating in the UK biobank. I think some of the signals that we find and analyze might be driven by this. So probably important not take any isolated signal too seriously.

I was a bit confused about the phewas results of this locus for example which I couldn't make sense of and included several UK biobank questions.

 

[tralfamadorian97]

I think the high genetic correlations with fibromyalgia, irritable bowel syndrome and self-reported CFS (rg > 0.7) compared to a much lower correlation for measures of intelligence and education (rg < 0.2) suggest that the majority of the signal reflects the diseases and not participation bias

Good point. The genetic correlations with fibromyalgia and IBS are indeed reassuring. I would be more reassured if DecodeME had strong genetic correlations with objectively measured molecular phenotypes. But given that efforts to find a ME/CFS biomarker have failed so far, it is perhaps unsurprising that DecodeME currently lacks strong genetic correlations with such objectively measured phenotypes.