Participation bias

Simon M · Mar 15, 2021

Split from The course of the illness for ME patients in Norway, 2021, Schei and Angelsen

The potential effect of sample biases on DecodeME
Summary: there is some evidence that this could be a problem, but is not clear how big a problem and it may be possible to adjust for possible biases.

IMPORTANT these are very much my own views and this is not an official DecodeME response.

Jonathan Edwards said: ↑

Let us say that there is a gene allele A that for some irrelevant hum-drum reason that may be very indirect means that people with this allele are more likely to volunteer for GWAS projects. It might be that they have an unusual blood group and are sought out by blood banks so are used to contact with healthcare systems. Even if when they get ME they cannot give blood they might be readier to volunteer for research. Let us say that 1% of the population have this allele. It might be an allele that goes with choosing science subjects at school. It could be all sorts of things.
Click to expand...

this is an important point and there is some evidence to back it up.

This new study in Nature Medicine finds genetically biased participation rates both for the UK biobank and the Avon mother and child longitudinal cohort. Genetic differences linked to higher intelligence and more time in education led to higher levels of up participation. High levels of body fat, and risk of Alzheimer's and schizophrenia, for instance, reduced participation.

The affects are generally quite modest, but could throw up spurious answers and the paper recommends trying to correct for these factors.

What isn't clear is how relevant this research is to DecodeME. In the case of both the biobank and the Avon cohort, people had already been recruited into large studies and the new study was ]measuring the effect of bias on people who either took part in additional activity, such as a mental health questionnaire or completed follow-up activity.

For DecodeME, we need to understand the risk of bias when recruiting from the public, primarily through social media and PR. And I'm not aware of any research that looks at this.

The keen volunteer issue.

Normally, you would expect a low response rate to adverts or PR stoires, and that could lead to a "keen volunteer" issue:

Jonathan Edwards said: ↑

because as more people volunteer the bias from 'keen volunteers' will drop off.
Click to expand...

but there is some evidence that we are going way beyond keen volunteers. Not least because we were offering people who have no treatments and who are often disbelieved about their illness, the chance to take part in a study that could find answers.

We have around 25,000 people already signing up to indicate their interest in the study, which I think goes way beyond any previous participation rate for an ME project in the UK. And response to a single new story with a link back to the decodeme website was extraordinary, again suggesting huge interest. More details in the quote box below.

In the case of both the UK biobank and the Avon mother and child study, people were primarily volunteering to help research in a general sense, and most people wete actually healthy. So the motivation is to help the greater good, rather than for personal benefit.

But our target audience are severely ill with a poorly understood illness, one with no good treatments and one where, due to the lack of understanding you suffered from prejudice from the health system and often society at large. We are offering people the chance to take part in the world's largest ME/CFS study that aims to help identify biological causes of the illness, which could lead in time to treatments as well as tackling stigma. Potentially, this provides a very strong incentive to participate, one that might go a long way to overcome the "keen volunteer" problem, but does it?

There is some evidence from our recruitment results to date. First, are the very high engagement levels. I think we are up to nearly 25,000 people in the UK who have a clinical diagnosis and have already signed up to signal their interest in the study, mostly recruited from social media posts through the ME online community. I don't think any other project has had such high level of participation from the community.

But what really struck me are the results from a single online article in a newspaper, a fairly basic piece based on our press release of the funding announcement. Crucially, this article included a link to the decodeme website. I would have thought a few hundred people signing up as a result of this article would have been a good result. We got over 5,000.
Click to expand...

What proportion of patients might DecodeME recruit?
As @Jonathan Edwards points out, a lot depends on what proportion of all UK patients join the study, and he starts with an assumption of 120k patients.

My guesstimate (not an offical DecodeME one): 67k patients in the UK with a diagnosis and meeting our criteria. Allowing for inevitable losses due to some people, not consenting, not complete a questionnaire or not returning the spit kit, we would need half of these patients to start the process in order to end up with 20,000 participants.

Warning! There are a lot of assumptions in this. IDetails in the quote box.

We are only looking to recruit people in the UK who have been given a clinical diagnosis. Let's sing their 100,000 of those. Let's assume that two thirds of these meet our criteria (clinical diagnosis reported plus meeting either IOM or CC C criteria). That means the total available audience is 67,000.

However, not everyone will consent, not everyone will complete the signup questionnaire and not everyone will return the spit kit (that certainly the experience of every other GWAS). If we were to assume an optimistic 90% consent rates, 85% questionnaire completion and 80% spit kit return rate, we would still lose nearly 40% of the people who started the signup process. (Actually, we will also screen out people don't meet the criteria, but I factored that into the 67,000 figure above).

Which works out to needing about 33,000 people with a diagnosis meeting and our criteria in order to end up 20,000 participants. We don't know to what ex-what role biases might play in those losses.

So we might have a total available audience of 67,000 and needs to engage 33,000 of those, around half. That suggests we are going deep into the patient population, particularly as even if the PR goes well, we can't expect everyone with ME to hear about the study.
Click to expand...

Hutan · Mar 15, 2021

Simon M said: ↑

but there is some evidence that we are going way beyond keen volunteers. Not least because we were offering people who have no treatments and who are often disbelieved about their illness, the chance to take part in a study that could find answers.
Click to expand...

I would have thought the relatively low effort involved in spitting in a tube and sending it off would go a long way to work against participation bias. It's interesting to see the problem being identified with the UK biobank, which presumably also has a low effort requirement. It's interesting to think about what groups of people with ME/CFS might be excluded/exclude themselves from the DecodeME study, and how that might be prevented.

If it is a problem with a 'spit and send' sample, then how much more of a problem must participation bias be in ME/CFS studies where people have to risk PEM (especially in studies with an exercise component)?

There must surely be participation biases also where many of the target recruitment population might surmise (from the study description or the names of the researchers involved) that a study is aiming to show that faulty personality traits, behaviours or past traumas cause ME/CFS, and therefore either actively avoid or support it.

Invisible Woman · Mar 16, 2021

N=1 perspective but I rarely volunteer.

Partly due to the effort involved

because I am very cautious about anyone who might want to write to my GP and thus add something to my medical records - note the GWAS allows you to deny access to medical records.

Partly deep suspicion and scepticism born of people who seem to be acting in good faith but the reality is quite different. I remember all pro PACE arguments.

I can see that participation bias could be a problem but we are also a fairly unique patient community in that much of the research done in the past few decades has done us harm. That may make much of the longer term ill less likely to participate without careful consideration.

There might also be an element of "voting with our feet" & demonstrating support for what is seen as honest, good quality research while avoiding poor quality studies or studies run by those with a history of outcome switching etc. For example the blocking of the proposed genetic study by Davies & Crawley but the endorsement of DecodeME.

As usual, when it comes to ME, nothing is straightforward!

Log in or Sign up

Participation bias

Simon M Senior Member (Voting Rights)

Hutan Moderator Staff Member

Invisible Woman Senior Member (Voting Rights)

Share This Page

Log in or Sign up

Participation bias

Simon M Senior Member (Voting Rights)

Hutan Moderator Staff Member

Invisible Woman Senior Member (Voting Rights)

Share This Page

Useful Searches