Open Online Study into suitability of existing research questionnaires for use in ME/CFS research, Tufts Uni, VanElzakker, 2021

The following info about the study is given on the second page at the link provided above

What is this study about?

The current study will test different questionnaires (questionnaires are also sometimes called surveys or scales). Questionnaires are a set of standardized questions. We are testing the validity of some existing questionnaires, which means that we are testing whether they ask questions in an appropriate, accurate, and understandable way. We are also testing whether certain questionnaires ask for relevant and understandable information about ME and CFS (Myalgic Encephalomyelitis and Chronic Fatigue Syndrome).

Many different questionnaires and surveys have been used to ask ME and CFS patients about their symptoms. Some of the commonly used questionnaires may have confusing, unclear, or irrelevant wording. One goal of the current study is to show ME and CFS patients several existing and new questionnaires, and to gather information about the actual symptoms of patients with an ME or CFS diagnosis. Another goal of the current study is to understand which commonly used questionnaires ask questions in a clear way or a confusing way.
The data collected during this study includes but is not limited to: racial or ethnic origin, biometric data, physical or mental health, sex life, and sexual orientation.

You are being asked to participate because you have a diagnosis of ME or CFS. We expect that over 1,000 individuals may participate in this study. Your participation in the study is expected to last about an hour. This study is supported by Dr. Michael VanElzakker's laboratory sundry fund.

The current study is not part of your medical treatment. We will not share your answers with your doctor, and your answers will not be used for your medical care. We are asking about your medical history to learn more about what ME and CFS patients experience, and to see if different questionnaires are useful and easy to understand. The information collected in this study is being collected for research purposes and not for medical treatment. The results of the surveys and the research will not be shared with you because they are being done for purposes of the study, not for purposes of treating you.

In the event that you suffer, or believe that you have suffered, a research-related injury, your medical expenses will be your responsibility or that of your third-party payer, although you do not waive any rights that you would otherwise have by signing this consent.

What will happen during this research?

If you agree to be in this research, your participation will involve completing approximately 12 online questionnaires, which should take about an hour. You will have the opportunity to tell us if any questionnaire wording is confusing or unclear. This is expected in this study because we have taken direct wording from existing questionnaires. You do not have to answer any question that you do not want to answer.

Because there are so many questionnaires, you will only be asked to complete a selection of them. If you were to complete all of the questionnaires we are testing, it would take several hours. Therefore, instead you will be randomly assigned to complete a selection of the total questionnaires.

We expect that a large number of patients will participate in this study. Most participants will not be paid. As compensation for your participation, after completion you will be given the opportunity to enter into a raffle for the chance to win one $20 VISA gift card per every 100 participants. Your responses may be analyzed and used to improve the way that ME or CFS patients are asked about their symptoms during research studies.

We will tell you about any new information that may affect your willingness to continue participation in this research.
 
Last edited:
Michiel Tack said:
To clarify: the survey is full of frustrating or stupid questions but If I understand correctly, this is not because the current authors have designed them as such. They are studying these problematic questionnaires and want patients' reponse to it.

Click to expand...

Which seems to show that the survey could have been better? I do have sympathy though, knowing the lengths that everybody in DecodeME has gone to in order to make all of the DecodeME communications as clear as possible, it's a challenging task.
 
Andy said:
Which seems to show that the survey could have been better? I do have sympathy though, knowing the lengths that everybody in DecodeME has gone to in order to make all of the DecodeME communications as clear as possible, it's a challenging task.
Click to expand...

Yes, in my opinion, his survey itself is inappropriately designed. He is perpetuating the bias that reliability is based on quantitative measurements of questionnaires - but this means that a lot of information is simply lost. The survey also doesn't even bother to attempt to measure the fundamental question - why are symptom questionnaires being used in the first place? Is it the symptom or the impact of the symptom on quality of life that is important - the latter question is not examined at all, because patients are not being asked about the specific relevance of each question.



I understand he is trying to improve things, but I fear his approach is misdirected.
 
Snow Leopard said:
Yes, in my opinion, his survey itself is inappropriately designed. He is perpetuating the bias that reliability is based on quantitative measurements of questionnaires - but this means that a lot of information is simply lost. The survey also doesn't even bother to attempt to measure the fundamental question - why are symptom questionnaires being used in the first place? Is it the symptom or the impact of the symptom on quality of life that is important - the latter question is not examined at all, because patients are not being asked about the specific relevance of each question.



I understand he is trying to improve things, but I fear his approach is misdirected.
Click to expand...

I started the survey and gave up when I increasingly found myself answering every question with 'prefer not to answer' because there wasn't an option that said 'unanswerable'.

Like the tweeter in the post above I was looking for open text fields. I understand the difficulty of working with them but if they're wanting to assess the usefulness (uselessness) of the questionnaires you really have to be able to comment, especially if you can't answer 'unanswerable' or 'ambiguous' or 'irrelevant'.

Occasionally there were two questions asking you to assess the preceding questionnaire: how many of your symptoms were captured (0-100%) and how clear/confusing the questions were (0-10). That's neither here nor there.

Note that I didn't finish the survey so there may have been more meaningful questions later.

Also, a 1-hour survey to be completed within 24 hours is pushing things for the more severe. And it would be helpful to have an indication of how far through you are.

I appreciate what they're trying to do here but I don't think this is the best way of going about it.
 
Snow Leopard said:
Yes, in my opinion, his survey itself is inappropriately designed. He is perpetuating the bias that reliability is based on quantitative measurements of questionnaires - but this means that a lot of information is simply lost. The survey also doesn't even bother to attempt to measure the fundamental question - why are symptom questionnaires being used in the first place? Is it the symptom or the impact of the symptom on quality of life that is important - the latter question is not examined at all, because patients are not being asked about the specific relevance of each question.



I understand he is trying to improve things, but I fear his approach is misdirected.
Click to expand...


Yea the standard approach if a techniques is flawed (questionnaires) is to design it out.

OK there may be some things you can't easily measure but is a poor method (questionnaires) better than nothing?
 
FMMM1 said:
Yea the standard approach if a techniques is flawed (questionnaires) is to design it out.

OK there may be some things you can't easily measure but is a poor method (questionnaires) better than nothing?
Click to expand...

I have already given the review of existing questionnaires a lot of thought and to avoid a Tu quoque trap, I'd first design a survey to review existing questionnaires using qualitative interview methods - asking participants what they think, designing a survey on how to assess the questionnaires, getting more qualitative feedback on the survey methods themselves and only then recruiting participants to answer the survey.

Instead Michael seems to have skipped a step, with the result is his survey suffers from similar problems to the questionnaires he is trying to get feedback on.
 
I've done and submitted the survey. It says each person will be given a random selection of questionnaires, so my experience will be different from other people's.

For some of the surveys there was an opportunity to say whether the survey captured my experience and/or symptoms of ME/CFS, but with others it didn't. The questionnaires I got included the Chalder one and another which was pretty impossible to answer sensibly. Only one actually seemed to be a reasonable attempt to assess my functional level, but in a very limited way. Most of them were crap.

I have no idea how the results of this can be analysed. Perhaps the only useful way would be to see whether the severity scores calculated from each survey for each individual correlate with the severity level they filled in at the beginning of the exercise. They will also see what proportion of participants thought the questionnaires accurately captured their experience, in which case, why didn't they ask that after every questionnaire.

I appreciate the aim of this, but I don't think the thousand or more hours of patient time are well spent here.

I think it might have been better to simply show participants all the surveys and ask on a simple scale whether they thought each survey makes sense and would reflect accurately their symptoms and functional level. You don't need to fill in most surveys to be able to gauge whether they will show something real and useful.
 
Snow Leopard said:
I have already given the review of existing questionnaires a lot of thought and to avoid a Tu quoque trap, I'd first design a survey to review existing questionnaires using qualitative interview methods - asking participants what they think, designing a survey on how to assess the questionnaires, getting more qualitative feedback on the survey methods themselves and only then recruiting participants to answer the survey.

Instead Michael seems to have skipped a step, with the result is his survey suffers from similar problems to the questionnaires he is trying to get feedback on.
Click to expand...

You're obviously much more knowledgeable than I am re this.

At a simple level you can switch questionnaires for actimetry/actimetry* but I guess it's not that simple for other indicators/symptoms.



*Noticed this review of actimetry (electronic activity monitoring) versus questionnaires
"Conclusion
The findings suggest that the measurement method may have a significant impact on the observed levels of physical activity. Self-report measures of physical activity were both higher and lower than directly measured levels of physical activity, which poses a problem for both reliance on self-report measures and for attempts to correct for self-report – direct measure differences. This review reveals the need for valid, accurate and reliable measures of physical activity in evaluating current and changing physical activity levels, physical activity interventions, and the relationships between physical activity and health outcomes."
https://ijbnpa.biomedcentral.com/articles/10.1186/1479-5868-5-56
 
Yes I think this would make sense
Snow Leopard said:
I have already given the review of existing questionnaires a lot of thought and to avoid a Tu quoque trap, I'd first design a survey to review existing questionnaires using qualitative interview methods - asking participants what they think, designing a survey on how to assess the questionnaires, getting more qualitative feedback on the survey methods themselves and only then recruiting participants to answer the survey.

Instead Michael seems to have skipped a step, with the result is his survey suffers from similar problems to the questionnaires he is trying to get feedback on.
Click to expand...
 
Very poorly designed IMO.

I don't understand why so many scientists put out poorly designed surveys.

At uni I had like 5 lectures on survey design, plus did a bit of reading. That's all it took for me to have a basic understanding of good survey design. Don't understand why so many scientists don't have this understanding.

For me this was within a module on "statistics and methods for social sciences" - is that the issue? Sociologists being taught this but not medical scientists?
 
The questionnaire is badly designed. Best if people not waste their precious energy on this.

I had to give up fairly early in. I just couldn't answer the question about year of illness onset and there was no option to explain. Yet I felt it was important that I had important information to contribute.

Also, seriously, giving people only 24 hours to complete? That means many people will not even be able to do it, given our cognitive and energy limitations, skewing the results.

I sent them an e-mail, suggesting they contact Leonard Jason's team, who've done much better with these things in the past. Now it only defeats its own purpose.
 
You must log in or register to reply here.
Back
Top Bottom