Questionnaires - design, validation and use in ME/CFS research - discussion thread

[bobbler]

It reads like a variant of patient-blaming, only with racist overtones.

Even if "catastrophising" actually exists (which is not a given, despite the researchers' peculiar lack of curiosity about it), so what? The patients would still be in pain even if they stopped.



I wonder if some respondents haven't gained insight into some of the issues yet? For patients at least, it can take a long time to develop an appreciation of the differences between fatigue, fatiguability, and weakness, and even then it can be hard to make clear distinctions. But an awareness of the differences does at least highlight the problems with some fatigue scales, which don't offer opportunities to rate them separately or even acknowledge they exist.

Worse than that, it has now been at least a few decades since scientific psychology realised that all the 'intelligence' stuff using things like IQ tests was not only racist but useless due to it measuring cultural issues and areas that white natives at good advantage would have drilled into them instead of actual tests of intelligence in some of them.

Those lessons really take no transferrable skills and the lessons themselves are pretty parallel and so similar they prety much slide across and direcly map-over to being able how you not only word your questions but being really poor at designing your questions vs your actual research question: in layman's terms 'missing the point' and 'measuring something completely different to what you claim/think you are'.

The fact that certain demographics might be the ones that smart most at the weird wording that doesn't make sense from some of the irrelevant questions on your survey doesn't indiciate pathology in them but is almost the equivalent of when in psychology you do the test where words are missed out and those who are most used to reading sentences like that don't even notice they are missing when they scan-read, but those who tend not to read stuff like that 'spot the error'.

I have real issues with people using wording and not having to test the semantics and meaning imbued from these and them making sense and being interpreted to actually asking what they claim they do: that is the first thing anyone doing any kind of market research or survey, no matter how informal, has to do (even if it is 'read this question, what does it say to you, does it make sense etc' to a few colleagues) - and yet for some reason the subject area doesn't have as standard a section where they have to include this testing of these tools in their methods section or at least appendix.

 

[MSEsperanza]

Just stumbled across Arthur Stone, researcher on 'Self-report of medical and psychological outcomes' when searching for more of Eccleston's and Amanda Williams' work.

Sorry currently unable to skim his papers but thought could be relevant for those of us working on the issue.

A. Stone's research seems to address problems with mostly pain but also fatigue scales , e.g. recall at different time points and patients' compliance with filling in paper / online diaries

Apologies for just leaving a link to his publication list here for now:

https://renaissance.stonybrookmedicine.edu/psychiatry/faculty/stone_a

 

[Ravn]

2. How are questionnaires validity and reliability tested? Do these methods result in useful questionnaires from ME/CFS patients' perspective?
Here's a paper and a chart found from a quick search:

Professor Kristian Sommerfelt talks about the Norwegian Functional Capacity questionnaire development at the IIMEC15 Conference & explains how they tested and validated it. Sounded more rigorous than some others we've read about



More discussion here

 

[rvallee]

Nice. Have just skimmed to get the general idea, but this looks a lot like what I proposed a while ago. Actually looking for it, 4 years. Ugh. Time passes, and life with it. From a thread about a similar scale called the Validation of the Pittsburgh Fatigability Scale.

This is something that comes up so often on social media, it's good that some experts are finally paying to it: it's not what you can do, it's what you can do without consequence, and can maintain. Don't even need to grow, to do more, merely what is sustainable without incurring penalty, without tradeoffs.

And there were a few similar comments today from the thread about a center in Norway for severe ME patients, that too often what simply happens is substitution of activity, which is what mostly happens in BPS program. They boast about patients being able to do more, but don't notice, or don't care, that it was in exchange for other things, or that it's an artificial increase if it's in an in-patient treatment where patients don't have to do chores that have to be done in normal life.

 

[Liii]

Video by researcher Leonard Jason who seems to focus exclusively of various questionnaires for assessing ME/CFS.

I don't know what I think about all this, but Jason seems to be a sensible and knowledgeable person in general. I think it was interesting to listen to someone who is so into this.

He seems to have been working on DSQ-2 which doesn't seem like a particularly good tool to me.

There is some criticism of FUNCAP55 which I didn't agree with. It seem sensible to me that FUNCAP asks about concrete activities. That seems likely to be influenced by random things like your mood when you take it.

 

[Utsikt]

There is some criticism of FUNCAP55 which I didn't agree with. It seem sensible to me that FUNCAP asks about concrete activities. That seems likely to be influenced by random things like your mood when you take it.

Jason has been criticising FUNCAP for not accounting for pacing, even though that’s built into the questionnaire. I’m not convinced he fully understands it, nor the flaws with trying to identify if someone has PEM by asking them if they have PEM, which is what the v2 was doing it the latest version I’ve seen.

I think a questionnaire will always fail at PEM, so we should move away from them entirely.

 

[Liii]

It would be interesting if someone did a study like this:

1. Have participants with ME/CFS. (Could be interesting with other groups also.)

2. Randomise in two groups.

3. Show Group A videos about people with severe ME/CFS.

4. Show Group B videos about healthy people doing vigorous activities.

5. Have the two groups fill in various questionnaires for fatigue and physical functioning. For example Chalder Fatigue Scale and SF-36.

6. Compare the two groups. That is, check if the participants compare themselves to the people in the videos when they rate for example if they have "high level of fatigue" or "very high level of fatigue".

This would provide quantitative evidence for our common critique of subjective outcome measures. For example the critique that is expressed in Grahams video.

Edit 1: It wouldn't surprise me if the difference between the groups would be in the same order as in most of studies of psycho-somatic interventions.

Edit 2: Someone of our psychologist allies really, really should do a study like this! Or maybe it has already been done? Surely I would have heard about it by now if that is the case?

 

[Chandelier]

Video by researcher Leonard Jason who seems to focus exclusively of various questionnaires for assessing ME/CFS.

I don't know what I think about all this, but Jason seems to be a sensible and knowledgeable person in general. I think it was interesting to listen to someone who is so into this.

He seems to have been working on DSQ-2 which doesn't seem like a particularly good tool to me.

There is some criticism of FUNCAP55 which I didn't agree with. It seem sensible to me that FUNCAP asks about concrete activities. That seems likely to be influenced by random things like your mood when you take it.

ME/CFS Science on Twitter about this lecture:

1) Just watched this lecture by Prof. Leonard Jason. His team in Chicago has been developing questionnaires and assessment tools for ME/CFS for several decades.

In this talk, he gives an overview of his main findings and also comments on FUNCAP.


2) Jason's team developed the DePaul Symptom Questionnaire (DSQ), one of the most popular tools to evaluate ME/CFS symptoms.

In early versions they found it was necessary to include both severity and frequency requirements.

3) A lot of controls report having ME/CFS-like symptoms such as fatigue, sleep problems or brain fog at a mild severity level. So that’s why the DSQ requires a level frequency and severity levels of 2 or higher.

Just asking if people have a symptom isn't specific enough.

4) Jason also mentions the DSQ-PEM to evaluate post-exertional malaise. He said they choose 5 items on PEM that were shown to select the widest net of patients.

(That might explain why it mostly works as a screening tool rather than an evaluation of PEM).


5) He mentions that they created a much larger PEM-questionnaire with 50-60 questions using phrasing and suggestions from patients but it became too long and unpractical.

They tried to distill it into a brief version, what has now become the DSQ-PEM-2.

dsqpem2.com
DSQPEM2.COM

6) His team also submitted a review on evaluating illness severity and burden (Cripe et al. 2026), showing that the SF-36 and Sickness Impact Profile has been used the most.

Asking "have your symptoms caused a 50% or greater reduction in your activity level?" also works well.


7) However, Prof. Jason is quite critical of FUNCAP saying it is not measuring the symptom exacerbation itself - PEM symptoms such as brain fog or fatigue - only the consequences of these symptoms.

He thinks these two should not be mixed or confused.


8) The FUNCAP activities that cause functional reductions are almost all physical. His studies found that patients also report other symptom exacerbation triggers such as emotional stress, heat, foods, chemicals, drugs, mold, etc. and these aren't considered.


9) Third, he argues that some people could have severe symptoms but might still be able to function. And if patients are not engaged in an activity, they need to imagine doing it which is less than ideal for a questionnaire.

10) I personally disagree with many of these later points and the proposed DSQ-PEM-2, see comments here:

bsky.app/profile/mecfss…
BSKY.APP

 

[Utsikt]

7) However, Prof. Jason is quite critical of FUNCAP saying it is not measuring the symptom exacerbation itself - PEM symptoms such as brain fog or fatigue - only the consequences of these symptoms.

He thinks these two should not be mixed or confused.

Well, duh, it’s a questionnaire that’s specifically designed to assess functional capacity.

8) The FUNCAP activities that cause functional reductions are almost all physical. His studies found that patients also report other symptom exacerbation triggers such as emotional stress, heat, foods, chemicals, drugs, mold, etc. and these aren't considered.

It includes mental tasks like reading, working, talking, and writing.

It’s also not meant to assess all environmental «triggers». Another strawman argument.

9) Third, he argues that some people could have severe symptoms but might still be able to function. And if patients are not engaged in an activity, they need to imagine doing it which is less than ideal for a questionnaire.

Well, it’s about your sustainable functional capacity. I’d argue that anchoring it to activities everyone knows is a good thing to allow comparisons between people, rather than asking about some arbitrary baseline years ago.

I don’t think Jason has learned anything from what members here have been trying to tell him.