S4ME: Submission to the public review on common data elements for ME/CFS: Problems with the Chalder Fatigue Questionnaire

In Hilda Bastian's blog, she mentions this review on fatigue scales by Lisa Whitehead: https://www.jpsmjournal.com/article/S0885-3924(08)00560-5/fulltext

Whitehead seems to indicate that the CFQ does not have the "ability to act as an outcome measure sensitive to change with disease progression or treatment".

 

Yes, the problem with the CFQ is how the questions are framed, it is a short term "how are you feeling" measure, designed to be as easily biased as possible.

 

It was designed as a diagnositic tool. I don't think they designed it to be biased. But its use as anything other than a diagnostic tool does lead to bias.

 

Added a link to the submission into the first post of this thread, https://www.s4me.info/docs/CFQ-Critique-S4me.pdf - this can be used by members and non-members to access a PDF version of the submission.

 

Excellent!

 

Just read this through again, and really excellent submission. Thanks to all those involved.

One thing really shouts out at me when reading this: The CFQ seems to be the product of some horribly muddled and uncritical thinking. Seems to epitomise something that has been designed by people not competent to do so; I really do believe that.

I'm an engineer not a medical expert, but one thing has to be true surely for all designs: The requirements must be clearly identified first, that the design is intended to fulfil. In my engineering world that means a requirement specification is first written, by someone qualified to do so, and that requirements spec is peer reviewed first. Only once the requirements have been ratified, do you then proceed with designing something that will meet those requirements, and then you get that peer reviewed too. The person/people writing the requirements might be the same as those doing the design, but often not, it depends who is competent to do what.

To me the CFQ smacks of something muddled together, with the "requirements" emerging post hoc out of the smoke as the design was being done; inevitably satisfying the designers' biases, because I suspect those biases are what mainly guided their design efforts.

As the submission says, any new questionnaire should be done completely afresh, from the ground up. To me that means getting the requirements identified first and peer reviewed; a process that itself might need several rounds of revision and review. Then and only then see about designing a questionnaire that meets the requirement. Else it could still be a mess.

This is something we could possibly consider looking at here in S4ME, if not already doing so. (Apologies if I've missed something along the way and that is already happening).

 

I’m not sure it was ever a “designed” questionnaire!! The questionnaire seems like something written late one night because it was going to be needed first thing next morning. Then once it had been used and achieved the desired results, so it was used again and again. Each time it performed as desired it was favoured more and more, and gradually it became the go-to questionnaire amongst a certain set of researchers.

Either that or we say it WAS designed, in which case the use of the term “usual” as a stand in for “when well” was an intentional feature, that could only have been designed that way if the real intention was to confuse. Like wise the anomaly of having an option to effectively select “better than when well” (ie “better than usual”) would have been an intentional design feature. Also the inclusion of statements usually linked to depression rather than fatigue becomes suspect.

If it was designed to confuse, then it really shows the researchers in a much worse light than if they simply created a rushed but faulty questionnaire which they then had to stick with because it proved so useful.

 

In my round about way that's what I was saying really . It should have been but it wasn't. I think all these folk consider necessary is to come up with some preconceived prejudice and then cobble together something that self-fulfils it.

 

Since I have become ill I have been asked to fill in a number of health questionnaires - I'm not referring to ESA or the like here.

In every case there were questions that were ambiguous and open to interpretation on the part of both the person filling it in and the person assessing you in a heads we win, tails you lose style.

Sometimes the ambiguity is subtly done. Done well enough that it's very hard to believe it is entirely accidental.

 

I've got this vague memory of Wessely saying he basically just jotted it down on the back of an envelope one night, but I can't remember where. Maybe it was in a podcast related to this paper? Maybe I've misremembered someone else making fun of this:

https://www.meassociation.org.uk/20...-fatigue-syndrome-journal-article-march-2012/

 

Well we recently learned that it was originally created by Simon Wessely so, yeah, it definitely was. Of course Chalder's, uh, "thinking" is also horribly muddled and uncritical but then the explanation for both is the same.

The whole thing about PROMs, about "measures" that aren't even measures need to be addressed and raised more broadly, especially as a critical component of the crisis of replicability. Questionnaires do not measure anything, even less so here. The language needs to be adapted to differentiate actual measures, precise, accurate and relating to the natural world, and subjective scoring systems. The CFQ is even less scientific than movie rankings and the average psychometric questionnaire is barely as good as Rotten Tomatoes' ranking system, sometimes about as good as "which Spice Girls are you?".

The foundation of science is measurement. If you aren't measuring accurately, precisely and reliably than you aren't doing science (yet). It doesn't mean it's not possible to measure the thing but weak unreliable scoring systems are not a substitute and the language must reflect that. No, fatigue was not "measured" to be higher or lower or anything, ever. We can't measure it, whatever "it" is since the meaning itself is highly confused, we can only rate it. The same way we rate a song or a BBQ sauce. It's an imprecise subjective value whose selection is highly influenced based on circumstances but especially so when the intent of an experiment is to influence it in one direction over another.

I understand that there is huge sunk cost and escalation of commitment behind the fiction that questionnaire scores can be a substitute for precise scientific measures but that's the worst possible reason to keep something that has shown itself to be flawed beyond any repair. CFQ may be the worst example of this. It's even less precise and reliable than the average questionnaire. Its use reflects a system that has veered off the deep end years ago and is coasting around in the dark wasting lives and not doing much else.

 

Another thing that strikes me about attempts to measure fatigue is that not only is it impossible to measure, we can simply rate as @rvallee says, it also assumes fatigue is one dimensional. It is not.

If we are to use the word "fatigue" to encompass such a wide range of feelings then we should at least acknowledge that there are different "qualities" to fatigue and record those too.

In this way @rvallee's rating would be more appropriate because some feelings of fatigue can be quite pleasant whereas others most definitely are not.

 

I generally have no trouble with the idea of ill intent from this group but before reading here my thinking was that the CFS represented a failure of expertise and education. The scales questions read like something a high school student might come up with (and not trying too hard) before being trained to think about how precision in wording matters to getting a clear response that is specific and sensitive to the matter studied.

The questions all reflect the lack of understanding / bias / prejudice that comes from a generally healthy persons POV (ie a lack of deep thinking about the ill person and Q's from their POV) Education and training to be an expert at this kind of thing should have yielded something way better than an 'anybody could have come up with this' questionnaire.

Then also, it could be that the education and training is the failure. But even so people who reach that level ought to be capable of (especially after seeing many patients) putting themselves in the position of the patient to imagine a better / more precise set of questions.

And maybe not have relied so heavily on them for all of their data would be helpful while we're at it.

 

Oh, yes! Post-exercise fatigue in a healthy body is basically a drug. Doesn't have to be fit. It can also hurt but then it will do both. It is a very pleasant experience, completely unlike the crippling exhaustion during illness. There are definitely variations between individuals but most people seem to enjoy the endorphin rush that is characteristic of a good workout.

I don't know how to separate those but this is an important distinction that is deliberately misrepresented. It's frankly as pathetic as taking intense emotions during a particularly competitive sports match and somehow arguing those expressions are of rage, despair or some other negative emotions. Things that look alike often aren't. Science is supposed to 1) be concerned with the difference and 2) work to reduce them being confused for one another. In BPS there are explicit deliberate efforts to create this exact kind of confusion and reduce complexity into meaningless oversimplicity.

The level of charlatanism required to deliberate create this confusion and misrepresent different concepts based on trivial similarities is incompatible with either science or even any profession. My own field of software development doesn't even have licensing boards or official certifications, it's even notorious for being lax on those, and this kind of misconduct would never be tolerated. That any part of medicine should have lower standards than this is unimaginable failure.

 

I hope you also sent it to Hilda Bastian for the Cochrane exercise review scrutiny group.

 

Yep, see https://www.s4me.info/threads/indep...led-by-hilda-bastian.13645/page-8#post-260994

 

As far as I can see from the submission, there seems to be no standardised way of converting the bimodal scoring of the Chalder fatigue scale to the Likert scoring and vice-versa?

 

I'm just reading the original submission now, and it's bang-on. I thank everyone involved for their excellent advocacy work.

 

Just re-read the original submission again, and I'm as always appalled by the naive incompetence so clearly evident in those who came up with / advocate the CFQ as a general instrument for measuring fatigue. Their obvious inability to think something like this through, the inbuilt heavily biased presumptions that inevitably load the results, and failure to comprehend its laughably blatant failings, casts massive doubt on their competence to do anything requiring clear uncluttered reasoning ... like being scientists for instance. Then add into the mix how these faux scientists have held sway over the health - and indeed safety! - of so many sufferers over so many years, it just beggars belief. It's medieval. And to me at least it feels downright criminal.