Unwilling or unable? Interpreting effort task performance in myalgic encephalomyelitis/chronic fatigue syndrome, 2025, Kirvin-Quamme et al

[Utsikt]

View attachment 26556

A very nice, tightly argued paper.

Looking at that excellent Figure 1, it's rather clear that there really wasn't that much of a difference between proportions of hard task choice anyway (the y axis). Most participants are choosing the hard task between 20 and 50% of the time. The regression line is pretty flat - the proportion of completed tasks does not have much effect on the proportion of hard task choices. If you exclude the 2 or at best 3 healthy volunteers who chose the hard task relatively often, there is not much difference between the 'healthy volunteers* and the 'ME/CFS patients* in terms of proportion of hard task choices.

If most of the healthy people look like most of the ME/CFS people in terms of proportion of hard tasks selected, then the measure doesn't have much discriminatory power, and probably isn't telling us much about ME/CFS.

Then, there was that healthy control who was excluded from the study because he was supposedly gaming the system - which was rather ironic given the task was supposedly about making choices to maximise the reward. If I recall correctly (and I may well not be), the reward system was set up so that the best strategy was to complete a few tasks with high reward and fail to complete the rest, maximising the chance that a high reward task would be randomly selected for payout from the completed tasks. I think the exclusion of that person who followed a reward maximising strategy changed the overall finding towards what Walitt wanted to find?

If I'm recalling that situation correctly, it is possible that some of the ME/CFS participants were following a 'work smarter, not harder' strategy, somewhat similar to what the excluded healthy control was. The two healthy controls with higher rates of hard task selection had perhaps failed to understand the implications of the reward scheme, instead being focussed on demonstrating how healthy and physically superior they were.

I can't recall now, but I think there may have also been some issues with the two cohorts in the effort preference test not being well matched on age and sex? Even without any complications arising from those possible differences, the experiment was an uninterpretable mess. The reduced physical capacity of some of the ME/CFS participants, deliberate strategising favouring low rates of completion, and the possible motivations of the healthy controls to demonstrate physical prowess all possibly affecting what people did in this very small study in ways that are rather hard to unravel at this point.

These are just comments fired off on the basis of vaguely remembered facts - so I'm very happy for the authors or others to tell me about things I have got wrong.

I remember reading something similar here:

The NIH Intramural ME Study: “Lies, Damn Lies, and Statistics” (Part 1)

The infamous intramural National Institutes of Health (NIH) paper on post-infectious Myalgic Encephalomyelitis (ME), a disease affecting many millions worldwide, purports to define the ME phenotype…

thoughtsaboutme.com

It’s too long for me to go through and check again. But game theory is one of my more favoured subjects (and something that was extensively covered in my education), and I remember being quite shocked at what was described, and thinking that the authors of the NIH study got it backwards.

 

[Peter T]

I'm looking forward to a day when people who are incapacitated by ME/CFS get effective treatment, rather than having to expend their meagre energy on refuting arguments like this.

A tangent, but I think Karen makes an important point here about those focused on understanding ME/CFS.

In general, researchers into and clinicians working with the condition are hyper focused on their own interests rather than understanding it as a whole. On the one hand the BPS approach is about singlemindedly imposing beliefs about psychogenic causation and/or imposing psycho behavioural interventions, completely ignoring most of what is known about the condition, and on the other the biomedical, the physiological and neurological interests tends to address just specific aspects (eg ‘it’s something in the blood’), which, though producing potentially useful data, leaves us with only limited overall understanding of ME/CFS.

With only a few notable exceptions such as Decode ME, it is only in such contexts as here, that are patient/sufferer dominated, we see a desire to sift all possible information sources to understand both the lived and scientific reality of the condition rather than bolster prejudged conclusions. In how many fields of medicine is progress in overall understanding of a condition dependent on patient scientists (or ‘activists’ if you feel threatened by us) and a handful of allies? This is all the more remarkable when you recognise the energy constraints we have to operate within.

This relevant here in that, had it not been hijacked by Walitt et al, the NIH Intramural Study was intended to do just that, to achieve a bigger picture.

[edited to add final sentence and correct some of my typos]

 

[DMissa]

I am not a psychologist. I could not understand a lot of the conclusions and psychology of the Wallitt paper which is why I have never really commented about it. It was over my head.

I understood this paper, start-to-finish, perfectly on a first pass. Really nice work making this accessible.

 

[petrichor]

This seems like a good paper.

The Walitt paper is odd. It appeared to be assuming that the extra fatigability in ME/CFS should be immediately apparent, when post-exertional malaise is a defining feature of ME/CFS. And then they choose one statistically significant result (and bring it up repeatedly) and conclude that indicates "altered effort preference" when there are multiple other fairly obvious explanations for that result (like finding the task harder). And they also had a much more statistically significant result that people with ME/CFS had a decline in button pressing rate on the easy task compared to controls, and other results that could be described as fatigability even by their specific and somewhat arbitrary definition, but they chose not to interpret those as fatigability

A general issue with the Walitt paper was the use of broad terms/claims to refer to specific results that don't have a clear interpretation. Eg. using "effort preference" to refer to the results of the task choice test, and "central fatigue" and "peripheral fatigue" to refer to other pretty specific measures. That just 1. Makes the paper confusing and difficult to follow 2. Leads to them making claims that go way beyond the evidence they have.

 

[Yann04]

This seems like a good paper.

The Walitt paper is odd. It appeared to be assuming that the extra fatigability in ME/CFS should be immediately apparent, when post-exertional malaise is a defining feature of ME/CFS. And then they choose one statistically significant result (and bring it up repeatedly) and conclude that indicates "altered effort preference" when there are multiple other fairly obvious explanations for that result (like finding the task harder). And they also had a much more statistically significant result that people with ME/CFS had a decline in button pressing rate on the easy task compared to controls, and other results that could be described as fatigability even by their specific and somewhat arbitrary definition, but they chose not to interpret those as fatigability

A general issue with the original paper was the use of broad terms/claims to refer to specific results that don't have a clear interpretation. Eg. using "effort preference" to refer to the results of the task choice test, and "central fatigue" and "peripheral fatigue" to refer to other pretty specific measures. That just 1. Makes the paper confusing and difficult to follow 2. Leads to them making claims that go way beyond the evidence they have.

In my opinion it’s not that if you’re a psychiatrist you magically understand it. It’s that the authors had decided on their conclusions before they got the data. They had decided how they wished to interpret the illness beforehand so they just picked out the result they could find that might back up that view and mostly sidelined the rest.

 

[Trish]

In my opinion it’s not that if you’re a psychiatrist you magically understand it. It’s that the authors had decided on their conclusions before they got the data. They had decided how they wished to interpret the illness beforehand so they just picked out the result they could find that might back up that view and mostly sidelined the rest.

I agree. It seemed so obvious that there was a predetermined conclusion from the fact that they chose to build the entire conclusion of the abstract around their invented term 'effort preference' despite it being a misinterpretation of the outcome of a very minor aspect of their research that only included a few participants. It was so clearly fake.

 

[Utsikt]

I am not a psychologist. I could not understand a lot of the conclusions and psychology of the Wallitt paper which is why I have never really commented about it. It was over my head.

It wasn’t over your head, it was over theirs. There is nothing to understand in psychology, 99 % of it is pseudoscience.

 

[rvallee]

It wasn’t over your head, it was over theirs. There is nothing to understand in psychology, 99 % of it is pseudoscience.

It's probably more accurate to say that most of psychology is generic to the point of being inapplicable in any particular context, and most of the rest is pseudoscience. Not much different from astrology. Will there be life events happening during your day, or did life events happen some time before you got ill? Sure, probably. Is that useful? Not at all. But life events will surely be happening on any particular day to most people. Most of the findings fall in the first category. They aren't wrong per se, but they are so generic that they can't be used for anything.

It's actually a lot like polygraphs. Sure, lots of people have this physiological reaction involving elevated heart rate (after all, the context is confrontational and it should never be assumed that it is honorable) and some sweating. Does it indicate lying? Of course not. Could it be? Sure, why not, but it can't confirm either way, and interpreting anything is up to personal biases and agendas.

Most of the pseudoscience is actually in interpretation. Especially as it applies to clinical psychology and the gaps in medical knowledge, most of it is. But overall it's mostly just not very useful, while the pseudoscience part of it is mostly harmful. The "effort preference" paper is a good example of spurious and biased interpretation veering into pseudoscience out of generic and otherwise useless testing of some random thing that takes zero account of context, especially as it clearly worked backward from the conclusion out.

 

[Trish]

It wasn’t over your head, it was over theirs. There is nothing to understand in psychology, 99 % of it is pseudoscience.

I think in the case of the Walitt paper, it's not psychology or psychiatry that you need to understand, it's the fact that there are people, including Walitt, who may be physicians, psychiatrists or psychologists who believe that pwME don't have either a physical or a mental illness. They believe we are not ill at all in the normal sense, but that our problems are psycho-behavioural That is, we are misreading healthy variations of symptoms as signs of disease, and are therefore avoiding effort. Hence their interpretation that pwME in the test they used had a preference for lower effort tasks. We have faulty 'effort preference', not inability to do the harder task, according to their false interpretation.

Which is not a medical or psychological explanation, it's pure prejudice. And especially insulting the pwME who made the huge and potentially harmful effort of volunteering to take part in such a gruelling study in which this particular test was just a small part.

 

[Utsikt]

They believe we are not ill at all in the normal sense, but that our problems are psycho-behavioural

I agree with your analysis about Walitt’s supposed beliefs, but this is just another branch of psychology.

 

[Sasha]

They believe we are not ill at all in the normal sense, but that our problems are psycho-behavioural That is, we are misreading healthy variations of symptoms as signs of disease

This has always baffled me, that practioners could endlessly hear people saying, 'Exerting myself makes me feel as though I have the flu' and not understand how abnormal that is. Feeling as though you have the flu isn't just an exaggerated feeling of being tired.

 

[Karen Kirke]

Actually, a really nice part of this project was getting to read plenty of fascinating psychology articles that I had not come across before (e.g. the ones we cited on motor speed and processing speed), and discovering how effort-based decision-making has been studied in other conditions like schizophrenia. What was quite apparent was that Walitt et al. deviated from the norm.

Along with working with Andrew and Kevin (both of whom have backgrounds in psychology/cognitive science/neuroscience), and getting a fantastic review from Jarean Carson, it has been a positive psychology experience.

Thank you to everyone who has posted and for all your kind words. I'm sorry that I don't have the capacity to engage with each one. This one makes all the writing and rewriting and rewriting and rewriting feel worthwhile - thanks @DMissa !

I am not a psychologist. I could not understand a lot of the conclusions and psychology of the Wallitt paper which is why I have never really commented about it. It was over my head.

I understood this paper, start-to-finish, perfectly on a first pass. Really nice work making this accessible.

Andrew and others will be able to engage more next week.

 

[Utsikt]

Personally, I’m curious about what «effort» actually is. How is it defined?

 

[Yann04]

Personally, I’m curious about what «effort» actually is. How is it defined?

In French we tend to translate the “E” in PEM to “effort” instead of exertional.

“Malaise Post-Effort”

I think it has the plus side of it being implied it applies to cognition aswell. (Atleast the word “effort” implies that in French more than “exertion” does in english).