Use of EEfRT in the NIH study: Deep phenotyping of PI-ME/CFS, 2024, Walitt et al

I think it would be most compelling to demonstrate the issue with the Effort task with data - people might find the logic of it hard to follow. If it can be shown from the data that the difference in the hard:easy ratio between patients and controls could be due to something else - specifically something like a difference between the groups in terms of the reward values and probabilities they were presented with - then that would be game over.

If not, then I think the key issue is the different starting point of patients and controls, in terms of it being a potentially fatiguing task for one group but not the other, render the task fundamentally invalid for comparing these groups, because it scuppers the reward system. One group has a penalty that they have to factor in when considering the rewards, the other doesn't.

If people are looking at the data, which I'm going to upload again to this message, note that HV F's data were invalid so you'll need to remove those rows.

 

And even when the rewards are higher, they are so laughably small that they are completely tokenistic for all intents and purposes.

My thinking about this worthless test, I don't think it's valid in any context, is precisely about token rewards. Video games provide the best example of this, and they are indeed very popular. People will go through all sorts of challenges and spend a huge amount of time and effort for what is usually nothing but a token reward. And there is a huge amount of psychology about gamification, in large part because it's so lucrative to the video game industry.

But what they don't account for here is the shift in priorities and the opportunity cost of going for a pointless tokenistic reward. In general, pwME are all out of fucks to give, our lives have been broken, we have lost more than most people can even imagine anyone could lose without dying, and here they were, presented with a pointless task for a bullshit token reward that carries an opportunity cost, a risk where going for the "higher" (but still tokenistic) reward can mean significant deterioration, in a context where there already is a significant demand on the body that could wreck them, possibly permanently, for simply having made the very hard choice of participating in such a grueling battery of tests.

So it's a bit like having someone stranded in a remote area with a broken leg, who have to carry on if they want to survive, then subject them to a stupid game of hopscotch to win a ribbon with a star on it, then conclude that they're probably a bit depressed/lazy/unmotivated, or whatever, since they're not interested in playing an easy game for an easy reward. People love rewards, even fake ones, if presented the right way, as a challenge, and these people aren't playing along, there's clearly something wrong with them, if you completely ignore the context in which they exist.

So for all the faults of this bullshit invalid test, the worst flaw the researchers did is completely ignore the context in which the patients were playing in, which already included the hard choice of participating in the study. Which is the same mindset that brings people with PEM to refuse to keep pushing themselves, and conclude that they must be lazy/unmotivated/delusional since there is clearly nothing wrong with them, if one doesn't believe in the illness, doesn't accept the patients' lived experience as valid, and decides to suspend disbelief about the totality and infallibility of medical testing, which can't even be said to be half-reliable in the best of cases.

Like everything else biopsychosocial, it makes sense if you don't think about it, and it takes all but a minute to see so many holes in the reasoning that no serious person could argue that it's a valid model. It takes beliefs, and we already knew about Walitt's beliefs, they are very clear, in the end that's all that went into the interpretation.

 

In case this hasn't been shared already:

 

Does he genuinely think they've "shown" that on the basis of a single p.04 result from 8 patients and 6 controls? While other results in the same trial show no "effort preference" effect?

 

I'm slowly catching up with this thread so may just be repeating what others have said, but this task is about calculating odds, not about how much energy a person can/is willing to spend. Surely "success" in the task is getting maximum reward for minimum effort. Also the task is cost free for a healthy person but not so, either physically or cognitively, for pwME. The only utility it might have with ME is showing cognitive disparity in making good calculations very quickly and repeatedly.

 

Yes. I'm trying to think of a simple way/term that might communicate it or what we can use to show up the issue with what Walitt has done/is saying - I'm wondering whether others can think of better 'frames of reference' eg common games others might know that we could use to explain it?? anyone up for some idea-generating?

The closest analogy I've got is taking a normal exam, and remembering the 'exam technique' stuff we used to get taught about not writing a long essay on the first few questions that often only are to warm up and have just one or two marks per question - because you'll run out of time for the big stuff at the end.

Well imagine the EEfRT as Treadway et al (2009) have designed it as like someone sitting down and doing a standard exam paper, with the usual different questions have different (max) numbers of points/marks associated with them, time is limited/tight and you do your best to get as many questions answered well as you can without wasting time on any. Imagine some of these questions say needed 200 word answers and others 30 (so you shouldn't give them much time)

And instead of them calculating your performance based on what mark you got vs the marking scheme, and maybe looking at other factors like how far through you got and where you might have fallen-down like under or over-answering questions vs marks available...... explaining it as individual differences in exam behaviour

Then instead someone comes along and takes that collection of descriptive stats as a tool and instead just marks the whole thing based on:
'out of those questions you did, how many did you write 200 words for' - and used that as a 'metric'

without any of the detail or pulling out the sensitivities like whether it was a question that needed 200 words, how many marks it was worth or not, how many questions you actually completed/far you got in the test etc.

and called that metric 'effort preference'

What would that mean?, without any more detail into whether someone completed more questions or wrote more words in total across the whole paper. Would that metric mean what the term 'effort preference' sounds like? Would it be valid like the person who might have invented the analysis tools to show different issues with exam technique had validated it for?



I'm aware that perhaps underneath it all some of these things might map out in analysis (or might not) if done as per the EEfRT models and methods, but that metric on its own isn't the validated part from Treadway and the new name is misleading vs the instructions and design of the tool?

 

Oh yes, I've used a filter and see the same, all the trial 1 have the same 'reward' whatever they chose in the hard column. And so on for all the trial numbers. Surely not and maybe it means the combo or something but either way..

I've done very scratchy and unscientific calcs

 

as a totally random one I've just noticed. HVB seems to have selected easy for nearly all of their high probability and low probability tasks. then strangely is more half and half of doing hard when it was medium 50:50 probability.

 

Because of the nature of the increased time to do each hard trial vs easy trial when you get to the later trial numbers there are fewer and fewer people who did them at all. If someone did 'too many' hard early on they wouldn't get to the later ones. And of course it is designed so that noone knows what is coming up - although I assume they were told the same info as in the Treadway version - the max and min reward values, so could have a sense just enough to get some sort of personal heuristic (eg whether a reward was so small it wasn't worth the extra time) once they got into it.

No strategy could really win given it was 2 trials selected at random - even if you were a super-fit finger-athlete, do you prevent the lowest of your 'win-trials' from being too low by always going for hard (particularly if it isn't a low probability one) or do you try and keep the mode/mean up by going with trying to move quickly through so you can get to the higher value ones and just going for quick and easy if it is £1 something. But of course you don't know how many are £1 something and how many are £3 something. That's the point.

It's interesting looking at an individual level by probability-level and I don't think I'm seeing things in looking at the different individual patterns that people are employing strategies. And that some of the participants are having to change take part way through. For example on medium there are a number of ME-CFS who look like they make a conscious choice around trial 11 to go for easy (unless it is a really high value) when it is 50:50. WHen you look at HVs then even then there seem to be a small group who do the same but later on (like late 30s) which could either be exhaustion or strategy.

To risk a sidenote but I don't think it is, we all have to remember that having ME/CFS has probably taught us a lot that isn't about illness. I could teach well people a lot about efficiency and wasted time/energy and so on.

The idea it is all down to fatigue rather than us having to make these sorts of daily judgements - and they accumulate as our disability does, or we 'get into' starting to look at life differently to be more efficient because we've other important stuff (including more committments than healthy people due to appointments, meds and so on) - is also a misnomer;

we might just be pretty used to looking at tasks like these and the calculations involved, having to find a way to see through the BS and pick a 'best-course'. And experience might also show as 'decreased activity' either in shock or puzzlement etc at what to do or just the thinking involved with coming up with a strategy.

 

it's exactly like broken-leg in the middle of nowhere hopscotch for a ribbon!

 

Just seen this - great work. I think it is exactly what we would expect:

EEfRT raw data analysis by @bobbler
Minimal difference on 12% win probability tasks where easy tasks are more logical
Likewise 88% win probability tasks were hard tasks are more logical
Significant though not massive difference on the 50% win probability tasks:
ME: 31% hard tasks
HV: 41% hard tasks

IIRC, Treadway also indicated it was 50/50 tasks where the difference was to be found. As far as I can tell, a lot of the complexity was added to make it hard for participants to find a simple (no-thinking) winning strategy

 

Isn't it because the reward value for the easy task is always lower than the reward value for the hard task (with the probability of winning being the same)?

For an easy task the reward was always $1 whereas for the hard tasks the reward was one of [1.24, 1.42, 1.6, 1.78, 1.96, 2.14, 2.32, 2.5, 2.68, 2.86, 3.04, 3.22, 3.4, 3.49, 3.58, 3.76, 3.94, 4.12].

 

It's never explicitly stated, but if you look in the supplementary data on page 9 of the file titled 41467_2024_45107_MOESM1_ESM.pdf there's a diagram which shows this (and I think it's more clearly set out in some of Treadway's work.)

I also struggled to understand the task for the same reasons as you, but I think it only makes sense if the easier option always has a lower payout?

 

Yup if it is the case and not an artefact of the datasheet (e.g. if each amount we see in the excel column actually equates to a 'pairing' of hard-easy eg $4.50-$3.50) then you are absolutely correct, he hasn't run the experiment at all because there should be no choice. Or it would be something completely different regarding strategy - as they only pick 2 win trials maybe there is something in the time you take on the different options of hard-easy but I mean it would be so minimal and so complex it would blow the whole point.

It would be such a fundamental thing I'm having to assume that it is just what is reported in the column that is the error for now, and not that it represents that he only showed one 'reward magnitude' on the test?

 

From Worth the ‘EEfRT’? The Effort Expenditure for Rewards Task as an Objective Measure of Motivation and Anhedonia Treadway et al:

... For easy-task trials, subjects were eligible to win the same amount, $1.00, on each trial if they successfully completed the task. For hard-task choices, subjects were eligible to win higher amounts that varied per trial within a range of $1.24 – $4.30 (“reward magnitude”). ...