Use of EEfRT in the NIH study: Deep phenotyping of PI-ME/CFS, 2024, Walitt et al

I don't think there is, as far as I know. It would probably be nice to have it all in one place, but I doubt the journal would publish it. It could be posted as a comment on the article or self published on a preprint server.

I'm not sure whether or not to ask about it. Also not sure if they'll be allowing questions from the people on zoom. I am tempted to though, it feels wrong to allow them to promote the findings without addressing the issues. I really was hoping to have the letter in before the symposium but it didn't happen. Do you have thoughts on this @EndME @Evergreen ?

 

I decided to submit two questions ahead of time. Here is what I submitted. I think it's unlikely that they'll include them but it doesn't hurt to dream.

1. You reported in the article that the PI-ME/CFS participants had significantly lower successful completion rates for the EEfRT's hard trials compared to healthy volunteers. The creators of the EEfRT state in their original validation article that the EEfRT data is not valid if participants are not able to consistently complete the task. Why did you proceed with interpreting the PHTC data as evidence of impaired effort preference when the data appears to be invalid according to the guidelines set by the task creators?
2. Your article is the first to frame the EEfRT as a measure of effort preference, despite there being over 70 previous articles using the task. In these other studies, the task is typically framed as a measure of reward motivation, effort allocation, or effort-based decision-making. What is your rationale for reframing the EEfRT as a measure of effort preference, rather than using an established interpretation?

 

Sorry for the delay, @andrewkq. I wasn't sure what to say. Here's what I would have suggested:
You measured what you call effort preference using the proportion of hard tasks chosen in the EEfRT task, and you found that people with ME/CFS chose fewer hard tasks than healthy volunteers. However, the hard task sounds very difficult for people as disabled as your ME/CFS participants were - indeed, you found that patients were less able to complete the hard task successfully than healthy people. Since the hard task was more effortful for your patient cohort than for your healthy controls, how can you reach any conclusions about whether effort preference is different in people with ME/CFS compared to controls? It sounds like comparing apples with oranges.

 

I anticipated this response from AIRIO (Agency Intramural Research Integrity Officer) at NIH after I filed a complaint on this test & Walitt, but sharing for visibility.

"I am writing in response to your email of March 19, 2023 on the Nature Communications article, “Deep phenotyping of post-infectious myalgic encephalomyelitis/chronic fatigue syndrome.” My office has assessed the allegations, and determined that they do not fall within the definition of research misconduct, in that you are not alleging falsification, fabrication, or plagiarism in this paper. Although there is a scientific difference of opinion about whether the Effort-Expenditure for Reward Task (EEfRT) test was appropriately used and appropriately discussed, this assertion does not meet the definition of research misconduct, as research misconduct does not include honest error or differences of opinion. Therefore, my office has closed this matter with no further action. As you may be aware, NINDS will be hosting a symposium about this topic on May 2, 2024 [https://mregs.nih.gov/ninds/f1p5-f2l3320], which is one of the remedies I believe you had proposed. We wish you the best."

Sidebar: Will see how tomorrow goes now.

 

"Although there is a scientific difference of opinion about whether the Effort-Expenditure for Reward Task (EEfRT) test was appropriately used and appropriately discussed"

Does this mean people at the NIH had different opinions on whether the EEfRT test was appropriately used/discussed?

 

@EndME I asked this question. Sharing reply.

“No, I could not say that. I could say that my office reviewed the literature (including all the material you sent) to confirm that this was, in fact, an ongoing matter of discussion, and that there had not yet been a consensus statement resolving the matter.”

 

All I see is plausible deniability..

 

I think this is framed well @Evergreen, would you be open to submitting it or having someone else from the thread submit it? I think this version does a good job of highlighting the logical issues without needing to rely on Treadway's perspective.

The email for submitting questions is mecfssymposium@ninds.nih.gov

 

Thanks Andrew. I'd be very happy for someone else to submit it. Just post on here if submitting so that it only gets submitted once.

So many on this thread pointed out that the hard task would be really hard for them, or harder for patients than for healthies, and for me, that's the crux of the argument. You can't compare effort preference if you don't control effort.

The task in any response is just to provide evidence for what patients can see at a glance - and yes, Treadway and other researchers have seen it too.

Edit: Didn't mean to make it sound like a response is easy! Making simple things appear simple is notoriously hard work.

 

For those who weren't able to tune in, they did answer (shortened) versions of my two questions. Here are quotes of what the moderator asked and the answers given.

My original question 1:

What the moderator asked:

Nicholas Madian's Answer:

As a reminder, here is the exact quote from the original EEfRT article we are referencing:

My original question 2:

What the moderator asked:

Brian Walitt's Answer:

 

Maybe they might at least get a hint that 'effort preference' was a remarkably stupid choice of term.
Involuntary effort is something you cannot test for and so not a scientific concept.
Psychological paternalism always shows through.

 

"to answer the question, consistently completing the tasks is not a requirement for a valid effort test administration"

But the problem is that the task likely required more effort for patients than controls. Lower completion rate is probably only a reflection of that, as was the proportion of hard task choices. Frustrating response.

 

Well done on getting answers to your questions, @andrewkq , very valuable.

Just highlighting bits from the quotes in @andrewkq 's post above:

From Treadway's paper - the creator of the task:

From Nicholas Madian's answer:

 

Thanks for submitting these, Andrew.

Re: Walitt's response, I don't recall whether there was precedent for using number of hard choices versus using difference in motivation effect.

Does someone recall?

Also there is a teleconference May 6. Does anyone know if they've been asked about whether they posed a hypothesis and tested for it. Versus data exploration?

And going forward has anyone pressed them to be transparent about the hypothesis leading into testing versus allowing 8 years of data exploration and then using p-values as if they are meaningful.

 

Could you say a bit more about what you mean by using difference in motivation effect?

I don't think I've heard anyone ask about hypothesis testing versus data exploration and the use of p-values explicitly. I would love to see them confronted about this. In the call yesterday, Walitt openly admitted that their hypotheses for the fMRI portion were only about the motor cortex and they were surprised that they were not significant and that the TPJ popped up instead. So he openly admitted to it being entirely speculative and not hypothesis driven, even though that is not at all how it was framed in the paper. It would be nice if someone could convince them to preregister their hypotheses in the future if they are going to use p-values but there's no way they would agree to that.

 

I don't think the 'study' deserves its own thread, it's just a huge muddle of academic echo chamber brain rot that tries to pin brain fog on pandemic measures, using very odd language that reminds me of microeconomics 101 classes and models of 'homo economicus', simplified models meant only for illustrative purposes and that aren't supposed to translate into the real world, as they simplify human behavior down to basically being simple automatons.

It reframes brain fog as simple lack of motivation, or whatever. Everything blamed on imaginary hardship during a pandemic that completely ignores the freaking pandemic itself:

Reference for "In addition, CFS and depressive disorders may mutually intensify each other. Anhedonia – a general lack of motivation and willingness to exert effort for goal-oriented behavior – emerges as a prominent symptom in both conditions" is:

To me it looks a lot like the principle behind 'effort preference', although the closest they come to using the term is:

It's not named as the EEfRT task, but it's described as pretty much the same thing:

The study:

Effects of Perceived COVID-19 Exposure and Action-Outcome Predictability on the Motivation to Invest Cognitive Effort
https://econtent.hogrefe.com/doi/full/10.1024/1016-264X/a000392

Abstract: Everyday life situations characterized by poor controllability because of restrictions and uncertainty about action outcomes may attenuate motivational states and executive control. This article explores the interaction of a prior experience with COVID-19 and the susceptibility to respond to a challenging situation with low action-outcome predictability. We assessed cognitive effort readiness as the willingness to invest in cognitively demanding tasks. Individuals with a COVID-19 history exhibited a more pronounced reduction in cognitive effort readiness after experiencing experimentally induced action-outcome unpredictability compared to controls. These results suggest a generalization of perceived loss of action-outcome control among individuals with a COVID-19 history. These findings contribute to conceptualizing and assessing the long-term consequences of pandemic-induced emotional and motivational problems.

 

I feel sad for the kids who underwent that testing presumably in the hope that they were contributing to some meaningful research.

 

Emotional and motivational problems .... If only