Unwilling or unable? Interpreting effort task performance in myalgic encephalomyelitis/chronic fatigue syndrome, 2025, Kirvin-Quamme et al

[Karen Kirke]

Bit unfortunate though that the commentary was not published in the original journal and that Walitt et al. were not urged to respond to it. I thought this was the original plan: is it possible to give some background on what happened? Did the journal reject it?

Indeed. Yes, we submitted our manuscript to Nature Comms as a Matters Arising. When you’re submitting a Matters Arising, you’re “strongly encouraged” to first send it to the authors, and we did this in July 2024. We had an extensive email exchange with the NIH team in July and August 2024, before submitting our manuscript to the journal in September 2024. We were advised in October 2024 that “another Matters Arising highlighting these criticisms is currently under consideration at our journal” and the two articles would “reiterate arguments”, so ours would not be considered. Ours was not sent to reviewers at Nature Comms.

To date, we are not aware of any Matters Arising addressing the Walitt et al. study that has been published. If the “This article is cited by” section at the bottom of Walitt et al.’s paper is comprehensive, then no Matters Arising articles have been published. However, we do know that three Matters Arising had been submitted by July 2024, as Dr Walitt advised us of this in our email exchange.

Ultimately, our paper is stronger for having gone through a longer journey to publication, as it went through improvements at each step. The first time it got reviewed, it got accepted, which is heartening. And we think it's found a good home in Frontiers in Psychology.

 

[Karen Kirke]

An SF 36 physical function of under 30 is lower than usual in clinical cohorts, I think. IIRC, the pace trial was mid 30s.

Yeah, you're right. I'm going to be obnoxious and quote myself Kirke 2018!

GETSET's sample was higher-functioning than in previous trials, with a baseline mean SF-36 PF score of 49, compared to PACE's 38 and FINE's 30.1,3,4 Housebound patients have been reported to have a mean SF-36 PF of 17.5

 

[Braganca]

We had an extensive email exchange with the NIH team in July and August 2024, before submitting our manuscript to the journal in September 2024.

Dying to know their reaction to the article..

 

[Karen Kirke]

There was a bit of conversation above about an email Prof Treadway sent to a forum user in March 2024. In September 2024, John Bolecek wrote here:

In an email, Treadway told me it is “important to confirm that both groups were able to complete the easy and hard tasks at equally high rates.” The ME/CFS participants’ lower completion rate for hard tasks could suggest those participants made a “rational decision” based on their ability, rather than a difference in “effort preference,” he said. Furthermore, he stressed the importance of individually calibrating button pressing speed to distinguish preference from ability, which NIH failed to do.

 

[Utsikt]

There was a bit of conversation above about an email Prof Treadway sent to a forum user in March 2024. In September 2024, John Bolecek wrote here:

Imagine a physics paper that concluded that the speed of light is now 310,000,000 m/s, and it turned out that they did not calibrate their watches or properly measure the distance.

 

[Turtle]

Yeah, you're right. I'm going to be obnoxious and quote myself Kirke 2018!

Nothing obnoxious about that. When it's the right answer you're allowed to quote yourself.

 

[Sean]

Nice work, authors. Thanks.

I do like brevity.

Also, good you got in a mention of inadequate statistical power. That is an important limitation of the Walitt paper.

I'm looking forward to a day when people who are incapacitated by ME/CFS get effective treatment, rather than having to expend their meagre energy on refuting arguments like this.

Thank you.

It is a disgrace that patients have to do the medical profession's dirty work for it, and all too often also get abused and defamed for our troubles.

What the hell are the 'pros' being paid for?

 

[ME/CFS Skeptic]

We were advised in October 2024 that “another Matters Arising highlighting these criticisms is currently under consideration at our journal” and the two articles would “reiterate arguments”, so ours would not be considered. Ours was not sent to reviewers at Nature Comms.

To date, we are not aware of any Matters Arising addressing the Walitt et al. study that has been published.

Thanks for the explanation! So it's possible that a critique in Matters Arising - Nature Comms, is still in the pipeline? Perhaps somebody could write to the editors at Nature Comms to ask about this?

If it got rejected after peer review, it would be interested to know what the critique said and who wrote it (and if it could be published on a preprint server somewhere).

 

[andrewkq]

Thank you all so much for the kind words. It truly makes all the hard work feel worth it. Echoing Karen, I wish I had the bandwidth to reply to everyone individually, but I’ve been pretty short on spoons lately.

@ME/CFS Skeptic We were definitely pretty disappointed that Nature Comms refused to consider the article, but they weren't willing to budge.

Dying to know their reaction to the article..

Honestly, it was pretty similar to their public reactions to criticism. In the end, they just said 'it appears we will have to agree to disagree.' Hopefully the other Matters Arising submission will eventually be published, and Walitt et al. will respond to that. I’d love to see a direct response to our article, but I’m not holding my breath.

Thanks for the explanation! So it's possible that a critique in Matters Arising - Nature Comms, is still in the pipeline? Perhaps somebody could write to the editors at Nature Comms to ask about this?

If it got rejected after peer review, it would be interested to know what the critique said and who wrote it (and if it could be published on a preprint server somewhere).

It's definitely possible. The editor told us in November 2024 that the other MA submission "will likely be published," which was part of why they wouldn't consider ours. But I would've expected it to be out by now. Unfortunately our team doesn't have any more info. It wouldn't hurt for someone to reach out, but I got the sense that they weren't really willing to discuss upcoming publications. I'm also curious if anyone else on the forum knows more about it.

I think it's sufficient clear evidence for Wallitt et al to be asked to retract their original paper. Is there any move in that direction? Has Walitt been contacted for a response?

Our team isn't pursuing a retraction. Personally, I believe the evidence could justify one, but I don’t have much hope that those with the authority to initiate or enforce it will see this as more than a matter of “differing interpretations.” I know that a formal misconduct investigation reviewed similar concerns and concluded that no misconduct occurred. Based on our prior conversations, the NIH authors don’t appear interested in retracting the paper, despite being made aware of the issues. That said, I hope others may be willing to take up the baton, and that our article might help strengthen any future efforts toward retraction or broader accountability.

 

[Utsikt]

Personally, I believe the evidence could justify one, but I don’t have much hope that those with the authority to initiate or enforce it will see this as more than a matter of “differing interpretations.”

I don’t get this. If there are multiple valid interpretations of the data, only presenting one as if it’s the only valid interpretation is in itself clearly a false statement, and therefore grounds for retraction or amendment.

And in my opinion, the different interpretations are clearly not equal - yours is far more in line with the data and evidence. So Walitt et al have not just lied by omission, they have also lied about what the data actually shows.

 

[Cinders66]

I don’t know if this paper has been discussed anywhere in relation to NIH Intramural Study on ME/CFS , or if this has any importance, but in 2020 John Hopkins research was studying effort preference and fatigue https://www.hopkinsmedicine.org/new...r-pinpoints-brain-regions-that-may-control-it “ Participants whose motor cortex activity changed the least, in response to fatiguing exertion, were the ones who were most risk averse in their effort choices and were most fatigued. This suggests that fatigue might arise from a miscalibration between what an individual thinks they are able to achieve and the actual activity in motor cortex.”

 

[Utsikt]

I don’t know if this paper has been discussed anywhere in relation to NIH Intramural Study on ME/CFS , or if this has any importance, but in 2020 John Hopkins research was studying effort preference and fatigue https://www.hopkinsmedicine.org/new...r-pinpoints-brain-regions-that-may-control-it “ Participants whose motor cortex activity changed the least, in response to fatiguing exertion, were the ones who were most risk averse in their effort choices and were most fatigued. This suggests that fatigue might arise from a miscalibration between what an individual thinks they are able to achieve and the actual activity in motor cortex.”

Found a thread here.

 

[andrewkq]

I don’t get this. If there are multiple valid interpretations of the data, only presenting one as if it’s the only valid interpretation is in itself clearly a false statement, and therefore grounds for retraction or amendment.

And in my opinion, the different interpretations are clearly not equal - yours is far more in line with the data and evidence. So Walitt et al have not just lied by omission, they have also lied about what the data actually shows.

When I first began working on this project, I assumed the process would be straightforward. There were so many misrepresentations of the data and clear examples of questionable research practices that I thought simply pointing them out would make the need for retraction self-evident. But my impression now is that the bar for what qualifies as clear-cut misconduct is set incredibly high. As long as there’s even a shred of plausible deniability, the decision-makers default to framing it as a difference in interpretation rather than deception. In this case, the journal, the NIH, and the original authors are (as far as I know) the only parties with the authority to retract the study, and all have strong incentives not to retract something they endorsed or were directly involved in.

 

[jnmaciuch]

When I first began working on this project, I assumed the process would be straightforward. There were so many misrepresentations of the data and clear examples of questionable research practices that I thought simply pointing them out would make the need for retraction self-evident. But my impression now is that the bar for what qualifies as clear-cut misconduct is set incredibly high. As long as there’s even a shred of plausible deniability, the decision-makers default to framing it as a difference in interpretation rather than deception. In this case, the journal, the NIH, and the original authors are (as far as I know) the only parties with the authority to retract the study, and all have strong incentives not to retract something they endorsed or were directly involved in.

That makes sense, though it is disappointing for the state of science as a whole. I think quite a large percentage of published papers would be liable for skewing interpretation of the data and misusing various measurements, and publishers would hardly want to make themselves accountable for that.

Is this paper going to be listed on PubMed? I couldn't find it there when I searched. If it's on PubMed, it can at least be listed as a "Comment on" Walitt et al. so that a search for the latter might bring people's attention to this commentary as well.

 

[Snow Leopard]

In my opinion it’s not that if you’re a psychiatrist you magically understand it. It’s that the authors had decided on their conclusions before they got the data. They had decided how they wished to interpret the illness beforehand so they just picked out the result they could find that might back up that view and mostly sidelined the rest.

They also lacked knowledge on how to test things properly across numerous sub-studies, the study of TMS excitability was flawed in numerous ways that wouldn't have happened if they actually bothered to read any research written by exercise physiologists in the last 10 years about how to do this properly.

As Karen noted, even with regards to 'effort preference' the Walitt et al didn't really know what they were doing and deviated from standard practises and interpretations.

 

[Trish]

On the question of retraction, there is surely also the option of an addendum or editors or authors note or correction being added to a published paper.

That could be done in this case perhaps, with a note simply saying something like 'the authors wish to amend their interpretation of the EEFRT test. The conclusion that the test demonstrated altered 'effort preference' in ME/CFS is not supported once the ME/CFS participants' capacity to complete the harder key tapping exercise is taken into account. We therefore withdraw the statement .... from the abstract and .... from section.... '

But of course they won't do anything like that.