Use of EEfRT in the NIH study: Deep phenotyping of PI-ME/CFS, 2024, Walitt et al

That one of applicability of subject sample is interesting as an old-chestnut we've seen highlighted as an issue eg in the Nice analyses for the 2021 guideline of therapist-delivered.

It is a really relevant issue for ME/CFS to get sorted. It does feel like it needs rules to be considered and ironed out.

Whether misdiagnosed or not, when does 'spontaneous recovery' also mean exclusion is appropriate is another question - made even more pertinent when yes the % isn't going to always be covered by probability or something you can glance off as 'x% of people recover' (I thought it was less likely if they've been ill for a longer time), perhaps more likely if the nature of the trial itself is such it would be at best a daunting consideration for those with more experience and worse illness (so knew where it might lead them due to bitter experience)?

 

I think this one is one of the key points, the invented 'metric' in itself. WHich of course was then combined with the hand-grip test as if it wasn't the invented one but the validated one.

 

Filled out the HHS Office of Inspector claim, as I hadn’t done that before (did AIRIO, HHS Office of Integrity, and submitted FOIA)

 

If anyone lives in the US wants to contact congress people, this site will tell you all your representatives.

 

I reached out to all of my elected officials right after the study release but the honest truth is that there’s not much they can really do under the current separation structure of NIH and Congress. I’d love to be wrong but that was my takeaway and also a reason that there’s a restructure proposal in place from one party in Congress at the NIH (amongst other concerns relating to oversight concerns)

 

I'm still reading through the 34 pages, but I noticed this in Treadway's README if you download the 2022 software.

"It should be noted that a proportion of subjects may exhibit a pattern of choosing all Easy or all Hard choices (aka “single responders”). Single responders must be treated with care, as they can exert significant influence on primary questions of interest. While we generally recommend against exclusion of otherwise valid data, we recommend that primary analyses of interest should be run with and without single responders to ensure that they are not responsible for creating (or suppressing) an important effect in the data.

 

That's wild. It sounds like they're just saying choose whichever option returns the largest desired effect.

Do they go on to explain how to decide whether or not to include single responders?

 

Makes sense. The sensitivity analysis carried out by people upthread, with the single responder included, showed that the effect was changed (in favour of the null hypothesis). This means that the main analysis was not robust.

 

Good catch. This reads a lot like poor design being blamed on users. The design of this test is very poor, it doesn't even apply to this problem, and how the NIH used it is even worse. They even freaking excluded the only player who actually followed the optimal strategy.

But no, it's the users who are wrong. Pffft. Unserious clown show.

 

Apologies if this is already in the thread. But I didn't realize until you said it RVALLEE that Control F actually tied for the highest amount of money of any of the heathly controls!

So I looked at the data with control F in and this is what I see. The numbers at the bottom are sums HV/17 and ME/15. Does anyone know why my ME number does not match Jeannette's? (HV is off bc Control F is still included) It is a PIVOT table, ID, using the SUM of "Value of Reward" and is filtered by "Reward granted Yes Is 1"

 

Also, here's a screen from the latest version stating you should choose your hard tasks carefully.

 

Perhaps because the 4 test trials were still included? Those have a negative trial number and should be removed in the analysis.

There is another issue that the rewards that were given are likely those that have a value of 1 for 'Reward_Granted_Yes_is_1' and for 'Successful_Completion_Yes_is_1' (instead of only 1 for 'Reward_Granted_Yes_is_1'). Explained here:
https://www.s4me.info/threads/use-o...s-2024-walitt-et-al.37463/page-33#post-537834

 

OK. Those 4 removed. And required both 'Reward_Granted_Yes_is_1' and for 'Successful_Completion_Yes_is_1' to be one leaves us with this:

Control F is not the highest paid HV, they go to the lowest paid at $22.74

 

Without having looked at what is happening, I think you are not using the correction notion of highest paid. Highest payout would refer to your end pay out, not the total sum of monetary gains. That is to maximise payout you tend to complete more tasks that have a high pay out since you only end up getting the reward of two won games from what I recall (that is if you're able to complete hard tasks, otherwise your strategy changes), i.e. you don't want to accumulate low monetary rewards and in particular never want to complete a task if your successfully obtained reward average is higher than the reward assigned to the current task and if that average includes 2 successfully rewarded trials. The details of different strategies and why it can be viewed as reasonable to exclude HV F were discussed earlier in the thread.

 

I am getting the same numbers at ME/CFS skeptic, but excellent point. Is there any data on the actual payout per person?

 

No I don't think so but since the selection of 2 rewards (that were actually paid out) was based on luck, we can look at the mean reward value and see how each participant maximised their actual rewards.

This is what I got after a quick calculation. Healthy volunteer F got by far the biggest mean reward.

 

Yes, the earnings for each of the 31 participants whose data were analysed are in the data available through MapMECFS. The excluded HV F does not feature as their data were not analysed. The highest earner (not including HV F) got $7.32.
In the paper they state

 

In part 2 of Jeanette’s blog, this bit (that @forestglip and @bobbler mentioned above) stuck out:

I don't think this is right. Participants won actual rewards, not virtual rewards, and the amounts won were much lower:

Edit: As @Theresa pointed out, Jeanette is adding up potential rewards, not actual winnings.

Healthy volunteers won an average of $3.83, patients won an average of $2.93. (Calculated from the EEfRT dataset in the neurophysiology subset of the data from the NIH study available via mapMECFS.org, accession code given at the end of the paper as: https://www.mapmecfs.org/group/post-infectious-mecfs-at-the-nih, see reference 2 below).

Patients did not win more than healthy volunteers on the EEfRT.

It's abundantly clear to me on a daily basis that I am not winning.

1. Walitt, B., et al. “Deep phenotyping of Post-infectious Myalgic Encephalomyelitis/Chronic Fatigue Syndrome.” Nature Communications. February 21, 2024. DOI: 10.1038/s41467-024-45107-3

2. Mathur, R.* & Carnes, M.U.*, et al. mapMECFS: a portal to enhance data discovery across biological disciplines and collaborative sites. J Transl Med 19, 461 (2021). https://doi.org/10.1186/s12967-021-03127-3

Edited to add sentence starting "Edit".

 

Is the virtual reward possibly what the participants theoretically won based on all the tasks while the actual reward is the payout selected from two random tasks? (I might be misunderstanding it)