Use of EEfRT in the NIH study: Deep phenotyping of PI-ME/CFS, 2024, Walitt et al

I've almost got this test working properly, and got far enough to get a sense of it. On both macOS and Windows the older version of the standalone installer for PsychoPy 2022.1.4 (macOS | Windows) seems to work.

The experiment I used is defined here, though I think it has bugs. I haven't delved into the code to see if there are obvious errors to easily correct and it's a semi-visual programming language that's probably a bit clunky to reason/troubleshoot. This is not the code that NIH ran, but should be broadly similar - enough to at least give a taste of what a trial is like. It gives points rather than dollar values. However, it seemed not to follow my choice for A:Easy vs B:Hard for some reason, despite my choice being made within the allowed timeframe, which I presume is a bug. It didn't successfully do the trial runs either.

Regardless, in the trials I did, I found non-dominant fifth finger tapping on the space bar was locally fatiguing: more to the extensor compartment (EDM) than the specific hypothenar muscles (FDMB). Being non-dominant it's not as well coordinated as my right second finger and you're having to tap many more times per attempt on hard. I only ran about 12 iterations initially. FWIW, there was no central element of fatigue subjectively and I can attempt to objectively support that comment, as I've now spent 4 hours reporting imaging for a busy orthopaedic outpatient clinic (remotely from home). I type my reports and my right hand is normal but my left is still a bit fatigued, but I'm still making effective progress.

I chose easy tasks to give some recovery time before considering trying hard tasks again. This was not a difficult decision and barely required any thought. My dominant index finger/arm did not fatigue over the trials I did.

My internal "software program" was —

if not(fatigue_has_abated) {
try easy_task
} else {
if reward_is_adequate {
try hard_task
} else {
try easy_task
}
}

I don't know if this would be practical, but I suspect if they had EMG sensors, fNIRS, lactate monitoring, MR spectroscopy etc, it might be objectively clear that the non-dominant muscle was peripherally fatiguing with the hard task. (Despite the handgrip findings claiming no peripheral fatiguability)

This thread is long, so I can't recall if anyone else has given it a go. If so what was their experience? You can roughly mimic it without the software by tapping the space bar 30 times in 7s for easy (dominant 2nd); and 100 times within 21s for hard (non-dominant 5th).

---
Other EEfRT repos on GitHub that may be helpful are —

eEfRT ("refactor of EEfRT task from UGA")
OSL_EEfRT_psychopy ("A modified psychopy version of EEfRT used in Oregon Sleep Lab")
EEfRTapp
EEfRTapp
effrt_model ("Behavioural model for decisions during the EFFRT task in schizophrenia patients")
EEFRT_code ("Jim's code for analyzing Effrt task includes methods for aggregating data from raw files")

 

OK so one of the other repositories is the one we want. It sounds much closer to the NIH experiment and uses a monetary reward.
Use OSL_EEfRT_psychopy ("A modified psychopy version of EEfRT used in Oregon Sleep Lab")

So the simplified instructions if you want to try this yourself

1. Download the standalone PsychoPy app for Mac or Windows and install the app
2. Download the experiment file from GitHub (file name is 220817OSL_EEfRT.psyexp)
3. Launch PsychoPy.app/.exe and click the open folder icon at the top left of its toolbar
4. Choose the 220817OSL_EEfRT.psyexp that you downloaded at step 2
5. Click the green triangle "play" button in the middle of the toolbar
6. Follow the on-screen instructions

Note you will likely have to grant system access for the PsychoPy application to control your computer's screen. This is open source software in wide use and I have no security concerns and was happy to run this on my machines, but of course nothing is 100% guaranteed.

 

There seems to be downloads here at the Tread lab: Tools — TReAD Lab

But as there is a form for access for one, and then a download for one which seems to be staggered including an MRI type thing I'm not fully sure what is involved and what you'd need to run it computer-wise etc.

 

No I agree with what you are suggesting. Whilst typing with my index finger and swift button pressing of the space bar is straighforward, it isn't at all with my little finger which clearly isn't designed/intended for those types of tasks. But also I think that different keyboard set ups would cause big variation for me as I have small hands and sort of would have to fold my other fingers out of the way or hover my hand for the little finger whereas the heel can be firmly planted as a pivot for the index finger. I seem to be having to turn my hand to the side to do the little finger tapping so am either using my hand or when using finger I can tell it is engaging muscles further up the arm like outer upper arm too etc.

Over the years I've had all sorts of keyboards on work computers that have been quite high space bars and given that is the key near the edge and my hadn would need to rest on the desk that variation makes a big difference along with how creaky it is (I've had some that were pretty rickety sounding when you hit eg the space bar which seemed to have a cheap spring). Flat keys like a laptop keyboard,however and being able to tilt it at pretty custom angles might be quite different

 

I don't think we'll ever get too detailed answers on this because the patients were likely not fixated in a position, so there'll always be variations amongst the positions, even if one tried to tell them to start off similar and comfort is a rather subjective feeling as well. That also seems reasonable to me, why should someone not be able to move to a more comfortable position that is more natural to them?

If with fatigue you mean in-game fatigue, i.e. fatigue that occurs as part of doing the game (rather then the whole study or general fatigue), then it seems like this wasn't the major driver for the results of the study (at least for what one can say with data on such a small cohort where all kinds of variances naturally occur and different things can balance out without showing up on the data).

 

I wonder if blood flow to the brain had any impact on the results. There is at least some good evidence to suggest that, when upright, blood flow to the brain is reduced in pw ME/CFS. Surely this would impact decision making as time goes on. I assume no effort was made to determine if this played a role.

 

Just thought I'd mention as a comparison a finger pressing exercise which many of us will have done.
I had my eyes tested 2 days ago including a fields test for glaucoma. The optometrist stopped me before the test for one eye was complete because, she said, I was getting tired and it was affecting the result. After a short pause I resumed. I felt tired.

The test involved for those who might be unfamiliar, sitting on a chair with chin on a ledge and a button to press in my right dominant hand. The task ( slightly different from earlier ones I had done) was to watch a screen with one eye covered where small pinprick lights appeared occasionally and sometimes faintly across various parts of the screen alongside an intermittent noise. But the light did not appear each time the noise sounded.

The task was to press the button only when the noise was accompanied/followed by a light. It takes a lot of concentration and I did get very tired quickly, more from the concentration than the button pressing I think, tired enough for the test to be stopped temporarily. The test is then repeated with the other eye.
I think pwme will find the concentration required to do these tasks quickly drain the energy available.

Of course the NIH sample of pwme too are those who are mild/mod rather than mod/severe. Do we know the levels of 'effort preference' for mild/mod pwme, as compared to mod/severe pwme? Apologies if this has been covered. I have read most but not all the thread.

edit: NIH substituted at beg of last para for clarity

 

Received this today from NINDS, mods feel free to keep or delete as you see fit:

“Your emails to Avindra Nath, M.D., at the National Institute of Neurological Disorders and Stroke (NINDS) and others at the National Institutes of Health (NIH) concerning the use of the term “effort preference” have been forwarded to this office for reply.

You asked that the term “effort preference” be removed from the recently published paper on the NIH myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) research study. You are concerned that the term will be used by researchers and healthcare providers to wrongly psychologize people who have this disease. We understand your concerns and would like to explain this term and how it is used in the context of the Nature Communications article.

The following website was recently established to provide information about the NIH Intramural Study on ME/CFS:https://www.nih.gov/mecfs/nih-intramural-mecfs-study. This portion of the site provides helpful answers to frequently asked questions about the study: https://www.nih.gov/mecfs/nih-intramural-mecfs-study-qa.

In our day-to-day lives, the brain must decide how to expend its focus and energy on the tasks that are presented to it. The expenditure of energy is interpreted by the brain as effort. Not all tasks require the same amount of effort, with some tasks being easier and others harder. Tasks are also not equally valuable, with some tasks having more reward and others less. Effort preference is a measurement of the decisions the brain makes of how to utilize its energy based on difficulty and value of a task. We are often not aware that these processes are happening.

In this study, a series of tasks were given in which people with post-infectious (PI) ME/CFS and healthy volunteers had to choose between doing an easy or hard pushing task. The tasks were repeated many times, with different reward values assigned for successful completion. Persons with PI-ME/CFS were more likely to choose the easy task over the hard task compared to the healthy volunteers. This difference in task choice was not influenced by the number of tasks they performed or the value of the tasks. All the factors that did influence the choices are not known, and, as is the usual case, many are not conscious.

The full biological explanation for the observed difference in effort preference in PI-ME/CFS is not known. However, in the study, effort preference was related to differences in the brain, including decrease in catechol neurotransmitters and decreased activation of the right temporo-parietal junction, which plays a role in evaluating and initiating physical action. Therefore, the term does not describe a psychological difference in people with ME/CFS and people without ME/CFS; it describes an actual abnormal finding in how the body’s neural system is supposed to function. Further analysis of the relationship between effort preference and these biological observations is planned.

We hope this information is helpful.”

 

It's not. There is no measurement here. There is only a misapplied test tortured to fit a pre-achieved conclusion that invents a bullshit term out of a sample so tiny that a single tester creates the significance they want. What ridiculous nonsense that all they can think of responding is explaining to you what you clearly understand. And as they say, they plan to do more of this, so they are unlikely to back down.

So now I think we have to consider whether the NIH is simply not a serious organization, and whether it would actually be preferable for this funding to stop. Because they are clearly doing more harm than good. Funding outside grants is one thing, but internally they are clearly not capable, and we can only expect more of the same from Walitt's interoception crap.

Not that we can stop them, but it's likely not worth bothering to ask for more in the future. I guess that was always the plan: fail miserably so that we stop bothering them entirely, having sent the clear message that they have no intention of doing anything useful, even accidentally, which they will then use to argue that they tried, but it's just not solvable.

 

> Persons with PI-ME/CFS were more likely to choose the easy task over the hard task compared to the healthy volunteers.

But if you can't do the hard task then there is no choice. It's the wrong verb.

It would be more accurate if they had said something like "Persons with PI-ME/CFS were less likely than healthy volunteers to do the more physically demanding task." And then it's almost reduced to a redundancy: "sick people can do less than healthy people".

 

well, it's certainly worth blogging about but haven't gotten around to it yet.

 

This response is infuriating (the whole message, not just this quote). Can someone correct me if I'm wrong here: they keep claiming in their communication about the study that effort preference was related to decreased activation in the right TPJ but as far as I can tell, they didn't actually test for this relationship. There's no statistical test reported that shows that these are related. They are just asserting that they are because they found both of them in the same cohort which is the most basic misunderstanding of science one can have.

This is what frustrates me the most. They are going to spend a decade dedicating all of their funding to effort preference research and they just won't publish the results whenever they don't find a relationship.

 

Are you not satisfied with the highly valuable definition of "Over a game of 35 rounds to opt out 2 times extra to take a hard task in a testing procedure in the setting of a clinical trial where sample sizes are small and variations high and where half of the people with ME/CFS actually opt to take hard just as many times as the median HV person and as such have no such preference, but where half of the healthy controls show exactly this same preference as well (i.e. being below the HV mean) and as such is a property that is equally common in both cohorts"?

 

Yeah I think they would likely push back on it being an arbitrary choice. It could be something to look for in the other EEfRT studies, whether others pick a cut point for what's considered an adequate completion rate.

 

It stands for Generalized Estimating Equation, it's the statistical model that Wallit used to analyze the EEfRT data. They're pretty complicated and much of the nuance goes over my head, but they allow you in this case to model the data at the trial by trial level, so each participant contributes ~50 observations to the model rather than each participant only contributing 1 summarized observation.

It's necessary for answering the complex questions that Treadway designed the task to measure (does the relationship between anhedonia scores and hard task choice behavior change based on probability of reward and reward value) but I'm actually not convinced that they are necessary to answer the effort preference question (do the two groups differ in their overall percentage of hard task choices). I'm pretty sure you can (and likely should) test this with a simpler test like a t-test or regression. But that's a more nuanced critique I'd need a statistician to review.

 

The lab I worked in ran 1 participant per day and the visit took the full day. We would review participant data quickly at the end of the session to make sure that there weren't mechanical issues screwing up the data, which is pretty common, but we would never flag someone as invalid based behavioral alone in the absence of a mechanical issue. It is considered bad practice to review the data together (i.e., run stats and compare participants) before all of the data has been collected (though not everyone subscribes to this). We would always pilot a new task to make sure it worked correctly and to make sure that there were implementation issues, which can save you from having to make adjustments once actual data collection has begun.