Use of EEfRT in the NIH study: Deep phenotyping of PI-ME/CFS, 2024, Walitt et al

Great, an economics task where your data is removed if you utilize the economically optimal strategy. How this kind of test can be seen as useful is beyond me.

 

I think it is worth while to look at issues at all levels. I tend to focus on the broad brush but I think the detail is equally important.

One simple thing I am wondering about:
The paper proposes the general interpretation that peripheral tissues are essentially normal and that the problem is that central signals either prevent activities or make them seriously unpleasant. That to me is a reasonable idea but Nath's comments suggest confusion between two different types of central signalling.

In flu, signals due to cytokines either make actions impossible (you actually fall over if you try to stand) or make actions induce unpleasant symptoms like nausea or 'fatigue'. My understanding is that in ME the situation is similar, in both forms, except that whereas in flu unpleasant symptoms occur within a couple of minutes (plus maybe more payback after an hour or so) in ME the reaction is delayed and can be very long lasting.

What the effort tests test for, however, is either unconscious conditioned responses or deliberate conscious responses, acquired because of experiencing the 'flu signals'. These responses would constitute a second and quite different set of signals. (There may or may not be valid evidence for such responses.)

We have two sets of signals we might call flu signals and pacing signals. What is blindingly obvious with flu and Covid is that flu signals cause the pacing signals. But suggesting effort task data provide an explanation implies that pacing signals are responsible for flu signals, or just are flu signals, which is unjustified.

Put differently, whether or not PWME make different effort choices provides no explanation for being disabled.

 

As far as I've seen the generation of the reward money is purposely not told to players so that optimal strategies are harder to find (this was written in a different EEfRT paper), furthermore I've never seen the distribution specified in any of the papers. In this case a truely optimal strategy would envolve estimating this distribution on the space of all possible distributions on ([1.24,4.30]), whilst also accounting for multiple other things (estimating your own abilities, what kind of rewards have already been paid out to you, what combinations have already appeared etc). The complexity of this will be far too high for any person playing the game. Something that the participants might know is that "Each level of probability appeared once in conjunction with each level of reward value for the hard task." (I believe this should probably say "at least once" and I would assume level refers to the 3 levels in to which the rewards are typically categorised), so participants can make some good decisions based just on that, as long as they have a good memory and have fully understood everything (which most pwME probably won't have in this setup).

However, the examiners and this is what other studies have done as well, might still exclude data from outliers that use "certain" strategies, especially those focused on maximising pay-out, by only focusing on completing hard and high pay out tasks, if even if they are not truely optimal in the classical sense. In this case the details start becoming important. How exactly and in which setting were the rules explained to the participants and on what kind of basis were exclusions made (pre-defined rules, post-hoc analysis focused only on one part of the analysis), this includes looking at whether other participants were not excluded even though they could have been based on different rules (for example what the one study that @Murph cited calls "non-systemic responding"), which in this case could imply an exclusion bias.

 

Another simple point is that if an experiment reveals the cause of some pathological state, you expect to see more or less complete separation of populations. If it really is the single cause, which is quite often identifiable, the separation must be complete.

Correlations and slopes and box plots and p values are interesting if you are just looking for a clue as to what sort of cause may be involved. But the interpretation in this paper seems to be that a cause has been found. Nothing of the sort.

 

Whilst this might be correct argument from a biomedical stand-point and should be more clear to researchers and reviewers of journals in general, I fear it might be harder to make this argument in the context of responding to the EEfRT part of the study. That is because the results of the EEfRT seem to be somewhat unrobust in all kinds of different trials and yet others have often drawn some rather strong conclusions themselves, so Walitt et al might be able argue that their drawing of conclusions is consistent with the literature, even if the literature itself is inconsistent.

I currently feel like focusing on the "unrobustness" of the results in combination with an wrongful application of the test could be a sufficiently strong argument, without having to get into a debate on whether any of the results, which these authors and many others have often presented, have ever meant anything to begin with.

 

Yes, I was not particularly suggesting using this as an argument in a critique. It was more just suggesting reminding ourselves how far away from any rational argument their interpretation of data like this is.

 

You seem to have it right, Trish.

Trouble is if you have a field of study where the ratings for a Michelin Star for a cheeseburger are judged by cheeseburger producers you are likely to get quite a lot of MacD*** outlets.

 

I agree. During activity started from a well rested state, my fatiguability symptoms are similar to those of 'flu, but aren't as severe at the outset.

Activity attempted during a PEM state produces symptoms just like 'flu. The pattern includes that odd stage during recovery, where an activity feels possible whilst at rest but proves completely unfeasible as soon as it's attempted.

 

this trips me up no end. In fact i'd say it accounts for the majority of my experience when not in PEM.

I seem to be a very slow learner

 

Me three.

 

I think I’m not unusual in having both fatiguability and PEM separated by time.

I know some researchers like to roll both concepts into one. But they are absolutely not the same!

The fatiguability is physically limiting right then in the moment, but it is the delayed PEM that concerns me more regarding my ongoing well-being, and it’s that which I monitor for future planning. (IME Slow burn fatiguability is the most PEM inducing.)

So in terms of this finger tapping exercise, I think it very likely that some pwME will be physically unable to complete harder tasks due to the fatiguability.

The weighing up of the PEM is, of course, another whole layer of complexity for pwME to consider as the tests continue.

Very different to the situation for the HV’s who don’t have to deal with either issue.

The test is therefore not comparable between the 2 groups. At least not as a measure of “effort preference”!

 

Oh and of course ongoing PEM can escalate what might be an okay activity when rested, into an activity that induces both more rapid fatiguability AND worse PEM.

 

precisely

 

As many of us have now said a bunch of different ways - its not an appropriate or fair comparison.

You cannot do a test for 'effort preference' when one set of subjects has a range of negative consequences to factor in when doing it (both in-real-time fatigability & the later 'payback' of PEM), and the other set has no negative consequences to factor in whatsoever.

Thats seems so obvious, it seems absurd to miss it

You cannot compare motivation or anything else when one set has only reward to deal with, and the other set have both "reward" & punishment. Its a game for one and a major life affecting set of calculations for the the other!
(and thats assuming the "+punishment" set are not too impaired to think clearly to calculate it in any case - which is unlikely!)

So the whole thing is just a complete joke AFAICS

(minor editing for sense & typos)

 

What ever triggers immediate fatigue may contribute to subsequent PEM, and indeed this rapid fatiguability may be more noticeable or have more rapid onset if already in PEM, but I agree my subjective experience is the two are distinct.

If not already in PEM or experiencing concurrent ’rolling’ PEM, if I am experiencing rapid fatiguability immediately stopping the activity and resting will see the developing symptoms fair promptly dissipate, which is different to PEM which is delayed and can paradoxically continue to worsen even when resting. Also initial symptoms of rapid fatiguability are more likely to be directly related to the activity, such as muscle ache or tremor, where as the symptoms in PEM may include those related to the activity but also includes other not directly related symptoms.

I have similar experiences with sensory hypersensitivities, for example if a smell or a noise starts triggering adverse symptoms escaping the stimulus sees an immediate and reasonably prompt diminution in those symptoms, but persisting in that situation sees the symptoms worsen and can also lead to subsequent triggering of PEM.

 

They either don't understand that there is such a strategy, or didn't consider it relevant. The simple use of hard:easy ratio as the final outcome says it all, since doing the test as they intended it to be used doesn't maximize earnings, and that's discounting the fact that the earnings are so pitiful that the real motivation isn't about rewards.

That Nath replied that using the optimal strategy is "not following instructions" says it all, when in fact this tester is the only participant who did it the right way to maximize their earnings, the only player to actually play the game, rather than do what the researchers expected.

"Don't hate the player, hate the game" was invented for that.