Performance Validity and Outcome of Cognitive Behavior Therapy in Patients with Chronic Fatigue Syndrome, 2021, Knoop et al

Abstract
Objective:
There is limited research examining the impact of the validity of cognitive test performance on treatment outcome. All known studies to date have operationalized performance validity dichotomously, leading to the loss of predictive information. Using the range of scores on a performance validity test (PVT), we hypothesized that lower performance at baseline was related to a worse treatment outcome following cognitive behavioral therapy (CBT) in patients with Chronic Fatigue Syndrome (CFS) and to lower adherence to treatment.

Method:
Archival data of 1081 outpatients treated with CBT for CFS were used in this study. At baseline, all patients were assessed with a PVT, the Amsterdam Short-Term Memory test (ASTM). Questionnaires assessing fatigue, physical disabilities, psychological distress, and level of functional impairment were administered before and after CBT.

Results:
Our main hypothesis was not confirmed: the total ASTM score was not significantly associated with outcomes at follow-up. However, patients with a missing follow-up assessment had a lower ASTM performance at baseline, reported higher levels of physical limitations, and completed fewer therapy sessions.

Conclusions:
CFS patients who scored low on the ASTM during baseline assessment are more likely to complete fewer therapy sessions and not to complete follow-up assessment, indicative of limited adherence to treatment. However, if these patients were retained in the intervention, their response to CBT for CFS was comparable with subjects who score high on the ASTM. This finding calls for more research to better understand the impact of performance validity on engagement with treatment and outcomes.

https://www.cambridge.org/core/jour...gue-syndrome/76FE3A1565C22165F1243522C9D53CD9

 

Locked psychiatric wards maybe?

 

"Performance validity" is a concept from neuropsychology. It refers to tests or analyses that are designed to "catch out" people who are faking, or just not putting in the required effort to give a score that reflects their ability. Knoop and chums are trying to justify chucking people out of their CBT and GET trial on the basis that their "performance validity" scores indicate they are not sufficiently engaged. The idea is that their lack of effort will mean they might not engage in the therapy, and might result in them not responding to CBT or GET. Obviously, the authors didn't find that, but of course, they conclude that "further research is needed, so that we can find the outcome we want in some other way."

While I'm sure that there are people who fake disability or try to exaggerate their deficits for monetary gain (perhaps especially in the US). It is very difficult to formulate a single test that can "catch them out". You have to pick something that can't possibly be genuinely impaired in your population. Else the whole enterprise is %$#@ed. The test they used here is a short-term memory test, which could very well be difficult for those with MECFS. So assumption not met. So people scoring low on this measure are probably those with more severe cognitive impairment. Hence the higher dropout rates.

They don't seem to understand that performance validity is an inference that you make across a wide range of different performance measures, not a thing you can assess with just one test.

 

Just one more thing they don't understand!

 

yes (I originally read it as 'restrained')

 

They are looking for an excuse to dump some people from their participant pool, so they can make their outcomes look better.

They didn't find it here, but rest assured, they will keep looking till they find something that, for whatever reason, allows them to dump people in a way that enhances their results. Its like a more sinister version of p-hacking.

 

I did once stop and refuse to continue a test. I was doing a treadmill test for checks on my heart. I think medical staff were looking for evidence of blocked coronary arteries, but that wasn't what was wrong with me. (It took a different hospital and nearly another year for the problem to be identified and fixed. The treadmill test was being done by the second hospital to try and identify the problem, but the hospital who found the cause of my problems was the third one.) The chest pain I had while doing the treadmill test was so intense that I just couldn't cope with it, so I stopped. This wasn't the first time I'd had such chest pain. I dread to think how that got written up in my medical records. But if I had continued I think I might have gone into heart failure.

 

Denominator-hacking? Cherry-hacking?

 

This

 

I'm not sure whether this paper was freely available at the time but it is now:
https://www.cambridge.org/core/serv...atients-with-chronic-fatigue-syndrome-div.pdf