Re-evaluating randomized clinical trials of psychological interventions: Impact of response shift on the interpretation of trial results, 2021, Verdam

Andy

Senior Member (Voting rights)
Abstract
Background
Effectiveness of psychological treatment is often assessed using patient-reported health evaluations. However, comparison of such scores over time can be hampered due to a change in the meaning of self-evaluations, called ‘response shift’. Insight into the occurrence of response shift seems especially relevant in the context of psychological interventions, as they often purposefully intend to change patients’ frames of reference.

Aims
The overall aim is to gain insight into the general relevance of response shift for psychological health intervention research. Specifically, the aim is to re-analyse data of published randomized controlled trials (RCTs) investigating the effectiveness of psychological interventions targeting different health aspects, to assess (1) the occurrence of response shift, (2) the impact of response shift on interpretation of treatment effectiveness, and (3) the predictive role of clinical and background variables for detected response shift.

Method
We re-analysed data from RCTs on guided internet delivered cognitive behavioural treatment (CBT) for insomnia in the general population with and without elevated depressive symptoms, an RCT on meaning-centred group psychotherapy targeting personal meaning for cancer survivors, and an RCT on internet-based CBT treatment for persons with diabetes with elevated depressive symptoms. Structural equation modelling was used to test the three objectives.

Results
We found indications of response shift in the intervention groups of all analysed datasets. However, results were mixed, as response shift was also indicated in some of the control groups, albeit to a lesser extent or in opposite direction. Overall, the detected response shifts only marginally impacted trial results. Relations with selected clinical and background variables helped the interpretation of detected effects and their possible mechanisms.

Conclusion
This study showed that response shift effects can occur as a result of psychological health interventions. Response shift did not influence the overall interpretation of trial results, but provide insight into differential treatment effectiveness for specific symptoms and/or domains that can be clinically meaningful.

Open access, https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0252035
 
Honestly, of all the factors that make this field of research a muddy mess, "a change in the meaning of self-evaluations" is probably the least of all problems. It's not the patients who change their minds, that rarely happens, it's that everything is arbitrary and detached from reality but that's never taken into account, with as little reliability as movie reviews, probably even less.

But focusing on the least of all problems is a convenient way of ignoring the entire herd of elephants in the room.
 
"a change in the meaning of self-evaluations" is probably the least of all problems.
I think it's still a big problem, as Graham's video and our discussions have shown. If you think your pain is pretty bad, and then you are told that actually it's not bad, it's actually pretty normal, you are likely to change your frame of reference.

I haven't read the paper yet, but I don't see how these researchers could have retrospectively reliably separated out the impact of a change in a frame of reference and the impact of either a real therapeutic benefit or a natural change due to time in trials. Isn't the difficulty of separating out the contributions of the various types of change in self-reported variables exactly the problem?
 
Note,
Verdam et al. said:
Although there is some previous research on the occurrence of response shift in measures of fatigue (cf. [3739]), those studies mostly pertain to fatigue in cancer patients and response shift induced by radio- or chemotherapy; and thus their results are difficult to compare to the results of our study, and not informative with regards to the interpretation of response shift due to psychological intervention.
 
If you think your pain is pretty bad, and then you are told that actually it's not bad, it's actually pretty normal, you are likely to change your frame of reference.

This happened to me. All it achieved in the beginning was giving me an inferiority complex. It didn't make the pain any easier to cope with.

I should add to this, that it also made me believe that a big chunk of the medical profession is malicious and really nasty, a belief that I've never really had much reason to alter throughout my life.
 
Back
Top Bottom