The pervasive problem with placebos in psychology: Why active control groups are not sufficient..., 2013, Boot et al.

Full reference:

Boot, W. R., Simons, D. J., Stothart, C., & Stutts, C. (2013). The pervasive problem with placebos in psychology: Why active control groups are not sufficient to rule out placebo effects. Perspectives on Psychological Science, 8(4), 445-454.

Link to fulltext:

https://sci-hub.se/10.1177/1745691613491271 (note this is a scihub link, so may needed to be updated from time to time).

Abstact:
To draw causal conclusions about the efficacy of a psychological intervention, researchers must compare the treatment condition with a control group that accounts for improvements caused by factors other than the treatment. Using an active control helps to control for the possibility that improvement by the experimental group resulted from a placebo effect. Although active control groups are superior to “no-contact” controls, only when the active control group has the same expectation of improvement as the experimental group can we attribute differential improvements to the potency of the treatment. Despite the need to match expectations between treatment and control groups, almost no psychological interventions do so. This failure to control for expectations is not a minor omission—it is a fundamental design flaw that potentially undermines any causal inference. We illustrate these principles with a detailed example from the video-game-training literature showing how the use of an active control group does not eliminate expectation differences. The problem permeates other interventions as well, including those targeting mental health, cognition, and educational achievement. Fortunately, measuring expectations and adopting alternative experimental designs makes it possible to control for placebo effects, thereby increasing confidence in the causal efficacy of psychological interventions.

 

This lays out the problem pretty well.
The suggestions for alternative designs make sense but are maybe bit pedestrian.
There is a suggestion that maybe somehow expectation bias could be 'factored out'y measuring it. That seems to me unlikely to be realistic. The whole problem with expectation bias is that it will move its goalposts whenever you try to ring fence it - pretty much as you are hinting for PACE and subjective outcomes?

 

Increasingly the bulk of CBT research in relation to ME has abandoned the idea of active controls.

It feels like the psycho ME/CFS researchers (deliberate ambiguity of terms) have abandoned completely any attempt to eliminate bias as they have moved away from even attempting to use objective measures and from meaningful controls. One explanation for this is that they are not interested in progressing science but rather perfecting a research design that will give the answers they want.

 

The explanation

 

One answer is that they don't have family/friends who are affected by ME/CFS, so they don't care (was going to use another word). To them, this is just about getting funded --- for (flawed) research --- a friend who works as an administrator in a university says --- if you can't be part of the solution, then make money out of the problem!

 

I'm not sure how this paper is relevant to trials of CBT/GET, such as the PACE trial.

The problem isn't "expectation"/expectancy effects. Indeed, initial expectations of APT were as good as CBT or GET. The problem isn't necessarily placebo effects either, but biases in reporting (response biases). A placebo effect assumed there is a real health benefit and many CBT/GET practitioners would argue that this is beneficial regardless of how it was induced.

 

Interesting article. Good to see some common sense questioning in the field of psychology. Thanks for posting it here.

It seems that most reviews don't see a lack of a control condition as a potential source of bias. They simply mention the control condition briefly and don't discuss problems with interpretation (such as whether differences are due to the intervention or non-specific effects such as expectancy, contact with healthcare professionals, etc.).

A good example of this is the draft report on the management of ME/CFS commissioned by the CDC.

 

The problem IS expectancy effects, @Snow Leopard (among other things). They are the primacy mechanism that gives rise to the response bias. As @Trish says, they really pumped up CBT and GET during the trials to create huge expectancy effects.

 

I disagree that it is the primary bias, though it is certainly one of the possible biases.

"Expectancy" is often discussed as a mechanism for inducing the placebo effect. What is often implied, is that is there are real (and clinically significant) biological effects that are induced due to mental beliefs. (beyond the demonstrable transient effect of endorphins, or the similar effect to diminish nausea, or trivial increases in dopamine) Hence they'd argue that this is a success regardless of the fact that it was due to "expectancy" effects.

 

This article is about the problems of controlling psychological research, including how ‘active controls’ may or may not help. These are issues that the original PACE experimental design was in theory set up to address, but undermined by their selective reporting and outcome shifting, over and above any deliberately introduced bias through newsletters and subject information sheets, telling them that GET/CBT are proven interventions endorsed by the Prime Minister no less.

My point was the original PACE design paid lip service to the issues this paper also examines, but more recent studies by BPS researchers in ME/CFS gave up even the pretence, with passive controls being used, with no attempt to compensate for any form of bias.

 

I think expectation bias is a useful term to cover a range of things including the problem addressed by this study. The patients were said to have as high expectations of APT as CBT or GET but once in the trial expectations would have been manipulated.

I do agree that there is a sense that the authors may not appreciate just how many problems are involved. They use expectation in video games as a model and there is a slight sense that they think this suggests that expectation can be measured and dealt with. Real life human interaction with vested interests involved is much murkier than that.

 

many CBT/GET practitioners would argue that this is beneficial regardless of how it was induced - But it's not a real benefit since it does not improve your health medium/long term. E.g. it wouldn't lead to a medium/long term increase in activity/ability to function normally - so is it really just a card trick?

 

Can't it also work vice versa? The transient effects make people believe that the placebo really works so they expect that they need just more of the treatment and that in turn makes them hopeful and let them revise the severity of their symptoms (even if they only experience repeated short term relief).

I think it's important to be aware of that short term effects and that, in unblindable trials, it's necessary to control for those effects by long term monitoring even if objective measures are applied (e,g, cognitive tests) or are objective measures are applied to back up subjective outcomes like pain (measures of how symptoms impact people's daily life/ movement/ activities).

It's interesting that (if I skimmed properly) the paper doesn't discuss that possibility but instead suggests that expectation bias can be controlled for by measuring it / additional statistical work. (Perhaps because they are more interested in short term interventions or more generally, in "increasing confidence in the causal efficacy of psychological interventions"?)

Edited for clarity.

 

The most pervasive problem in psychology trials of ME is not having a control group with chronic illness. Any effect they find could be because of the effect of being constantly unwell. Using just sedentary controls only looks at one aspect of life so is seriously inadequate.

edited to add

Years ago, in the early days of CFS, a trial was set up to see if people with CFS were more likely to assign symptoms to a physical cause than healthy controls. They explicitly said they did not use a chronic illness as a control because people with chronic illness are known to be more likely to assign symptoms to a physical cause!!!

 

Indeed and the other aspect of using sedentary controls are these people are sedentary by choice. That may also have an impact on how they think & respond to questionnaires.

 

That's true. But I think does only apply for etiological research (epidemiology and pathomechanisms) not in clinical trials, i.e. reserach on treatments/ interventions as discussed here?

 

Ha.

You ask them if they are ok with magical healing crystals benefitting from "expectancy effects", you will find a very different answer.

I've been looking at the "endorphin release" literature, and its pretty dodgy. It is also very short-lived (a few hours at most). So no, not going to fly here.

 

Yes, but the investigators only interpreted the expectations before people started the interventions, so they conveniently didn't include that the interventions themselves incorporated encouragement of positive expectations.