Criticism of PACE trial from Daniël Lakens

[Esther12]

Via google translate:

To check everything up again, I also mailed my questions about Pace to Daniël Lakens, assistant professor of Applied Cognitive Psychology at Eindhoven University of Technology.

Lakens calls the Pace research dubious and meaningless. 'It is a very bad study, which must be completely ignored.' The Pace trial, he says, has not given us any knowledge. Sheets also gives an analysis of my mail to him: 'I get the feeling of your mail you currently do not really realize how problematic the study is, that the criticism is exaggerated, and that it is not too bad. I think the criticism of social media is exaggerated, but unfortunately the study must be ignored. '

https://translate.google.co.uk/translate?sl=auto&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=https://demonitor.kro-ncrv.nl/artikelen/hoe-ga-je-als-journalist-om-met-een-controversiele-wetenschappelijke-studie&edit-text=&act=url

Any Dutch speakers got a better translation?

Daniël Lakens is a psychologist who writes a lot about how to design and interpret studies, problems in psych research, etc. http://daniellakens.blogspot.com/

The article this is taken from is pretty annoying, and is part of an annoying series discussed here: https://www.s4me.info/threads/dutch...-objection-to-cbt-get.5521/page-2#post-101089

It sounds like Lakens may have saved it from being much worse.

For some journalists, it just doesn't seem to occur to them that they should take the time to examine the evidence, and pursue the truth independently of any authority figure they can defer to. I'm sure we've been spoiled by Tuller's work on PACE, but to have journalists do a whole series on the controversies around the treatment of ME/CFS, yet seem to have so little interest in the details of the dispute, is pretty frustrating.

 

[ME/CFS Skeptic]

Any Dutch speakers got a better translation?

To double check, I also e-mailed my questions about Pace to Daniel Lakens, university lecturer of applied cognitive psychology at Eindhoven University of Technology.

Lakens calls the Pace-study dubious and meaningless. ‘It is a very bad study, which must be completely ignored.’ The Pace trial, he says, has not brought us any knowledge. Lakens also gives an analysis of my mail to him: 'I get the feeling from your mail that at the moment you don't fully realize how problematic the study is, that the criticism is exaggerated, and that it is not that bad. I think the criticism on social media is exaggerated, but the study unfortunately has to be ignored.’

Google translate isn't that bad apparently.

Daniel Lakens might be another name for Tullers list? I like the fact that he uses the word dubious (the exact word in Dutch is almost identical: 'dubieus', so no nuance differences in translation).

 

[Rick Sanchez]

That the study is unblinded is inherent in the interventions studied. Of course, that can always give bias, but in this case the bias can work in two directions. If there were already many doubts about the effect of CBT when the trial started, then it could be that participants assess the effects of CGT too negatively. So in this discussion I am not convinced that non-blinding overestimates the effect.

This is absurd.

So to rationalize that there was not any bias, Olaf Dekkers, actually makes the assumption that many patients willingly enrolled in PACE back in 2005 despite being against CBT? Is this what the defense of PACE has come to? Isn't it obvious that a study is garbage if you have to make up wild assumptions to defend it?

And what does does ''there can always be bias'' even mean. There WILL always be bias. It's the responsibility of those conducting a trial to minimize this bias as much as possible. Why does he think medicinal companies spend gazillions on placebo-controlled studies?

 

[Esther12]

This is absurd.

So to rationalize that there was not any bias, Olaf Dekkers, actually makes the assumption that many patients willingly enrolled in PACE back in 2005 despite being against CBT? Is this what the defense of PACE has come to? Isn't it obvious that a study is garbage if you have to make up wild assumptions to defend it?

And what does does ''there can always be bias'' even mean. There WILL always be bias. It's the responsibility of those conducting a trial to minimize this bias as much as possible. Why does he think medicinal companies spend gazillions on placebo-controlled studies?

Wessely tried to make that point in his mental elf blog - I expect Dekkers got the point from there. Really, those sorts of concerns about bias just illustrate the problem of using subjective self-report questionnaires as a primary outcome. This article talks about 'not all outcomes showing benefit for CBT', but it fails to make clear that CBT failed to lead to an improvement for any of the trial's three more objective outcomes: the 6 minute walking test, a test of fitness, and employment outcomes.

PACE did collect data on expectations prior to treatment (those in the CBT and GET groups were given positive claims about the efficacy of their treatments after this data was collected during their treatment):

Adaptive pacing therapy

Treatment is logical: 134 (84%)

Confident about treatment: 114 (72%)

Cognitive behaviour therapy

Treatment is logical: 115 (71%)

Confident about treatment: 91 (57%)

Graded exercise therapy

Treatment is logical: 135 (84%)

Confident about treatment: 112 (70%)

Specialist medical care only

Treatment is logical: 79 (49%)

Confident about treatment: 65 (41%)

 

[Rick Sanchez]

Wessely tried to make that point in his mental elf blog - I expect Dekkers got the point from there. Really, those sorts of concerns about bias just illustrate the problem of using subjective self-report questionnaires as a primary outcome. This article talks about 'not all outcomes showing benefit for CBT', but it fails to make clear that CBT failed to lead to an improvement for any of the trial's three more objective outcomes: the 6 minute walking test, a test of fitness, and employment outcomes.

PACE did collect data on expectations prior to treatment (those in the CBT and GET groups were given positive claims about the efficacy of their treatments after this data was collected during their treatment):

Adaptive pacing therapy

Treatment is logical: 134 (84%)

Confident about treatment: 114 (72%)

Cognitive behaviour therapy

Confident about treatment: 91 (57%)

Treatment is logical: 115 (71%)

Graded exercise therapy

Treatment is logical: 135 (84%)

Confident about treatment: 112 (70%)

Specialist medical care only

Treatment is logical: 79 (49%)

Confident about treatment: 65 (41%)

It's interesting that Dekkers doesn't even seem to distinguish between subjective and objective outcomes at all. It's as if all outcome measures are created equal in his world. As long as the majority of outcome measures are positive, that's all that matters to him.

 

[Woolie]

Lakens is an important figure in the movement for the improvement of psychological science (SIPS). He has great statistical skills, and has proposed some very significant innovations that I think have great promise for improving research quality. There aren't that many people I can say I truly admire in the field of Psychology, but he's definitely one.

Usually, the SIPS folks concentrate on experimental studies - particularly social science and personality - they don't usually touch anything clinical or treatment-related. Their strength is in the statistical and methodological, there isn't a lot of expertise amongst the group about issues relating to theory, so I was quite pleasantly surprised that Lakens sees the problems with PACE.

 

[Esther12]

I dread to think how much worse this series would have been without Lakens being so critical of PACE. He even seemed to call them out for trying to impose their own narrative on the criticism of PACE... but even that didn't seem to make much difference.

I can be annoyed by people like Lakens not speaking out on the problems with PACE, and that group seeming to leave a lot of the heavy lifting to patients who lack any real authority, but I'm definitely very grateful for this intervention.

 

[Rick Sanchez]

I dread to think how much worse this series would have been without Lakens being so critical of PACE. He even seemed to call them out for trying to impose their own narrative on the criticism of PACE... but even that didn't seem to make much difference.

I can be annoyed by people like Lakens not speaking out on the problems with PACE, and that group seeming to leave a lot of the heavy lifting to patients who lack any real authority, but I'm definitely very grateful for this intervention.

Honestly.

I loved the

The Pace trial, he says, has not given us any knowledge. Lakens also gives an analysis of my mail to him: 'I get the feeling of your mail you currently do not really realize how problematic the study is, that the criticism is exaggerated, and that it is not too bad.

And the journalist is just like

Herp derp, smell you later Lakens, enough quotes from you. Got some BPSers to talk to. Thereby just clearly confirming Lakens misgivings. Honestly made me laugh.