The study has now made it onto the SMC website -
Reanalysis of the PACE trial
http://www.sciencemediacentre.org/reanalysis-of-the-pace-trial/
Thanks for posting this,
@Eagles.
Actually, its not too bad as a defence of the PACE trial - given that its tough to make an argument here to save it. Some of the points are actually reasonable:
The authors in this paper seem to have selected the most extreme analysis to make their point: for example by making adjustment for 6 comparisons where 3 or 5 comparisons are also described, and focusing on 52 week data only.
The first bit is true - we did use a tough method of correction. We could have used 3 comparisons.
However, the second bit of that sentence is blindingly stupid - 52 weeks is the primary endpoint of the trial. It's the
primary endpoint. It's the PRIMARY endpoint.
The authors rely heavily on p-values and thresholds for statistical significance when reporting results; this is quite an out-dated approach and information about the magnitude and precision of treatment effects is missing in most cases.
We could have presented odds ratios (obviously not confidence intervals, they're only for continuous data). But this isn't what you said you'd do in the protocol. We did
what you originally said you'd do. That was the whole point.
The authors have made little attempt to uncover the reasons for protocol deviations in the PACE trial or the point at which they were made; trialists could have been invited to comment.
Well, I don't think that's defensible. We carefully examined every published statement made in justification of the changes. No more can be expected, especially not from such a hostile group. Look what it took to get that little data sample we worked on - imagine trying to get them to actually answer our questions!
The new paper may give the impression that all, or almost all the evidence on CBT and GET comes from the PACE study, and it says that “it seems unlikely that further research based on these treatments will yield more favourable results”. In fact, CBT and exercise therapies have been investigated in several other studies, and these have been reviewed in Cochrane reviews. The latest such Cochrane review (of exercise therapies, from 2017) includes eight studies other than PACE, and does come to positive conclusions about some aspects of effectiveness of exercise therapies.
Translation: even if our study wasn't that impressive, there's all those great other positive studies out there.
Are you referring to the previous, lower-quality poorly controlled studies that gave inflated effects due to their lack of appropriate controls - the ones you were trying to supersede with your better controlled "definitive" study? Its a bit disappointing to be back there in 2007 again. Alas that's the way of artefactual effects in Psychology. The more you control against them, the more they disappear!
For me, I love a good academic argument. Its the stuff that moves science forward. This defence is not a bad effort, but pretty easy to counter. I wish I had a cleverer adversary. That would be much more fun.
