When PACE-Gate Meets Sample Size, 2025, Sepúlveda

[forestglip]

When PACE-Gate Meets Sample Size

Nuno Sepúlveda

[Conference paper]

Abstract
PACE-Gate is the name by which the case surrounding the high-profile PACE clinical trial became publicly known. This case ended up in controversy due to a change in the pre-trial primary endpoint and a re-definition of the treatments' efficacy.

The present paper concerns a different angle of the case: the statistical argumentation of the reported sample size calculations. The analysis of the per-trial research protocol and the post-trial statistical analysis plan revealed inconsistencies between the theoretical assumptions underpinning the sample size calculations and the prior beliefs of the trial's research team. The reported sample sizes also seemed inaccurate for not accounting for multiple pairwise comparisons contemplated in the trial's objectives.

In conclusion, the statistical argumentation of the trial sample size is suboptimal. The question is whether PACE is either an exception or the norm with respect to incongruous sample size determination.

Link (New Frontiers in Statistics and Data Science) [Paywall]

Most pages are available to read on Google Books.

 

[Nightsong]

This study identified four problems in the sample size calculations of the PACE trial. First, the trial protocol did not report the name of the test that supported these calculations. Second, the prior beliefs about the treatment efficacies suggested the use of one-sided tests rather than two-sided tests.

Three, the sample size calculations did not account for multiple treatment comparisons included in the trial’s objectives. Four, concerning the losses-to-follow, there was an inconsistency between the MCAR assumption used in the sample size calculations and the MNAR mechanism that the authors acknowledged to be more appropriate for the statistical analysis. In conclusion, the reported sample size calculations were suboptimal.

...

To conclude, the pre-trial research protocol includes the following statement at the end (Ref. [7], p. 18):

This paper should not be used as the protocol for executing the study, and is not the complete protocol considered to be ethically satisfactory by the relevant MREC [Multi- centre Research Ethics Committee], having been subject to shortening and revision to enhance communication for publication.

The above statement is somehow a safeguard for the authors to modify the trial protocol according to their will. It is again reasonable to ask about the intrinsic value of a pre-trial research protocol. It is worth remembering that the decision of making trial registration mandatory and accessible to the public was exactly motivated by recurrent episodes of selective reporting, especially, the clinical trials with the potential to reach mainstream clinical practice

 

[rvallee]

The above statement is somehow a safeguard for the authors to modify the trial protocol according to their will. It is again reasonable to ask about the intrinsic value of a pre-trial research protocol

And especially notable that this is the reason, pre-registering their protocol, Horton gave for fast-tracking publication. Which is a special kind of con: technically they did pre-register it, they just didn't do what they pre-registered.

 

[neurophusion]

And especially notable that this is the reason, pre-registering their protocol, Horton gave for fast-tracking publication. Which is a special kind of con: technically they did pre-register it, they just didn't do what they pre-registered.

The PACE trial investigators also fiddled with the registration to remove the original primary outcomes.

From an earlier version:

Endpoints/primary outcome(s)

1. The 11 item Chalder fatigue questionnaire, using categorical item scores to allow a categorical threshold measure of “abnormal” fatigue with a score of 4 having been previously shown to indicate abnormal fatigue.

2. The SF-36 physical function sub-scale, counting a score of 75 (out of a maximum of 100) or more as indicating normal function.

From the version last edited in 2015:

Primary outcome measure(s)

1. Is APT and SSMC more effective than SSMC alone in reducing (i) fatigue, (ii) disability, or (iii) both?

2. Is CBT and SSMC more effective than APT and SSMC in reducing (i) fatigue, (ii) disability or (iii) both?

3. Is GET and SSMC more effective than APT and SSMC in reducing (i) fatigue, (ii) disability, or (iii) both?

4. Are the active rehabilitation therapies (of either CBT or GET) more effective than the adaptive approach of APT when each is added to SSMC, in reducing fatigue, in reducing physical disability?

5. What are the relative cost-effectiveness and cost-utility of these treatments?

 

[rvallee]

2. The SF-36 physical function sub-scale, counting a score of 75 (out of a maximum of 100) or more as indicating normal function.

Which despite the lie about it being too high so they had to push it down, is still significantly below normal function. 75 on SF-36 physical function is definitely not a healthy person.

 

[Jonathan Edwards]

I am not sure that a faulty power calculation would alter the validity of results. It would call into question the ethics of setting up the trial but the statistical significance of the results does not depend on someone judging how likely they are to get a significant result in advance?

The abstract seems to miss the main point that PACE was invalid simply because it was open label and subjective. The fiddling with outcome measures is a minor issue in comparison. The re-definition of recovery does not affect the data, just the language of presentation.