I saw
@Lucibee's recent
post on Bluesky about Rebecca Goldin's article on stats.org:
In 2011, researchers announced that PACE, the largest treatment trial in the history of chronic fatigue syndrome, had been a great success. That seemed like great news since there is no known cure for this devastating disease that affects over a million people in the United States alone. Now, it...
web.archive.org
Rebecca Goldin stated:
"However any change in a statistical plan brings an elephant into the room: why did the authors decide to use the continuous scores? There is little evidence to support the claim that the authors purposely did this because their original protocol yielded no results. The PACE authors point out that the continuous results are easier to interpret; they consist of a direct comparison of groups and which treatment had more success as reported by the patients. But why did the authors make this decision mid-trial, rather than at the beginning, and why didn’t they also provide the results based on the original criteria, or at least conduct a sensitivity analysis to identify how dependent the outcomes are on these different ways of describing the data? What invalidated their original planned analysis, even if the authors determined that comparing improvement provided a 'better' analysis?"
In the comments section [Lidia Thompson] mentions that the PACE trial changed the primary outcomes because of null results from the FINE trial, which
employed almost identical primary outcomes compared to the original primary outcomes of the PACE trial before the changes. There is no direct evidence for this motivation from the TSC and TMG meeting minutes, but a deep dive into the chronological order of events demonstrates highly suspicious circumstances:
* TMG (page 3) 13 February 2008: Analysis strategy presented after 18 months of development. "The TMG thanked [X] and [X] for the enormous amounts of hard work put into the creation of this document and to congratulate them on their achievement in producing it."
* TSC (page 9) 29 April 2009: FINE trial results were presented at the FINE trial TSC on 13 May 2009 and will be presented again to the PACE team day in June 2009. The TSC approves the PACE analysis strategy principles but give extra time to approve the PACE analysis strategy text.
* TMG (page 3) 23 June 2009: "Analysis Strategy Group: The ASG had met prior to the TMG and had approved the final Analysis strategy."
It was announced that the PACE team day was a success and the FINE trial results were presented there.
* TMG (page 2) 4 November 2009:
Primary outcomes suddenly changed. "A few concerns were raised that making a change at this stage may invite criticism." "The originally planned categorical scores will also be reported in the main paper, as a secondary analysis, reflecting clinically important differences."
This raises the question about whether the original primary outcome results were ever calculated. When the ME Association submitted a FOIA request for the results in accordance to the original primary outcomes, they were
informed those results were never calculated. So either they were dropped at some stage without that being documented in the TMG and TSC meeting minutes, or QMUL lied because those results were unfavourable (a significant offense under FOIA law which makes it less likely). As others in this thread have noted, there were additional meetings by groups which were not documented in the meeting minutes for the TMG and TSC.
