As is my way, I've only pulled out the slight problems with Levin's slides. For a lot of it, I was impressed that he seemed to have a good grasp of the details, and it seemed that he had dug into it and was mentioning problems that don't get a lot of attention. Thanks a lot to Levin for all his work.
PACE was actually worse than it sounds here:
This happened because the criteria for recovery were changed, months after data collection was completed, sometime between when the Statistical Analysis Plan was finalized and the trial results were published. The investigators claimed this was done before “looking at the data” and with the approval of the ethics oversight committee, but...
The Statistical Analysis Plan did not include the PACE trial's new recovery criteria. We now have the TSG & TMG minutes available to us, and there is no sign of approval for the revised recovery criteria:
https://www.s4me.info/threads/pace-trial-tsc-and-tmg-minutes-released.3150/
The PACE researchers have been deeply evasive on all this, classing a request for clarification of the timing of protocol changes from Matthees as 'vexatious', but it really looks to me like the PACE recovery criteria was devised after they had already been analysing trial data:
https://www.whatdotheyknow.com/request/timing_of_changes_to_pace_trial
In their 2013 paper they say: "We changed three of the thresholds for measuring recovery from our original protocol (White
et al. 2007) before the analysis, as explained below." It is perfectly reasonable for people to assume that this means before they had begun analysing trial data, but that's not what they say. They say before
the analysis, and so, potentially after other analyses.
After the 2011 PACE paper was published a number of patient groups made an FOI request for outcomes laid out in the trial protocol:
https://www.meassociation.org.uk/2011/05/6171/
QMUL responded by saying that some of these outcomes, including the protocol specified recovery outcome, were exempt from the FOIA as they were due to be published in an academic journal. This would put the date for changing the recovery criteria as being after the data for the 2011 paper had been analysed, and this paper included the 'normal range' outcomes for the SF36-PF and Chalder Fatigue Questionnaire that were a core part of their later recovery criteria:
http://www.meassociation.org.uk/wp-content/uploads/2011/06/FOI+from+Queen+Mary.pdf
edit: I think Levin may have gone a bit wrong there as he just assumed that PACE had done the basics of what competent, honest researchers do. The evidence we currently have available to us does not support that assumption. We can't yet be certain of when the PACE recovery criteria was changed, but it seems that it was after the final TSC & TMG meetings.
• The revised endpoints were subjective, self-reported outcomes, not the original objective measures
PACE dropped actometers as a potentially objective outcome, but this was before the protocol was published. The protocol primary outcomes and recovery outcome both relied only on subjective self-report outcomes.
For years they rejected requests from patients for the release of the findings for the originally specified endpoints as “vexatious.”
I can't think of an example of them rejecting those specific requests as 'vexatious'... I doubt PACE would want to challenge this point though, as it brings up cases like the Matthees one above, where they had unfairly classed his request as vexatious.
I really like how this talk took the time to present some of the PACE team's responses, and then pick them apart. I can see why some people don't bother, as their responses are so rubbish, but I think it's worth doing.
Going through that was deeply cheering - PACE really is a piece of shit, isn't it?
. 
