PACE trial TSC and TMG minutes released

From the Trial Steering Committee meeting of 29 June 2005 (ie. several months *after* recruitment of patients had started - so the trial was underway - but still a full 16 months before the text of the protocol was submitted to BMC Neurology):

"The analysis plan will be written once it is felt no further amendments to the protocol are likely" (page 7)​

I'm not sure how I'm supposed to interpret this statement other than as an open admission that the protocol - supposedly agreed before the trial began - was being routinely tweaked after the trial had started and that the TSC fully expected this state of affairs to continue for some time.

From what I can tell from the TMG minutes, the Analysis Strategy Group didn't even *start* work on the strategy until October 2006 (the same month the protocol was finally submitted to BMC - what a coincidence!) by which time the trial had been underway for over 18 months. And yet incredibly the MRC don't seem to have had a problem with any aspect of this.

 

I hadn't thought of that. Another interpretation is that they wanted it absolutely clear on the record that the TSC had approved of the changes laid out in the statistical analysis plan (which failed to include the deviations to the recovery criteria) before analysis commenced.

Annoying. I wonder why that would be. It could be that in involved Action for ME?

 

This tickled me. From the minutes for Trial Management Group Meeting #30, 11 March 2009. Page 8

14. Final PACE team day 2009
This is on Wednesday 17th June 2009. Topics for presentation during the academic morning were discussed...
...Two suggestions were made for the social afternoon:

a) Treasure hunt around London.
b) Boat trip to Hampton Court.​

I can just imagine the discussion...

"So, any ideas for the social afternoon?"
- "How about a treasure hunt around London?"
"Great idea! But what would we hunt for?"
- "Errr... statistically significant results?"
{tumbleweed}
"No-one here finds you funny. I hope you know that."

 

As data from the step test was never published, in combination with the above fact, this leads me to wonder: was adverse event data associated with this part of the trial (the step test) reported still? If not then was the step test data not reported as a means of hiding associated adverse events in the trial?

 

Not only an unblinded trial which primed participants on the effectiveness of CBT and relied on subjective outcomes measures, but also one in which patients volunteered to join because they were already convinced of the potential effectiveness of CBT and GET and became depresssed if they did not get these treatments. This really is a textbook example of how not to conduct a clinical trial – or how to ensure a positive result for an ineffective treatment.


https://twitter.com/user/status/977849873667121152

 

There would have be no end of gossip about how the patients did if it bore any relation to the way clinical staff normally behave under such circumstances.

 

I think this is a good illustration of how one has to realise that clinical trials are episodes in real life and that being part of a trial is a good way to play merry hell with all sorts of beliefs and emotions. It also highlights the fact that nobody was under any illusion that specialist medical care was anything other than - zilch.

 

Could be. Seems odd that no-ones found more detailed discussion about the protocol deviations. There also seemed to be some COI statements missing from that release.

Thanks for putting that stuff together.

General point: My instinct is to be cautious with making criticisms of the basis on these minutes until we have a good idea of exactly what happened. If we're trying to push the UK academic community to go against their self-interest, and take action on the problems with PACE, it's probably worth being aware that many of them will feel they have skeletons in their closets, and feel some sympathy for PACE having TSC minutes released like this. There are already so many problems identified with PACE that it could be counter-productive to risk making further criticisms that are not rock-solid.

It looks like the minutes further support the view that they failed do get TSC approval for the Recovery criteria used in the psych med paper. Maybe that's the point to focus on now? That paper was always the key candidate for retraction, and Robin Murray's response to concerns was scandalous.

 

I think the explanation is that we don't have minutes for the meetings of the Analysis Strategy Group and it seems likely (to me at least) that these meetings are where all the protocol changes were proposed/discussed/agreed. We know that the TSC attended the meeting of the ASG that took place on 4 September 2009 and that deviations from the protocol were discussed at this meeting - this is almost certainly the meeting that the TMG minutes for 4 November 2009 are referring to in the section you highlighted.

As we don't have the minutes for that meeting we don't know which specific changes were discussed, what justifications were given or whether the TMG voted on that date to approve any of the ASG's proposals. And as we don't have the rest of the ASG minutes we don't know whether the TSC's attendance at this meeting was a one off or if they made a habit of attending the ASG's subsequent meetings. Perhaps it was at one of the meetings of the ASG that the TSC approved the analysis strategy - they certainly don't seem to have given it approval at any of their own meetings...

Without access to the ASG minutes it's going to be impossible to work out how the changes happened. To be honest it probably wouldn't be much easier with the minutes... But I think the minutes that have been released so far show that the official trial records do not support the PACE team's claim that the changes they made from the protocol were correctly approved by the TSC.

 

Apologies if this has been posted already:
https://twitter.com/user/status/977494101959565314

 

We did. In response to an FOI request the MRC made this statement in a letter dated 12 October 2016:

"The Trial Steering Committee joined a meeting of the Analysis Strategy Group held on 04 September 2009. It was at this meeting that the change in the analysis of the Chalder Fatigue Questionnaire was discussed and agreed i.e. the planned secondary outcome measure of the scale using the Likert (0,1,2,3) scale would become part of the primary outcomes instead of the bimodal (0,0,1,1) scale."
​

Presumably the details of this discussion - and the fact that this change in analysis was agreed by the TSC - appear in the ASC minutes, because there's certainly no mention of the TSC taking this decision in their own minutes. So did the TSC agree any other changes at this meeting, or at any subsequent ASG meetings? Without the ASC minutes there's no way of knowing.

How many other decisions did the TSC make that aren't recorded in their own minutes in any way?

 

I have to say that I am so impressed by the mastery of the facts some of you have here. I cannot cope. I am hoping for a simple precis once the dust has settled. Keep at it.

I work a lot with solicitors on medicolegal cases and you remind me of the very best team I have worked with - like very smart indeed. And they would pay me a grand for any one of the nuggets in the last few posts - not because they had not already thought of it themselves but the rule was that I had to come to that conclusion myself, even if with a bit of help in terms of making sure I actually read all the words in the documents in front of me. Helped by some nice biscuits with the coffee, I usually did. And with good reason. You do not build a case worth hundreds of millions on dodgy arguments. Barristers are not that smart, they are just good at verbal sparring. It is the nice quiet friendly solicitors who know how to unravel a stitched up argument.