A general thread on the PACE trial!

It was clear from the beginning that as an open trial with subjective outcomes it was dead in the water, but my understanding is that the pushback from organisations like MEA was on grounds that were pretty debatable.

 

I don't know, perhaps people like @Dolphin would recall? Certainly the earliest document in the Trial Management Group meeting minutes - from June 2002 - references previous meetings and a protocol. So there were drafts pre- June 2002. At a meeting in September 2003, the primary outcome measures were already nailed down:

But the criteria for positive outcomes were changed both between this point and the published protocol, and between the published protocol and the published results, if I've understood correctly

Outcome measures are measured twice, before therapy and after therapy, to see whether someone improved.

Predictors are measured once, before therapy, so that you can see if any of them are useful for predicting whether someone improves or not. For example, you might want to know if people who are sick for less than 2 years and more likely to improve than people sick for longer, because it could help you direct services towards the people most likely to benefit. So their interest in actigraphy was whether a particular activity pattern - eg boom and bust - predicted whether someone improved or not in the trial.

Some things are both outcome measures and predictors, because you might want to know whether someone's depression improved with therapy AND whether being depressed at baseline made you more or less likely to improve with therapy.

The predictors from the published protocol of the PACE trial were:

Sorry I don't understand this bit.

They didn't seem to see any problem with having only subjective primary outcome measures in an unblinded/unblindable trial. Nor do countless other researchers. Their original goal was to see which was better between CBT and GET.

 

My guess is that it was about negative patient feedback on exercise and CBT, and possibly about physical vs psychological, but I haven't found the relevant documents.

 

Also since FINE was mentioned, a timely reminder that this is the trial that (IIRC) had a primary objective outcome that was swapped for a subjective one, and where the published paper had the famous quote about how "the bastards just don't want to get better". It was a terrible offensive trial of no credible or scientific purpose and it should never have happened.

It was also a complete bust, showed no positive results, and yet still formed the basis for a NHS module on CFS. The whole thing was even more shambolic than PACE, and I still maintain that PACE was fraudulent in the way it was reported, had a piss-poor design and served no legitimate professional or scientific purpose either.

So opposition to both was well-warranted, they served no legitimate purpose and ultimately set everything back.

 

Maybe I'm misremembering, but I think that the 2007 guideline actually mentioned an upcoming trial that would prove it correct, likely referencing PACE. It was really all set up together, with the main piece of evidence happening in the future, so that piece of evidence could not fail, hence they were allowed to blatantly cheat.

 

Thank you.

Interesting - so maybe check the previous Guideline.

 

The working group that produced the 2002 report to the Chief Medical Officer included Shepherd, Pheby as well as Chalder, Chris Clark (AfME), White. Names we'd know like Anne Macintyre, Speight and Weir were also involved. So these quotes might give an idea of what early objections to the PACE and FINE trials would have been:
(full report here http://www.erythos.com/gibsonenquiry/docs/cmoreport.pdf)

 

In the US in 2015, the CDC was still saying that PACE-style GET could help some patients. CDC dropped that in 2017 as a result of advocacy on all the issues with trial design - including how they selected patients.

Regarding the diagnostic criteria - wouldn't there be a question of whether its scientifically legitimate in this specific case to generalize the exercise recommendations from cohorts who were not required to have PEM to people who all have an abnormal response to exertion?

 

This would presumably be at the stage of planning to set up a trial. The quotes are quite interesting in that they suggest that those with inside knowledge of ME/CFS saw CBT and GET as pretty unlikely to help on the basis that people had already tried exercise mostly and not managed and that CBT was a bit of a theoretical muddle.

It is interesting to note that this is only a few years (?4) after Chalder first proposed an optimal method for CBT for ME/CFS - without having done any trials of course.

 

That can be raised as a concern for those wishing to have advice specifically for those with PEM but the trial would still be entirely scientifically legitimate - on its own terms. There are always caveats about subsets.

 

So in a very general sense, I can see the point that ME is a "subset" of the condition of "chronic fatigue." And presumably, there are many different conditions that have chronic fatigue as a feature. But PACE was not claiming to be studying exercise in all such conditions. They were specifically claiming to be studying the group of people who have ME.

I can imagine one reason they are objecting to NICE requiring PEM is because they know NICE's new criteria scientifically invalidates the way they've been studying ME all along...

Edited to add
Of course, this on top of all the other problems with study design and conduct

 

Exactly. The CFS/ME Working Group began their work in 1998, culminating in their report to the CMO published in January 2002. The first minuted PACE trial meeting was in June 2002, and those minutes reference earlier meetings. In July 2004, the MEA publishes this: ME Association. MEA calls for PACE trial to be scrapped. ME Essential, July 2004: 91: 3–4. The PACE trial management group meeting minutes refer to a campaign to stop the FINE and PACE trials from 2004. Recruitment for the FINE trial starts in February 2005 and for the PACE trial in March 2005.

 

I’ve always thought the timing of the PACE protocol changes need to be seen in the context of the earlier-finishing FINE trial.

The protocols for the FINE and PACE trials had the same criterion for improvement on the SF36PF scale (having agreed to swap protocols in a May 2003 TMG meeting). The FINE protocol confirms using the 11-point Chalder questionnaire, but does not specify what will count as improvement on that scale.

This table, from Wilshire et al 2018 (full reference below) shows the switch in criteria for improvement on the SF36PF and CFQ that was made between protocol and publication of the PACE trial ie 2007 vs 2011:



From: Wilshire CE, Kindlon T, Courtney R, Matthees A, Tuller D, Geraghty K, Levin B. Rethinking the treatment of chronic fatigue syndrome-a reanalysis and evaluation of findings from a recent major trial of graded exercise and CBT. BMC Psychol. 2018 Mar 22;6(1):6. doi: 10.1186/s40359-018-0218-3. PMID: 29562932; PMCID: PMC5863477.

On 29th April 2009, the PACE statistical analysis plan is discussed at a TSC meeting, but there is no mention of changing the criteria for improvement at this point.
On 13th May 2009 the FINE trial results were presented to the FINE TMC.
On 17 June 2009 the FINE trial results were presented to the PACE TMG.

The FINE trial results as published in 2010:


From: Wearden AJ, Dowrick C, Chew-Graham C, Bentall RP, Morriss RK, Peters S, Riste L, Richardson G, Lovell K, Dunn G; Fatigue Intervention by Nurses Evaluation (FINE) trial writing group and the FINE trial group. Nurse led, home based self help treatment for patients in primary care with chronic fatigue syndrome: randomised controlled trial. BMJ. 2010 Apr 23;340:c1777. doi: 10.1136/bmj.c1777. PMID: 20418251; PMCID: PMC2859122.

The FINE trial summarised their findings as follows:

On 4 November 2009, less than 5 months after the FINE results were presented to the PACE TMG, TMG meeting minutes note changes to the analysis plan:

White et al. describe the change in 2015 in BJPsych Bull. 2015 Feb; 39(1): 24–27:

 

The PACE trial claimed to be on chronic fatigue syndrome and their definition did not include PEM. That is scientifically legitimate. The term ME had always been ambiguous and confusing, with a variety of interpretations. I think the present term ME/CFS is very useful but I am increasingly doubtful that ME was a very useful medical term. It tended to assume a causal homogeneity that we don't have support for.

 

But by Boolean algebra, if ME is the intersection of several symptoms, one of which is fatigue, then it will describe a subset of the set of people with fatigue.

 

I am sure that is right. Tom seems to be the person who saw the scientific problems long before anyone else. My thought was that this may not have fed through much to messages and campaigns run by organisations like MEA. The message I see from then is that PACE was not appropriate because it was known that ME was not psychological and that people found exertion intolerable.