BMJ: Pressure grows on Lancet to review “flawed” PACE trial

The long-term FU paper is important because it highlights other flaws in their methodology that aren't mentioned in the original papers.

For example, we know that in order to avoid early drop-outs from participants not getting the therapy they wanted (due to randomisation), participants were told that they could switch therapies after they had completed the first 12 weeks of the trial. What some of these patients then found was that not all the therapies were then available - APT was not offered as a second-stage therapy; Edinburgh lost its GET therapist. We also don't know at what stage patients received their second therapy. Did they really wait the full 12 months so as not to contaminate the main trial results? Or did they 'cheat' to keep participants interested?

Had this been properly planned, they should have done a more complex cross-over trial (with a more rigid plan), so that they could at least stand a chance of analysing the results. That would also have meant they could use fewer participants, but would have also introduced far more complexity. By introducing an element of choice, they completely ruined that possibility. Combining CBT and GET from the outset might have made that easier.
 
Trish said:
But what seems the most significant thing to me is that if GET/CBT are so good, why did the patients in those groups stay stuck at their low 'slightly improved' level of functioning. The logic of GET on someone who is deconditioned would be the longer you keep doing it, the fitter and healthier you get, so all the GET group should have been fully recovered, not stuck at the 'slightly improved' stage.
Click to expand...
Exactly. My wife would have returned to her full fitness very soon after having the infection which triggered her ME, because the same as always she never gave in to it; just kept on striving through, as always has been and always is, her way. She has never been deconditioned, she's had nothing to recondition from. In fact not just PACE, but the experiences of the ME community, are tangible evidence that deconditioning is not the problem, because I suspect many PwME do naturally what is needed anyway to recondition. Have any surveys been done with this in mind?
 
Last edited:
Deconditioning theory completely wrong in my case too, as in many cases, as I had had PVFS after a flu bug 3 times before for between 6-10 weeks and recovered to full fitness each time. So I just thought 'here we go again' and did exactly the same things as before but never recovered and here I am over 22 yrs later!
 
Lucibee said:
For example, we know that in order to avoid early drop-outs from participants not getting the therapy they wanted (due to randomisation), participants were told that they could switch therapies after they had completed the first 12 weeks of the trial.
Click to expand...
How does that work then? Genuine question re trials methodology. Allowing trial participants to change horses in mid-stream feels like a recipe for skewing results, but I guess it must be an accepted practice else even PACE would not have got away with it. I thought the released data only showed any one participant allocated to a single trial arm throughout the whole trial? If any given participant were to switch trial arm, how would the stats have coped with that? Presumably the process would have had to be randomised to minimise skew, but then participants could not have been given a choice anyway. Confused by this.
 
Barry said:
How does that work then? Genuine question re trials methodology. Allowing trial participants to change horses in mid-stream feels like a recipe for skewing results, but I guess it must be an accepted practice else even PACE would not have got away with it. I thought the released data only showed any one participant allocated to a single trial arm throughout the whole trial? If any given participant were to switch trial arm, how would the stats have coped with that? Presumably the process would have had to be randomised to minimise skew, but then participants could not have been given a choice anyway. Confused by this.
Click to expand...
It would contaminate the results if patients moved between arms.
 
adambeyoncelowe said:
It would contaminate the results if patients moved between arms.
Click to expand...
Well yes, that was my assumption. But I'm interested to know, as seems to be implied, if it is an accepted trialling practice - if it is then it would only be so if there are well established methodologies to cater for it, and null out any potential skew. Maybe there is, I'm not qualified to know.
 
Woolie said:
What they mean is that patients will always carry with them the psychological weaknesses that make them prone to "unexplained" illnesses. So next time they get themselves into a tizzy, there's no guaranteeing they won't go off again.

Its sometimes hard to understand their logic until you view things from their belief system.
Click to expand...
As a bit of an aside: sometimes CBT and GET talk about relapse prevention. This can sound good. However what they tend to mean is that patients shouldn’t restrict their activities with new symptoms i.e. they are trying to stop patients relapsing into what they see as maladaptive behaviours (and beliefs).
 
Woolie said:
Finally, there's the argument that its a valid goal to "accelerate" improvement, get people feeling better faster, even if they would have found their own way eventually.
Click to expand...
To which the response is look at the slopes of the improvement rates for each group at LTFU, particularly between 52-134 weeks (the last, and longest, measurement interval), and at their overall shape (i.e. from 0-134 weeks).

APT & SMC are improving at faster rates than CBT & GET on both primary measures. Furthermore, CBT & GET are tapering off in an asymptotic manner, while APT & SMC are not.* In other words CBT & GET appear to be following the standard decay curves that you would expect from non-specific confounding effects washing through over time.

Not only are APT & SMC improving at a faster rate between 52-134 weeks, but compared to the previous interval of 24-52 weeks their rates of improvement actually increase over that time, while for CBT & GET they decrease.* This may be the most important information in that graph.

If you extrapolate out the rates between 52-134 weeks, clearly APT & SMC are going to outperform CBT & GET over time. I know that is not the most robust line of argument as we don't know if and how the rates may change past 134 weeks. But it certainly doesn't look good for CBT & GET, given their distinctly asymptotic form.

Interestingly, GET is the closest to flat lining of all groups on both measures. Clearly exercise therapy was the least successful treatment at sustaining any effect.

*CBT on fatigue is almost linear between 24-134 weeks, but it does inflect slightly at 52 weeks (i.e. continues on an asymptotic path, thus reducing, if only slightly, the rate of improvement between 52-134 weeks). But even if it was linear the rate is still less than for APT & SMC.
 

Attachments

  • PACE Long-term follow-up graph - Figure 2.jpg
    PACE Long-term follow-up graph - Figure 2.jpg
    155 KB · Views: 5
Lucibee said:
Had this been properly planned, they should have done a more complex cross-over trial (with a more rigid plan), so that they could at least stand a chance of analysing the results. That would also have meant they could use fewer participants, but would have also introduced far more complexity. By introducing an element of choice, they completely ruined that possibility. Combining CBT and GET from the outset might have made that easier.
Click to expand...

Of course a big factor that is never mentioned in any of the papers as far as I remember is whether the patients actually complied with what they were being told to do in the therapies. That's why it was criminally negligent, in my opinion, not to have them wearing actometers throughout the trial.

Lucibee said:
For example, we know that in order to avoid early drop-outs from participants not getting the therapy they wanted (due to randomisation), participants were told that they could switch therapies after they had completed the first 12 weeks of the trial.
Click to expand...

Was it really 12 weeks? I (mis)remembered it as 12 months. Haven't go the energy to search it out. Someone else will know.
 
Woolie said:
I'm very worried about this little side-step into psychologisation that we see in Kerr's letter. We actually don't know if psychological stress can cause reactivation of herpes viruses, but we know for damned sure that immunological stress can (e.g. a serious new infection).

It really worries me, the way people can't seem to let go of the psychological nonsense, even when they've seen how wrong it can be.
Click to expand...

I vaguely remember an interview/comment from J Kerr from a few years ago (when he'd largely stopped publishing on CFS) that seemed quite odd to me. I've been trying to hunt it down, but can't remember enough about the details to find it. It didn't relate to any sort of psychologisation, but it did leave me with some concerns about how careful he was with the claims he made.
 
Barry said:
How does that work then? Genuine question re trials methodology. Allowing trial participants to change horses in mid-stream feels like a recipe for skewing results, but I guess it must be an accepted practice else even PACE would not have got away with it. I thought the released data only showed any one participant allocated to a single trial arm throughout the whole trial? If any given participant were to switch trial arm, how would the stats have coped with that? Presumably the process would have had to be randomised to minimise skew, but then participants could not have been given a choice anyway. Confused by this.
Click to expand...

Yes - that's what the data showed (single arm allocation throughout the trial), but the accounts we have from those on the trial tell a slightly different story. There are expectation biases introduced by doing this too. If you are entering a trial knowing that you are on a therapy that you didn't want, and that you will get the therapy you wanted if you complete the first phase, that is inevitably going to lead to outcome biases - maybe by saying it was more helpful than it actually was so you don't get penalised and refused the treatment you wanted all along, or downplaying improvement so that you are not too 'recovered' to try another therapy you think might be better. It's really terrible practice to offer it as part of the trial (which is what they seemed to do - participants referred to the trial as the "PACE trials" (plural)). It should have been kept separate if they were going to do it at all.

Trish said:
Of course a big factor that is never mentioned in any of the papers as far as I remember is whether the patients actually complied with what they were being told to do in the therapies. That's why it was criminally negligent, in my opinion, not to have them wearing actometers throughout the trial.
Click to expand...

There's not even a record of how much pts had agreed to do, let alone whether they reached their targets.

Was it really 12 weeks? I (mis)remembered it as 12 months. Haven't go the energy to search it out. Someone else will know.
Click to expand...

There were 12 sessions - just checked and they were biweekly - so first-stage therapy would have been completed in 24 weeks (~6 months). The long-term FU paper doesn't say at what point any second-stage therapies were taken up. [They made it sound in the FU paper like it was post-hoc pt choice, and not something offered at the start of the trial.] If it was within 12 months, then that could be a big problem. Obviously they are not going to report that. With the trial stretched over 5 years, I'd be amazed if it didn't go slightly arwy at any point.
 
Last edited:
Pretty sure additional therapies were only offered after the trial endpoint (52 weeks/12 months). Those wishing to find evidence of "cheating" in the form of swapping patients between arms pre-52 weeks are probably going to be disappointed. There may have been some, but there may not, and no way of knowing.
 
Woolie said:
Pretty sure additional therapies were only offered after the trial endpoint (52 weeks/12 months). Those wishing to find evidence of "cheating" in the form of swapping patients between arms pre-52 weeks are probably going to be disappointed. There may have been some, but there may not, and no way of knowing.
Click to expand...

Unless I've misunderstood, I don't think it was that sort of swapping that was being discussed. While treatments of choice that broke randomization were only provided after 52 weeks, a few participants have said that they were 'offered' when they were deciding whether to be part of the trial. ie: one of the inducements for being a part of PACE was that potential participants were told that they would get quicker access to their treatment of choice (even if randomised to a different treatment at first) than if they just went on the waiting list. I think that there was something in writing from the PACE team confirming something like this was happening, although I can't remember the details now.

I don't think that this is an important enough point to be worth including in brief summaries of the problems with PACE, but it is something that seems to have affected how some participants engaged with the trial.
 
Esther12 said:
I don't think that this is an important enough point to be worth including in brief summaries of the problems with PACE, but it is something that seems to have affected how some participants engaged with the trial.
Click to expand...

i agree. For a brief summary we probably have plenty of clear material. The problem is not that we don't have enough to criticize, it's knowing which points to include when.



Lucibee said:
If you are entering a trial knowing that you are on a therapy that you didn't want, and that you will get the therapy you wanted if you complete the first phase, that is inevitably going to lead to outcome biases - maybe by saying it was more helpful than it actually was so you don't get penalised and refused the treatment you wanted all along, or downplaying improvement so that you are not too 'recovered' to try another therapy you think might be better.
Click to expand...

However, this is such a clear example of introduced bias. How can it be possible for them to be so ignorant of human nature? (That's a rhetorical question).
 
You must log in or register to reply here.
Back
Top Bottom