Blog: The PACE Trial: How a Debate Over Science Empowered a Whole Community [Carolyn Wilshire/ME Association]

Or so they allege, and by whatever meaning of "analysed" is used. We can't yet verify, but I very much doubt this to be true. A quick partial analysis would have made it obvious that they would report failure and they likely stretched the meaning of "analysed" to mean final, validated analysis.

Even Wessely said they changed it because otherwise no one would have "recovered", which they could only have known from at least making an initial run at the numbers. Not that we can do much about this until it can be proven, but given their past behavior it is pretty much guaranteed and will eventually be shown from a combination of internal communications and possibly whistleblowers.

 

That's not what he said...

But it is clear from the steering committee notes that they changed the outcome measures and thresholds before the formal analysis because of the weak results of the FINE trial (and potentially an overall feeling by those who saw patients that results were not strong) and they were worried that the PACE trial would have non-significant results too if they used the original analysis plan.

 

Surely it's time for recovery to mean the same across all illness. Use cancer definitions. @sTeamTraen

 

They didn't need to look at the data - FINE data would have flagged up problems.

Eta - responded prior to reading rest of thread

 

Not exactly, but he did say this:
"They changed the recovery measure because they realised they had gone too extreme and they would have the problem that nobody would recover."

https://twitter.com/user/status/847468025120907265


ETA: SW's subsequent clarification:
https://twitter.com/user/status/848125525482774530

 

I can't remember the fine mind which pointed it out but it deserves to be repeated.

If you are doing a study to see how many tall children there are in a school, you don't have to see the results to know that changing the definition of tall so that it is a few inches shorter will give more tall children.

Whether or not they had seen any data or even collected any data, lowering the definition of recovery towards the sicker end of the scale was going to give them more people with a positive outcome.

It is a basic fact of numbers and has nothing to do with how many people agreed it should go ahead.

 

You can tell the deferential type. It needs authority of the Lord Chief Justice to point out the distinction between criticism and abuse.

 

What SW doesn't seem to understand (or maybe he does, idk), is that it doesn't matter *when* you make the change (before the trial, mid-trial, after the trial, before/mid/post-analysis), it's the fact that it was made at all.

And it doesn't matter that it was recorded either. CONSORT reporting guidelines are there to make sure that what you have done is adequately documented so that others can see what you've done and check it, maybe even replicate it. Following those guidelines *does not mean* that what you have done is correct. And yet, that is what so many seem to believe.

We and they know what they were expecting to happen, because they document it in their sample size calculations - 60% improvement in CBT, 50% in GET, 25% in APT, 10% in SMC. (Take that, equipoise!).

That he confirms that the change was made "before the analysis" and not even mid-trial is even worse for them, because it indicates that they possibly waited until all the data were in before they made the change.

Why didn't anyone query it? Why did the MRC and trial steering group let it happen? Well I guess if there is an indication that unless a particular change is made, your 5-million quid will have been for nothing, then that's a fairly strong incentive. They will have had to have found a way to believe it was the right thing to do. (Ironically, that seems to be the only discussion that wasn't documented, so who knows whether that was the incentive.)

"Rounds and rounds of peer review." Keep going round until you get the right answer.

 

I think that they know exactly what they're doing with that language: "To be clear the change was before the analysis. It's on page 2228."

Of course the criteria was devised before the analysis using the criteria. That doesn't mean it was devised before other analyses using the same data (including the 'normal range' ones done for the Lancet in 2011, which were described as 'post-hoc' there and went on to form the basis of their recovery criteria).

While they didn't describe their recovery criteria as a secondary analysis in their recovery paper, they have since done so elsewhere. Responses to FOI request would also seem to indicate that this was a post-hoc analysis... though they seem to do have done all they can to avoid making this clear to readers of their paper.

 

And during a fully UNblinded trial what are the chances (I mean, really!) of the research team NOT discussing how things are going. I just cannot imagine they would not discuss some of this stuff informally. And I most certainly do not believe that the investigators would have had anything like the integrity to disregard such information. At best they would have been unconsciously biased by it, and at worst would have very consciously accounted for it ... guess which option I favour!

 

But the more likely it were done once outcomes were predictable, the more damning it is. And I believe the fact they cite the changes were agreed before any formal analysis was done, is a 100% feint. The study design meant they had bags of information to hand, accumulated throughout the trial, including anecdotal within the research team, that things did not bode well; and plenty of similar information of what would change that. It bordered on a scam.

 

Which is a steaming pile of garbage. It was weak enough as it was by relying strictly on self-report, but I would not consider 85 to be recovered by any stretch. Improved, sure, with ongoing limitations but otherwise capable of nearly living a normal life. It's the kind that allows you to work half-time and otherwise have a genuine quality of life, but even at that it's only remission as every long-term follow-up showed that the questionnaire effect is not sustained and is merely an illusion.

But 85 on that scale is simply unacceptable for 20 year-olds as a definition of being in full health. 100 is full health, full stop. They could have allowed for some slack in their definition, accepting that it is a state of of some disability, but noooo, they had to make extraordinary claims of a magical permanent cure, or "within normal range", which was pure fraud.

 

Cancer's too hard in my opinion. The cancer may have gone (for now or for ever) but they treatment may have left the patient unable to function as before or with other weird problems that aren't adequately understood.

and on recovery in general, with ME and CFS there is a brain washing possibility that the subject may be told that they are simply getting"old" or de-conditioned to explain lingering symptoms giving a self-censoring reply to the question "have you recovered" that may be hard to stop and that the patient may be barely aware of.

 

I think long term patients can have a distorted view of recovery would be like.

They want to be well again and begin a new chapter in their life where can forget about the time they were ill. They also have been ill for so long they don't remember what normal feels like.

Their life is adjusted to chronic illness and they may think that they're pretty close to nomal if they just have a good week where they can do their usual activities with little difficulties. But if they tried to do even half as much as a healthy person of the same age, they would quickly realize that they're still very limited.

The idea that patients have an exaggerated illness perception does not fit at all with my experience and I live in a family where others are also ill. That's yet another instance of reality being the opposite of what CBT/GET people claim it is.

 

Chat on some FB pages today re The Last Leg ( new series starts tonight) - PACE, DWP, kids and GET - could this break into mainstream ?

 

Any specifics?

 

No. A lot of parents commenting and separate post re PACE and DWP benefits denials.
I' ll watch tonight and see if there is a mention / reference.