A general thread on the PACE trial!

PACE [edit: health economist, semi-statistician] Paul McCrone was a signatory of the 2018 letter claiming that using GET "is based on good evidence from multiple studies and randomised controlled trials showing that these treatments are safe and useful for some patients": https://www.bbc.co.uk/news/newsbeat-44004882

If he's going to do that, he has a responsibility to have investigated the concerns raised about bias.

edit: I was sure someone had posted the full text of that statement, but I couldn't find it just now.

 

Whoops. As a health economist he did all the stats work on their PLoS paper though (it says "Performed the experiments: PM. Analyzed the data: PM."): https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0040808

Would a health economist would have even more of a responsibility to think about those issues?

 

Goldsmith, one of the PACE trial statisticians, did her PhD on the data. On the one hand, this makes her vulnerable: she doesn’t want to be excluded from the project. On the other hand, she probably spent more time thinking about the data than a lot of statisticians do with a lot of data sets.

 

@Lucibee, don't have statisticians a mathematical background where you come from? I studied maths, and I call maths students that specialized in statistics and stochastics statisticians. Maybe that's not correct in English? Maybe that's incorrect at all? What background does one need to be called statistician? I view statistics as highly complex - I was a looser. I would think you need good training and interest.

Personally I don't view statistics as a service, but as a central position in trials. I read others's experiences, and I know how it is in universities - so it's quite possible that, in order to get payments, a certain portion of statisticians "don't look too close". In the industry, in fact, I think that's quite normal.

I stand by my opinion: If someone applies maths to reality, he has to check whether the assumptions of a theorem, lemma, corollary etc. hold - or else your result is wrong with a certain (high) probability. There are several examples from engineers who used simulation programs without properly understanding the theory behind it (who can blame them, humans are not machines; I would be a null in engineering). (Differential equations e.g., especially boundary value problems - the starting point of many simulations in engineering, physics... - are highly complex.) It is known broad areas of physics don't do that, and they realized themselves that's a problem. Where I made my PhD they tried to establish collaborations between physicists and mathematicians, or crystallography and maths, or.... Sometimes people studied maths and physics, which is perfect for theoretical physics but obviously not for everyone. (I couldn't have done it.) No matter where you go, it's never just psychology, chemistry or physics, you need collaborations for good research, in my opinion.

I know this is utopia, that's just my opinion.

 

I'd like to know which of the ME charities PACE trial author A L Johnson (Tony Johnson) had a "familial involvement" with (my bolds and formatting)

"

Chronic Fatigue Syndrome (CFS) (with P. White, T. Chalder (London ), M. Sharpe (Edinburgh)) CFS is currently the most controversial area of medical research and characterised by vitriolic articles and websites maintained by the more extreme charities supported by some patient groups, journalists, Members of Parliament, and others, who have little time for research investigations. In response to a DH directive MRC called for grant proposals for investigations into CFS as a result of which two RCTs (PACE and FINE) were funded and have started despite active campaigns to halt them.

I am part of the PACE study, a multi-centre RCT comparing cognitive behaviour therapy, graded exercise training, and pacing in addition to standardised specialist medical care (SSMC), with SSMC alone in 600 patients; it is funded by MRC, Chief Scientist's Office (Scotland), DH, and Department of Work and Pensions at an estimated cost of £2.7m.

I have been fully engaged in providing advice about design of PACE and I am a member of both Trial Management Group and Trial Steering Committee. I am not a PI because of familial involvement with one of the charities, a perspective that has enabled me to play a vital role in ensuring that all involved in PACE maintain absolute neutrality to all trial treatments in presentation, documentation, and assessment

"

(taken from a MRC Biostatistic website using the Wayback method)

 

Magical Medicine alleges @P409 that he is the son in law of Elizabeth Dowsett, formerly medical advisor and president of the ME Association.

 

Thank you @chrisb

I had forgotten he featured there.

The correspondence is interesting as it sheds light on what the statisticians/designers behind PACE were thinking

"“Throughout his reply, Johnson uses the terms: ‘In designing a clinical trial (of CBT/GET) we have to estimate the number of patients’; ‘Estimation essentially requires a guess at what the results will be’; ‘In guessing what the results may be…’; ‘The assumptions we make…’; ‘Broadly, we assumed that around 60% of patients in the CBT group would have a ‘positive outcome’ at one year follow‐up….’; ‘We speculated that….’,

I'd guess that any statistician working on PACE would have faced a career limiting choice if they strayed away from this obvious influence

 

(you probably know all of this and I'm just including it for anyone who hasn't read the old PACE minutes)

A look at the PACE Trial steering committee minutes shows that they were keen to distance serious adverse effects away from being included and also that short term effects were expected (i.e the admission by Chalder that people could get worse and then better)

10. It was noted that severe adverse events (SAEs)(e.g. a patient having a stroke) was
not necessarily severe adverse reaction (SARs) to treatment. Therefore, the
procedure for notifying every one of severe adverse reactions did not apply to all
severe adverse events. It was also noted that SARs need to be operationalised into
mild, moderate d severe. Finally, it was important to discriminate SARs of the
supplementary therapies from SARs to USC. The definition of SARs in this trial
is complex and requires further consideration

and

11. The data monitoring committee safety role would require it to monitor for
deterioration of participants in a particular group, as judged by outcome data. It
was noted that there needs to be agreement between the Pls, the Chair of the TSC,
and the DMC about under which circumstances the trial might be stopped.
Action: Pls, JD and DMC to meet in September

and

o) Professor Darbyshire led discussion about how to define
`improvement'. Professor Dieppe stated that in order to identify
`damage' by any treatment arm, it would be important to know how
patients receiving no treatment would be expected to progress. The
question was asked `how soon will you know if a participant is getting
worse?' to which Professor Chalder responded that previous research
has shown that it cannot be determined if people are getting better until
at least six months after the end of therapy (i.e. a year after therapy
has begun). CBT and GET may both make a patient worse before they
begin to improve. Professor Sharpe clarified that there is a difference
between transient and persistent deterioration. It was felt important that
the DMEC be aware of this short term differential effect.

ACTION 11: Professor White to add into section 10.3 (monitoring
adverse outcomes) a defined drop in SF36 score.

ACTION 12: DMEC: An explicit definition of deterioration should be
produced before the first review by the DMEC next year. At six months
and one year after the trial opens for randomisation, the DMEC (and
statisticians) will review SAEs, CGI and SF36 scores to see if there Is a
normal distribution. In addition, previous trials will be reviewed to aid
categorisation of deterioration.

and

q) Section 14 on adverse events was carefully reviewed as this has
undergone substantial revision since the last TSC meeting. It was felt
that a `new' disability might be irrelevant in the context of PACE

 

[My bold]

This seems to be at the heart of much of the problem. Those who might have the skills to ensure adequate safety standards are met (the correct running of medical trials is a human-safety issue), operate in an environment where the odds are stacked against them. If avionics projects were run in this way there would be aircraft dropping out the skies with terrifying frequency. I know we primarily focus on ME research issues here, but it seems to highlight the tip of a very grubby iceberg.

 

Yes.

 

I can't get past the fact that the statisticians let them away with treating the SF36 as a normal distribution. This is entirely in the domain of statistics, never mind common sense. It is not bias or anything complicated, it is simply WRONG.

It is akin to having a microbiologist involved in a trial who did not point out that they were dealing with bacteria not viruses, something basic to the profession.

If the statistician was not there to look at the statistical methods they used what on earth was he paid for?

 

No. Not all statisticians have a maths background. I didn't. And maybe that's why I feel utterly inadequate. But actually, you don't need a maths background to understand many of the concepts in statistics.

That was only important when working out the "normal range" that White used to change the recovery criteria. It's not so important for other analyses (because of central limit theorum - I think that's been discussed elsewhere on here). Again, we don't know whether they were consulted on that, or what they said. If White reasoned that it was a clinical decision, they wouldn't have had a say anyway.

 

Started sorting out stuff on my pc and found this letter from White to the Lancet in response to a letter from Prof Hooper in 2011 (don't seem to have his original letter but I'm sure it's out there somewhere!).

(haven't checked to see if it has been posted already).

 

Thanks for posting - I thought I'd copy and paste the text, to help it turn up on google searchers for anyone checking context to a quote:

 

From the letter:
2. Fast track publication (page 6) - It is not for us to comment on the editorial practices of a highly respected international journal.

Noticed this recent exchange on twitter, and thought I'd just add them to this thread.
https://twitter.com/user/status/1054081949734785024

https://twitter.com/user/status/1054083621798195200

https://twitter.com/user/status/1054307802372546561

 

I think someone needs to provide citation for their claim.

How "standard" procedure is it for papers to be fast-tracked simply by having submitted a trial protocol? Because that would be incredibly broken since it seems to suggest that all you'd need to do is submit the protocol and benefit from less scrutiny in peer review, even though it's certainly assumed that submitting the trial protocol is itself standard practice (or I sure damn hope so!). That is definitely not how any of this works, especially when the trial itself deviated from its protocol on its primary outcome.

Can someone who is not blocked by Sharpe ask him for numbers to back up his claim? What % of papers whose trial protocol was submitted benefit from fast-track (and thus less thorough review)? And wouldn't that be worrying if so many papers got this treatment, essentially bypassing the fundamental process of peer review?

And maybe get Horton to comment? Since Sharpe is definitely suggesting something that would heavily damage the Lancet's reputation if it were true.

 

So fast tracking had nothing to do with the fact that NICE were waiting for the results to decide whether or not to review the guidelines(?)

 

This twitter thread started off with the Cochrane withdrawal, but turns into a discussion about PACE. And since several here are blocked by prof. Sharpe, thought it might be of interest. I had to split it up in three posts in order to show all the tweets:
https://twitter.com/user/status/1052984697196818433

https://twitter.com/user/status/1053961447846363136

https://twitter.com/user/status/1053998157888135170

https://twitter.com/user/status/1054483078033358849