A general thread on the PACE trial!

Peter Tatchell tweeted a support for ME-patients today
https://twitter.com/user/status/982612726131326977


Which was noticed by among others Michael Sharpe who responded with
https://twitter.com/user/status/982650673618735104

https://twitter.com/user/status/982648866146144256


Edit To Add:
https://twitter.com/user/status/982702998525825025

 

Has anyone attributed all the political and social issues surrounding ME to a single trial?

Without White guiding them they're really in trouble, aren't they?

 

@Lucibee Table 3 in the supplementary material to Collin and Crawley’s 2017 paper demonstrates exactly the effect you suggest, where a decrease of 2 points on the Chalder Fatigue Questionnaire, used in PACE as indicating improvement in fatigue, can occur when the patient has deteriorated.

The patients who reported themselves “very much worse” (n=11) at 1 year follow-up after specialist treatment had a mean change in their Chalder Fatigue Questionnaire score of -2.27, with 95% CI of -4.77, 0.22.

The patients who reported themselves “much worse” (n=23) at 1 year follow-up after specialist treatment had a mean change in their CFQ score of -0.26, but the 95% CI was -2.81, 2.29.

Similarly, the 95% CI for those who reported themselves “a little worse” also includes -2:

The patients who reported themselves “a little worse” (n=51) at 1 year follow-up after specialist treatment had a mean change in their Chalder Fatigue Questionnaire score of -1.24, with 95% CI of -3.08, 0.61.

The patients who reported themselves “no change” (n=65) at 1 year follow-up after specialist treatment had a mean change in their CFQ score of -2.28, with 95% CI of -3.59, -0.96.

It is only when patients reported themselves better that we leave the -2 realm, with mean changes in their CFQ scores of:


A little better (n=157) -5.90

Much better (n=92) -11.0

Very much better (n=25) -14.9


I can’t find a way of linking to the supplementary material or copying the relevant row of the table – but it’s well worth a look.
The Collin and Crawley 2017 paper is here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5513420/

Please do check the original source before quoting these figures or interpretation as I have brain fog!

Someone with number skills could look at the FOI dataset from PACE and create a table showing how the PACE data compare to this. Mean change in Chalder Fatigue scores at 52 weeks according to patient-rated Clinical Global Impression scores. Any takers? I think we'd expect bigger changes, but the pattern/trend seems likely to be in the PACE data too.

 

I still don't think there is any value in looking at "change in CFQ score" because CFQ already measures change. But if you take what they've done, then their definition of improvement (decrease of 2 pts or more) is well and truly bust for the reasons you outline.

Because CFQ is such a useless measure, it is very difficult to draw any solid conclusions without knowing exactly how each participant interpreted it and used it. And of course, the investigators will have had their own interpretation, which will be different again.

 

Basically like presuming to build a solid foundation with jelly.

 

I agree, and I've read your blogs on the CFQ. I think it's nice to have NHS data to demonstrate what happens when you try to use a tool that can't measure change to measure change. There are some who only get convinced when they see numbers, no matter how strong or incontrovertible the argument. I think most who read this literature never actually look at the scale itself, and just blindly accept it as an accurate measure of fatigue.

And as @Barry says, we're then building on jelly. And what we're building is a house of cards.

 

@Evergreen OKies. I've done a quick and dirty data summary...

Graph:


Data summary:


Not sure it tells us very much, but intriguing nonetheless!

 

Thanks for taking the time to do that, @Lucibee .

I think that's 'cos it's wobbly, like all good jelly!

More seriously, I think it tells us a lot. If we assume that there is some worth in the Clinical Global Impression scale, then we'd expect a good outcome measure to have some semblance of a predictable, sensible relationship with it. We'd expect the CFQ scores to go down when the person says they improve on the CGI and go up when they say they get worse on the CGI. We'd expect some kind of a diagonal line in your chart above, instead of a splodge.

It seems that the CFQ scale cannot detect negative change in fatigue (meaning increased fatigue). It's skewed towards detecting, or creating an artefact of, positive change in fatigue (meaning reduction in fatigue). So if the desired result from a study is reduction in fatigue scores, and no harm aka increase in fatigue scores, then the CFQ is the way to go. You can't lose.

A change of -2 on the CFQ is clinically meaningless in the PACE data.

 

https://twitter.com/user/status/983854042500780033

Replying to:
https://twitter.com/user/status/982888678170222592

 

https://twitter.com/user/status/989221359556362241

 

On Sharpe's comments regarding funding David Tuller, a quest editorial - http://www.virology.ws/2018/04/16/t...-is-not-a-conflict-of-interest/#disqus_thread

 

Last week, I wrote to my ex-boss (Richard Horton) about my concerns regarding The PACE trial, and in particular, the way the review process was handled.

Over the weekend, I got a response. They won't budge on this. As far as The Lancet is concerned, PACE trial is "science". So I suspect that any criticism of it is therefore treated as anti-science.

I don't know where to go with this now. But I can't stop. If I wasn't activated before, I am now.

 

I wonder what this guy would make of PACE:
https://en.wikipedia.org/wiki/John_Bohannon

"
Who's Afraid of Peer Review?
Main article: Who's Afraid of Peer Review?
In September 2013, he submitted a fake scientific article to a large number of fee-charging open-access publishers,[18] revealing that less than 40% were living up to their promise of rigorously peer-reviewing what is published.[19] This approach was criticized by some commentators as well as by some publishers of fee charging journals, who complained that his sting only targeted one type of open-access journal and no subscription-based journals, damaging the reputation of the open access movement.[20][21][22][23]

Misleading chocolate study


The study. Click for bigger version and original PDF file.
Publishing under the name Johannes Bohannon, he produced a deliberately bad study to see how the media would pick up their findings. He worked with a film-maker Peter Onneken who was making a film about junk science in the diet industry with fad diets becoming headline news despite terrible study design and almost no evidence.[24]"

 

Revealing the truth ... the damage was already there but people were less aware of it.

 

Rational criticism of science is a part of science. To treat that as anti-science, without instead making rational arguments, is the actual anti-science.

 

Others have already tried this route and got nowhere, I'm afraid. They simply don't accept it as valid scientific criticism. Once they've put you in the flat-earther, anti-vaxx, climate-denial, alien-conspiracy-theory, anti-science camp, it doesn't matter what you say or how valid it is. It's all treated as nonsense. Reputation is everything is science. If you have no reputation, nothing you say will be taken seriously.

Which is a shame. The best scientists I've encountered are those I've met on here.

 

As I said to @dave30th a while back, in so many words, when you hit a brick wall you cannot go through you then have to try and come at the problem from a different direction, rather than tackle head on. Maybe building on the political angle etc for more powerful allies, so there will come a point the Lancet etc simply cannot ignore all who criticise them.

 

Another thing I haven't mentioned (yet).

When I published my first blog on the PACE trial (2 years ago), Sonya Choudhury got in touch with me to ask if I would speak at the CMRC conference about it. I declined, because I felt that such an unscientific study had no place at a scientific conference. I think I also would have been thrown to the lions had I done so, knowing what I now know.