A general thread on the PACE trial!

Sarah · Apr 30, 2018

Lucibee said: ↑

Last week, I wrote to my ex-boss (Richard Horton) about my concerns regarding The PACE trial, and in particular, the way the review process was handled.
Click to expand...

Thanks very much for doing this, @Lucibee

adambeyoncelowe · Apr 30, 2018

Lucibee said: ↑

Another thing I haven't mentioned (yet).

When I published my first blog on the PACE trial (2 years ago), Sonya Choudhury got in touch with me to ask if I would speak at the CMRC conference about it. I declined, because I felt that such an unscientific study had no place at a scientific conference. I think I also would have been thrown to the lions had I done so, knowing what I now know.
Click to expand...

Would you speak there now, given its professed change of direction?

Lucibee · Apr 30, 2018

adambeyoncelowe said: ↑

Would you speak there now, given its professed change of direction?
Click to expand...

I really don't know. I'm also not a very good speaker!

Jonathan Edwards · Apr 30, 2018

Lucibee said: ↑

Reputation is everything is science. If you have no reputation, nothing you say will be taken seriously.
Click to expand...

I think this is unfortunately true.

I have been lucky enough to gain a reputation. Not maybe the sort that gets you elected as president of cosy academic institutions, but I am regarded as maybe the most successful clinical researcher of my generation at UCL. People to whom Horton would defer, like Mark Walport or Keith Peters know that I do original high quality science and that when I speak my mind (as I tend to) I have good reasons. Up until now I have worked on the principle of responding to requests for comment. I think the time has come for me to be more proactive - if I do have some leverage I guess I ought to use it.

I agree that the methods used so far have not yet lanced the boil, although in this situation I think every letter and email helps to erode misplaced confidence even if that is not obvious. The next step for a brick wall is either to creep round the side or drop a nuclear weapon. There are some plans afoot and of a fairly nuclear kind. But details will have to wait.

Indigophoton · Apr 30, 2018

I've never before found the idea of the nuclear option so appealing...

Barry · May 1, 2018

Adrian said: ↑

It always concerns me how much science works by reputation rather than actual analysis. I suspect its getting worse if for no other reason than the volume of papers - too many academics have to publish too many rubbish papers to meet numbers rather than doing and properly reporting quality research.
Click to expand...

The scientists' equivalent of Chinese whispers it almost seems.

Adrian said: ↑

In the end I think that it is important that the Lancet is on record of having been told what the problems are. They are taking a huge risk with their already questionable brand.
Click to expand...

Absolutely. Putting their wrongdoings on public record is very 'laudable' of them; they will regret it in due course I suspect. Such an audit trail will prove essential and invaluable I think.

Barry · May 1, 2018

Indigophoton said: ↑

I've never before found the idea of the nuclear option so appealing...
Click to expand...

I'm sorry, I've lost the thread here. Which nuclear option are you referring to?

Indigophoton · May 1, 2018

Barry said: ↑

I'm sorry, I've lost the thread here. Which nuclear option are you referring to?
Click to expand...

@Barry, the one Jonathan apparently has tucked up his sleeve,

Jonathan Edwards said: ↑

I agree that the methods used so far have not yet lanced the boil... The next step for a brick wall is either to creep round the side or drop a nuclear weapon. There are some plans afoot and of a fairly nuclear kind.
Click to expand...

Barry · May 1, 2018

Indigophoton said: ↑

@Barry, the one Jonathan apparently has tucked up his sleeve,
Click to expand...

Thank you . In which case...

Indigophoton said: ↑

I've never before found the idea of the nuclear option so appealing...
Click to expand...

... it might be more relevant whether the PACE crew find the possibility of a nuclear option appealing or not. Depends what that option is I guess.

Amw66 · May 1, 2018

Link to interesting article on inherent bias in RCTs
https://www.tandfonline.com/doi/full/10.1080/07853890.2018.1453233

Tom Kindlon · May 6, 2018

https://twitter.com/user/status/992820252579975169

Sean · May 6, 2018

Funny how Michael Sharpe has plenty of time to respond to easy questions on twitter, but the PACE team (which I assume included him) failed to respond to Chris Ponting now deputy chair of the CMRC.
Click to expand...

I'm sure there is a joke in there somewhere.

Sly Saint · May 21, 2018

No idea if this has been posted somewhere.
It is a letter submitted in 2015 from the Independent Commission of Freedom of Information call for Evidence (don't know what the outcome of this commission was?).
It's on page 137 of 212 https://assets.publishing.service.g..._Evidence_Responses_from_individuals_A_-D.pdf

The letter reads:

Dan Clarke
The Freedom of Information Act is vital for revealing to our elected representatives the
extent to which they have been misled by other parts of the British State. They are not able
to do this work on their own and need an informed and engaged public to help.
I am going to provide an important and current example of this.

The biopsychosocial model of disability, and the assumptions that underpin it, have played
an important role in the reforms made to disability benefits over the last twenty years. It has
served to distort policy maker's judgements about people's capacity to work when they have
health problems, and led to the adoption of unrealistic criteria for ESA. The exaggerated
view of the extent to which disability is maintained by psychosocial factors amenable to
management and personal control has led to massive additional pressure and burdens being
placed upon people with disabilities. Lord Freud has described the importance of the
biopsychosocial model in the house of Lords, and circulated reports extolling it's benefits.
However the evidence base in support of this model is weak, and has been misrepresented.

As an illustration of these problems I encourage you to look closely at the results from the
PACE trial[1]. This is a trail which examined the efficacy of biopsychosocial rehabilitative
approaches for Chronic Fatigue Syndrome, an area where the biopsychosocial model had
been claimed to be of particular value. This is the only medical trial to have recieved funding
from the DWP and Mansel Aylward and Moira Henderson of the DWP were both observers
for the trial's steering committee[2], while one of the trial's Primary Investigator's, Peter
White, is an advisor to the DWP[1]. The results from this trial were presented in a way which
gave a misleading view as to the efficacy of the interventions tested, and the potential for
those with health problems to recover by altering their own cognitions and behaviours.

The trial was necessarily non-blinded, and compared patients receiving four different
interventions. Those in the group receiving just Specialist Medical Care received
little therapist time, while those receiving Adaptive Pacing Therapy were actually encouraged to
only do 70% of what they felt comfortable doing. Alternatively, those receiving the
biopsychosocial interventions of Cognitive Behavioural Therapy and Graded Exercise
Therapy were told positive claims about evidence showing that these interventions would
lead to improvements in health. This means that there will be problems with response bias
for self-report outcome measures, and it is these that were used as the primary measure of
the trial's results: as the DWP has found, running courses which leave people with
disabilities more likely to self-rate themselves as 'positive' about finding work is much easier
than running course which leave them more likely to actually find any sort of sustained
employment.

Beyond the problem of bias, there were also problems with the factual claims about what
patient's questionnaire scores should be interpreted as meaning. The PACE trial's published
protocol defined 'recovery' as requiring an SF-36 Physical Functioning (SF36-PF)
questionnaire score of at least 85 out of 100, while the trial's entry criteria required a score of
65 or under, which was taken to indicate that patients' fatigue was disabling[2]. Yet the post-hoc criteria for recovery which was eventually used to allowed patients with an SF36-PF
score of 60 to be classed as recovered. This change was justified by the claim that a
threshold of 85 would mean “approximately half the general working age population would
fall outside the normal range.”[3] In fact, the data cited showed that the median score for the
working age population was 100, less than 18% of the general working age population had a
score under 85, and 15% had declared a long-term health problem[4,5].

An SF36-PF score of 60 was claimed in the Lancet PACE paper to be the mean-1sd of the
working age population, and thus a suitable threshold for ‘normal’ disability[1]. They had in
fact used data which included all those aged over 65, reducing the mean physi
cal function score and increasing the SD[4]. For the working age population the mean-1sd was over 70, requiring patients to score at least 75 to fall within this ‘normal range’[5]. Also, the trial's
protocol makes it clear that the thresholds for recovery (including ≥85 for SF-36 PF) were
intended to be more demanding than those for the mean-1sd, reporting that: “A score of 70
is about one standard deviation below the mean... for the UK adult population”[2].

Even using the loose post-hoc criteria for recovery, only 22% of patients were classed as
recovered following treatment with specialist medical care and additional CBT or GET[3].
Regardless, the BMJ had reported that PACE showed CBT and GET “cured” 30% and 28%
of patients respectively[6], a Lancet commentary claimed that about 30% recovered using a
“strict criterion” for recovery[7], and a paper aimed at NHS commissioners stated PACE
indicated a recovery rate of 30-40% for CBT and GET[8,9]. Such misstatements of fact to be
should not be allowed to go on affecting how doctors treat patients, how funding decisions
and policy are made, or the information that patients are provided with before deciding
whether to consent to particular interventions.

Beyond questionnaire scores of dubious reliability, the PACE trial also provided data on
rates for employment, disability benefits and insurance claims that should be less prone to
bias. This data showed biopsychosocial rehabilitation did not lead to improvements in
employment levels, and that all groups reported "receipt of benefits due to illness or disability
increased slightly from baseline to follow-up" [10].

A separate meta-analysis of actometer data from CBT trials for CFS also found that CBT
was able to lead to improvements in questionnaire scores in non-blinded trials, but not to
improvements in the amount of activity that patients were actually able to perform[11]. Sadly,
the PACE trial dropped actometers as an outcome measure, although they were purchased
and used at baseline[12].

Recent evidence from a large study of NHS CFS/ME specialist services indicated that
reported results for CBT and GET are even poorer than those reported in PACE (where
considerable effort could be put into patient selection), and that centres offering CBT and
GET achieved marginally worse results than centres that merely offer ‘activity management’[13]. We do not currently have compelling evidence that CBT or GET are more effective medical interventions for ME/CFS than homeopathy, despite some of the claims made by proponents.

The changes to the outcome measures used in the PACE trial were justified by inaccurate
claims and have been misleading to others. Government policy makes should be basing
their judgement upon a rigorous examination of the evidence, when too often they seem to simply trust those academics who are telling them what they want to hear. The DWP part funded this trial, and policy makers should not be basing their decisions, or understanding of the value of the biopsychosocial model of disability, upon spun data. They should have access to the trail's protocol defined outcomes, or else from their own analysis of the raw data - done by those who have not built their careers upon claims about the value of the biopsychosocial model.

When it comes to the DWP and the biopsychosocial model, it can be difficult to tell if politicians have been misled or if they are happily misleading others. Regardless, unfounded claims and spin have a serious affect upon the lives of others, and politicians need to recognise their responsibility to look more seriously at the available evidence. However much they may like to believe that there are treatments able to lead to recovery for many of the people currently claiming disability benefits (for those with mental health problems "up to 90 per cent" could return to work with treatment according to a senior government source in the Telegraph![14]), and that steep cuts to welfare spending can be made without harming desperate and deserving people thanks to wonderful new psychosocial management techniques, a more critical and honest look needs to be taken at the data from those whose careers are built upon claims about their ability to successfully treat patients.

Recently, American academics and journalists have started to recognise the problems with
the PACE trial and the way in which results from it have been misrepresented. David Tuller,
academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley has written a series of critical pieces following a years long investigation [15], and this has led to an open letter from a collection of academic calling for an independent re-analysis of data from the PACE trial[16].
Throughout all of this, the Freedom of Information Act has played a vital role in allowing informed criticism to take place. So much so, that the researchers and their university successfully campaigned to have the Freedom of Information Act changed, to make it easier to avoid releasing important data. The University proudly reported an outcome of the trial being that "the IT Act of 2014 includes an exemption to the existing Freedom of Information Act 2000, to exempt current research from the FOI act requests, so long as release of data can be shown to be detrimental to the research."[17] Since then the Information Commissioner's Office has decided against QMUL's refusal to release annonymised data from this trial, as fortunately a request for information was made before the exemption came in to force. When commentating on this case James Coyne, one of the world most widely cited psychology professors, argued that greater openness with scientific data was necessary as evidence of widespread misrepresentations and spin within science could no longer be ignore
d. He quoted bioethecis Jon Merz arguing that "The litany of excuses not reasons – offered by the researchers and Queen Mary University is a bald attempt to avoid transparency and accountability, hiding behind legal walls instead of meeting their critics on a level playing field. They should be willing to provide the data for independent analyses in pursuit of the truth."

Do you feel confident that our elected representatives are able to unpick the spin and
misrepresentation that takes place within science and the civil service on their own? If not,
you should be committed to strengthening the Freedom of Information Act so as to empower
citizens to aid in the pursuit of truth which necessarily underpins worthwhile policy
development. Even with the Freedom of Information Act in place, it has been hard work to
gather evidence of the problems which has afflicted government policies towards disabled
people over the last two decades, and it now seems that it is American academics and
journalists who have taken time to express concern about this British scandal. There is a
culture of deference within British society which allows those in authority to live in a fantasy
world of their own creation. A small collection of people with shared prejudices and assumptions
can set about funding poor quality research which supports their own beliefs, justifies the policies which they wish to pursue, and venerates their efforts-without even realising that they are doing anything wrong. Greater openness with data and information would help those who do not share their prejudices and assumptions to more effectively challenge the distortions of thought and perception which can otherwise do so much to harm others - particularly to those at the bottom of society who are least likely to have their own concerns represented within the corridors of power.

In the last few weeks, a new PACE paper has been released which showed that, even using
only self-report outcomes, the addition of CBT and GET to patient's medical care led to no
improvement over the control intervention[19]. Nonetheless, this was presented by the trial's
researchers as being a big success for CBT and GET. Even with this relatively transparent
spin, I am still not confident that civil servants and politicians keen tobelieve in the value of
rehabilitative approaches would have been able to recognise the true significance of the data released.

Giving people greater ability to be well informed and speak honestly about the important
political matters which affect theirlives is a good thing and worth the cost of the Freedom of
Information Act. I do not think that the problems related to the PACE trial (of £5,000,000
immediate cost, affecting NHS funding decisions of many millions more, and shaping government disability policies) could have been uncovered without the Freedom of Information Act. Do you?
Click to expand...

Lucibee · May 21, 2018

Just came across this on Lancet TV. (Sorry in advance...)

https://www.youtube.com/watch?v=jYRjB_t8dYs

large donner · May 21, 2018

adambeyoncelowe said: ↑
Would you speak there now, given its professed change of direction?
Click to expand...

Lucibee said: ↑

I really don't know. I'm also not a very good speaker!
Click to expand...

Ill go there and speak for you, I can winge the hind legs off a donkey but I am also a bit of a thickhead.

Well the offer is there anyway if people want a car crash rant.

MeSci · May 21, 2018

What a strange interview. Sharpe looked down a lot, as though he was reading. And a strange reaction to the end - sort of co-conspiratorial? As though he were saying "Was that good enough?"

Most unconvincing.

Lucibee · May 21, 2018

(Apologies if I put anyone off their tea. There are another two videos that I couldn't watch. )

Snowdrop · May 21, 2018

I know that there are some videos out there about what is wrong with PACE and they a very good and clear about the flaws but I'm wondering if maybe a similar styled video (talking head) answering questions that respond point by point to this video would be a really good investment.

That is explaining for example how the 'rigourous' oversight failed. And a concise and cogent account of the mid trial changes the recovery threshold maybe with a brief powerpoint overlay.

These things have been done as I said but I think that there is room for multiple ways of presenting the same important points. So in this case we'd need an 'authoritative' figure who calmly presents the other side of that story.

If only we had more people outside of sick community members who could assist with this sort of thing.

large donner · May 21, 2018

Lucibee said: ↑

Just came across this on Lancet TV. (Sorry in advance...)

https://www.youtube.com/watch?v=jYRjB_t8dYs

Click to expand...

"comments are disabled for this video"

You dont say!!

Andy · May 21, 2018

I'm not offering but how about doing a video version of our PACE Briefing Document? https://www.s4me.info/threads/science-for-me-pace-briefing-document.3140/

Log in or Sign up

A general thread on the PACE trial!

Sarah Senior Member (Voting Rights)

adambeyoncelowe Senior Member (Voting Rights)

Lucibee Senior Member (Voting Rights)

Jonathan Edwards Senior Member (Voting Rights)

Indigophoton Senior Member (Voting Rights)

Barry Senior Member (Voting Rights)

Barry Senior Member (Voting Rights)

Indigophoton Senior Member (Voting Rights)

Barry Senior Member (Voting Rights)

Amw66 Senior Member (Voting Rights)

Tom Kindlon Senior Member (Voting Rights)

Sean Moderator Staff Member

Sly Saint Senior Member (Voting Rights)

Lucibee Senior Member (Voting Rights)

large donner Guest

MeSci Senior Member (Voting Rights)

Lucibee Senior Member (Voting Rights)

Snowdrop Senior Member (Voting Rights)

large donner Guest

Andy Committee Member

Share This Page

Log in or Sign up

A general thread on the PACE trial!

Sarah Senior Member (Voting Rights)

adambeyoncelowe Senior Member (Voting Rights)

Lucibee Senior Member (Voting Rights)

Jonathan Edwards Senior Member (Voting Rights)

Indigophoton Senior Member (Voting Rights)

Barry Senior Member (Voting Rights)

Barry Senior Member (Voting Rights)

Indigophoton Senior Member (Voting Rights)

Barry Senior Member (Voting Rights)

Amw66 Senior Member (Voting Rights)

Tom Kindlon Senior Member (Voting Rights)

Sean Moderator Staff Member

Sly Saint Senior Member (Voting Rights)

Lucibee Senior Member (Voting Rights)

large donner Guest

MeSci Senior Member (Voting Rights)

Lucibee Senior Member (Voting Rights)

Snowdrop Senior Member (Voting Rights)

large donner Guest

Andy Committee Member

Share This Page

Useful Searches