Cochrane review and the PACE trial

In the debate on the PACE trial yesterday, the Cochrane review was again cited as validation of the PACE trial results
".........scientists as part of normal research collaboration, including the internationally respected research organisation Cochrane, which independently validated the findings."

I thought it would be useful to have a dedicated thread to discuss this/post relevant info.

Exercise as treatment for patients with chronic fatigue syndrome
"This review is an update of a previous Cochrane review from 2004, which showed that exercise therapy was a promising treatment for adults with CFS. Since the review, additional studies investigating the effectiveness and safety of exercise therapy for patients with CFS have been published."

http://www.cochrane.org/CD003200/DEPRESSN_exercise-treatment-patients-chronic-fatigue-syndrome

(I see that it comes under depression........hmm).

eta:
old article
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003200.pub4/full
most recent
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003200.pub7/full

 

They refused data to Keith Geraghty and 3 of them sat on the cochrane review.

https://twitter.com/user/status/966251723638665216


They also cited that the authors have published responses to criticism, but were not able/didn’t manage to publish anything meaningful in the PACE special edition JoHP. Can’t remember their reason though.

 

It's sort of hard to see the Cochrane review as having independently validated the PACE findings, when on page 31 of the decision by the ICO tribunal on the releasing of the PACE data, Trudie Chalder said that "all three of the PACE principal investigators sat on the [Cochrane] review panel". http://informationrights.decisions....iversity of London EA-2015-0269 (12-8-16).PDF

 

Were there any other major holes in the govt response?

 

I think we need to stress over and over again that the main criticism of the PACE study concerns the basic design flaws. They used subjective outcomes in a study that was not blinded, and they conflated chronic fatigue with ME/CFS. Yes, they also manipulated the results to boost the subjective improvement and recovery rates, and they screwed up the sub-group analysis of different diagnostic criteria. However, even if we disregard the manipulations and “mistakes”, the conclusion still is that there is no improvement in objective secondary outcomes to back up the modest increase in subjective scores. It is a null result.

The Cochrane reviews on cognitive behavioral therapy and graded exercise therapy for ME/CFS are meta-analyses based on studies with the same design flaws. Pooling over multiple studies reduces statistical errors, but does not change the inclusion criteria in the studies and does not convert subjective outcomes to objective ones. We need to emphasize that there is no evidence of objective improvement in the Cochrane reports.

 

This is also a concern

https://twitter.com/user/status/966335904439787522

 

I strongly believe that the Cochrane review is a mega-important issue for us to focus on at the moment. Even more so than PACE.

The PACE authors will present the Cochrane review to NICE and claim that it upholds their own findings.They will have a strong case that will need careful, analytical and rigorous rebuttal.

We won't easily be able to challenge the use of CBT and GET as long as the Cochrane review stands.

Either we have to challenge the Cochrane review or we have to make sure that NICE understand that the Cochrane review relied on the Oxford criteria; meaning that it doesn't investigate ME/CFS. The Cochrane review shouldn't claim to examine chronic fatigue syndrome while relying on the Oxford criteria; it should claim only to investigate 'fatigue'.

It does also seem to be a conflict-of-interest if the pace authors sat on the Cochrane review panel. (Although I'm not quite sure what that is.)

Importantly, there are many technical flaws and factual errors in the Cochrane review which have been highlighted in the letters submitted and published in the latest version. These need to be followed up and pursued, vigorously.

 

That is so brilliantly and succinctly put @Bob.

 

It might be worthy to highlight the extreme vetting the NIH is doing for the intra-mural study. Dr. Nath reports have to screen 200 suspected ME/CFS patients, just to find 20 study participants. Also, the purpose of the formation of Common Data Elements is to ensure that trial outcomes can be compared.

I used to work as a therapist using an evidence-based model. Because the model is evidence-based, agencies all over the county dish out huge grants to employ agencies that use the model. Every case had tremendous amounts of data collection and reporting. But, guess who generated the assessment forms, analyzed the data, and published? The same group of psychologists who created and patented the model. It was obvious working from the inside how skewed their interpretation of positive outcomes was. Evidence-based psychology is junk science without quality control, objective measures, and independent review.

I'm afraid we live in an era where facts are not reliably facts, since they are manufactured outcomes...PACE being the worst example.

 

Am I right in thinking that this is in effect the penultimate review and that another review is 'in the pipeline'. I was asked to review a new version with authors including I think all of White, Sharpe and Chalder some time last summer. I made it clear that I thought this was poor quality. What I do not know is what has happened since.

 

The PACE trial is now seen as a big piece of shit, null results on a subjective non blinded study with a made up criteria. If the Cochrane review supposedly backs up the findings of the BPS crew from the PACE trial doesn't that mean that the review itslef found all the other papers to be a big bag of shit too?

One cant use Cochrane review to advocate GET and CBT using PACE and visa versa, if PACE is now being seen as a null trial.

It seems to be that the BPS crowd are saying, "Ok well PACE the £5 million definitive trial may be crap but our pre PACE prejudices that we couldn't prove via PACE are backed up by other studies, after all we authored a lot of those other papers then sat on the Cochrane review panel".

 

The other issue is that the Cochrane mental health group which lays claim to ME/CFS was co-founded by Simon Wessely. Some members of the group likely continue to have a vested interest in the outcomes of their own reviews (CBT/GET clinics and/or insurance ties), and I also suspect that Wessely and others have maintained an influence in it.

 

There is a new review in the pipeline which will analyse individual patient data for exercise therapy. [Exercise therapy for chronic fatigue syndrome (individual patient data)] At the moment only the protocol has been published, which was authored by the PACE authors. I suspect you were asked to review this review.

The existing review (exercise therapy for chronic fatigue syndrome) was first published years ago and was republished a couple of years ago with new data. It doesn't analyse individual patient data but only looks at effect sizes. The review includes substantial errors, untruths, and outcome switching etc. (Similar to PACE but maybe even worse?) The PACE authors aren't officially listed as reviewers for this review.

 

Unfortunately the PACE trial hasn't been retracted. This means that the study and its results are still a legitimate part of the scientific literature even though it is a contested paper.

The main complaints about PACE are that the data have been promoted and interpreted incorrectly; that it uses subjective outcomes; that the recruitment criteria aren't appropriate; and that the basis of the therapies aren't appropriate. On the other hand, no one has actually contested the data itself; Only the interpretations of the data have been challenged.

Cochrane can legitimately claim that it is appropriate for them to review the PACE data in an analysis of subjective outcomes from nonblinded trials that investigate 'fatigue' (using the Oxford criteria). I think it's legitimate as long as they clarify that bias is inheritant in all of the studies and that it doesn't investigate chronic fatigue syndrome. I don't think it is legitimate to claim that the review investigates chronic fatigue syndrome when the studies recruited patients based on a 'fatigue' criteria.

The Cochrane review is an analysis of subjective outcomes from non-blinded sstudies using Oxford criteria, or similar, for recruitment. In that respect it is no better than PACE. It is legitimate (as long as it is transparently stated in its aims, which it isn't) to carry out such an analysis of psychological therapies to treat 'fatigue', but it does seem like a rather pointless exercise, and it isn't relevant to ME/CFS.

Our task needs to be to challenge the Cochrane review directly, and/or to make sure that NICE understands the failings of the Cochrane review in relation to ME/CFS.

i.e. the failings are that it is completely irrelevant to ME/CFS patients, because the studies investigated used non-ME/CFS recruitment criteria; that the trial investigates subjective outcomes from non-blinded studies, so there is inherent bias; and that there are flaws and errors in the Cochran review, which have not been corrected.

Conflicts of interest could also be raised but this is a bit of a tricky area, and will probably be impossible to challenge unless we manage to bring down the entire scientific establishment.

 

re Oxford Criteria I recall another discussion:

quoted from: https://www.s4me.info/threads/iime-...recommended-treatments.1949/page-3#post-34540 #54 & #56

 

This succinct article "Time to Reject the PACE Study" criticizes both PACE and the Cochrane review of PACE as well:

The Cochrane review conflicts of interest in particular relate to the role of Dr Lillebeth Larun, which Prof James Coyne details in these blog articles.

 

Lets do it!

Thanks, I'd forgotten about some of those Coyne blogs. What was the COI stuff? Was it just her involvement with the IPD review? There's so much to read at the moment - my eyes have been complaining.

 

It might be easier to just bring down the mental health Cochrane group, especially in specific regards to ME/CFS. The various Cochrane groups seem to act with fairly little oversight, so most of the blame can land on that one in particular without bringing the wider Cochrane brand into the discussion too much. Though Cochrane is failing to enforce quality controls, and their failure to deal with a rogue group would suggest that they have some serious faults in general.

 

This is the most recent:
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003200.pub7/full

25 April 2017
(I added it to my first post shortly after I'd posted the excerpt)

This was their reply to Coyne:
http://community.cochrane.org/news/cochrane-and-conflict-interest

eta: the PACE trial authors also responded but the full article is behind a paywall,
anyone have access to it?
"
Controversy over exercise therapy for chronic fatigue syndrome: continuing the debate
In a recent Round the Corner, Mitchell commented on a Cochrane Review of exercise therapy for chronic fatigue syndrome (CFS). One of the trials included in that review, and discussed by Mitchell, was the PACE trial. In this month’s Round the Corner we are publishing a response we received from authors of the PACE trial (Chalder, White & Sharpe), together with Mitchell’s reply. Ed."

Sept 2017

http://apt.rcpsych.org/content/23/5/288

 

My brain is the organ that complains the most about the workload (isn't it ironic that when you are too ill to work in a salaried job, you actually find you work longer hours due to involvement with ME/CFS research and politics than you ever did in regular employment).

I only briefly glanced at Coyne's blog, but this is what he says about the COI: