Cochrane Exercise Review Withdrawn - Individual Patient Data

I am always a bit confused when it is said White et al were reviewers on the 2017 meta analysis as I don’t see their names there? Thank you.

 

Thank you @Jonathan Edwards for your explanation.

Essentially the reviewers reviewing their own work. Fox - hen house, and all that.

Cochrane appears in a bad light as well, having offered co-authorship for the meta-analysis. Shocking that a trusted institution would be involved in this.

Makes one wonder what else may be uncovered.

 

Confusing @Amw66. I read that a GET review was shot down. This review is the individual data review. Who did the shooting? Cochrane?

This is starting to look like an Abbott and Costello routine of "Who's on first, What's on second." Thank you in advance for your explanation!

 

Cochrane's reputation? This whole scenario would seem to put Cochrane's reputation at risk in general, would it not?

 

Cochrane had clearly decided not to publish the IPD review itself--that was the one that had been sent around and gotten scathing outside reviews. They had published the protocol for that IPD review several years ago. Presumably, there was no point in keeping the protocol published if the review had already been rejected and wasn't going to be published.

In terms of next steps, my sense is that Cochrane is now being methodical in moving forward. They have promised (in comments on the review about whether it should be in mental disorders) to come up with decisions soon. Given the withdrawal of the IPD review protocol, it's possible Larun et al now see where things are heading and will be willing to withdraw the exercise review at the same time everyone puts out a nicey-nicey statement that all have agreed the illness should be moved out of mental disorders to X section of Cochrane.

 

I hope you ar right @dave30th. It may be relevant that the withdrawal of the IPD review has nothing to do with complaints. It was on straight peer review. Maybe Ms Kelland should ponder that.

 

How much is known of the scathing reviews have any details been published or leaked, I'd love to know what was said; I'm sure it was well deserved ?

 

@wdb I think you can rest assured that it was said, whatever it was you hoped was said. If not by me by others.

 

Things I hope were said:

An attempt to mislead the public.
Data so embarassing they can not be shown to independent scientists.
The authors pretend they have more than a placebo effect.
Junk science that doesn't meet basic quality standards required by Cochrane.

I would love to read these "scathing" comments.

 

Can I ask again what is meant when it says people like White were involved in the Cochrane review? Was he involved in the 2017 one as well as this protocol that has just been taken down, or were he, chalder and White just involved in the 2014 one where the protocol had been put up and the review though not published had received a lot of critical reviews? Thank you.

 

The PACE folks were co-authors on the IPD exercise review protocol and the unpublished review based on it. As I understand it, that's because Cochrane allows those who provide trial data to be used in IPD reviews to be authors on those IPD reviews. They were not authors on the basic exercise review that has been recently under dispute. However, that doesn't mean they weren't involved advising or in some other way behind the scenes.

 

White looked very involved in the IPD protocol - a co-author and the funder of a meeting for devising the protocol.

For the 2017 one (which was really published in 2014, but it gets a new publication date when feed back is incorporated) it says this:

"We would like to thank Peter White and Paul Glasziou for advice and additional information provided."

I so wish that I knew what was going on at Cochrane. The removal of this protocol has to be a good sign, but I've still got a sense that we're going to get a nasty surprise from them at some point. While Cochrane's getting criticised from a range of people already, and Wessely's influence continues to spread, I'm sceptical that they're going to be willing to do things that will fundamentally challenge the myths he's built around himself.

I've been wrong about the last couple of Cochrane moves though, so I should probably stop trying to make predictions. It is possible that they're just clearing up some of the most indefensible problems in order to adopt a fall-back position that will be just as harmful for us, but with fewer outright absurdities to raise concern about with others.

 

Has somebody perhaps suggested that white was asked to review, i.e. referee, the April 2017 review? That would be entirely plausible. There would then be no mention of him as an author.

 

@Jonathan Edwards, sorry, no, I was just surmising as was just trying to work out the bps’s lot involvement in the Cochrane papers. Esther though did point out above White was thanked for his advice on the 2017 paper.

@dave30th and @Esther12, thank you, finally understand.

 

So, are you saying that the fact the Cochrane removed the IPD doesn't really change anything, or, worse, indicates that Cochrane never intended to publish the individual patients data in the first place? So, the removal of the protocol is not necessarily a positive thing?

 

I don't think this was David's intended meaning. I agree with David that it looks as if 'Cochrane', by which we actually mean the editor in chief David Tovey, is playing things by the book and methodically moving in a sensible direction. It may be unreasonable to expect him (as a person) to do more and this looks to me like a clever way to indicate that he understands what needs to be done.

I think maybe we should view Cochrane not so much as a group of people with a political agenda, although historically there was an element of that. but more as a public library with Tovey as the librarian.

The way a public library goes ultimately depends on the people using it. There is a building and a librarian who has certain rules to stick to but both the choice of books and the rules ultimately depend on what the users want. So in a sense the problem here with peer review has not been Cochrane itself but the community of people who use Cochrane - those who write and referee the reviews. Tovey can only try to apply rules consistently.

It looks as if there is a rule that says that once a review is accepted by peer review and published it can only be withdrawn by agreement. There is every reason to think there is not going to be an agreement. If librarians overrule rules they can easily be fired so Tovey does not necessarily have elbow room. If someone is allowed to take out six books instead of four they cannot then be fined for taking out too many books.

BUT, if, as has happened, Cochrane realised that peer review was not working well and decided to send out a review to a wider range of referees, and the standard peer review system yielded a resounding No Thanks, then Tovey cannot be criticised for withdrawing that review permanently. He is following the rules to the letter. Nobody can cry foul about patient activists or him being biased.

So I actually think this is a hugely positive step. It confirms that in good faith Cochrane changed their peer review policy, as we were told they had, and have now stuck to to the results. Thanks to Kate Kelland everyone who might be interested has been made aware that Cochrane would also prefer to withdraw the original review. That leaves that review with no credibility in the context of NICE or whatever.

 

So does that mean there is every reason to think it will never be withdrawn?

 

I guess that is possible. However, if another review is commissioned it would presumably replace the one now in place. The only problem with that is that it is doubtful that anyone will be motivated to do another review who is not biased towards making it sound positive.