Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

I'm happy to look into it further, but as the review has not been retracted it seems that Cochrane is standing behind the conclusions.

He is not technically correct here. Cochrane have not retracted because their rules did not allow them to if the authors disagreed. What is happening now is not so much standing behind conclusions as trying to pretend the mess on the floor isn't there.

 

I so wish I could remember the name of the German theatre company doing the very funny non-verbal show at the Edinburgh Children's Festival, where they made a mess on the floor with paper and then tried to conceal it with more and more paper. I'd like to ask them where they got the idea.

But anyway, who's going to do the review of Cochrane's rules to establish whether or not they ensure the worst science is firmly embedded as mainstream?

 

Well @dave30th certainly enjoyed ripping up a copy or two of PACE...

 

I am planning to write to COPE (of which Cochrane is a member) to complain (https://publicationethics.org/complaints-and-concerns-about-our-members) The new Cochrane withdrawal policy was changed suddenly in August 2019 with no transparent decision-making process behind the change (https://documentation.cochrane.org/display/EPPR/2019/10). I think it was decided in a hurry because of the pressure by me and others to withdraw the CBT and Exercise reviews. Of course I will never be able to prove that.

Now one of the very limited reasons given for withdrawing a review is a "serious error" which means following the conclusions might cause harm to patients. (https://documentation.cochrane.org/display/EPPR/Withdrawing+published+Cochrane+Reviews)

On the one hand this makes it look like Cochrane care about patients. On the other, they rejected my request to withdraw the reviews because they decided there was no specific (ie. statistical, or other technical) error. So the fact that harm may be caused to patients by their publications because they are out of date and/or poorly conducted, or contradict more recent thorough evidence reviews, as done by NICE in this case, is not their concern.

This doesn't seem to be ethical publication practice to me, especially for an charitable organisation whose motto is "Trusted evidence....informed decisions...better health". I am fairly sure COPE will disagree and think it's all absolutely fine, but you never know. Even if they do agree, they can only make recommendations to their members, which their members can ignore. Hey ho. Keeps me off the streets.

 

That's a good idea...how do I go about flagging it with them?

 

I will give it a go!

 

Thank you for your efforts!

 

Perhaps it wasn't transparent to you because you weren't part of the organization? A Cochrane policy like this one takes a lot of consultation and debate among the 50+ editorial groups and others before a major editorial policy is changed, and they don't tend to be changed at short intervals as a result.

This one was part of the whole process of updating policy, not just the issue of withdrawals. And it took was under discussion for years, as I recall. (As the policy around withdrawals was directly relevant to my work when I was at the National Library of Medicine, and to my PhD studies, I used to check in every now and then to see if it was getting close to being finalized.)

I think, to be honest, the backstory of these reviews and Cochrane isn't that there's a history of them being so sensitive about them that they would do something drastic organization-wide because of them - but more that they weren't a priority, I'm afraid. Most healthcare issues suffer from this, everywhere, not just Cochrane. There are many healthcare issues that attract more attention, powerful stakeholders, and controversy than ME/CFS.

 

Yes. I understand what you're saying. I guess then I am suggesting the withdrawal policy was possibly finalised in a hurry. I don't think it was a good policy change as it now allows out of date reviews to remain in a trusted database of what is assumed by most to be the best available evidence. Cochrane authors used to make a commitment to update the review every two years, unless that Cochrane policy has also changed. Now if they decide they don't want to update the review, then that's also accepted as ok. There has been no effort from Cochrane, as far as I know, in the case of the CBT review to recruit a new author team. Or to approach Vink, who has already done a review of CBT, to Cochranize his.

There is no information about when a review is being updated because of its outdated methods and assumptions about the condition, or so out of date it shouldn't be used, displayed on the public-facing version of the reviews. This is the version that comes up first in a Google search.

 

I don't see any indication that it was hurried, nor any link with these particular reviews. They didn't need to suddenly change the policy to not withdraw these reviews - they'd been not withdrawing them under the previous policy, too.

 

To be honest @Hilda Bastian, for those of us who have got fed up with Cochrane being disingenuous about these reviews there is every reason to think that this decision was disingenuous as well, even if it is a coincidence. If Cochrane wants to restore any credibility for itself it needs to start making valid decisions. One thing they could usefully do is get rid of GRADE and RoB2, which are clearly unfit for purpose. The more I look at this the more the whole system looks to have lost the plot.

 

David Tovey announced the withdrawal of the exercise review under the old policy and then changed his mind because of pressure from the authors. He said explicitly that the GRADE rating of "moderate" for the quality of the evidence for the fatigue outcome was not defensible.

A month after I and two patients met with Karla to beg her once again to withdraw the review rather than publish the amendment, the policy was changed. Of course it could be coincidental. It doesn't really matter. I still think it was a change in the wrong direction. And not just for people with ME but for everyone.

 

Cochrane withdrew the Chinese herbal medicine for CFS review in 2018 under the old policy because it was out of date. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD006348.pub3/full

When I asked for the CBT for CFS review to be withdrawn for the same reason - it's even more out of date - they said the policy had changed so they wouldn't. https://www.cochranelibrary.com/cds....pub2/detailed-comment/en?messageId=261273171

 

I have now made a submission to COPE via https://publicationethics.org/facilitation-and-integrity-subcommittee. I have covered the specific issues around Cochrane's refusal to withdraw the two CFS reviews, or to independently review my appeal against that decision. And more generally about having editorial notes such as "do not use for decision-making" on reviews which are not visible from the plain language summary and author conclusion sections of the review which are made accessible on Cochrane's main website. This is the version which pops up first in a Google search. I view this as a clear breach of publication ethics. It will be interesting to see if they agree.

 

PS - please don't Tweet about the fact I have made a submission to COPE as one of the condition is that they are left alone to get on with investigating...if they do investigate of course

 

And they begin in such a nice manner:

"high quality" should well include also to go through open questions and even contradictions, especially when they have already been discussed.

If not, it must be considered to be of low quality -


Maybe I am naive, but the review is not much better done than by cribbing.
(@Hilda Bastian)

 

This was the strategy BMJ used to hide for over a year the fact that the Lightning Process study was being examined for serious ethical and methodological violations. They put an editorial note on the paper, but it was on a completely different page that no one would have thought to look at. So no one reading the paper knew there were problems. I complained about this directly to Dr Godlee and she wrote back and said the BMJ's IT systems made it challenging to move the notice to the same page as the article itself. Yeah, right.