Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Wow, how did all the crap about "moderate" make it through to the final publication, given all that discussion?

 

https://twitter.com/user/status/1266908377893896192


Capable of withstanding rough treatment from readers is exactly what a protocol for a systematic review has to be. And glue is apt for what should follow if a systematic review is to be trustworthy: the reviewers should stick to the protocol, with transparency and very solid justification for any time they don’t. We should be far more serious about how to achieve all of this. It’s critical to confidence in a systematic review’s integrity.

https://absolutelymaybe.plos.org/20...bout-bias-in-protocols-of-systematic-reviews/

ETA:
A fundamental problem with the original exercise therapy and ME/CFS protocol, it seems to me, was evident author bias in the text about both the intervention and the condition. Some of that bias in the text is actually encouraged by the current structure and guidance for Cochrane protocols, and it shouldn’t be.

 

Have just tweeted an interaction I had with Hilda late in 2018 -I had forgotten about it, and it includes some brilliant commentary from an anonymous person. It should be added to Hilda's commentary/critique sheet. I will ask her to do that.
https://healthycontrolblog.wordpres...ember-october-2018-via-absolutely-maybe-blog/

 

Responses on Twitter from Hilda Bastian to some comments here.

https://twitter.com/user/status/1266989998638678019

 

welcome to the forum @Hilda Bastian

 

Thanks to Hilda Bastian for taking so much interest.
My understanding was that the individualised patient data review I reported on was produced with the intention of addressing perceived weaknesses in the review still in place and would have superseded it. So it seemed to me to be an update of the same review - except that, presumably because the reports from a wider range of referees were so negative - the IPD review was pulled and never appeared.

 

Very insightful and necessary discussion. It only marginally concerns us but only because this is about the much broader issues in evidence-based medicine. That line about author bias, though, absolutely perfect. The underlying problems are as fundamental as it gets and are far from unique or even particular to the excessively poor evidence base for ME-BPS.

There really does seem to be a kind of suspension of disbelief that anything produced from EBM can only be good since they follow a protocol, never asking themselves whether the protocol is any good or if it was even followed at all, which we actually know is not even close to be the case. This is the kind of thinking that must happen if medicine is the least bit interested in solving the crisis of replicability.

Very little of the internal Cochrane discussion reflects an understanding of the magnitude of the problem, even when they involved the editor-in-chief. This thinking here is what's needed moving forward. I'm very much looking forward to what shape it takes as applied specifically on the issue of ME, especially the fundamental problem that even asking the question of exercise therapy in a disease defined by significant deterioration from even mild exertion is completely absurd and all problems derive from that original failure of asking the wrong questions for the wrong reasons and with the worst possible execution possible by the very worst possible people to do the job, followed by the very worst possible excuses not to fix that egregious mistake.

 

Thanks!

 

Hi, @Hilda Bastian!

yes, Cochrane rejected the IPD review, based on scathing comments from Jonathan and others outside the usual suspects on whom Cochrane had previously relied to vet these seriously flawed CFS reviews. And then it also withdrew the published protocol for the IPD review.

I reported on that here: https://www.virology.ws/2018/12/10/trial-by-error-and-cochrane-makes-another-move/

Apart from Cochrane: I would love you to look into the Lightning Process study published in 2017 in a BMJ journal--Archives of Disease in Childhood--that now has a 3,000-word correction appended. More than 70 experts from Columbia, Berkeley, Stanford, UCL, Queens Mary University of London, etc signed a letter of protest to Fiona Godlee at BMJ.

The investigators from Bristol University recruited more than half of the participants before trial registration, swapped primary and secondary outcome measures after gathering data from these participants, and published the findings without disclosing these changes. BMJ did not retract the study but let the original findings stand while acknowledging all the errors I had documented on Virology Blog.

This flawed study is now being cited by Norwegian researchers to justify a study of the Lightning Process there.

Here's an article about the BMJ study that I wrote last fall for STAT:
https://www.statnews.com/2019/12/13...awed-chronic-fatigue-syndrome-research-paper/

 

Thanks, Dave - will read.

 

thanks for the response! much appreciated.

 

I agree and challenge the assumption that updating a review which asks the wrong question is appropriate. https://healthycontrolblog.wordpress.com/2020/05/22/alternative_proposal_for_overview/.

The CBT for CFS review - another wrong question for the wrong reasons by the wrong people - is still up saying CBT is effective. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD001027.pub2/full.

There has been no move by Cochrane either to update or amend the CBT review as they have done with the Exercise review. The Chinese medicine one has been withdrawn (https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD006348.pub3/full), as part of a "strategic portfolio assessment of all existing and planned reviews and protocols, including those relating to Chronic Fatigue Syndrome (CFS)" But the Exercise and CBT reviews have not been withdrawn as part of this "strategic portfolio assessment". It is inconsistent.

 

If the wrong questions have been asked, what would be the right questions?

One good question is: How can doctors best support patients with ME/CFS even when there's no proven treatment yet?

 

Indeed! This is why Cochrane reviews (other than overviews) are often not useful to doctors or patients as they tend to focus on individual interventions or intervention types. If it's not a drug, device or surgical procedure, there seems to be an assumption that it probably doesn't do any harm, even if any benefit is marginal or unproven.

Patients and doctors are interested in the range of options available to them, and the risks and benefits of each when compared with each other. Cochrane reviews force people to look at individual reviews in isolation, each with a different author team, different inclusion criteria in terms of diagnosis, outcomes etc. And sometimes biases in favour of the intervention under scrutiny which is near impossible to spot. It would be better to do evidence overviews focussed on patient (or public) populations, defined by diagnosis, location, health system, patient-important outcomes, resources, language etc. NICE are not perfect, but they may end up doing a better job with answering the question above. I hope so.

 

Something like a condensed, practice-oriented IOM report would be very useful.

PS: there is a primer for pediatric ME/CFS which I like

Note the bolded parts.

https://www.frontiersin.org/articles/10.3389/fped.2017.00121/full

 

Thank you @Hilda Bastian for your article, you highlighted many relevant issues. Hopefully the Cochrane IAG will correct them so that the new review is not biased towards positive views on GET, which it currently is.

If possible at all, some input from @PhysiosforME and Workwell seems quite crucial as they are physiotherapists with an expertise on ME.

In the absence of a treatment, the very least doctors could do would be to heavily insist on the importance of pacing. They could provide leaflets from national associations (or translated documents if no national association exists in a given country) and recommendations of books on pacing. This is simple, easy to implement, but it isn't done yet (especially not in France...).

 

Unfortunately, the PACE trial effectively prevented that from happening even tho it has never been acknowledged that the APT supposedly being 'tested' is not what is generally considered by patients as pacing.
Subsequently, 'pacing' has been incorrectly interpreted as a kind of 'gentle' GET in many quarters including AfME (a UK charity) and most, if not all, of the 'specialist' clinics.

 

It maybe warrants a thought experiment. Imagine the PACE authors really had proven to be good scientists, and reported honestly, openly and fully about their findings in 2011. How different would the ME/CFS landscape be today?

 

Hard to predict but:

The forced separation of children with ME from their family, because they decline (further) GET treatment would most likely disappear.

The admission that CBT/GET as line of research has ended would mark the end of an era and probably stimulate and vitalize other research efforts in various ways.

Health insurance companies would try to find some new creative ways to delay or deny payment to patients with ME/CFS.

There would probably be some new quackery trying to establish itself.

PS: I forgot a really important effect. There would be a shift in how the illness is perceived by society.