Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Apologies for just popping in.

Don't know where else to put this so leaving some notes here regarding caveats when critiquing most common features of some researchers' claims about trial methodology and alleged benefits from therapist-delivered treatments for ME/CFS.

- Objective outcomes:

It's true that objectivity with regard to outcome measures is always a 'spectrum'.

And even an indisputably objective outcome like death needs to be interpreted: Is it caused / delayed due to the investigated treatment or due to an other cause?

So, there is a hierarchy of robustness with regards to objective outcome measures and those outcomes need to be analyzed properly, but on the other hand, there are subjective outcomes that clearly are subjective, and only subjective -- both subjectively reported and only subjectively assessable.

The point that certain objective outcomes are not entirely objective is not valid with regard to differentiating between subjective and objective outcomes in the context of reliable trial methodology.

- Improvement versus recovery

This point is somewhat related to objective outcomes like going back to work or school.

It there is no cure, of course every relevant improvement should also be counted as benefit, even if it doesn't lead to restored working ability.

I think that needs to be taken into account when evaluating treatment trials that claim to have found evidence for improvement of symptoms but not for restoring working ability.

If there were any reliably measurable, relevant improvements, that actually would be better than nothing. But behavorial treatment and physiotherapy ME/CFS trials I'm aware of didn't show evidence of improvement beyond a short-term placebo effect in a minority of patients.

I think though that selling placebo as evidenced-based treatment could be counter-productive to coping with a life-altering illness and could cause long term mental suffering and other harm.

Perhaps it would be worthwhile to discuss what people with ME see as relevant improvements if they can't go back to work or school?

This post has been copied to start a new discussion thread:Clinical trial outcome measures of improvement and recovery in ME/CFS - which ones are useful? Discussion thread

 

That sounds a lot like quality assurance. I love QA, in software development it is a critical part of the process. One that few appreciate, because all they do is point out mistakes. Like it's their job. Which it is!

I see no evidence of quality assurance anywhere in the process of clinical evidence or medical practice. QA does not end on release, it is a continuous process that not only works with feedback but is proactive in gathering information, all information, so as not to miss anything. It is obsessed with not deleting information. The rare good kind of obsession.

This process does not exist in medicine, or I have seen no evidence of it anywhere. I personally think it's the systemic change that is needed, one that assumes that mistakes will be made, rather than categorically rejecting the premise of there being a mistake as vexatious and bound for the shredding cabinet.

Sure would be nice. Life-saving, even. In software we do this for junk like marketing campaigns and we do it proudly, because we know we make mistakes all the damn time. The reverse approach, I must say, is very, very bad. Boeing learned the same, but they have financial pressure. Medicine has no such pressure, doesn't even take the main stakeholders, the patients, into account at any point. What an odd system.

 

I think the Powers-That-Be might dispute the idea that patients are the main stakeholders in medicine. They would suggest that the people reaping the profits are.

 

https://twitter.com/user/status/1381662201690345472

 

Her answer: I'm sorry - I know it's inordinately slow: certainly didn't expect it to be. Not going to pre-empt the report, I'm sorry.

 

Robert Saunders: Thanks. Can you clarify, are you referring to a progress report or the new review of exercise therapy for ME/CFS? If it’s the former, what is holding it up? This was billed as a “pilot project for engaging stakeholders (such as consumers...)”. As a consumer I feel let down.

Hilda Bastian: Progress report. I'm sorry to have let you down.

 

Did we misunderstand the part about this being part of some sort of pilot project for patient engagement? Because it looks identical to the old patient engagement of there being none whatsoever. Still in the wrong category (which of course is intentional and meaningful). Still the wrong definition and framing. Still harmful advice based on awful evidence, all blocked by people with no stake in the matter. Still astrology-level pseudoscience with conclusions detached from even the mediocre evidence.

However it's Cochrane who let us down. Or more accurately is failing in their mission, whatever it is because frankly I can't tell at this point. This is a strictly volunteer thing and a big ask for one person who is genuinely busy with important things, which I can't fault, Bastian's work on vaccine evidence really is important and influential in the right way. But it's true that very few people can actually take on this task and it must be hard to get people interested to contribute to it, tangible career risk and all. Makes for very broken incentives.

5 years and nothing to show for it, with an arbitrary veto from the only people who stand to lose from it, and not even any significant loss. With millions of lives hanging in the balance this is just terrible all around, a system broken beyond any hope of repair.

 

I'm going to post all the information about this that I can find, because the whole thing has been as clear as mud so far.

On October 2, 2019 - Cochrane announced publication of the amended version of the Exercise review, and also made this statement (https://www.cochrane.org/news/cfs):

On 13 February 2020, Hilda Bastian was announced as lead for the Independent Advisory Group (IAG): https://www.cochrane.org/news/appointment-lead-independent-advisory-group

We were promised further updates - but nothing so far has been forthcoming. Many of us emailed the group, but have not received any further information.

This is the document that Hilda Bastian posted on 9 April 2020 (which I've only just seen): https://community.cochrane.org/orga...eholder-engagement-high-profile-reviews-pilot

The relevant section on the IAG is here:

As far as I'm aware, the members of the IAG have yet to be appointed. So the process hasn't even started yet.

Has anyone actually signed up for updates and heard anything? I emailed the IAG directly in March last year, thinking that would trigger updates, but that was probably a mistake, since I have not heard anything since. From Hilda Bastian's recent tweets, it seems that nothing has actually happened yet.

 

I signed up for updates on the 10th March 2020, and am yet to receive any updates via that route (as opposed to my brief email correspondence directly with Hilda middle of last year where she confirmed that there was no update).

 

I have.

No.

@Hilda Bastian
I understand that there are situations where priorities shift.

Not even letting us know in a formal manner, inform us on the webpage etc is what really annoys me, though.

If you want to take people with ME seriously, then you have to do something that takes us seriously.

At the very least, why can't you just put a simple notice on the webpage and let subscribers know?

I don't understand why you need to finish a report to let people know that there will be a substantial delay.

Also, I don't understand why you need a report to let us know if people have been appointed to the IAG.[*]

Ideally, you could consult patients how to proceed if you won't have sufficient time for your job for a while.


Edit: [*] Or if invitations have been sent -- see this post

 

yes i signed up; no i haven't heard anything

 

I signed up and have never heard anything at all. I have written several times to the IAG address with suggestions. I have received acknowledgement every time, and once, in the early days, a reply from Hilda to my questions - will dig that out.

Cochrane have rejected my request to withdraw the "current" reviews (Exercise and CBT) despite the fact they could cause harm to patients if the findings were followed. I sent details of that to Hilda.

I have submitted a complaint about Cochrane's refusal to withdraw the reviews, or to independently review this decision when I appealed against it, to the Committee for Publication Ethics (www.publicationethics.org). I did this a month ago, but have heard nothing back from them yet.