Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Please could you confirm that I understand correctly that, here on an ME patient forum, you have negatively described our communities efforts to highlight the flaws in the current Cochrane review of exercise for CFS as "pounding", and used this as a reason why Cochrane, as an organisation, are reluctant to take responsibility for updating this review.

 

Except, as you noted in your evidence to the Scottish Parliament: “these trials generally tell us nothing useful. However, the PACE trial at least may tell us more – that the theory behind the treatment is actually wrong.” If Cochrane were to come to the same conclusion, that would be useful.

@Hilda Bastian Can Jonathan’s evidence to the Scottish Parliament be added to your list of documents?

Should the NICE document which grades all the outcomes from exercise studies be added to the list of documents too (https://www.nice.org.uk/guidance/gid-ng10091/documents/evidence-review-7)?

 

I'll consult on the terminology "activists": obviously, as someone who themselves has a long history of activism, I don't consider it a negative term. And I've saw no sign that any of the people involved in this discussion see it that way - and there is great concern for the quality of the review. Indeed, had they had the views described here, there wouldn't be an activist in the author group at all.

Wanting to ensure that the scientific work is not just of high quality, but also widely agreed to be of high quality, isn't a negative in my view.

If they wanted to control the process, the advisory group wouldn't be an independent one, I wouldn't have been given this role, I couldn't have chosen the advisory group's members, I'd have had no involvement in the editorial arrangements or the author team. Communication would be handled by the Cochrane communications team, with me needing permission to do something as at high risk for criticism as participating in this Forum (and in a truly controlled situation, a request like that would no doubt no denied - I'm going here by my experience of having had requests in other organizations in other contexts denied).

If everybody agreed on that view of the evidence, it would be a very different situation. But they don't. If they did, multiple versions of NICE reviews, the AHRQ review, and the CDC one, wouldn't have come to different conclusions about evidence that was included in each of them.

 

I never wanted the review to be done at all! Still don't. I have now asked twice why Cochrane using a different process to do the same job that NICE have just done for all interventions for just one intervention is going to add useful knowledge that we don't already know. Why is the Cochrane process better? If the Cochrane process explicitly fed back to funders of any poorly designed studies of exercise that might possibly be identified to inform them that public money had been wasted, that would be quite useful I guess.

 

It's not tone policing from Cochrane. I don't see activist as a pejorative term - I have described myself as an activist, for many years primarily as an activist, and done so with pride. I see what you mean, though, so I'll consult about a better way to describe this, and I welcome suggestions (and I've just added your language of "patient community" to a working list). I agree that the language matters, it always does, and there should be a way to describe the situation that at least lessens these concerns, and doesn't tap into the language that people in this complex terrain use to be derogatory about each other.

 

I think it’s clear that most of those with vested interests in promoting CBT/GET as safe and effective treatments are unlikely to agree with criticisms of PACE methodology etc. The best we can hope for is agreement between independent scientists who understand trial methodology (like Jonathan) and expert patients. (The £600 Jonathan and I offered in 2018 to any mainstream independent scientist who is willing to defend PACE as “good quality” science remains unclaimed.)

Also, please can my BMJ rapid response on harms of GET be added to the document list (https://www.bmj.com/content/371/bmj.m4356/rr-0)?

 

I appreciate activism from your perspective is not a negative nor pejorative term @Hilda Bastian & why should it be?

However in our world the term has been used to smear & silence us. The label "activists" has had the effect of making it acceptable to ignore our views as unreasonable , demanding & uneducated. Reports of threats made by ME activists were used, without proof given, to paint the ME community as mentally unstable, in denial as to the cause and treatment of our condition and so forth.

Patients who spoke out, reasonably and politely, about harms caused by treatments and the poor quality of research were dismissed as militant ME activists and accused of putting researchers off working in the field.

In their defence at the PACE tribunal, one of the witnesses speaking on behalf of QMUL likened ME patients (who requested data through proper FOI channels) to animal rights activists and, if memory serves, the type of climate change activists who used to threaten violence and damage property.

All the more insulting because those very "activists" are barely able to get themselves to the loo or brush their teeth. In fact one such "activist", Alem whose FOI request was successful, has been so very severely ill ever since is still, to the best of my knowledge, entirely bedbound and dependent on care from family. So the term activist can seem.....taunting.

I'm not saying this to embarrass, or claim your perspective of activism is any less valid than ours. I am saying it to demonstrate that your perspective and the perspective of someone who hasn't been involved in the community is very different.

This lack of understanding and knowledge highlights to me many of the concerns raised by others. It isn't necessary or even expected that every member of the IAG will have that background perspective and expertise in both the problems of research into ME and how that influences the daily lives of patients but we, the patient community, do need to be confident that at least some of them do.

 

Thanks for pointing that out - I'll get it updated next week. That was the original wording, and as the report makes clear, there is going to be a new protocol now. Originally, I'd thought a sensible way to start was with pointing out problems in the original protocol: but assembling and working on a distillation of all the problems people had already pointed out was a better way to go, I think (and that's what we've done).

I'm not sure what you mean about Cochrane saying they would have a new review: I've only ever seen them talk about an update.

 

Thanks! (I won't respond to everything here, especially where I've commented on the issue.)

I take your point. However, there has also been lots of public criticism of how Cochrane excluded and misinterpreted or disregarded ME/CFS stakeholders. It was a trajectory that had to change, and consumer authorship is a part of doing that decisively.

I wouldn't have proposed anyone for authorship unless I was convinced they would bring keen scrutiny to the methodological work, and both an interest and competence in understanding the critical feedback that will come about the author team's decisions. I agree it's impossible for people to assess that themselves when a person hasn't been publishing analyses and comments.

I agree that some people see consumer authors as the way you described, or as patient representatives, rather than as equal co-authors. Like many perceptions, these are ones that we're stuck with for now, but I hope will be changed over time.

I know that would have been a popular choice in the ME/CFS community. I considered this deeply and agree they would make a great contribution. Other people would make different choices obviously, all along the way here. Based on my experience as a systematic reviewer and participating in systematic reviews with multiple people with strongly held opinions, in my opinion there were others who were surer bets to be effective at the range of aspects of this task.

The commenting system will still be available for the protocol and review. The IAG's role in engagement or distilling discussion points does not mean that comments will be paid less attention to than they otherwise would be. But given our role in this process - and that will be expanded on in the next report - it's essential for us to be able to audit as we go along, to make sure issues don't fall through the cracks and get due consideration: and for us to judge whether they have been addressed satisfactorily in our opinion.

I think inevitably the first report of a group on stakeholder engagement focused on the stakeholder engagement thus far. I think that the analysis of previous critiques of methods etc and establishing a system to ensure they are considered is a major undertaking that isn't "about making everybody happy". We will be publishing a major litany of criticisms without addressing the validity of those criticisms or the counter-arguments made to them: that is not a recipe for making everybody at Cochrane happy, for example.

I have no doubt that it's impossible for any review to be considered free of methodological error by everyone: there are disagreements about everything, and even some of the critiques from the ME/CFS community are contradictory. I'm sure there will still be comments and critiques afterwards. Our role won't be over when the review is published, although it won't be more than a few months. The commenting system doesn't get closed off for any Cochrane review. I think if there's been thorough discussion of a point, and the reasons for a different choice have been made clear, presumably pointing to that discussion would be the response. I don't believe the new editorial team would disregard and dust off new insights, but we'll certainly be advising Cochrane on this issue at the end of the process.

(Edit: I had broken the link that showed what I was responding to at the top - fixed that.)

 

Goodness, no! I don't agree I said anything like that. And Cochrane is obviously updating this review, so I can't imagine why on earth you'd say I said they were reluctant to update it. On the other hand, while it makes me sad, I think it's understandable that when there's a hot potato, and it's not unambiguously in one group's court, that people don't rush to grab it. That there's a group who have embraced it, and for the right reasons, is highly admirable I reckon - good on 'em!

There is no way the vast majority of people in these communities use that style of activism: there's a huge diversity in how individuals engage in activism, all along its very long spectrum of styles and tactics. But some do it, advocate it, and/or applaud when others do it. I think the ME/CFS community is subjected to extraordinarily unfair demonization because of that, as though it's everyone. And I've criticized that strongly, describing it as "a collective ad hominem attack". I'm not going to make it myself! And if I indeed believed it was characteristic of the whole community, I wouldn't be putting myself into this position. There are people who love to argue and fight, but like most people, I'm not fond of putting a giant target on myself and walking onto a battlefield when it's not even an issue I have a stake in - I don't have ME/CFS, none of my family does, I'm not a member of Cochrane, and I had nothing to do with that Cochrane review until I started to criticize it myself.

That I have opinions about effective advocacy is not news and shouldn't surprise anyone, any more than it should surprise anyone that I'm critical of the demonization of an entire community because of the behavior of a few.

 

Thank you! The first one doesn't mention the Cochrane review, but I'll definitely look closely to see if there are points that are not already included.

The NICE document will definitely go into the category of reviews that aren't in accord with the Cochrane review, and I and others are definitely going through it several times to see if there are new discussion points that haven't been raised before.

 

Thank you - I will be really careful about the use of the term in future in that context: I'm sorry about this.

 

Thanks, Robert! I'll go through it carefully for new points and literature, and re-visit how to handle things that are not specifically about the Cochrane review.

In the meantime, just a reminder that documents that aren't specifically about the Cochrane review can be added to this open spreadsheet.

 

Yes, I agree it's one of the discussion points raised in the past, in a document already included.

This review is going ahead - and it won't be expanded to cover all interventions about the condition. If that were on the table, my personal advice would be not to leave this as it is for another few years: and as retraction is not being considered, that would be the effect.

There are many Cochrane reviews on topics reviewed by others, and that will continue to be the case. I think there can be advantages and disadvantages to reviews by different groups: I don't believe any one "brand" is always better. I have read the NICE review, but haven't analyzed it yet, and there isn't a protocol yet for the Cochrane review.

 

This.

 

What's this "behaviour of a few" I keep hearing about? What did these "few" actually do? I genuinely don't know, but it must be pretty damned awful. Did they try to occupy public spaces, damage property, break into researchers' offices or homes, or place researchers in fear of physical harm or for their lives? Given the limited energies of people with ME, that last one would be one pretty good confidence trick.

I certainly hope the dreadful abusive "few" does not turn out to be people who were angry on twitter or who asked angry questions at conferences or who wrote letters critiquing published articles. Or worse still, those who made FOI requests for data from various trials. Those are all to me, entirely understandable responses to the situation.