Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

[cassava7]

I think inevitably the first report of a group on stakeholder engagement focused on the stakeholder engagement thus far. I think that the analysis of previous critiques of methods etc and establishing a system to ensure they are considered is a major undertaking that isn't "about making everybody happy". We will be publishing a major litany of criticisms without addressing the validity of those criticisms or the counter-arguments made to them: that is not a recipe for making everybody at Cochrane happy, for example.

The admission that some Cochrane members are reluctant to scientific criticism being shared openly, for everyone to review, is seriously concerning for the organisation's credibility as a whole. They should be keen on promoting post-publication peer review (à la PubPeer), not opposing it.

There was always going to be a diversity of views about exercise in this group: the other clinician, Todd Davenport, is very firmly of a different opinion. I was in favor of Julia Newton, and very not-in-favor of some others. I was in favor of her because of what I heard about her from others.

While I do not know the grounds on which you favored potential picks, relying on hearsay is problematic, especially when there is (recent) evidence that the person being considered conducted poor quality work on the matter at hand. The studies involving Dr Newton that I mentioned exhibit some of the same methodological shortcomings that prompted the review to be updated in the first place. Certainly this should be grounds for choosing someone else?

I would add that, depending on whose opinions might have been collected and had the methodological issues not been known, hearsay alone could have led to some proponents of exercise therapy for ME/CFS being selected as reviewers.

 

[Sean]

The last response to one of these FOIs was that the data could not be supplied because Peter Denton White had retired and there was no one else at QMUL who could extract the data......
miraculously this data has been retrievable many times over in the past year by PDW and associates to produce yet more papers on CBT/GET.

How is this not blatant fraud/perjury/false statement?

 

[Sean]

Transparency is something Cochrane need to address.

 

[Adrian]

possibly the most frequently used emoji on the forum.


The last response to one of these FOIs was that the data could not be supplied because Peter Denton White had retired and there was no one else at QMUL who could extract the data......
miraculously this data has been retrievable many times over in the past year by PDW and associates to produce yet more papers on CBT/GET.

And its worth pointing out that the step test secondary outcome from PACE has not been published

 

[Caroline Struthers]

Yes, I agree it's one of the discussion points raised in the past, in a document already included.

This review is going ahead - and it won't be expanded to cover all interventions about the condition. If that were on the table, my personal advice would be not to leave this as it is for another few years: and as retraction is not being considered, that would be the effect.

There are many Cochrane reviews on topics reviewed by others, and that will continue to be the case. I think there can be advantages and disadvantages to reviews by different groups: I don't believe any one "brand" is always better. I have read the NICE review, but haven't analyzed it yet, and there isn't a protocol yet for the Cochrane review.

When you analyse the NICE review, will you critically appraise its scope and methods (such as use of GRADE) or check the data and reproduce the analyses - or both?

 

[cassava7]

Were researchers who work on the quality (methodology, reproducibility or replication) of trials of exercise and physical activity in chronic illnesses asked to participate, either as reviewers or to sit on the IAG?

Someone who is familiar with ME/CFS and the evidence base for exercise therapy in CFS is Rosie Twomey, and while others in the field may not (immediately) be, they have likely seen the methodological issues that have been raised with the review elsewhere and understand them.

 

[Invisible Woman]

Again, I don't want to hassle you @Hilda Bastian but I am wondering if it's the case that you were unaware of the politics around patient involvement when it comes to ME. We are not your typical patient group thanks to the lobbying, smear campaigns and gaslighting that has gone on for so long.

It is quite usual for the go to person when discussing ME in the UK to be either a member of the establishment, or someone with close ties to the establishment. Generally the BPS pro exercise point of view is given dominance.

It is also true that the establishment by way of the BPS school have tried to pressure medical experts who also have ME to remain silent. Even going to the extent of making unfounded complaints and accusations against them to their employers. One example is Dr Keith Geraghty.

When trying to diminish or distract from valid criticisms made by the likes of Geraghty the establishment have claimed that patients with ME should be excluded from positions of potential influence because if a conflict of interest.

Patients with ME, be they expert patients, or experienced researchers who happen to have ME do not have a conflict of interest. They have an interest in rigorous, high quality research that yields accurate results with honest and open reporting.

Within the ME community we have many experts. In this forum alone we have doctors, nurses, university lecturers, teachers, statisticians, psychologists, IT & data specialists etc. There is a wealth of expertise well versed in the issues unique to this community.

Hence our suspicion when someone with close ties to the establishment has been chosen, with little to no input from us, to represent us.

 

[rvallee]

It's not tone policing from Cochrane. I don't see activist as a pejorative term - I have described myself as an activist, for many years primarily as an activist, and done so with pride. I see what you mean, though, so I'll consult about a better way to describe this, and I welcome suggestions (and I've just added your language of "patient community" to a working list). I agree that the language matters, it always does, and there should be a way to describe the situation that at least lessens these concerns, and doesn't tap into the language that people in this complex terrain use to be derogatory about each other.

Patient experts is probably the preferable term, as it also implies that not every patient participation is positive, the participants need to have minimal fluency on the topic, this is not something to learn on the fly, as one comment above explained it can actually be worse than no participation since it's too much information to absorb for a pwME.

Activist being pejorative is a specific thing in chronic illness, I don't think there's working around the fact that it's been used in bad faith to dismiss legitimate complaints. It's a positive term that's been corrupted in the context. But so have almost every relevant term, including most of the technical terms, including fatigue, so it's a special situation to have to work around common language because that language has been distorted specific to the context, as we have been reminded lately when there was a vague mention at a recent CDC/NIH conference call.

It's amazing how much damage can be done against a patient population by spreading vague mentions of unspecified misbehavior through the media. The consequence has been that anything we do is bad on principle, because we are bad militant activists, even if people like Richard Horton, who badmouths us as bad activists, proudly wears the activist mantle. Even when we literally do nothing but post scientific papers in a discussion. It's just weird like that.

 

[rvallee]

It is the Cochrane community that has been amateur. When Cochrane founder members (more than one) come out with the claim that sometimes you can just tell that a treatment works, from clinical experience, it becomes clear that they really don't understand the basics. It would be nice to educate them but they need to eat a bit of humble pie first.

Or a co-founder boasting about the superior powers of his mind for overcoming... 95%+ odds... one whose specialty is clinical evidence yet believes in the magical powers of his willpower and is willing to go on national television in multiple countries to spread the good news about how anyone can use exercise and positive thinking to beat.... 95% odds. I'm not even sure making the same claims about healing crystals would be any more humiliating and disqualifying.

Oddly enough, about the only time I saw significant public criticism of Cochrane was following the announcement of the retraction-that-never-happened and people like Clare Gerada were using their very prominent platforms to trash Cochrane and basically threaten their survival if it happens. That clearly rattled a few cages.

 

[rvallee]

Were researchers who work on the quality (methodology, reproducibility or replication) of trials of exercise and physical activity in chronic illnesses asked to participate, either as reviewers or to sit on the IAG?

Someone who is familiar with ME/CFS and the evidence base for exercise therapy in CFS is Rosie Twomey, and while others in the field may not (immediately) be, they have likely seen the methodological issues that have been raised with the review elsewhere and understand them.

I would say Todd Davenport fits that description.

 

[MEMarge]

I think @Sly Saint nailed this process in an earlier post

"a report about setting up a committee to debate how to write a report to answer an enquiry about a previous report on a review of an update of a review that was out of date but needed updating but was ok to leave in place while they reviewed how to set up a committee to review the evidence, stir twice and repeat."

 

[Hutan]

@Hilda Bastian, I sorry to have to keep going on about this, I'm really tired of this and I'm sure you are too.

But you still haven't explained how you came to select Kay Hallsworth. I think we've established that she expressed an interest in being on the writing team. And then, as there is absolutely nothing online that would indicate that she has a particular interest and expertise in ME/CFS research (other than she has ME/CFS) and you yourself are not an expert in ME/CFS research and politics, you must have asked around. I just can't see that any informed patient or patient advocacy organisation would have said 'ah, yes, Kay. She is exactly the person you need to have on the writing team to represent patients and inform the rest of the team about the problems with the ME/CFS exercise research, definitely the best person for the job'. I expect they would have said 'who?'. So what organisation was it that supported Kay for the job? She's a patient representative on a process that should be accountable to stakeholders, particularly the stakeholders with the least power and to who the outcome matters most. I think you must be able to say, even if only in general terms, who endorsed her. If I'm wrong, and a credible patient advocacy organisation endorsed her, please do say.

I think we have established that Kay has links with BPS proponents including Wessely and parts of the military interested in managing contested illnesses. If Kay has never met Wessely and White and Etherington, and does not support the statements they have made about ME/CFS and GET, then could she please make a clear statement about that? If she has met any of them and/or does support their views, I think she should also state that, so we all know where we are.

Meanwhile, it seems that you have chosen a BPS proponent to represent people with ME/CFS on a review that was prompted by people who view the BPS proponents as having caused people with ME/CFS great harm. I've appreciate you engaging here, but I'm not getting any clear sense from you that you understand how problematic that is. From here it looks as though you have done something that goes against all that you stand for.

If you hope to make this process credible, there needs to be an expert patient on the writing team. I'm sure that S4ME could help you locate a suitable person.

 

[Hutan]

I'm trying hard to think of an analogy that might resonate with you @Hilda Bastian.

The best I could come up with is a time when women did not have the vote. Suffragettes have worked hard, developing coherent arguments why women should have the vote. As a result, a committee is to consider the issue.

When deciding who should be on the committee, the decision is that the suffragettes have been altogether too noisy for them to be included. So, to show some sort of balance, the husband of a suffragette is included. And, in a gesture expected to win great acclaim, a woman who isn't all that interested in the issue but thinks that probably the status quo is fine and really those men who don't agree with universal suffrage only want what's best for women, is also added to the committee that otherwise consists of men.

And when the suffragettes complain that the committee is not representative and will not come to a good decision, the committee says 'ah, there you go again, being altogether too noisy. That's why we couldn't involve you. Don't worry, go look after your children, you can leave the thinking to us.'

(sorry, minor edits)

 

[Hilda Bastian]

@Hilda Bastian , I realize you have replied to questions about some of your wordings that are unclear.

I still have difficulties, though, to understand them in another way than meaning PWME and their advocates bear joint responsibility for all ME related topics being unpopular, instead of taking into account that a network of people who have the power to influence decisions that directly -- and badly -- impact PWME's lives shaped that image of PWME:



Could you spell out who you mean is" pounding at a group of people constantly"?


It doesn't help me understand those wordings if you say you don't mean the term activists to be pejorative, maybe others do. It still seems that you think if others mean it to be pejorative, that's also co-caused by some PMWE and their allies.

I don't believe PWME and their advocates bear joint responsibility for the unpopularity of ME topics, or for the stigma that flows from all being judged by the behavior of a minority. I don't believe criticizing, or being persistent in criticizing, is a problem: not for me, and not for Cochrane either.

There's such a huge spectrum of choices about actions - many have been raised here, and there are many more. I believe each can have consequences - both ones that are desired, and ones that are not. Even when people believe they are doing the right thing, and with the best of intentions and best of skills, they can cause harm. That's so for communication and advocacy efforts as it is for other interventions.

Like you, I don't see activism as a bad thing. People can be activists or advocates and still be scientists / civil scientists, and if the siuation warrants it, keep both roles apart. E.g. S4ME forum members who are on the NICE guideline committe had to pause other/ public involvement in ME advocacy for the time being on the committee.

Still I don't see S4ME as a forum primarily dedicated to activism. I think it pretty much is what it states to be:

https://www.s4me.info/help/Values/

Yep - and the language of activism, advocacy, and taking action is widespread globally in ME/CFS communities, including as self- or group description - though obviously not universal. I haven't been talking about S4ME specifically, but yes, totally agree of course that taking action often isn't the primary purpose of a group - and there are some that are purely support and/or fundraising, and never try to mobilize opinions or responses and spur actions on any issue whatsoever.

 

[Trish]

I don't believe PWME and their advocates bear joint responsibility for the unpopularity of ME topics, or for the stigma that flows from all being judged by the behavior of a minority. I don't believe criticizing, or being persistent in criticizing, is a problem: not for me, and not for Cochrane either.

There's such a huge spectrum of choices about actions - many have been raised here, and there are many more. I believe each can have consequences - both ones that are desired, and ones that are not. Even when people believe they are doing the right thing, and with the best of intentions and best of skills, they can cause harm. That's so for communication and advocacy efforts as it is for other interventions.




Yep - and the language of activism, advocacy, and taking action is widespread globally in ME/CFS communities, including as self- or group description - though obviously not universal. I haven't been talking about S4ME specifically, but yes, totally agree of course that taking action often isn't the primary purpose of a group - and there are some that are purely support and/or fundraising, and never try to mobilize opinions or responses and spur actions on any issue whatsoever.

I'm sorry, Hilda, this is far too vague for me to grasp what relevance it has in this situation. What behaviour? What minority? What actions? What consequences? What harm?

If any of these questions have any relevance at all in this specific situation, please explain.

If not, it's just another pile of innuendo directed at us that should not be raised when discussing this subject, as it gives the impression that there's something bad coming from people with ME that harms our case for better science.

 

[Hilda Bastian]

If someone is a researcher and has published a bit, the best way to judge someone's ability to do all this I think is to read their publications, not asking others about the person?

There are some discussions on Julia Newton's research papers on the forum.

Do you know that it is one of the main purposes of S4ME to rigorously discuss research, no matter if it's done by people we like as persons and advocates or not?

I think a range of abilities are needed for specific roles - and there are abilities in common across multiple roles, but some that are specific. For example, you don't need particular values and beliefs to provide a really valuable peer review. What you find out about people in interviews, or from people they've worked with, or asking what their track record is with community groups and so on is irrelevant to peer review if they have the technical knowledge and analytical and written skills you require: but it's highly relevant to people in an author team.

 

[Hilda Bastian]

The admission that some Cochrane members are reluctant to scientific criticism being shared openly, for everyone to review, is seriously concerning for the organisation's credibility as a whole. They should be keen on promoting post-publication peer review (à la PubPeer), not opposing it.


While I do not know the grounds on which you favored potential picks, relying on hearsay is problematic, especially when there is (recent) evidence that the person being considered conducted poor quality work on the matter at hand. The studies involving Dr Newton that I mentioned exhibit some of the same methodological shortcomings that prompted the review to be updated in the first place. Certainly this should be grounds for choosing someone else?

I would add that, depending on whose opinions might have been collected and had the methodological issues not been known, hearsay alone could have led to some proponents of exercise therapy for ME/CFS being selected as reviewers.

I did not say that some Cochrane members are reluctant to scientific criticism being shared openly for every to review. I just pointed out that if I wanted to make everybody at Cochrane happy, I wouldn't start off with a long list of unvetted criticisms without responses to them, from the starting point of looking only for critiques not supporters' articles or favorable commentaries about how great it is. I have faced no headwind from Cochrane for doing this, much less opposition.

Edit: PS - Cochrane does promote post-publication peer review, and is better than most journals at publishing it. (Doesn't mean that every editor and author is good at responding to it, though. That will be something the IAG will make recommendations about down the line.)

 

[Woolie]

he stigma that flows from all being judged by the behavior of a minority.

Not again! Please let's not promote this "behaviour of a minority" meme unless we can at least define it, in terms of what actions this appalling group committed, even if not in terms of who they are.

You see, this idea of making a split between a "dangerous minority" versus "nice grateful patients" was created by those defending exercise and psychotherapeutic interventions for CFS. They wanted to create the impression that only a small and generally unhinged minority were critical of these treatments, and that the majority of "nice" CFS patients were eager for these treatments. The "minority" were somehow ruining everyone else's chances of getting them. It worked very effectively, until the some folks in the UK created a petition to remove GET from CFS treatment recommendation, which was signed by more than 10,000 people in the UK. Which is a pretty big "minority".

The "minority" group was accused of many things, but most were hearsay, and were reported by those who created the "dangerous minority" meme in the first place. I am yet to hear any genuine reports of unacceptable behaviours from this group - other than perhaps some reports that a patient was angry or rude in some context.

If there's evidence for an abusive, nasty, dangerous minority, then I'll accept that, but can we please stop assuming that until we know.

Patients with ME are also constantly talked down to about how we "should" and "shouldn't" campaign, not only is this a way to shift the responsibility back to the patients for the lack of change ("we would have listened if you'd been nicer about it! but we won't because you're mean!"),
but it super patronising. First of all, we are a bunch of patients, not a lobby group. Every patient has the right to respond in whatever way they like to the situation they find themselves in (as long as its legal). Second, we are not a coherent organised group, so we cannot police a few patients who lose their temper on twitter.

Imagine if white people were instructing BLM activists on how to be "nice' in their advocacy? Implying that their lack of niceness is the reason why the abuses continue? How appropriate would that be? Did we see AIDS campaigners in the 80s being told that the reason they're not getting treatment research is because they are not nice enough about their campaigning (they really weren't nice, and that turned out to be a pretty good strategy for them in the end)?

Please, no more "dangerous minority" talk (or even any "angry minority" talk) until there's data. Everyone - both patents and healthies - can share specific ideas as to what might work best in advocacy. But these must be made in good faith, not as a technique for blaming us for the situation.

 

[Robert 1973]

That I have opinions about effective advocacy is not news and shouldn't surprise anyone, any more than it should surprise anyone that I'm critical of the demonization of an entire community because of the behavior of a few.

What's this "behaviour of a few" I keep hearing about? What did these "few" actually do? I genuinely don't know, but it must be pretty damned awful. Did they try to occupy public spaces, damage property, break into researchers' offices or homes, or place researchers in fear of physical harm or for their lives? Given the limited energies of people with ME, that last one would be one pretty good confidence trick.

I certainly hope the dreadful abusive "few" does not turn out to be people who were angry on twitter or who asked angry questions at conferences or who wrote letters critiquing published articles. Or worse still, those who made FOI requests for data from various trials. Those are all to me, entirely understandable responses to the situation.

I very much hope that @Hilda Bastian will answer this question.

I don't believe PWME and their advocates bear joint responsibility for the unpopularity of ME topics, or for the stigma that flows from all being judged by the behavior of a minority

I'm sorry, Hilda, this is far too vague for me to grasp what relevance it has in this situation. What behaviour? What minority? What actions? What consequences? What harm?

Hilda, you have now been asked a version of this question at least 3 times and haven’t answered. Please can you tell us who these people are and what they have done? If you are not willing or are unable to tell us, please can you explain why?

I don’t speak for the forum but I’m fairly sure that if there are people claiming to speak on behalf of people with ME who are behaving inappropriately or unethically we would all like to know who they are and what they are doing so that we can all condemn their actions and dissociate ourselves from them. However, repeatedly referring to them without saying who they are or what they are doing denies us this opportunity and perpetuates prejudices. That in itself is an unethical behaviour in my view.

I am grateful to you taking the time to answer questions on here, and very much hope that you will not continue to dodge this one.

[Edit: crossed with @Woolie ’s post above]

 

[Hutan]

I think a range of abilities are needed for specific roles - and there are abilities in common across multiple roles, but some that are specific. For example, you don't need particular values and beliefs to provide a really valuable peer review. What you find out about people in interviews, or from people they've worked with, or asking what their track record is with community groups and so on is irrelevant to peer review if they have the technical knowledge and analytical and written skills you require: but it's highly relevant to people in an author team.

Sorry Hilda, could you be clearer please? Are you saying that you accept that among the ME/CFS patient experts there are some with useful technical knowledge and analytical and written skills, including those you specifically named upthread, but among all of those ME/CFS patient experts, there are none who are nice enough to work in an author team? And that is why you had to appoint a patient representative who does not appear to have technical knowledge but whom you judged to be nice?

[edit - crossposted with Woolie, we appear to both be interested in ideas about being nice. )