Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

https://mecfsskeptic.com/nice-guideline-committee-and-grade-methodology/

'.....Another recognizable name is Signe Agnes Flottorp, Research director of the Norwegian Institute of Public Health. Flottorp recently was at the center of controversy when she defended research into the Lightning Process, a non-medical training program that combines concepts from Neuro-Linguistic Programming, Life Coaching, and Osteopathy. Critics have denounced the Lightning Process as pseudoscience or quackery.

Other authors of the rapid response are known for their leadership in evidence-based medicine. The most important name in this regard is Canadian Professor Gordon Guyatt. Guyatt was one of the authors of the GRADE handbook....'

 

Thanks. There are people who feel exactly the same way about authors you may approve of (although I realize it's also of course possible that you think they're all unsuitable). [Edit: just realized the subsequent post makes it clear that at least one person definitely gets a tick from you.]

A group could have been assembled in which having absolutely no prior position or involvement with the condition or research related to exercise, too. That's no guarantee of a good review, either. I don't believe lack of content knowledge is an enormous asset, or that relevant biases are ever absent in anyone - and biases can certainly develop and change during the research process. Biases are always there, and therefore you need better ways to deal with it than finding people most would agree are blank slates at day zero.

Even if a so-called bias-free group of people with zero involvement or research interest were appointed, that would also result in backlash. Anything Cochrane did, or didn't do, was going to face support from some, and backlash from others. I don't believe avoidance of backlash should determine research agendas.

 

If my own interests were front and center for me, I would never have written a word about ME/CFS. That's ineffably sad. It is literally because that is the case that I am doing it.

 

I fear that having knowledge or important awareness of context is being interpreted as having "biases".

I am concerned about an approach that is giving equal weight to different "feelings" people have. The point is, who is right? Are there serious methodological problems or not, for example? One cannot take these points on board and then say, oh well, yes but there's a different view as well. Someone is wrong.

 

I agree that knowledge and awareness of context is critical, and it's an asset, not a bias. Sometimes, people need to change their minds - in the group charged with the task, and in the people who will accept or reject what they write: it's not only about what's in a GRADE table or a meta-analysis.

So what do you do when people are deeply entrenched in opposed camps of people who have convinced each other without any doubt that they are right, and they each have an influence on the ground, where everyday decisions and actions are happening. Something has to shift, including in people's minds - and not just in one country. I don't believe you get there by people not talking to each other, not listening to each other, and not being seen to do so.

Some people aren't going to be convinced of something if they can't see that their point of view has been fairly and competently considered and clearly addressed. And it shouldn't be made easy for those who will just work to discredit any process that doesn't come to the conclusion they want. Critically, for the huge numbers of people who aren't in either camp but are hearing both sides, they need to be able to see for themselves that the attempts to discredit a document aren't valid. When either of those camps say to them, "ignore that - here, I can show you why they were biased and it's just about being overtaken by a lobby, not science - don't believe them - look over here instead, this is who you should believe" - it shouldn't be super, super easy for that argument to work.
Edit: PS: Aaarghhh! I "both sides"d it!!! So hard to resist binaries... Who aren't in *any* camp but are hearing *all* sides.

 

For reference on this, here are some things I've written specific to this:

• The evidence base, the Cochrane review, and the AHRQ review in February 2019
• The amendment to the Cochrane review in October 2019
• On bias in protocols in May 2020 - which was around the time I was deep in the process of gathering, and developing an analysis of, critiques of the methodological and other issues with the Cochrane review.

People will be able to invited to let us know if there is anything we have missed.

I wouldn't say this is a case even of both sides - I think there are more than 2. And "oh well, there's truth on both sides" is not how I see this. But I do believe, there is no shortage of people who have no doubt they are right on all sides.

 

It's number 24.

 

I suspect that Hilda's reference to 'pounding' may be related to the insistence by most of us here that subjective outcomes in unblinded trials leads to worthless pseudoscience, and that she sees that there are those that support such trials who have differing views on this issue. I think she feels that if those differing views are excluded from the process then the final product will be open to attack from that side. Hence I think she feels that we overplay our hand, that it feels to the other side that we are the dogmatic ones who won't listen to their reason.

Hilda, if I am way off the mark here, please say so. I certainly appreciate your involvement in this forum.

So, is there a way we can increase the attention that the process will pay to the problem of subjective outcomes in unblinded trials? Should we search for good peer-reviewed articles that address this issue (for example, the Albuterol, placebo, sham acupuncture paper) and make sure they appear before the writing and advisory groups? It seems so obvious to me and, I think, most of us here, but maybe we need to support it with hard evidence.

 

Thank you - and everybody - for the time and thought you put into them. I will at times be slow getting to them, but I read them with care and appreciation for the thought and effort. Not responding doesn't mean it's ignored.

I was a consumer advocate for decades, including leading a cross-issues national organization for a long time; I've got a highly active Twitter account and write for some very high profile magazines on highly controversial topics: I have a lot of experience of being called into question, I fully expect it, and I learn a lot from it. I believe a group of 8 authors with both Mary and Kay, is a far, far better group than the group of 6 it would be without them, which was the other option. I think they're both deeply impressive, and I have a lot of confidence in them both. I'll write more about the issue of the process in a methods report, as I've already discussed.

Research authorship: When people have jobs on offer, that can go through that process - although as we all know, that doesn't necessarily affect the outcome; and even when that occurs, it doesn't mean every one of the authorship positions were advertised. People generally sometimes put out the word they are considering authors in different ways and to different circles, and we did that, with an email address for people to send enquiries to - just as we notified that I was going to be establishing an advisory group. People and organizations certainly understood that they could propose people, because they did - and others asked if they could put their own or others' names forward, and they did. Or they just asked how was this being done. S4ME, for example, is one of the groups that asked to be considered for the IAG (to which I of course replied that they were).

How individual authors came to be on a research team is not something that's typically discussed. If there was honest transparency about that, I reckon it would mostly be "I asked some like-minded friends and/or people I wanted to be associated with and/or people I was obliged to ask".

I've said that I don't think it would be helpful for me to be more specific about what I believe is counterproductive, and so I'm not going to do it. My views about effective advocacy and criticism are public in many places, but it's clear to me now that I should not mention them here. I can delete that specific post, and I'm happy to do that if that would be helpful.

I hope for this too - and I'm sorry about the additional specific burden this means when it affects people's health on top of everything else. It's a major part of why I decided last year to invest heavily on capturing the things people already have said so eloquently over the years, and trying to honor it - as well as using the process to help us consider how we can reduce the burden of contributing, within the constraints we have.

Edit: fixed formatting screwup and added a missing verb and the end of an accidentally incomplete sentence.

 

Thanks! Not about any specific issue nor about criticism or criticizing the same thing persistently. However, the "is there a way we can increase the attention...?" does go straight to the question of effectiveness.

 

Thanks. (Though it's risky to assume that my views in June 2021 on any point are the same as they were in 2019, unless I said so. People learn things, make mistakes, and change their minds, including me.)

I didn't choose members of the author team to ensure that any specific methodological was given preference.

I agree it's important to keep that clear, and I have a different view on what research is needed now than I did in 2019.

There was no separate selection for criteria for patients as potential authors and "not having demonstrated any methodological expertise" was not a criterion.

I agree that reconciliation is neither required nor possible: I didn't say it was. Indeed, what I wrote makes it clear I presume people will still be making the argument - that's biased, don't look at it, look at this.

Thanks. Yes, even the best-intentioned and best-skilled person can do harm. I agree with you on what a key strategy here is: it's utterly vital to really hear people with ME/CFS and their great wealth of knowledge and experience. While it took a long time, hindered by a pandemic, to get to this point, listening and reading that is a large part of what I was doing*. The opportunities for feedback on this will enable people to see where something hasn't been fully appreciated, or it's been overlooked, or new insights emerge.

* For reference here, from the report:

The re-consideration of this review is due to the commitment many people made to articulating and publishing their concerns about its contents and conclusions – as well as pushing for change and inclusion of people with ME/CFS. In particular, Robert Courtney and Tom Kindlon submitted influential major detailed comments that Cochrane published, but did not adequately respond to.

So the starting point for the update of the review was the need to deeply engage with “historical" stakeholder critiques. This part of the process provides a framework to ensure that issues are thoroughly addressed from the outset, to assist in consultations and incorporating ongoing feedback, and to systematically address the points raised.

 

I agree that it can have dreadful consequences for something that requires diversity of thought and will affect a lot of people. That's why we didn't do it that way. In case anyone's wondering: I am pretty sure I chatted with David Nunan briefly socially at a conference once. I interviewed people who reached the stage of being prospective consumer authors, but had never met any of them prior. I have never met any of the other authors (with apologies if they ever came up and said hello and I don't remember).

For the IAG: I got to know George Faulkner in the discussions at Cochrane that led up to this process. For the methodologist positions on the IAG, I am going to rely on people I have known a long time, in whose expertise and integrity and listening/communication skills I have enormous confidence, who I know will be both diligent and have zero reluctance to tell me when I'm screwing up, and who I know will engender a lot of confidence in members of the Cochrane community.

 

One point that might not have been made yet (I haven't been able to keep up with the discussion in this thread) is that this process isn't just about our patient group.

It is a pilot programme and if successful, should in principle be used to revise or guide the development new meta analyses covering other conditions where there are similar methodological problems and ideally, improve the quality of non-pharmacological clinical trials in the future.