Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

I think in the past @Jonathan Edwards has said that subjective outcomes in unblinded trials can sometimes be OK, provided they are only secondary, and that primary outcomes are objective; the objective outcomes can then be used to assess the validity or not of the subjective ones. But I may have remembered this wrong.

 

Just as a point of clarity:

Obviously, I can't comment on discussions or outcomes, but I can reassure you that we see everything. There are no secrets.

The summaries are presented because we're often dealing with, as you can see from the evidence reviews, hundreds to thousands of pages of evidence and material at a time.

The summaries are provided in addition to giving us access to the raw info those summaries are based on, not instead of.

 

https://www.s4me.info/threads/adver...ome-2021-white-etherington.20767/#post-347678

 

I took it for granted that I didn't need to repeat that the 3 organizations of people with ME/CFS nominated their own representatives to the IAG and that it would be obvious I didn't know people with a clinical background in ME/CFS. I didn't know people from those organizations of people with ME/CFS and I had never met the people they nominated till they joined. I had never met either of the people appointed for clinical background either until I interviewed them.

I am increasingly comfortable with not having taken chances with methodologists, watching this unfold in the last few days. I didn't want to risk having people who would patronize, bull-doze, just not bother too much, etc. And these had to be people with an extremely deep familiarity with Cochrane's methods (which of course really increased the odds that we would know each other).

Yes, I agree, a lot was vested in me in this role. And no, that doesn't mean the editor-in-chief picked someone she'd have in her pocket, which I think is the implication of the comment. She explicitly picked someone she wouldn't have in her pocket, and she knew it. In fact, I was genuinely shocked that it was me that she asked - and I made very sure she understood the risks and implications of that choice before I accepted.

I don't think the setup was ideal either, but when I saw the tasks ahead that needed to be done to get there even by early 2022, that's the call I made. As I've said before, I had discussions with people with ME/CFS beginning when this was announced in October 2019, and came to decisions and negotiated arrangements that I believed were the best plan within the constraints. The IAG will make recommendations about all this at the end, when we've seen what seemed to work well and what didn't.

 

This is an example of what I was thinking of:
https://www.s4me.info/threads/uk-pa...ay-24th-january-2019.7826/page-16#post-138969

But having reread that I think I may have been taking Jonathan's comment out of context a bit, because for ME/CFS there are better, objective, outcomes that can be used instead. But I still think it might be unwise to suggest removing mention of all unblinded/subjective outcome trials on that basis alone, but better to clarify that they should not be considered unless all the belt-and-braces safeguards are being employed; in the case of the trials we have concerns with, they never have any such safeguards, and the investigators never even seem to have any awareness of the need for them. This would avoid walking into a trap of our own making.

 

Apologies for self-quoting. Searching for references to Barry's posts I stumbled across a somewhat related post that seems still to be relevant:

The addition about the review authors was made back then.

 

The composition of the author team is far from perfect, but I think there are pretty strong limits to how perfect it can get. Other reviews, such as NICE and the Dutch health council both had a mix of people with different views and backgrounds on them, but they still both concluded that GET should not be recommended. There is an advantage to having a mix of people on a team, as that gives the final review more credibility, rather than a perception of being loaded with people that will lead to a foregone conclusion.

I think that people's points in this thread are all pretty much valid, and there are probably things that could have been done better and could still be done better in the process of this review. And it isn't easy to predict what the final conclusion will be. I've had a look at some previous reviews of people on the team, and in order to accept the criticisms of PACE, they would need to consider things which they don't seem to usually consider when reviewing evidence. But I'm not sure how possible it would be for there to be (mostly) people on the team who do usually consider the kinds of criticisms made against PACE.

None of the people on the author team seem to have outright conflicts of interest, and there doesn't seem to be anything to suggest they're ideologically opposed to criticisms of GET trials or to criticisms to the universal utility of exercise, so there doesn't seem to be much reason to believe they'll outright reject those criticisms.

The specific inclusion of a "non-activist" patient picked for differing experiences with exercise on the author team is unusual and a bit strange, and I assume that has something to do with cochrane wanting to maintain the pretty significant amount of credibility and respect it has among people who might assume the review is pandering to "activists". I agree with other people that that's weird and not a very good look for cochrane, and may indicate some questionable views in cochrane. But Kay does seem like a fairly good pick, regardless.

I think that people should still keep expressing their concerns about the composition of the team and other things in this thread though, because Hilda is the Lead on the IAG and the IAG was set up to ensure and facilitate responsiveness to stakeholder concerns.

 

Comment from 2000 on the review Behavioural treatment for chronic low‐back pain quoted above -- didn't check the revisions so no idea how the author team from 2010 (when Henschke joined them) responded to it, also don't know Henschke's current stance on that:

Comment on the Back Group behavioural review (The Cochrane Library 2000, issue 2)
Note: This comment may relate to a previous version of this Review.

The commentator thinks there is a discrepancy between the author's presentation and discussion on the studies, and their conclusions. The commentator states that the results indicate that behavioral therapy gives a moderate effect only with back patients that receive no other treatment, but that effect is not proved in comparison with other treatment modalities. The commentator goes on to suggest that the whole treatment effect of behavioral treatment in this patient group is a placebo effect and no real treatment effect. The commentator states, "Behavioral treatment should not be recommended outside the setting of clinical trials."

Reply
We indeed clearly state in the review that behavioral therapy only gives moderate effect in the comparison with no treatment or waiting list control and that there were no differences identified when behavioral therapy was compared to other treatments. Moreover, comments like this and ongoing discussion regarding the potential beneficial effect of behavioral treatment (among other in guideline committees) prompted us to disaggregate the analyses regarding the comparison of behavioural treatment versus waiting list control, no treatment or placebo for this up‐date. The main reason for this is that we wanted to assess what components or types of behavioural treatment were effective. This update shows that especially a combined cognitive‐respondent therapy or progressive relaxation are effective for short‐term pain reduction.

We disagree with the commentator when he states that, "Behavioural treatment should not be recommended outside the setting of clinical trials." In our opinion many treatment modalities are used for chronic low back pain and there is not one treatment option that has been proven beyond doubt to be more effective than other treatment modalities. It is unlikely that the effectiveness of behavioral treatment is merely a placebo effect. As we have stated in our recommendations, we do agree that more research is needed in order to improve the understanding what specific patient populations benefit most from what kind of behavioral treatment.

Contributors
Commentator ‐ Ottar Grimstad
Criticism Editor ‐ Dr. Alf Nachemson

Declarations of interest:
No conflict of interest declared.

Edited to clarify: didn't check the revisions so no idea how the author team from 2010 (when Henschke joined them) responded to that comment from 2000 also don't know Henschke's current stance on that.

Apologies for causing extra confusion.

 

Which is what I said in my preceding post (and subsequent post in no way meant to countermand) ...

The point being that subjective outcomes in unblinded trials alone can never pass muster.

You may well be right, and maybe I've not fully appreciated this.

Which is surely an argument to not unconditionally ignore trials with subjective outcomes in unblinded trials, but to assess whether those outcomes may have some value in the context you speak of, even if that value is to show they are not reliable. The "safeguards" I was speaking of are the ones you also mention - having the trial data necessary to make those assessments.

 

This is worryingly muddled.
That many treatments are used for back pain is a fact, not something to have an opinion ion about.
That there is not one treatment proven to be more effective than another is a reasonable opinion - but in what way is it a rebuttal of the criticism? It isn't.
That it is unlikely that 'effectiveness' is merely a placebo effect makes no real sense and is not a fact but an opinion. It ought to be justified in some way.

 

Yes, I know it's on the list. That wasn't my question. I think I suggested at some point that Vink and Vink-Niese be invited to Cochranize their study to convert it into a Cochrane review. I think there are precendents for this. But of course Vink and Vink-Niese would have to agree which is by no means a given. Would that also be considered once you and the IAG and the author team have read and analysed the NICE review, the Vink Vink-Niese review and any other reviews of the evidence for exercise that already exist? The Cochrane process seems to be a process of re-inventing the wheel with many more hands on deck and potential spanners in the works (excuse shameless mixing of metaphors) than is necessary to make the simple point that the evidence that exercise is safe for people with ME/CFS and also has a useful positive effect (if objective outcomes measures are considered) is non-existent.

 

Thanks for your excellent posts, Hutan. In expressing your views so clearly you’ve saved me a lot of energy.

I’m a bit puzzled by this. Why does it matter to engender confidence in members of the Cochrane community? Perhaps I am misunderstanding what you’re saying but are we not in agreement that the “Cochrane community” has completely failed people with ME/CFS by its inability and/or unwillingness to adequately address the issues raised by Robert Courtney, Tom Kindlon, George Faulkner and others for so long?

As I see it, if you and your team do a good job a lot of people in the Cochrane community are going to be exposed for having overseen multiple failures.

Yes, I fear this may be the compromise/bodge some BPS activists and the Cochrane community might be looking to achieve: allowing GET to be dropped for well defined ME but keeping it for more general chronic fatigue and ignoring the methodological problems with the trials which could (and should) lead to the collapse of an entire house of cards – ie the withdrawal of multiple Cochrane reviews for unevidenced therapist-delivered interventions across the board. For me, such a bodge would be completely unacceptable.

Many of us are here because we’re fighting for our own lives, but in many cases our eyes have been opened to problems that go far beyond ME/CFS. My great hope is that the work that has been done by “ME activists” will eventually benefit a far wider patient population, and that those who have worked so hard to expose the methodological problems in studies of interventions which can’t be blinded will eventually receive due recognition from the establishment for their contributions to science and medicine.

 

A compromise wouldn't do justice to people with ME/CFS because as long as the core issue is not adressed, there will be a never ending stream of placebo therapies that cause harm in all the ways they cause harm. LP and ACT are already lined up to take the spot of CBT/GET, should they be discredited. The door for placebo therapies must be closed for good.

Edit: so a good question for this review is, how will it protect patients, policymakers and clinicians from placebo treatments being presented as effective?