Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Perhaps cochrane will significantly downgrade unblinded trials that rely on subjective outcomes. Based on NICE, that does seem like a possibility. However that's not the point I'm trying to discuss here.

 

There is nothing the least bit 'radical' about this. it is a matter of known fact that such trials are unreliable so there inso problem discounting them. Every medical student is taught this. such trials are routinely discounted for drug treatments so why not for non-drug treatments?

 

The problems that Cochrane is locked in to the GRADE system which has a set of rules that do not reflect the facts about reliability so is bogus.

Not sure what point you are wanting to discuss if not this?

 

Sorry, I had not read all the comments before responding initially.i agree with Hutan.

Systematic reviews are supposed to have rules that make them comprehensive and consistent. That makes sense to everyone. But unfortunately the grADE system used since around 2010 also has rules about how you judge reliability and I think that is something quite different. judging reliability should not be based on rules.it should be based on the facts we have about reliability. we know that unblinded trials with subjective outcomes are unreliable - that is why blinded trials were invented. So having a system that does not discount them is just bad practice.

the central problem with Cochrane is that it arose from a group people biased towards non-drug treatments. They repeatedly admit this. The question is whether they are prepared to overcome it in the context of showing themselves capable being consistent when it comes to application in reviews.

 

So I think there's a bit of miscommunication going on here, where my specific points and query may not be being completely understood. My basic question is, assuming that cochrane doesn't completely discount unblinded trials with subjective outcomes, how can those two key related problems with the PACE trial still be taken account of? Those two key problems being 1, that it is an unblinded trial with subjective outcomes, which is substantially made worse by the fact that 2: the treatment in PACE and other trials entails an "treatment-inherent cognitive bias" ie. patients are told to do things like ignore and dismiss their symptoms, which is likely to influence how they answer subjective questionnaires.

Given that there is a possibility that cochrane may choose to still give some weight to unblinded trials with subjective outcomes, I feel like this is something worth discussing.

I was a bit vague about the kind of solution I might be looking for. So I'll suggest one, in which those two problems could still be taken account of to some extent, even if cochrane decides not to discount unblinded trials with subjective outcomes, and being consistent with the use of GRADE and the limitations of systematic reviews.

In the GRADE guidelines, one of the guidelines on rating the quality of evidence relates to inconsistency. That is, inconsistency of relative measures of effect across studies and outcomes. When there is a large inconsistency GRADE suggests searching for an explanation. One of the strong indicators that there is this effect of "treatment-inherent cognitive bias" in PACE and other trials, is that the measures of effect in the subjective outcomes is very inconsistent with the lack of significant difference in objective outcomes, and long term followup. So, in accordance with GRADE, an explanation can be sought for that, and that explanation would be something along the lines of "the treatment may lead to a change in subjective reporting, but not real world improvement". And then PACE and other similar trials can be substantially downgraded on the basis of that. Whilst still being systematic and replicable, in accordance with GRADE, and taking account of those two key problems.

 

Yes, you can use GRADE to shift your decision in whatever direction you like- which is why it is so ridiculous. NICE decided to downgrade on the basis of selection criteria I think. You could think up a reason under almost any of the categories. But that needs people who want to do that. If people want to get positive analysis they Willard if not they won't. We can think of ways to get a negative analysis with GRaDE but that doesn't really help. If you stick with GRADE you sendup with what the assessors like. The alternative is to abandon GRADE but the people involved are much too obsessive todo that I think.

 

I'm honestly not in a position to really fully understand how systematic reviews and GRADE work, and what's acceptable, hence why I was/am trying to start a discussion. I think your response here partially answers what I was thinking about. NICE says in this document: https://www.nice.org.uk/guidance/gid-ng10091/documents/evidence-review-7 that the main reasons for downgrading were bias (from lack of blinding and subjective outcomes), indirectness and imprecision, by the way.

 

don't know if this has been posted(?) I haven't read it yet.
Report for June
https://community.cochrane.org/orga...igh-profile-reviews-pilot/cochrane-exercise-0

 

Thanks, I was curious about the selection of the last two IAG members.

 

Hmmm... unlikely that someone for whom GRADE is so central to her work will be able to acknowledge the massive flaws that lead to very low quality "research" to be judged arbitrarily to fit a pre-sought outcome by teams biased for that purpose, how easy it has been to manipulate it simply based on who is present to apply or not its principles. Because frankly the very basic premise of EBM is what has failed massively here, where most of the flaws are, in just how arbitrary this all is, turning this way and that depending entirely on who is present and their personal and professional agenda.

But that's just based on precedent, hoping to be pleasantly surprised. Could be a teaching moment, although those are very far and few in this field.

 

"The areas of expertise that have been suggested are wide-ranging, from people who have severe ME/CFS or care for people with ME/CFS, to people who have recovered"

Strange placement of "people who have recovered" here. People with severe ME/CFS are not on another side of an argument or contrast with "people who have recovered". What is the expertise or experience of "people who have recovered"? Anecdotal recovery stories? Are they to be taken to have "expertise" on recovery, or did they simply happen to recover? Reminiscent of Paul Garner's talk of listening to those patients who recovered in order to get well.

I feel this contrast of severely ill patients with "people who have recovered" diminishes severe ME/CFS. Would the same framing be used with cancer?

 

I know Miranda quite well - she used to be my boss.

 

My view is that there have been clues to how you assess the outcome of interventions for some time. E.g., as someone with a slight interest, I'd come across a study of American soldiers, who'd been in one of the Gulf wars, which monitored their phones --- moving around/not moving around --- whatever --- to objectively assess how they were.
Fluge and Mella used actimetry in the rituximab study.

I just don't grasp how come someone can use poor quality outcome indicators when better ones are available - presumably cost [EDIT "of e.g. actimetry"] is a factor , but if the evidence gathered using questionnaires is so poor then why do/publish the study?

Also, I think that dropping actimetry monitoring in the PACE study, with the support of the embedded charity Action for ME, was pretty much a low point in terms of the Government's treatment of people with ME. A study aimed at assessing CBT and GET was turned into a charade and the losers were people with ME/CFS and their families.

My question is how can you get this s--t published but unfortunately posts here indicate that you can get a lot of poor quality research published.