Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

The problem with the typical clinical trial of CBT/GET research illustrated in this example of various treatments for asthma (except we have no treatment that meaningfully improves objective outcomes, so ignore the albuterol).

 

I'm glad you're finding it helpful - thanks! And I'll check that out - thanks!

Epidurals are done late in labor for a variety of reasons, so that's when it needs to be tested, and it needs to be tested on the type of pain it's meant to relieve. (The C-section aspect was because of the question people had about whether having an epidural increased the risk of C-section because of interfering with the physiological processes of birth.) I totally agree that there are different issues related to the specific issues with this intervention and ME/CFS: but that's not what I was debating. It was about an absolutist statement which I was challenging, on the more general point of things being more complicated than the discourse was allowing for, and that people shouldn't be disparaged because they disagree with a simplistic global position about a very complicated issue.

 

Well, that'll teach me to use an obstetrics example without getting up-to-date on it first - in terms of whether you could have the epidural early you & Trish get to say "OK boomer" about that! But that wasn't a trial of placebo epidurals for pain relief: they all had pain relief until the second stage. And that doesn't change the point I was making.

 

Even that has to come with serious caveats though, because what they say will still likely differ significantly for unblinded versus blinded. So even if wanting to know nothing more than what people say, then you might need to dig deeper into why you are interested in what they say.

Not so sure about that? Unless the respects you are interested in are not particularly reliant on good science.

 

However, I did just prove that I didn't have an epidural

"...the trial might still have some value...I don't think anyone is suggesting otherwise": actually, the point I was responding to was someone saying exactly that, and that anyone who disagreed with the statement was either incompetent or conflicted. It seems to be a standard part of a discourse - not that everyone says it, but I've seen it said often, by multiple people.

What people say and the utility of the treatment can be the same thing: eg if the intervention is purely pain relief. Or it can be so integral to whether the treatment is worthwhile, that it rises to the status of a primary endpoint. There can be objective measures that are better than subjective ones, of course, and there can be big reliability differences among methods of measuring subjective outcomes. But even if a specific outcome in a trial is problematic, that doesn't, on its own, necessarily condemn the trial to complete "valueless-ness" if it's not the only outcome. (I'm not talking about any specific trial here.)

 

@Hilda Bastian Years of clinical trials on ME/CFS point to the above criteria as being the correct ones to look at in ME/CFS. If a clinical trial is truly meant to help patients -- that is, it tests whether a treatment is safe for these patients and has an objectively measurable positive effect over the long term -- can it really be argued that they are not?

I think these criteria apply to other chronic diseases too. It seems to me that they are actually being used in diseases that are well known by the medical field. For example, even though I have no references, I believe that testing drugs for cancer or multiple sclerosis is done that way. But with chronic diseases that are not well understood -- ME/CFS, fibromyalgia, IBS, etc -- we are left with poorly designed clinical trials.

Therefore, the reasonable course of action seems to use these criteria in the review protocol. If the review comes up with an empty result, excluded clinical trials can definitely be discussed, with an indication/explanation of what criteria they failed to meet. In doing so, these trials are being objectively assessed and the discussion still values them.

A silver lining with an empty result is that it can implicitly say: "it's time to do better for these patients". This alone can prove to be very helpful for them -- moreso than piling up trials with low quality evidence --.

ETA: If a subjective outcome is used in a trial, then the authors of the trial should provide robust evidence that the tools used to measure this outcome actually measure it correctly (where 'correctly' is defined according to all of the aspects that the outcome entails). This brings us back to the issue of questionnaires, which, for some (or many), there is no such robust evidence and instead serious flaws can be found. One example is the Chalder Fatigue Questionnaire as previously studied by @Lucibee, @Michiel Tack and others. In this case, the trial should at least be marked as low quality evidence.

 

The thinking seems to be that it is okay to do open-label clinical trials with subjective outcomes if there is no objective marker of disease severity. A sort of "we will do the best we can given the situation".

This is an emotionally appealing narrative but it doesn't make sense because all the lack of objective marker of disease severity does is making it hard to unambiguously demonstrate that a treatment is worthless. This isn't a serious effort to find treatments for illness, it's about appeasing patients and giving them hope and protecting them and doctors from the feeling of despair and powerlessness that comes with not having a treatment. Dealing with the emotional problems that come with severe untreatable illness is important but it shouldn't be done while pretending to be treating the illness. And I worry about the harm that could come from repeatedly treating patients with placebo therapies, for example disappointment, even worse hopelessness, waste of money and lack of trust.

 

It's also important to consider that there is a difference between placebo effects and reporting biases. The former would be actual reduction in pain (in the case of trials of analgesics/anaesthesia), rather than a change (bias) in pain reporting behaviour. Both placebo effects and reporting biases can be influenced by medical and psychological interventions, and double blinding is a form of minimising the effects of both, to determine the underlying effect of the intervention. Placebo effects (on pain for example) tend to disappear over time for chronic conditions and reporting biases also tend to disappear over time (typically gone by 2 year followups).

 

So how is that any different from suggesting PACE established that GET is relatively safe for ME/CFS patients? Genuine question, not sarcasm.

 

This has developed into is a huge problem as researchers are not using objective measures in unblinded RCTs. And they have not been pulled up for it either via their institutions or via the peer review process.

The researchers into CBT and GET then mix up 'feeling better' (which small/modest improvement is inevitable pretty much after a face to face intervention where someone was nice to you) and 'being better' (recovered/cured/able to do largely as they please with no or few symptoms). The later is 100% what patients want/would like. The former has been used ad nauseum by researchers in RCTs of CBT and GET. Poor.

Small/modest improvements in subjective measures/questionnaires has then been 'sold' as a meaningful result - when all it is, is a demonstration of the placebo effect. And it doesn't get anyone back to work or health or anything close to what the patients would consider recovery. This is a classic example of the psychological process of substitution. Human's are good at it. And slow, poor at identifying it and calling it out.

Joan Crawford
Counselling Psychologist

 

No one wants to point out the emperor has no clothes because it means we would necessarily throw out a lot of evidence for non-pharmacological therapies for many different conditions. It means that many people would have to admit they were at least partially wrong (from policy makers, to therapists, to scientists).

Fact is, most non-pharmacological/non-surgical therapies only show efficacy for subjectively reported symptoms and not objectively measured functioning or disease parameters. This is something the evidence base of these therapies shares with the evidence base for alt-med interventions.

 

The thing I've been debating is partly whether subjective outcomes in an unblinded treatment trial can ever justifiably be a primary endpoint, but more importantly this part of what he said - that if it's not, the trial is valueless (not just that particular bit of outcome data):

If Jonathan and the others arguing trials should be discarded so easily don't really mean that literally, that's great.

There are times when the only way of trying to assess whether a treatment is having a beneficial effect is by asking people, e.g. how much pain they are in.

I disagree that the epidural example is making the point you suggest. Since all the women had been on active analgesia until the second stage, when being randomized to placebo meant the analgesia was withdrawn for those in that group, I think it's hard to argue the women were truly "blinded" for that part of the trial. And even if they were on the placebo, they could get analgesia, and the data were analyzed on intention-to-treat, not whether they had analgesia or not. ("During the second stage of labor, participants with excessive pain were allowed be changed to an unblinded infusion of epidural medication.") And even if it that one trial showed what you're arguing for those 50-ish minutes, this wouldn't mean that the dozens (or whatever there are now) of trials of epidurals for pain relief that didn't have placebos were valueless, and epidurals haven't really been shown to reduce pain in labor.

 

If these serious issues can be so concisely summed up, one can only hope the new Cochrane review board will quickly reach the same conclusion. It's so desperately needed