Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

I know others will disagree but selecting the wrong cohort of patients is just as fundamental. Even if a body of research is perfectly conducted, its conclusions about efficacy and safety in a particular disease are meaningless if the cohort includes an overrepresentation of patients with some other disease.

 

Yes. Another possible loophole in PACE occurred to me, regarding PEM. Without round the clock activity monitoring, PACE almost certainly never captured participants' activity levels during PEM, meaning even the "objective" activity readings were badly clipped at the bottom end, effectively chopping off all the lower readings from the graph; readings that would be very low indeed. So any averaging of activity level readings would be heavily skewed upwards from reality.

If a trial is run without any insights into PEM, then one of the most likely reasons for participants to not attend sessions would be PEM. Who is really going to be attending a session in the midst of a PEM bout?

Moreover, if a pwME knows they have session coming up then they will be tempted to do what they need to avoid PEM at the session. This goes beyond simple activity exchange between session / non-session activity, because participants are likely to "over swap" in order to perform on the day.

I wonder how much of the PACE data actually has information from bouts of PEM? I suspect the whole issue of PEM is brushed under the carpet.

 

Ideally the outcome measures should relate to things that a relevant to the patient in the real world. Examples, might include: ability to work (or equivalent); actigraphy over a decent time period (a week or two) and followed up on at 12 / 24 months; reduction in medication usage (measured independently via GP records over time); absence or reduced biomarker of disease (independently measured); improvement on VO2Max 2-day testing; neuropsychological testing by independent assessor and so forth. I think this is one area that patients can have more input - where they have been notably absent in the past.

We use the short term measures like the 6 min walking test pre and post NHS pain management programs - where pain reduction/cure is not an expected outcome - it's about helping patients to cope well despite ongoing pain. This measure generally improves modestly over a pain management programme usually - but by a small amount and nothing like compared with age/sex matched controls. So, this type of data can be used to demonstrate an effect (in context) but there is the possibility that it could be used in a misleading / over optimistic way. Occasionally, we will have patients who were enduring through their pain and persisting with a lot of physical activity despite pain (usually those early on in their condition) who will score lower afterwards - but it is because they are now pacing themselves better. Context is important and interpretation.

Having secondary subjective outcomes measures, alongside objective ones, in a trial is very helpful as it gives a wide range of functioning (activity, physical, emotional, mental health, self efficacy) that would also expect to improve. It's informative and provides a backdrop to support an intervention and could help translation into clinical practice from research.

 

Dont want to go down this rabbit hole except to say that PACE's pronouncements about comparison across definitions were meaningless because they manipulated those definitions - reportedly for London and definitely for CDC, where PACE only required the "any 4 of 8 symptoms" part for 1 week, not the 6 months required by Fukuda. So who knows what conditions they actually evaluated.

 

The innuendo is still there Hilda -'adamantly'. I hold the position because it has been empirically verified thousands of times and is it is in effect a tautology. What we mean by a subjective outcome is one that can suffer spurious influences if people have a prior idea what the answer should be and an unblinded trial is one where they do. I will clarify my original comment in a minute but the suggestion that I hold this view through some sort of prejudice or vested interest is to commit exactly the impugnment that you were complaining of in others!

Thanks for replying and making clear your arguments - I and everyone else here really appreciates that. But the argument here is irrelevant again. As others have said, nobody would be as stupid as to imply that the presence a single subjective endpoint makes a trial valueless - why should anyone be so irrational (that impugnment again?). And I have already dealt with the red herring about subjective endpoints not being valueless. I actually put it in capital letters last time to try to make sure it got read! Of course they are important and valuable - if they are free from bias - which they will be in a blinded trial. Another red herring I get is that blinded trials are hard to do for therapist-delivered treatments. Indeed, that highlights the weakness of the trials we have. It does not mean that it is OK to treat inadequate trials as if they were somehow adequate. Reliability of evidence is a measurable factual matter, not something that shifts with your needs. Of course sometimes the subjective endpoint is more valid. Any fool can see that. But that is not the point.

Let me rephrase my original comment to give the meaning that is transparent to all members here but you seem to have difficulty with. It was I think self-evident in terms of the context in which the comment was made.

The use of subjective outcomes (alone) from unblinded trials to judge the usefulness of a treatment is valueless.

The problem with the exercise review is that the trials included either do not have primary objective endpoints that would be adequate (which relates to the superiority of subjective ones we have agreed on) or they were not meaningfully controlled because the psychological framing of the comparator was notcomparable. That is a different issue and one Ernst has interestingly commented in the context of the Lightning Process recently. I remember noting that one study had fitness as an objective endpoint that improved, but, as we agree, fitness is not a good index of whether or not someone feels less ill, which is what matters. OK the context is complex, but the bit about the combination subjective endpoints and lack of blinding is not. As other members have pointed out all sorts of things about both the trials and the review made things worse but we are discussing this one point.

I don't understand that comment. If you were not trying to defend the value of subjective endpoints- presumably against a suggestion that they were inferior - why do you keep mentioning it? I am baffled here.

As someone else has pointed out this is now about harms and clearly falls outside the meaning of my original comment in the relevant context. Yes trials can have value in showing harm. PACE would actually have great value in showing that CBT and GET do not work usefully if it were not for the dreadful mess about recruitment and generalisability. PACE is of value in showing that the theoretical model was wrong - if it was right the objective measures would have paralleled the subjective ones and they definitely did not.

The first paper I ever wrote was a retrospective review of a surgical procedure for pain in the wrist. It was an unblinded study with a primary subjective endpoint. Its value was in the humbling learning experience for a Dr J Edwards who later realised he had made every mistake we are now discussing. But it was valueless as a guide to the usefulness of the treatment.

But that is not a sensible reading of my original comment - as I think everyone else here would agree. It makes no sense. I think I did actually use the term 'primary outcome' initially and I am sorry if I did not put it every time. But everyone else here is up to speed on the need for primary outcomes prespecified, the problems of multiple analyses and so on. When I say an unblinded trial with a subjective outcomeI assume that people will understand me to mean a primary outcome or an outcome used to decide whether or not the treatment is useful - that was the problem with the review.

I would highly recommend reading many of the threads here.We have had along discussion about combining subjective and objective endpoints into composite prespecified primary outcomes. Rheumatologists have done this for 25 years with the ACR criteria of improvement RA. You have to use a multiple threshold system where improvements in subjective features are combined with objective endpoints that corroborate their validity. It is probably the only way to assess many treatments for ME. It does not completely deal with the problem of bias (especially in very large trials if endpoints are defined statistically) but mitigates quite well.

I am afraid that you are straw-manning here, Hilda. It is a needless diversion from the work in hand - to try to get a Cochrane review that is fit for purpose.

 

Indeed they are not. Orthopaedic trials have been notorious for recommending procedures based on the sort of flawed follow up I used in 1973.

Another thread that is relevant here but I think would distract is the CCI one. There is no doubt that both doctors and patients may appear to believe in dramatic benefits from operations that almost certainly do nothing specific.

But Hilda was not arguing that pain was a valid measure of success in unblinded trials, rather that other valid measures may be included - as I think we can all agree on.

 

At the risk of distracting with CCI, I just read on Twitter that in Italy, a doctor claimed multiple sclerosis was caused by a problem in the neck vein called called chronic cerebrospinal venous insuffficiency (CCVI). Patients underwent surgery and some claimed to be cured. Only a few specialists were able to make the diagnosis and perform the surgery (just like with CCI). In the end, multiple studies failed to support these claims, like for example this properly controlled clinical trial https://n.neurology.org/content/91/18/e1660.

I think these stories are important because the general public and many researchers do not seem to realize how easily a convincing illusion of treatment efficacy can arise. I also see parallels with the lightning process story unfolding in Norway at the moment.

Unfortunately demanding properly controlled clinical trials for ME/CFS can easily lead to being called an unreasonable extremist.

 

Presumably by this you meant to say "running an unblinded trial". As @Jonathan Edwards has noted before, if the trial is properly blinded then subjective primary outcomes are OK, albeit better if can also be ratified by objective outcomes.

 

Like: I can't get the brakes fixed until Monday so it must be OK to drive the car today.

 

A behavior we saw play out in the FINE trial, which posited that "there is no disease, you can aim for maximum recovery":

https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-6-132

Being emotionally committed to an intervention as a purveyor or researcher of that intervention is highly problematic. This is the case for the BPS model of ME, it is an opinion, a belief system, it feels personal, proponents feel it should work because they want it to work and refuse to accept when it doesn't.

Science works by failing to falsify a hypothesis. Here there is no hypothesis, only assumptions held as true and all contradictory evidence is simply dismissed.

 

Really good summing up.

Yep. If ME/CFS really was a deconditioning problem and pwME's behaviour needed to be changed in order to fix it, then everyone here would have been running around and jumping up and down again a long time ago. But to coerce and brainwash people into undertaking a physical treatment, for an illness totally different to that on which the trialled treatment was modelled, seems bordering on criminal medical negligence, or worse. When the modelling is that wrong, you cannot have anything like the necessary confidence that harms have been detected or identified, especially if the tell-tale signals are misconstrued as normal aversion behaviour for the illness model presumed. If compensation lawyers ever get involved here, I would think that to be a key part of any argument.

Yes, as has been said many times, and recently in this thread, PACE actually does show very effectively that the deconditioning model is wrong. If the science had been done and reported honestly then we could have all congratulated the authors for proving that ... because they have. Strangely ... they reported differently.

Which of course it has been ... it's just that none of the b*ggers are listening who should be! And the bell ringers get told to shut up and stop making a nuisance of themselves. Hopefully if Cochrane manages to now get to grips with all this, then that might change.

 

as we see from the authors latest publication they appear to think PEM = 'Boom and bust'

https://www.s4me.info/threads/patte...chalder-sharpe-white-et-al.15313/#post-263730

 

Spot on.

I use the quote from the FINE trial when i do my critical approach to MUS talk with professionals. Just jaw dropping stuff.

 

Yes, I also would very much like to endorse that view @Hilda Bastian. Your robust engagement here in seeking to get to the truth of things really heartens me you will do the same within Cochrane. I'm also well aware that no matter how capable you are (and I'm sure that is 'very') there is still a steep hill to climb for you, and not just on the science aspects we mostly witness here. I imagine some of the other things you need to achieve within Cochrane must feel like having to turn the proverbial oil tanker around.

I sometimes think, when in the midst of robust discussions like this one here, that if we were all discussing this in the same room, then things would clarify much more easily.

So your willingness to engage, and above all your determination to do what is right, is hugely appreciated by us here. Sincere thanks.

 

Which again reinforces their misconception of deconditioning I imagine.