Assessment of the scientific rigour of RCTs on the effectiveness of CBT and GET for ME/CFS: a systematic review (2019) Ahmed et al

The problem is the flogging neither killed the horse, nor proved that the flogging was a bad idea.

A high quality study that focuses entirely on meaningful objective outcomes will (mostly) settle the debate. It may seem like a waste of time and money since we know the result, but...

 

I just had a quick glance at this paper (I'm failing to keep up with what I want to read at the moment) and noticed that they rated O'Dowd 2006 as having been blinded.

I remember looking at that trial ages ago (it's reported in a 140 page pdf), but can't remember much about it now: https://njl-admin.nihr.ac.uk/document/download/2001335

O'Dowd et al. say:

That's a kind of blinding, but I'm not sure how content in the EAS and CBT groups differed and I don't think it's really a 'double-blind' trial if the therapists are giving very different information, models of illness, expectations etc to one group than the other. Balanced plausibility is of some value, but will not counter a lot of the biases that we want to use blinding to help take care of.

It looks like there wasn't a significant difference between the CBT and EAS groups for most (all?) their outcomes, but they tried to put a positive spin on that by saying that the CBT group reported significant improvement and there were some significant differences between the CBT and SMC group.

Maybe I should have started a new thread on this paper, but don't have time to read it properly now.

edit: Kindlon already responded to the O'Dowd paper:

From O'Dowd:

 

Evergreen BPS. Could be turned into a sticker and slapped on every single trial, study, paper or even random thoughts.

 

@Tom Kindlon also made a lot of notes on that O'Dowd study here:

https://forums.phoenixrising.me/thr...p-programme-odowd-et-al-200.29241/post-445932

He also posted this on pubmed commons, though that system has since been taken down:

I also thought I'd link to O'Dowd discussing this trial, it includes the expected positive spin, plus some info on EAS that makes it sound very annoying:

https://www.csp.org.uk/frontline/article/addressing-needs-people-chronic-fatigue-syndrome

Sorry for going a bit OT in a thread about what seems to be a very important new paper. When reading systematic reviews I always end up getting caught up in going back to the individual studies.

edit: Another annoying thing that is now off0line was O'Dowd describing PACE results like this: "The effectiveness of CBT and GET is moderate (30% recovery, 60% significant improvement)"

It was on-line here:
www.avongpeducation.co.uk/handouts/2012/neurology/CFSME.pdf+&cd=6&hl=en&ct=clnk&gl=uk

 

Would that study - of CBT and GET to convince patients they are not ill and should ignore symptoms when they exercise - be ethical in people with ME?

 

It is not in any useful sense blinding. The concept of double blinding arose for drug treatments where patient and assessor are the key sources of bias. But for decades drug trials have been more rigorous than that. The treatment administrator has to be blinded too - so bottles of infusion came with code numbers the significance of which was only known to a pharmacist somewhere else who had access to a brown envelope with the code in.

So a standard drug trial is triple blind at least. For therapist delivered treatments the crucial person to blind is the treatment deliverer - the therapist, as Keith raised when discussing trial methods. So the O'Dowd trial comes nowhere near meaningful blinding and the risk of bias is pretty much as big as ever.

 

So a high quality study would be 300, 500 or 1000 people? Knowingly given them a treatment that might make them worse?

Seriously I think there are major ethical, concerns here. The genie is out of the bottle ...there are reported harms from this ...even if one may think the various surveys are inflated.

I also can’t really see how anyone can do an objective follow up study on graded exercise without doing graded exercise on test subjects ? So by it’s nature as a follow up you are repeating the study but with different outcome measures. That is if the objective is to prove something about the earlier studies.

One more person that is harmed because of GET is one too many in my book. I feel we need to tread with extreme caution and ask ourselves of any study ...what benefit does this bring to our collective knowledge and is it worth harming someone (whether they give their consent or not)?

If it’s just winning politics or the “debate” ..it’s definitely not worth it imo.

And there is also the small matter of the arguments. To date all the position against GET have been around calling for the withdrawal because of the potential harms. If we then encourage a further study ..doesn’t this undermine that argument?

I think there are far more sensible ways of winning the debate that doesn’t require us to knowingly make people’s condition worse.

 

I agree and have given this opinion to NICE. There are ethical problems here that were never addressed initially and remain to my mind insuperable within anything like the current style of trial.

 

I hope the Karl Morten team has data that will show GET to be useless/harmful.

Then we can finally move on from CBT/GET.

 

Trials assessing CBT for psychosis seem to report less positive results when the researchers assessing patients symptoms has been blinded. I expect that psychosis is a condition better suited to external assessment than CFS, but I don't want to be too down on any sort of attempt to reduce problems with bias. I don't think that this trial should be rated as 'blind', but it could be that if CBT trials used blinded assessors that this would help reduce problems with bias and maybe that should be rated as some form of semi-blind? I guess it would be good to have more data on how this form of blinding affects subjective outcomes and their correlation with more objective outcomes. Generally, I think that re unblinded trials with subjective outcomes I can be more 'this is a serious problem' and you're more 'this is a load of rubbish'.

 

I agree that attempts to mitigate problems need to be acknowledged. But if one wants to give benefit of doubt in relation to a trial of this sort I think one needs to look around and see what the evidence is that the authors are sensitive to the level of problem. I don't know a lot about this particular study but the development of CBT for ME/CFS in general makes my hair stand on end in terms of lack of insight into problems.

Blinding is never a formality and it is interesting to see just what lengths researchers now go to in order to ensure blinding. Jo Cambridge and I got involved in a cardiovascular study looking at immunology. The cardiac people were well funded by an organisation that had strict rules about blinding of studies. That meant that Jo Cambridge was sent a set of coded samples and was not allowed to see the code even afterwards. She had to send her data to a third party who knew the code and already had the statistical methods set up on a computer so that all she could do was press a button and carry out the predefined procedure. No re-jigging of anything was allowed. And this was supposed to be a study of objective data on an objectively defined cohort and controls. And we could see exactly why they were as tough as this. All the time people bend the rules. Outliers get removed. Experiments get repeated 'just to check' and so on and so on.

For a blinded assessor I suspect it would be pretty easy to guess which group the patient was in just by the way they sat in their chair and smiled politely, or not. It would be easy to tell if the patient was 'keen to help' (test group) or just 'happy to provide answers' (control group). There has been little or no discussion of this sort of issue over the last five years of debate over the trials. Maybe it has been looked at in other illnesses, as you suggest, but nobody has discussed it in responses to critique. The obvious reason for that would be that it is a bit of a can of worms. Hence my being asked to remove comments about blinding from my J Health Psychol manuscript.

 

I think using blinded assessors is an easy way to improve things a little, but it doesn't change the crux of the matter: the fact that both patients and therapists know which is the intervention group and that patients receiving GET and CBT are being trained to interpret their symptoms in a different manner. I think that's the main problem. So the fact that the assessor was blinded, like in the RCT that Wessely did (Deale et al. 1997), doesn't really change much in interpreting the results IMHO.

Sometimes it's hard to find a way around the problem of subjective outcomes in unblinded studies. Because I assume it's not always possible to use objective outcomes. Suppose you have a psychological intervention that aims to change the mental wellbeing of patients, nothing more. That would be an outcome that is plausible but difficult to put into objective measures and thus to measure. I suppose science simply isn't capable of measuring these kinds of effects yet...

Really? What happened? Did the reviewers of this article (PACE team response shows a disregard for the principles of science) ask you to remove critical comments regarding blinding, or am I confusing things?

 

Yes, my memory is a bit vague now but one referee required the removal of two crucial paragraphs about problems with blinding. I did not remove them.

 

So I think we can agree that GET (Graded Exercise Therapy) is bad, and not something to be repeated.

But why does nobody else seem to think there is potential use in investigating graded exercise (or just drop the graded part to make it just exercise)?

In theory, sedentary but otherwise healthy people can steadily increase their exercise, their body will react to this change in activity and therefore become fitter and more capable of handling this increased activity, whereas for patients they can safely exercise (or if not exercise then exert themselves) up to, what I call, their individual sustainable exertion limit but our bodies can't react to this change in activity in the same way - it's this inability for our bodies to react to exertion that could be investigated with a exercise/exertion approach.

Note that while the word exercise might conjure up images of people being forced to go for runs or some other high-exertion form of exercise, what we could be talking about is isolating a particular muscle with an exercise which is then repeated - the hand grip 'exercise' that the CureME/Biobank team have published on could be investigated in this way.

Care must, obviously, be taken with this approach in order to ensure, as much as is possible, that patients don't over-exert themselves past their sustainable exertion limit, and any patient taking part should have the potential for serious setbacks if they do over-exert themselves highlighted clearly to them.

But I think biomedical investigations of this sort could be very useful and am interested in hearing reasons why they might not be, other than an instant, all exercise is bad, reaction.

 

I have no problem with exploring activity or exercise outside of the theory-laden context of GET.

 

The very first "Perspectives" I read (89/90 or thereabouts) had an article which talked about "The Glass Ceiling" which said that ME patients can improve their fitness but only to a certain point. This is very difficult for outsiders to appreciate.

I have managed to do much more (on good days) by gradually increasing what I did but it has taken years! A sudden infection can take me right back.

When I talk about increases it is of the order of walking unassisted the length of a room once a day for a few weeks then doing it twice a day and so on.

I did not have a heart rate monitor at the time but I think what I was doing was keeping below my anaerobic threshold.

Just as the BPS theories have denied us access to therapies that help people with other illnesses come to terms with the losses of chronic disease they have prevented us from making the small improvements that can make such a difference when you are as limited physically as we are.

 

Which is why I made the point of informing patients at recruitment. And I don't think that most of us on the forum would be suitable participants, but I'd argue we have to accept that we are, on average, a more seriously affected group of patients, and that there are patients out there who could take part more safely.

Sorry but I'm not talking about Graded Exercise Therapy, and this is the thing that I think people are getting stuck on. The study would not be looking at using increased activity as a therapeutic regime, potentially following the PACE model where setbacks are to be worked through, it would be attempting to look at what differences there are, or aren't, between patients and controls in response to a reasonable and limited exercise challenge.

But you are a severely affected, bedbound patient, so this result isn't surprising - your sustainable activity limit is very low. Severely affected are unlikely to be taking part in a study - and while the severe should be studied, it doesn't mean that looking at the less severe, who will be better able to tolerate such a challenge, isn't worthwhile.

Edited to correct spelling.

 

You're welcome, and please don't challenge yourself physically any more than you absolutely need to again, leave that to those of us who are more able to cope with it.