IiME letter to Mark Baker (NICE) re: CBT & GET as recommended treatments

[Barry]

Null comparator is my do-it-yourself term, for which I apologise. However, I am not sure that there is an official term and there should be. It is an indication of how much one learns from these discussions that I only really got the idea of the difference myself while thinking about the PACE problems.

As an example: I used to stain tissues with fluorescence-tagged antibodies to tissue proteins that under a fluorescence microscope showed exactly where the protein was with a beautiful green line or red blob (technically known as immunohistochemistry). A null comparator would be to look at a section without adding any antibody. And that was important because it turns out that some parts of tissues fluoresce without even putting any stain on. But if we were not sure that our fluorescent antibody was reliably specific for what we were looking for we would compare it with using other fluorescent antibodies that should give other patterns or no pattern because they were directed against other proteins or nothing relevant (often called an 'isotype control'; you used an antibody of the same isotype family to make sure that if there was non-specific binding you would get it with both).

So a control in science is a comparator (comparison with something else) that allows you to exclude the possibility that your test result is due to a non-specific aspect of the experiment. A null comparator is the simplest comparator where you are comparing with doing nothing. So the standard medical care in PACE is a null comparator. An adequate control would be a 'fake' therapy delivered by an enthusiastic fake therapy therapist with all the extra messages that went with CBT or GET. In simple terms a 'placebo' is by implication a control, whereas waiting list is a null comparator. There is a world of difference, which is why we have placebo controlled trials.

Many thanks for that @Jonathan Edwards.

In my efforts to better understand:-

I guess the intervention outcomes unavoidably incorporate effects from all aspects of the intervention, including some you are not trying to measure, but you cannot discriminate between. So the controls, as best you can manage, give you outcomes due to the intervention facets you are trying to discriminate from. Then you can look at the difference.

I think you are also saying there will be some things that are not part of the intervention at all, but nonetheless can have an effect. They may be obvious things such as a standard treatment that can/must not be denied to patients, and/or may be things not at all obvious, but must not be presumed non-existent. So the notion of a null comparator allows these non-intervention influences to be measured and then nulled out of the results.

There is also the issue of random noise, which I imagine is what the statistical significance is there to deal with.

So with PACE, where (I think this is the case) they considered standard care to be their control arm, it wasn't necessarily so because it was not part of the intervention being evaluated. But the aspects of the intervention that should have had controls to discriminate what caused what ... was not really addressed.

But I'm not entirely sure I haven't got the wrong end of the stick here ...

 

[Skycloud]

Hope these are ok

 

[Barry]

"Therefore, I cannot accept at this stage that reports of flaws in one study invalidate the results of all the studies in this area."
[My bold]

This may not be so at odds with today's stakeholder impressions as it might seem. To me the above statement sounds like someone not yet acknowledging to change course, but making provision to do so. These people do have a way with their words.

 

[NelliePledge]

"Therefore, I cannot accept at this stage that reports of flaws in one study invalidate the results of all the studies in this area."
[My bold]

This may not be so at odds with today's stakeholder impressions as it might seem. To me the above statement sounds like someone not yet acknowledging to change course, but making provision to do so. These people do have a way with their words.

Good point @Barry probably wouldn’t be good for NICE to concede this straight away no doubt the BPS machinery is in operation trying to maintain the status quo and they would have a lot of hassle from the BACME people on the lines of you’re leaving us without guidance. If the community can mount well supported campaign publicly and engage NICE through the charities lobbying activity that GET/CBT needs addressing now that pressure becomes harder to resist. And NICE can say they had no option in the end but to take on board what people with ME are saying. They’re not going to go as far as saying don’t do GET but I believe if we can get the review we could achieve at least a warning message about dangers being added to current wording.

 

[Jonathan Edwards]

But I'm not entirely sure I haven't got the wrong end of the stick here

I think you have got what I meant.

What I did not point out is that for a trial to be regarded as 'controlled' the controls must be adequate. And an experiment often needs several controls to control for different confounding aspects of the intervention context. So a trial with controls is not necessarily a controlled trial. Just as a bike with just front brakes has a control but is not a controlled vehicle because it cannot stop suddenly without somersaulting.

I think things are confusing because nobody spells this out much. So comparators where you do not expect to get a result are called negative controls. Comparators where you expect to get a result, either the same as your test or maybe of a different but positive pattern are called positive controls. We often talk of running a set of negative controls and the 'null comparator' of e.g. putting no antibody on would probably be regarded as the ultimate negative negative control, so it gets called a control. But as indicated in the paragraph above on its own it does not make the experiment a 'controlled experiment' or at least not an adequately or meaningfully controlled experiment.

 

[Jonathan Edwards]

"Therefore, I cannot accept at this stage that reports of flaws in one study invalidate the results of all the studies in this area."
[My bold]

This may not be so at odds with today's stakeholder impressions as it might seem. To me the above statement sounds like someone not yet acknowledging to change course, but making provision to do so. These people do have a way with their words.

That is a fair point. Baker seems to be saying 'Please do not ask us to jump the gun. We have to follow a consistent procedure and the procedure so far has not led to a conclusion that weaknesses in PACE alter the analysis. That conclusion may change with further deliberation with a new committee but that has to go through due process.'

But I think it would have been better to say it like that rather than use the opinion that NICE has agreed should be reviewed as a reason for not acting.

It must be strong for NICE to be faced with a request to remove recommendation of a treatment because patients do not want it. Most of the time they are trying not to recommend things that people want, because the evidence is inadequate. If nothing else they must be aware of the deafening silence from all those patients who benefited from CBT and GET. Being a bureaucratic organisation they have to stick to their rules but it must seem a bit surreal even to them to stock to rules nobody wants.

 

[Forestvon]

Our recommendations were based on a body of research which preceded the PACE study. In our most recent review of the guideline, and cognisant of the controversies regarding the interpretation of the PACE results and methods, we assessed the evidence with, and without, the PACE results and the trial made no difference to the conclusions. Therefore, I cannot accept at this stage that reports of flaws in one study invalidate the results of all the studies in this area.'

But didn't all these preceding studies use the the discredited Oxford criteria anyway, not requiring PEM.

 

[Graham]

Interesting! The PACE trial was heavily criticized for claiming an effect when, in truth, they proved that CBT was ineffective. The use of the Oxford criteria was irrelevant there because PACE showed that for both the "London ME" and the "depressed" patients, neither CBT nor GET produced any real results. It was funded as a large study to provide definitive evidence, which it has. As such then, it invalidates the previous small studies.

Although, of course, that isn't true either. All of the previous studies showed that CBT was ineffective when tested objectively (there was one trial that claimed success, but had a high dropout rate). The evidence is consistent. The conclusion is that the committees were incapable of accepting the truth.

 

[Adrian]

Interesting! The PACE trial was heavily criticized for claiming an effect when, in truth, they proved that CBT was ineffective. The use of the Oxford criteria was irrelevant there because PACE showed that for both the "London ME" and the "depressed" patients, neither CBT nor GET produced any real results. It was funded as a large study to provide definitive evidence, which it has. As such then, it invalidates the previous small studies.

Although, of course, that isn't true either. All of the previous studies showed that CBT was ineffective when tested objectively (there was one trial that claimed success, but had a high dropout rate). The evidence is consistent. The conclusion is that the committees were incapable of accepting the truth.

We should also remember that FINE had a null result and was done after the NICE guidelines.

 

[Sly Saint]

One point that I don't think has been raised but comes up time and again, is the fact the CBT for ME/CFS is quite specific:
https://www.kcl.ac.uk/ioppn/about/difference/22-CBT-for-chronic-fatigue-syndrome.aspx
"
CBT for chronic fatigue syndrome
Our researchers were among the architects of a bespoke talking therapy for chronic fatigue syndrome, which is now one of two treatments recommended in the UK.

"
CBT for CFS is based on the premise that the way people cope with the symptoms may contribute to their illness. ‘People often believe that if they don’t rest, their symptoms will get worse’, says Professor Chalder, ‘yet extended periods of rest can make people feel more tired and unwell by weakening muscles and disturbing the body clock.’

‘We think that what starts the fatigue is not the same thing that perpetuates the symptoms,’ she says. ‘People might initially develop the fatigue as the result of an illness, such as a virus, or after a period of stress. But once triggered, the fatigue is maintained by other factors, including some coping styles.

‘Beliefs and attitudes towards illness are important in many physical and mental conditions. In CFS, fear that exercise or activity may make symptoms worse can hinder recovery and inadvertently perpetuate the symptoms.’ "

NICE do not seem to take this into account/make it clear in their literature, and when they and others do comparisons (ie say that CBT is used as a coping mechanism for many other diseases)
it is left up to those in the ME community to constantly point out the differences.

At the very least NICE should acknowledge that this form of CBT is not suitable and should immediately be replaced with a form of self-help/talking therapy that is offered to other patients with chronic illnesses.

 

[Jonathan Edwards]

But didn't all these preceding studies use the the discredited Oxford criteria anyway, not requiring PEM.

That in itself does not make the studies bad science. It just means that any conclusions have to be applied to an Oxford defined population. I think the Oxford criteria are a poor way of defining a population but they do not make the studies invalid. I would agree with the PACE authors that this is not an argument against validity. The arguments against validity come in the trial design.

 

[Barry]

"CBT for chronic fatigue syndrome
Our researchers were among the architects of a bespoke talking therapy for chronic fatigue syndrome, which is now one of two treatments recommended in the UK."

In fact it is clear the same term, CBT, should not be used for both! If it were named more accurately FPIBC (false physical illness belief correction) or something, the eternal conflation of these two very different treatments would not be perpetuated time after time.

 

[Barry]

I think you have got what I meant.

What I did not point out is that for a trial to be regarded as 'controlled' the controls must be adequate. And an experiment often needs several controls to control for different confounding aspects of the intervention context. So a trial with controls is not necessarily a controlled trial. Just as a bike with just front brakes has a control but is not a controlled vehicle because it cannot stop suddenly without somersaulting.

I think things are confusing because nobody spells this out much. So comparators where you do not expect to get a result are called negative controls. Comparators where you expect to get a result, either the same as your test or maybe of a different but positive pattern are called positive controls. We often talk of running a set of negative controls and the 'null comparator' of e.g. putting no antibody on would probably be regarded as the ultimate negative negative control, so it gets called a control. But as indicated in the paragraph above on its own it does not make the experiment a 'controlled experiment' or at least not an adequately or meaningfully controlled experiment.

I think there is much abuse of the "evidence", "controlled" and "RCT" terms, and not always accidental perhaps. If PP could cite the evidence from a RCT (the gold standard as he gloated) supporting his LP, something is seriously amiss.

The word "evidence" is invariably taken to mean what it should mean - highest quality evidence. But evidence, be it clinical trials, police investigations, engineering investigations, whatever, is the culmination of a detailed and complex process, where the integrity throughout that process is paramount for the quality of evidence. If the controls are lax or just badly thought through, then any evidence appearing at the end is doomed. People citing "evidence" and "RCTs" seems to be used as cover for rubbish science. Which is a huge shame, because there must be so much good science being done. A bit like a nice pind of beer is acually the result of a pretty complex process - anything amiss along the way and it will be rubbish.

 

[Jonathan Edwards]

In fact it is clear the same term, CBT, should not be used for both! If it were named more accurately FPIBC (false physical illness belief correction) or something, the eternal conflation of these two very different treatments would not be perpetuated time after time.

Quite so, although maybe CBT is false illness, or life situation, belief correction and the treatment for ME is false illness, or life situation, belief inculcation (the prior belief being more realistic).

 

[MEMarge]

In fact it is clear the same term, CBT, should not be used for both! If it were named more accurately FPIBC (false physical illness belief correction) or something, the eternal conflation of these two very different treatments would not be perpetuated time after time.

You obviously need more time at the Bristol NHS acronym clinic Barry!

Seriously though, the fact that recommendations for CBT in NICE are based on "correcting false illness beliefs", needs to be spread widely.
PwME do not have a problem with counselling or similar supportive therapies (whatever the current in-terminology) for those who want/need help coming to terms with sudden, significant disability in their lives, as many people with other disabling long-term conditions do.

 

[MEMarge]

There was a lovely neurophysiotherapist at our table yesterday from a northern clinic. She is the "service lead"(?correct terminology, but she's in charge of service provision at their clinic). Their part-time CBT person does not do FPIBC (see above) and they'd recently done a survey where nearly all their PwME had found it helpful. They also do useful things like writing letters and helping with PIP applications.

Contact details are available for anyone wanting to move into the area, or find suggestions on what to include in an NHS service for people with ME. 50% of their clinical time is spent on home visits.

Am hoping, and will be encouraging her, to be involved further in the Guideline Development.

 

[Sly Saint]

Their part-time CBT person does not do FPIBC (see above) and they'd recently done a survey where nearly all their PwME had found it helpful.

Unfortunately, this would simply be regarded as a positive for use of CBT for ME.
Which is why NICE needs to make it crystal clear that the Chalder style CFS-CBT is not appropriate and should not be used.

 

[Forestvon]

That in itself does not make the studies bad science. It just means that any conclusions have to be applied to an Oxford defined population. I think the Oxford criteria are a poor way of defining a population but they do not make the studies invalid. I would agree with the PACE authors that this is not an argument against validity. The arguments against validity come in the trial design.

But I understood that NICE currently requires PEM as a symptom so shouldnt use as evidence the results of research that doesn't.

 

[MEMarge]

Unfortunately, this would simply be regarded as a positive for use of CBT for ME.
Which is why NICE needs to make it crystal clear that the Chalder style CFS-CBT is not appropriate and should not be used.

Absolutely, this is what we were explaining, why we were so anti the PACE/Guideline CBT. We were near her, so this was more of a private chat.

 

[Invisible Woman]

Absolutely, this is what we were explaining, why we were so anti the PACE/Guideline CBT. We were near her, so this was more of a private chat.

And here we have an issue:

Are the good ones, using CBT as a support and adjustment therapy, doing so on the quiet? Are they doing so knowing it isn't actually what the guidelines mean and want to keep their heads down in case someone notices and demands they inflict the toxic kind?