Monitoring treatment harm in [ME/CFS]: A freedom-of-information study of National Health Service specialist, 2019, McPhee et al

Abstract
The use of graded exercise therapy and cognitive behavioural therapy for myalgic encephalomyelitis/chronic fatigue syndrome has attracted considerable controversy. This controversy relates not only to the disputed evidence for treatment efficacy but also to widespread reports from patients that graded exercise therapy, in particular, has caused them harm. We surveyed the National Health Service–affiliated myalgic encephalomyelitis/chronic fatigue syndrome specialist clinics in England to assess how harms following treatment are detected and to examine how patients are warned about the potential for harms.

We sent 57 clinics standardised information requests under the United Kingdom’s Freedom of Information Act. Data were received from 38 clinics. Clinics were highly inconsistent in their approaches to the issue of treatment-related harm. They placed little or no focus on the potential for treatment-related harm in their written information for patients and for staff. Furthermore, no clinic reported any cases of treatment-related harm, despite acknowledging that many patients dropped out of treatment. In light of these findings, we recommend that clinics develop standardised protocols for anticipating, recording, and remedying harms, and that these protocols allow for therapies to be discontinued immediately whenever harm is identified.

Paywall, https://journals.sagepub.com/doi/10.1177/1359105319854532
Sci Hub, https://sci-hub.se/10.1177/1359105319854532

Many thanks to the authors, @Graham , @Tom Kindlon , @Brian Hughes

 

Good work! And concerning.

I can't see it in the paper itself, so forgive me if it's there and I've missed it, but how many of the 38 responders said that they provided *any* information/guidance to patients about safety and risks of GET? Were there any that admitted that they provided no information?

Are the responses from the centres available in supplementary form somewhere?

 

1 (out of 38) clinic reported having guidance to determine harm

"Harm may be defined by the patient as [it] could take different forms … If patients rate their health as worse or much worse we review the written information to assess if this is related to the treatment or to other causes, such as a deterioration in co-morbidities or a life event"

"A number of other clinics reported that they used scales such as the 36-Item Short Form
Health Survey (SF-36; Ware and Sherbourne, 1992) to measure improvement and deterioration. However, no clinic said how or whether they used any such measure to determine whether a patient had been, or might have been, harmed.

Overall, 18 clinics (47%) responded that they gave no written guidance to staff about harm detection, nor did they report giving any verbal guidance; 19 clinics (50%) gave responses that did not address the question."

 

Until the clinics accept that CBT/GET can cause harm they will continue to attribute any deterioration/drop out to 'something else'

eta: cross posted with ukxmrv

 

Guidance to determine harm is Q2, I was asking about Q1, which was about the information provided to patients.

If you're going to do a paper on clinic responses to 2 FOI requests, the main data needs to be in the abstract, so it would be useful to know how many clinics actually responded to those questions with any affirmative data at all.

 

Rather than read it from the perspective of someone looking for evidence of harm, it might be better to read it from the perspective of someone who has read repeated assurances from specialists and clinics that (as most suggest) the therapies are safe. There were many examples where questions were not answered properly, or at all, but nothing to give any strength to such an assurance. The overwhelming effect is that clinics do not monitor their outcomes in such a way that they can make any such statements.

In question 1:
15 said "setbacks" may happen: 12 said that symptoms may temporarily worsen: there was a general implication (sometimes quite specific) that if so, it was because the therapy had not been followed correctly. None said that there was a chance of ending up worse, none that this could be long-term, one that there was controversy over safety/risks (it wasn't a strong statement), and two gave criteria for the patient to report worsening. Overall, the clear emphasis was on how safe and effective the therapies were.

Interestingly, all clinics said that they followed NICE guidelines, but the variations between them were quite large.

 

Thanks, @Graham.

It wasn't clear from the table that the 12 responses about "symptoms may temporarily worsen" came from within the 15 responses about setbacks, and I didn't want to make that assumption.

Did that mean that 23 centres didn't provide any information about potential setbacks or worsening and gave the impression that the treatments were completely safe, or did they fail to provide enough information to even make that assessment?

There are wider questions about informed consent here. That "they placed little or no focus on the potential for treatment-related harm in their written information for patients" is key. But it would be useful to know the numbers for "little focus" and "no focus", as it would do for assurances about safety.

I'm thinking along the lines of, "Whereas X clinics provided patients with assurance about safety, Y clinics did mention the potential for treatment-related harm and Z clinics did not mention the possibility of harm/setbacks at all."

 

And I didn't answer that properly, did I? It isn't in the report, but going back to the original spreadsheet, I count 4 centres that said "symptoms may temporarily worsen" that did not say that "setbacks may occur": the other 8 said both.

You may puzzle over why our analysis wasn't clearer. It was simply that the responses were incredibly difficult to sort out. Even the simple question of "how many patients did you treat" gave us answers like "120 patients were referred to us": I think it was a couple of centres that gave us the number referred and the number treated, and they very broadly matched the reports from Newcastle and London that around half of patients referred to those centres, after careful testing, were found to have other conditions. (The percentages varied from year to year, but "around half" will do for this purpos.e) Of course some centres had no specialist, so presumably did not attempt to diagnose. Fifteen answered the question clearly, eight were unable to tell us, one declined to tell us and fourteen gave us a number that didn't actually answer the question.

I tore my hair out in frustration with many of the responses!

I can't generalize about the 23 that didn't give such information about setbacks etc. without trawling back through their responses, but they failed to mention any hint of setbacks, temporary or otherwise. Some were clear that the therapies were completely safe, and that any problems would have been the patient's fault (they weren't as explicit as that!). Some didn't say anything. It was like herding cats. I thought the questions were pretty straightforward.

One difficulty was that eight centres (possibly ten) treated patients on an entirely individual basis, so were unable to tell us much at all about what went on. (That is my analysis from the spreadsheet – the number wasn't in the report). So we cannot say that they were not told these things, but equally, the centres cannot claim that they did.

I'll go back to my original comment: the answers overall were so muddled, vague and varying, that it is not worth trying to decide what they actually said (unless you want to end up with my hairstyle). Much more relevant is the fact that they did not have the information to declare that these therapies are safe.

 

@Graham So, just to confirm, did all the information in Table 1 come from just 15 (of the 38) centres?

 

It seems there might be quite a lot more unpicking to do!

I guess this shows the difficulties inherent in FOIA requests. One needs to be able to ask the sorts of general questions that you've asked, so that you can then work out what more detailed and specific questions to drill down on.

It would be nice to think that the centres would be happy to engage in a more detailed survey along those lines in the future, now that you have more of an idea of what to ask. But I'm somewhat skeptical.

Organisations who understand and welcome the ability of the general public to ask such questions are usually helpful in supplying their answers. My OH does this for his organisation, and if a question is badly put, he will often volunteer a more suitable question that will provide the answers they are actually looking for, and will explain why that is so. Obstruction and obfuscation help no-one, and can be quite dangerous if the information provided is then misconstrued.

 

Good to see this out. It provides important evidence, even if 'evidence of absence of evidence'.

 

No, my comment above explains that 4 of the 12 that answered that "symptoms may worsen temporarily" did not mention "setbacks", i.e. they were not in the 15 above. You cannot assume any data is a subset of another unless we explicitly state so.

If it had been possible to simplify the data more clearly, we would have done so, but it was a tangled mess. There was very little consistency. If you are trying to report back on this, all I can do is refer you to my original comment. I do not think you could ever produce an FOI to find out exactly what clinics were doing: our aim was much more specific – to find out whether there was a background of analysis to clinics' and specialists' claims that these therapies were safe, and our conclusion is that there isn't such evidence.

Can you have a more specific question than "How many patients did you treat in 2016 (or nearest year)?"

We would not like to comment on whether the avoidance in answering our questions was deliberate, or whether it reflected their own woolly thought processes.

Some centres gave us the bare minimum of sentences, with curt replies: some wrote extensively, but wandered all over the shop. I guess that the truth is that there was a great variation in attitude. Only a handful were relatively helpful, clear and precise.

I'd also like to remind you that I'm not an academic, so my explanations here may not match research standards.

 

Great idea and well executed. Many thanks to all those involved!

That's also interesting.

 

Absolutely- if you’re not looking for data, you’re not going to get any.

Would be interesting to know the drop-out rates. They may be indicative of ‘something’.

 

4%-44% in the 7 clinics that reported them (Table 2).

In the line below that, one clinic reports worsening symptoms of more than a few days (etc) in 65% of patients. The only other clinic reporting that data gave a figure of 0%.

The loss to follow up also varies widely - 4% to 75%. But did any clinics provide info on how long FU was? - yes - up to 12 months. Again, 12 clinics said they followed up pts but then most couldn't say what their LTFU rate was. *sigh*

 

Thank you

 

This was one of the hardest pieces of information to sort out. We did ask for follow-up after the treatment had finished, but it seemed as though many clinics took the last session of treatment with a "So how do you think it is going?" enquiry as a follow-up. That is my personal interpretation of the responses!

I don't think that many (perhaps one?) clinics carried out any form of follow-up that (a) left a gap between the last treatment and the follow-up, (b) asked questions that could be compared with baseline responses, (c) were asked by folk other than those who actually carried out the treatments, and (d) were capable of being put together and analyzed.

Believe me, it was a struggle to decide what many of them were actually reporting, which is why I keep emphasizing that some of these statements are just my personal impressions, trying to give you a realistic picture, but warning you against making clear-cut statements to that effect.