Effectiveness of a symptom-clinic intervention ... multiple and persistent physical symptoms, 2024, Burton, Deary et al

Diagnosis

See the post upthread for the list of questions in the PHQ-15. I note that women have to answer an additional question about periods, and I think most menstruating women would answer at least 'bothered a little' about period symptoms. So, the PHG-15 and the diagnosis approach makes it more likely that women will be included in the sample.
Edit - later in the paper they say they adjusted scores when people left that line and/or the one about sexual activity blank.

So, they were looking for people who had a code for a physical symptoms syndrome, had been referred to specialists at least twice in the preceding 3 or so years, but did not have a medical diagnosis thought to cause symptoms. That leaves a great gaping hole for people with persistent symptoms with a biological cause that has not yet been identified. And these researchers thought it was ethical to instruct doctors to 'negotiate' a made up explanation basically amounting to 'your brain is wrong, it got muddled up somewhere along the way, and so you only think you are unwell, but actually you are fine, so, get on with things'.

GPs were able to veto someone being invited into the study. I'd be very interested to see the gender split on the veto'd versus non veto'd.

Depending on what the patient information sheet said, there may have been some substantial self-selection. It would be interesting to know how the intervention was described.

 

Randomisation; masking
A big deal is made of randomised allocation of people to the treatments. But that is rather insignificant when the participants knew whether they were allocated to the active treatment or 'nothing'.

Just noting that quote about some interviews of participants they did - the trial did include people whose primary symptom was fatigue. I guess if ME/CFS isn't diagnosed, you don't have to follow the NICE ME/CFS Guideline.

They audio recorded all the symptom clinic consultations. That will be a gold-mine as they refine techniques to convince people their symptoms can safely be ignored, and if ignored well enough, will go away.

A lot of measures were taken - it will be interesting to see if results for all of this are reported on:

 

It looks as though harms were reported by participants, but I assume that if the participants stopped participating and weren't contactable, then the harm wouldn't be reported. There is no indication that medical records were checked during that following year or that there was any feedback from the participant's own GP, and health care use was only gathered by self-report from the participants who remained around to still be participating.

I always get a bit worried when investigators start messing around with the data. Does this mean 'Oh, this doctor didn't seem to have such a good impact on improvements as all the other doctors, so we will improve the scores of his patients a bit'? Or, older people didn't seem to improve as much as younger people, and our treatment sample had more older people than the control sample, so we'll just improve the results of all old people. It seems like a black box of opportunities for data manipulation in the direction you want. I think the onus should be on researchers to make the control group a good match so adjustments aren't needed. And I don't understand the rationale for adjusting for the impact of the individual doctors 'treating' the participants.

Ugh

 

7837 people identified from GP records
891 excluded or not sent a participation pack - mostly due to the GP saying they aren't suitable e.g. because their symptoms probably are due to a real disease, probably also some GPs would know some patients would not be receptive

6946 sent the participation pack
5688 did not return the pack!!
1258 assessed for eligibility

904 were excluded, due to not being eligible or not wanting to proceed

354 randomly assigned to the treatment or control. That's about 5% of both the people identified from GP records and of the people sent a participation pack. There is such a selection bias there. If this treatment turned out to be wildly successful in this patient group, that selection bias would need to be borne in mind - this trial would not be evidence of the intervention being successful in the vast majority of cases.

 

Of the 354, 176 were assigned to usual care and 178 to the intervention
135 assigned to usual care made it to 52 weeks; 144 in the intervention made it to 52 weeks.

The numbers used in the Intention to Treat Analysis were those still present at 52 weeks, so did not include people who withdrew or were lost to followup. Numbers who withdrew and were lost to followup were similar in both groups and not very large; more withdrew in the control group.
3 of the people in the control group did not have baseline data; presumably that was imputed from their 3 month data.

That's interesting. So, one GP dropped out with no reason given and had seen some participants, and there is nothing said about what happened to their patients and the data.

As expected.

All the better to (tr)eat you... (reference to the wolf in red riding hood)

 

Well yes, and even then 'conceivably' is doing a lot of heavy lifting. As @rvallee would say, the giant spaghetti monster in the sky is conceivable, it doesn't mean it is likely.

As Nightsong said the Toussaint paper suggested the standard error of measurement was 2.3, which seemed to be suggesting that that was an estimate of a minimal clinically important difference:

And this paper found an improvement of 1.82. So, that's not actually clinically important. But they argued that because the 95% confidence interval of -2.67 to -0.97 includes the minimal clinically important difference, they could still say their intervention helped.

So, I would say 'in a subjective measure in an unblinded study of highly selected individuals that cannot be assumed to be representative of the disease population, comparing a group that got something to a group that got nothing, and with some dropouts and who knows what they did with their model "adjustments", even then, they did not achieve the bare minimum that could be conceivably called clinically important.'

 

I think, just assuming a normal distribution with that mean and that 95% confidence interval, that there is a 13% chance that the true mean is the supposed clinically important difference of -2.3 or more. Which makes sense if you look at the reported 95% confidence interval of -2.67 to -0.97. -2.3 is well to the end of the range. A question for people with better statistics knowledge than me, (ignoring all the other problems with this paper and just looking at the maths), can the researchers credibly claim that this study shows that the intervention achieves a minimal clinically important difference?

If you set that along side the harms that they have not properly assessed and the cost of providing the intervention, I think there is a lot of smoke and mirrors and no real benefit.

The authors seem to be suggesting that, for the people it works for, it works really well. I haven't seen the data on that yet.

 

Results

There were people who didn't attend the initial session or stopped attending the sessions, and that will probably have affected the results, but the rates of participation (for the 4 sessions) aren't too bad.

No statistically significant difference at 13 weeks or 26 weeks.

Table 2 gives the means for a range of other measures at 52 weeks, for the two groups. It is concerning that the numbers of participants varies quite a lot for each measure - missing data creates opportunities for inconvenient results to be lost. There is no analysis of statistical significance or minimal clinically important difference. There is no comparison with baseline measures. There is no discussion of these other measures in the results section at all.

Here's one: PGIC Patient Global Impression of Change
Usual care 1.8
Intervention 3.2

It therefore looks as if the usual care people were reporting that their health had, on average, 'much improved' tending to 'very much improved, while the intervention people reported that their health had, on average, 'minimally improved' tending to 'no change'. I find that hard to believe, and I wonder if data got entered in the wrong columns. But, it's either not reflecting well on the utility of the intervention or it's not reflecting well on the reliability of the paper.

I'd appreciate it if someone could check that PGIC finding. It might be worth poking around in the other data there.

A very low percentage of participants reported their health-care data. I haven't looked at the differences between group, but the authors are suggesting that the differences are minor.

On adverse events - some of the participants turned out to have medical conditions that may have explained their 'persistent physical symptoms'.

 

yes, sorry if it wasn't clear. Improved relative to the outcome in the 'do nothing' group.

 

Thanks Lucibee.

It looks as though they have completely changed the way the PGIC is scored, as well as using two different methods of collecting scores. And even then, the intervention doesn't come out well. Bear with me, it's confusing, and I hope I haven't confused myself. But, if I haven't, then it confirms my impression of the work of these people - it would be a joke if it didn't have an impact on people's lives.

PGIC - Patient Global Impression of Change - so it's an important measure

Here's the approach I found when googling PGIC:

And that's fair enough, there is the full range from very much improved, right through to very much worse. A low value is good.


But the top part of the PGIC that is reported in this study's appendix, as copied in Lucibee's post above, reverses the order:



But look - the range is essentially only from 'no change' to 'wonderful'. And, they don't say how that scale is scored. I reckon just that scale alone sums up all the problems with the psychosomatic medicine proponents. But, incredibly, these authors have managed to add layers of obfuscation and confusion to that.



The second scale in the PGIC scale appendix does have the full range:


But that is a 10 point scale, and it runs from better to worse, so lower numbers are better.


It's all a mess - we don't know what scale was actually used.

To remind ourselves of the numbers that were reported in this study for PGIC:
Usual care 1.8
Intervention 3.2

So, if the bottom scale in the appendix was used, the average outcome reported by the usual care people was better than the average outcome reported by the intervention people. !!

If the top scale in the appendix was used and a logical scoring symptom running from low to high was used, from 1 to 7, the average intervention score of 3.2 is still 'A little better, but no noticeable change'.

 

Maybe write to the authors and ask how the figure was calculate. They shouldn't be reporting a figure without the clear source.

 

How can something be 'a little better but no noticeable change'? If the change isn't noticeable, how do they know it's a little better?
That scale that only allows one negative/no change and lots of variations of positive should be illegal. It's not a research instrument, it's the cheats version of market research.

 

Yes, that looks right.



There are different versions of the PGIC. I don't understand how any scientist could look at the version these authors used (the version above) that so obviously minimises declining health and any harm from the intervention and has vague, difficult to differentiate descriptions and think that it was ok to use. It suggests that all of the literature using the PGIC is suspect until proven otherwise.

P.S. even the instruction on the questionnaire serves to underline to the people in the 'care as usual' group that they didn't get care at the special clinic, and so leads them to a default of 'no change'.