UK:ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services, 2023

This is an aside and isn't to do with any of the PROMS authors here, however I just needed to get it out because I couldn't quite believe it (certainly given what we know/is acknowledged now). From the Crawley and White (2013) paper: Treatment outcome in adults with chronic fatigue syndrome: a prospective study in England based on the CFS/ME National Outcomes Database | QJM: An International Journal of Medicine | Oxford Academic (oup.com)

They have detailed below how they used a complex 'GEE model' in order to apparently adjust the scores where follow-up questionnaires were returned late - in order to be precise in working out what the 'fatigue level' would have been at 12 months. They say that some teams obtained data at additional follow-up points (6 and 24-months) so I guess they used these to back-fill the model. And I don't know either how many of the sample would have had this done nor how 'far off' they were 12months. But it seems a pretty astounding approach given they are using retrospective data, but on something that hasn't actually been researched before?? ie it isn't like they've got 25 trials worth of data covering all these time periods to show some curve.

Yet I can only assume they think they are being really diligent by doing this.

"Follow-up interval
Each team sent out follow-up questionnaires at 12 months. Variation in when the questionnaires were sent and delays in return of 12-month follow-up questionnaires led to variation in the exact time of follow-up. Also, some teams obtained data at additional follow-up points (e.g. 6 and 24 months). To maximize the availability of follow-up data for our analysis, we determined a margin of follow-up either side of 12 months. We did this by fitting fractional polynomial generalized estimating equation (GEE) regression models of fatigue against time.21,22 This method (implemented in Stata as fracpoly combined with xtgee) compares a linear GEE model with the best-fitting first and second degree models, each containing fractional polynomial terms (from a pre-defined set of integer, fractional and negative powers) for time. The differences in deviances between the linear and 1st-degree model and the first and second degree models are tested using a chi-squared test and the resulting P-values indicate whether the change in outcome over time is linear or whether it has a more complex shape. We inspected a plot of predicted values of fatigue against time from the model with the best fit to determine an appropriate follow-up interval in which observed fatigue scores could be assumed to be representative of the scores predicted at 12 months. In patients with more than one follow-up assessment, the closest to 12 months was used."

This post has been copied and discussion of this paper moved to:
Treatment outcome in adults with CFS: a prospective study in England based on the CFS/ME National Outcomes Database, 2013, Crawley et al

 

Adding up these dates. I'm starting to wonder whether instead of 'repeating' it is more 'a continuation' ie those who thought they were on the 'more balanced' side (not necessarily outside their bubble is that confirmed) were planning these PROMS because they 'thought GET needed tweaking' from eg 2011. Then in 2013 start saying about measuring.

And if these PROMS papers actually go back to 2014/15 for planning stage...

I don't know if these PROMS for ME Association are basically the same questions or if they have properly updated with all the latest research and developments in the new guidelines that include the quality assessment of what research was low and very low quality and so on.

But I can see how some might kid themselves that the issue isn't the treatment per se but the variation in those who do it 'doing it differently' etc and so becoming over-focused on delivery because of those beliefs.


The thing is that just like in the MEA article from 2011 ( Physiotherapy journal urges gentle approach to prescribed exercise for people with ME - 19 January 2011 - The ME Association ) Jessica Bavinton clearly thinks she is being super-gentle and then I'd guess thinks that 'it doesn't harm' because she doesn't see anyone harmed. But as we all know that is self-perception. I can see why some might think they'd want to show this off by having measures showing up those other clinics who 'obviously aren't as careful in delivery'. Except the issue isn't just the 'survey' but stuff surrounding who fills it out. Hence the MEA survey had 50%+ of 900+ people who did GET saying it made them worse. But noone fancied returning to the clinic to tell Jessica to her face and carry on with appointments to be able to complete her survey frankly if they did get worse.

And / but the same issues with why those who might have got worse or disagreed with her self-assessment weren't on her radar would still blight any other PROM unless they were directly tackled - eg dropping out because you get iller meaning that you can't submit your data, feeling perceived threats if you claimed something had harm/being worried about the impact on other things for you if you upset someone - all of which mean that it doesn't really matter what the questions are in the survey if you aren't fixing whether something is 'externally valid' because the sample is being skewed.

Plus of course not everyone goes or can go to a clinic.

And these things clearly are to be used to influence not just 'delivery' but what treatments are actually offered etc. ?

 

Oh yes and (sorry brain is picking up on things then train of thought spots another to summarise).. to draw attention back up to near the top of this, with the paper that included CRawley and White (2013) explicitly saying 'physical function' in the abstract conclusion:

"Patients who attend NHS specialist CFS/ME services can expect similar improvements in fatigue, anxiety and depression to participants receiving cognitive behavioural therapy and graded exercise therapy in a recent trial, but are likely to experience less improvement in physical function."

but then making claims on 'CFS' in the conclusion itself's wording:
"Although NHS services are moderately effective in improving fatigue in patients with chronic fatigue syndrome, they are much less effective in improving physical function than similar treatments delivered in the PACE trial."

In the statistical analysis section:
"Our primary outcome measures were fatigue and physical function. Our secondary outcome measures were anxiety, depression and pain. "

"We defined clinically useful improvements within each person as having a difference of 2 points on the total Chalder Fatigue Scale and a difference of 11 points on the SF-36 physical function subscale between the baseline and follow-up measurements. We chose these cut-offs because they equated to 0.5 SD of the distribution of the baseline measurements."

It is that phrase stating clearly 'likely to experience less improvement in physical function' having been admitted back in 2013 by a paper with both of those on it.

And what now from their definition of 'physical function' vs what fell into the rest they defined in the conclusion as 'CFS' but in the abstract as 'fatigue, anxiety, depression' (and tackled by that old CBT) would be the bit that was actually deemed as making up an accurate description of 'ME/CFS' these days ?

And the big difference in that time being quite a significant change in the understanding and defining of what the condition is - ergo: that first phrase is correct because it will always be for that point in time '[some] patients who attended the CFS/ME services'

but I'm unconvinced by claims of having PEM required for data collected on any of those dates. Or an accurately defined version of it. It is interesting that eg the papers with Gladwell make a point of claiming they had PEM required for papers where the data would have been before eg 2018 (then references the 2007 guideline) despite the papers being published 2023 and 2024.

So yes, I've found it quite interesting spotting this 2013 paper and then looking at Gladwell's work on PROMS that include critique of the CFQ which focuses on the 'fatigue' measure and suggestions of needing better PROMS. But were begun much earlier than it might seem given the recent publication dates and both collected data prior to the new guideline.

 

@dave30th

 

This is all a bit complicated for me but am I right to think that maybe during 2011-2017 the Bristol group were finding 1. their studies were struggling to confirm an effect similar that claimed by PACE and 2. they were aware of increasing pressure from patients to tone down or abolish GET.

The perceived solution was to find an outcome measure that would continue to validate a form of 'pacing-up- or 'GET-light' that was their justification for existence.

The flaw of course is in thinking that PROMs of this sort are of any use as a way to validate a treatment. They might possibly be a way of showing why one centre got better results than another but comparisons of that sort are fraught with difficulty and only really make sense in surgery where hard endpoints are available - like survival or ability to walk.

 

Precisely! I also referenced Crawley's separate paper (also published in 2013) in respect of 5 centres from the National CFS/ME Outcomes Database. The Crawley and White paper later that year does the same thing but for 6 centres. Similar results. I think the Correction in the White paper and the Ethics approval (same in both) was probably a result of @Dave blogs on the 'service evaluation' questioning the ethics.

Good find, I'll have a proper look later, but just from the section you quote:

"I wonder whether (sorry if you already pasted it) the following article from MEA that does include Gladwell among others adds some interesting context - it is from 'just before' the PACE results were published and talks of how Nice is minded to review its guidelines based on a recent consultation... but is waiting on the results of the PACE trial first. Physiotherapy journal urges gentle approach to prescribed exercise for people with ME - 19 January 2011 - The ME Association "

While all the focus has been on CBT and GET, it hasn't really understood what they're doing in Bath with "Activity Management" as they called it then and mindfulness CBT.

I don't think the Bath service were providing GET, I could be wrong.

Bath at the end of 2011, when I attended the adult service, it was group 'activity management' and the mindfulness CBT.

What Gladwell describes is the the 'pacing up' which as they call it today. That, in combination with mindfulness CBT (ME specific) is what finished me off in 6 weeks and left me bedbound for a year and severe ever since.

On entering the clinic 6 wk course, I was already working from home on a phased return to work. My HR/OH department were following the Occupational aspects of the Management of Chronic Fatigue Syndrome: National Guidelines (2006).

Occupational Aspects of the Management of Chronic Fatigue Syndrome: a National Guideline (nhshealthatwork.co.uk)

Prepared by Chalder and Nickie Catchpool (OT of the adult service, Bath - and still is), with external advice from White and Sharpe. It references Chalder's research among others, with 10-20% weekly increases in return to work pointing out that employees with CFS/ME would fall under the Disability Discrimination Act 1995 (now incorporated in the Equality Act 2010).

 

Take this comment, for example from the 2011 MEA article you found:

"It seems that when people have had a poor experience of graded exercise therapy they felt they hadn’t had permission to step back,’ he says from the 2011 MEA article. [my bold]

This is NEVER being taken into account in research, or in commentary, and should be every time reference to any form of 'pacing up' or rehabilitation is discussed:

1. When you attend clinic you sign a consent form for them to obtain access to medical records from GP, and liaise with employer's HR/OH. Your GP, clinic and employers are all discussing you. They knew what they were doing! They still do
2. People still trying to stay in work while attending the clinic were, therefore, receiving a double hit of pacing both at clinic and increasing work activity/hours by 10-20% per week, and for all we know still are.
3. Feeling that "they hadn't had permission to step back" does not take into account job security pressures. HR departments were advised by their national organisation the CIPD CIPD | The Professional Body for HR & People Development to following these 2006 national guidelines. I remember reading an article by the CIPD particularly referencing the economic cost burden that 250,000 pwME would have on employers, particularly small ones, if the guidelines were not followed. Can't find it on the web now.

What has happened since the new guidelines in 2021, is they have used the fact that the guidelines suggests pacing but doesn't overly define it.

APT (PACE style) Pacing = rehabilitation
CBT (ME specific) = Mindfulness
HR guidance as per usual.

Nothing has changed in respect of pacing activity in the national conversation. I think the new PROMS (including the severe patients referred for re-ablement in the community) is so that Crawley, White, and other service providers can use the National CFS/ME Outcomes Database to repeat their research papers of 2013 in the hopes of finally providing evidence of better outcome using the same techniques!

This is why I passed on the invitation from MEA to participate.

 

Correct.

ETA: not just Bristol. White did the same outcome research as Crawley (both published in 2013).

 

OK @Maat sorry that I'm basically repeating what you've laid out, but yes it does seem to be quite a clear path.


I can see where you have the feeling that White might be involved in the PROMs somewhere behind the scenes too.

I may have missed out some of the papers that you have mentioned. Picking it up from (I think) the ones post-PACE being released

We first have:

Then with Gladwell involved we have two biggies that look like (because of dates they published them) 2023, 2024 but were actually started before 2016 and 2018:

I can't help above but note the title of the paper 'PROMS to represent the functional limitation of people living with ME/CFS' vs the paper from 2013 with White and Crawley realising that GET/CBT wasn't effective for physical function. As measured by/defined as 11 point improvement in SF-36.

And then...

So THIS ONE then is looking at the other 'primary outcome' for the paper with White and Crawley in 2013: fatigue

And is suggesting that the PROM for that apparently also doesn't work.

I've linked to the thread on s4me above, because that one has just begun and I'm hoping there will be comments there on what detail is found on that.

But yes the more I look the more it seems these have come from / are linked to those prior papers and the conclusions.

 

It was/is @Maat it has just taken me this long to get with the programme on that post, by thinking out loud and doing a lot of pasting phrases next to each other from each paper (I'm a bit 'visual' like that apologies to those who got it first read for all the posts that might seem like repeats).

And I'm still needing to break it down into time chunks (@Maat has managed to square the chain from further back and forward)

 

That's the slight of hand they pull though @Trish They teach you about establishing your baseline, staying below your energy envelope, but in their activity homework you (in collaboration prompted by their suggestions and 'living to your values') actually increase your activity weekly! In PACE it was an arm of the trial on it's own, not with CBT, ignore your symptoms.

 

Here's a visual for you

The Code is More Like Guidelines (youtube.com)

 

I haven't fully broken it down either it's far too much reading, and am not sure I've found everything. But I think together we've all found enough to prove the 20 year repeat which seriously brings doubt on whether valid consent was obtained for the PROMS research - because if it results in further harm - it was not not

Impartiality of information
“Information about your options, and your treatment, is an important part of your care and you should be easy to understand, clear, accurate, impartial, balanced, evidence-based, and up-to-date.”

Handbook to the NHS Constitution for England - GOV.UK (www.gov.uk)

ETA: PROMS

 

Sort of yes.

It seems the MEA news article from 2011 shows that at that point things were rather 'on the edge' for GET/CBT with physios having to talk about 'toning it down' and 'the issue being other physios being gung-ho and not gentle enough' (with Bavinton and Gladwell being said physios who of course saw no harm because they were really careful in delivery) because Nice were talking about a consultation following MEA surveys etc.

Then @Maat reminded me that at the same time as PACE there was another trial/piece of research with White and Crawley that was published in 2013 which was based on clinic data. That research had primary outcomes of 'fatigue' (CFQ) and 'physical function' (SF-36). And basically said that CBT and GET weren't effective for 'physical function' in clinics, which was different to what was found in the PACE trial. So suggested 'urgent measures of delivery in clinics were needed' on the basis of that disparity.

Then I've realised that Gladwell's 2023 paper was actually kicked off before 2016, and that focuses on new PROMS to measure: physical function - the thing there was no effectiveness for from clinics in the 2013 paper

and Gladwell's 2024 paper was actually kicked off before 2018, and that focuses on the CFQ /Chalder Fatigue Questionnaire , talking about PROMS being needed to replace it.

So yes it very much smacks of PACE being a saving grace 'just in time' and when the 2013 (White and Crawley) showed different results in clinics to this Gladwell has been part of studies not long after focusing on their two measures 'being the issue'. And suggesting the need for 'measures of delivery'.

Except given they started pre-2016 and pre-2018 then published 2023 and 2024 one mightn't realise how 'pre-guideline' they are.

It could seem almost like they knew they couldn't rely on PACE quite a long time ago and needed to get some 'other measures' in eg PROMS

and by 'they' then yes as @Maat says it would seem to not just be Gladwell across the various of these PROMs papers , but clearly instructed to be done/needed by the 2013 paper by Crawley et al which has Peter D White as the third author:

"Although NHS services are moderately effective in improving fatigue in patients with chronic fatigue syndrome, they are much less effective in improving physical function than similar treatments delivered in the PACE trial. This requires urgent investigation to determine whether it is due to differences in the delivery or the content of treatments offered by services."

 

I think the purpose of the 2023 and 2024 papers was to update the earlier papers in respect of the use of patient data following on from the GDPR on data in
General Data Protection Regulation - Wikipedia#

Relevant also is the fact that Crawley's paper in 2013 looked at employment/benefit outcomes which showed no improvement. As PACE was DWP funded this was not a good position to be in.

Now how in the hell are newly diagnosed patients supposed to know all this! It's taken me 12 years to collect it all.

 

How are we supposed to push back on any of this?