UK:ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services, 2023

All quiet on the PROMs front?

 

A good day to bury news…
Hopefully they will be obsolete by the time they’re finished. I can dream!

 

Can anyone give me a super short rundown of what the project is about?

 

Thank you! That sounds like a complete farce. I understand why you hope they’ve dropped it!

 

I think it was initially sold as something that would develop better measurements 'of ME/CFS' and if I remember this wasn't always clear and consistent across different media in different times because at first it wasn't made clear it was about clinics and research and datasets, but then it was and by the end there was talk of it being possible to share it across other services from clinics etc.

at some point there was a hint of it would be happenning anyway, so better to have one developed with these people involved rather than done to us etc.

they released it in 3 parts, with many months between each, and the MEA advertising the link to the survey for each so quite a lot of people just filling it in assuming it was a good thing. But I don't think we ever really got good descriptions of what each of the upcoming next parts would be so that people filling in the first would know how 'all three' was somehow going to be measuring something altogether.

But PEM was a massively 'missed/ducked/not done' aspect. It kept being promised that 'a later part' was going to focus on it or cover it on its own to those being asked to fill in the early ones, and then when that part came it I don't think even called it PEM and wasn't necessarily capturing it.

And no - even though there were many months gap to the next part, there weren't I don't think any 'results' given for the previous parts so people could see how what they'd filled in translated into something else.

And even the first one was a black box because I think they were using psychometrics (which I assume means they were 'prospecting' by finding which answers coincided most with certain outcomes or something similar to work out which items should be on a questionnaire because they were 'predictive'), so were just producing really long questionnaires without anyone really getting explanation why certain questions/answers were on there (and I'd guess others weren't too) and how they were really going to be assessing them before being asked to fill it in.

Which to me is an informed consent issue.

And does that mean people have been led into completing first parts on the promise that it is a proper questionnaire that covers ME/CFS ie an illness with a cardinal symptom of PEM - and then once they can't remove their data from those former parts they get to see the later ones where PEM is supposedly the main thing being covered and it isn't what was expected?

There were also some interesting things explicitly to do with the consent - in that I don't know whether it was at part 2 or part 3 but people suddenly had a new different list of things to agree to than had been the case for the initial questionnaire (and if I remember correctly on some ways of accessing it this wasn't easy to get hold of to read etc?)

Apart from some fuzzy blurb, suggesting something which would be hard to check or verify due to the black box nature of that approach and also there are questions as to whether it would be the most justified method on its own anyway - before you start thinking about issues like ceiling and floor effects (can't improve or can't get worse because the scale limits it) and how important something is that is listed vs something that wasn't listed so couldn't have been ticked as a symptom.

There were also issues with ambiguity or the wording of questions. As there are with too many surveys on ME/CFS - so I guess that isn't a surprise.

 

One issue that was also flagged was the length of questionnaires (and wording) vs accessibility for those who were more severe, and a debate about whether long questionnaires are harmful slowly 'grew'.

https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-10#post-515932

this post, particularly later on, covers a bit of what they are going to be using it for: https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-10#post-515951

and this is Trish's letter with a really good overview of the issues by the second part which was the PASS (supposed to cover PEM I think): https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-11#post-516708

the next link you may or may not want to read, but was when Sarah decided she wouldn't be interacting any further: https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-12#post-517050

 

Yes the plan was to gather information which could be used as a large dataset on ME - something NICE 206 identified as a gap. Sounds good, right? Sounds like the sort of thing the ME association should fund.

Except - the way this dataset would be generated was by the project group of Sarah Tyson and 2 NHS employees being given a grant to create ME “PROMs” Patient ReportedOutcome Measures which is a metric the NHS uses.
So they were paying a grant to 2 NHS staff to invent an NHS patient outcome database which would magically help pwME by being a dataset. And Sarah, who is retired from University of Manchester due to ME. She has ME so is the expert on all things ME.

 

post where Trish goes over the initial article/pos

question about PROMS project and whether it is part of the delivery plan: https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-19#post-524606

useful point by Hutan that if they are using surveys done in clinics only seeing those least ill then then it is a circular issue if they are being used to inform future development of said care because it won't include those who they are not serving currently/the severe: https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-20#post-524861

MrMagoo then succinctly puts their finger on what the issue is of PROMS being designed to measure what they want to be delivering and measuring not what ME needs (in our eyes): https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-22#post-525330

and the posts following that I think are useful.


There is then quite a lot of posts sequentially from me on this one: https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-22#post-525535

which pick up on Maat pointing out an issue.

And me then realising that the idea of having a different measure (a PROMS) is something Gladwell had been selling in research he'd been doing almost straight after the following paper in 2013 which included Crawley and White, and which explicitly ordered as an outcome that the measures needed to change and remove SF-36 (because they couldn't get an effect from that) and change the chalder fatigue scale to PROMS (as CFQ only had a small effect)

Crawley et al (2013) was actually another arm of the PACE trial (just done in clinics and so reported just slightly later) - so it literally seems to go all the way back to that result being shaky [and maybe they knew it was temporary?] - and didn't find that there was any increase in the physical function measure (SF-36), unlike PACE.

Even though some of Gladwell's papers (which suggest PROMS or critique the Chalder Fatigue Scale) seem younger as if there was a break - you will note through going through these posts that he had a number of pieces of research that were started off and done and then spent a lot of years sitting around before they were then published.

 

Thank you all! I think I’ve got the gist of it now.

 

Mainly because I've been reminded of it by Hutan's post that was responded to in one of these. Which references a paper on MS that specifically compared the sensitivity of a PROM vs SF-36 in picking up on/measuring worsened disability: The MSIS-29 and SF-36 as outcomes in secondary progressive MS trials, 2022, Strijbis et al | Science for ME

and underlines that the PROM misses this

 

and because I've been looking back through the whole thread and was thinking it was worth mentioning the issue (or perhaps to bps 'finding' which seemed to become a tactic) that Heins et al (2013): The process of CBT for CFS: Which changes in perpetuating cognitions and behaviour are related to a reduction in fatigue?, 2013, Knoop et al. | Science for ME
showed where Fatigue increased with Objective Activity, but decreased with Perceived Activity in a cohort who had been under false-beliefs CBT. ie when their perceptions had been distorted into thinking they were doing more than they were.

So a measure that asks people to do a third questionnaire on their activity only by subjective retrospective measures after they may well have been attending a clinic focused on certain 'psych /re-education / embedded ideas' and that will perhaps be passing on this 'progress' to other parties and might be seen as an indicator of whether you have been compliant etc. too is an issue.

Particularly when, as the Heins et al (2013) paper seemed to do with its 1-4 (slow-fast and non responders to CBT), you might well be skewing your sample by those who attend a supply-led clinic only being those who find it helpful rather than harmful and only those in this group and are well enough etc to provide their feedback via these surveys, at these timepoints.

Apologies I couldn't get these quotes to work on multiquote so had to cut and paste without the reference to the post, but they are about 5 posts down: The process of CBT for CFS: Which changes in perpetuating cognitions and behaviour are related to a reduction in fatigue?, 2013, Knoop et al. | Science for ME