Evaluating the ability of patient reported outcome measures to represent the functional limitation of people living with ME/CFS, 2023, Jones, Gladwell

[Andy]

Aim
To identify activities which people with Myalgic Encephalomyelitis (ME)/Chronic Fatigue Syndrome (CFS) report are impacted by their condition, and evaluate the ability of measures of function used by National Health Service (NHS) ME/CFS Services to represent these experiences.

Method
122 participants completed ME/CFS Service questionnaires reporting activities that they had reduced or stopped doing, as well as Patient Reported Outcome Measures (PROMs). These data were coded using the International Classification of Functioning, Disability and Health (ICF) using established linking rules. Matrices identified the agreement rate between the outcome measures and the participant-generated list. Activities which could not be coded against the ICF were grouped using content analysis.

Results
Responses from participants related to codes from nine subsections of the ICF. The PROMs used by the ME/CFS service had agreement rates between 58% and 62.5% with the participant-generated list. The content analysis identified a range of activities that were meaningful to participants that they could no longer do. These included holidays and day trips, accessing the community independently, and sustaining activity into the evening. These were not captured in either the ICF or the service’s outcome measures.

Conclusion
The list generated by participants referred to a wide range of activities, including some not captured by the ICF. Comparison against the outcome measures suggests that the measures used in many NHS ME/CFS services nationally capture patients’ experiences moderately well. However, there are activities that patients value that are not captured by these measures.

Open access, https://www.tandfonline.com/doi/full/10.1080/21641846.2023.2175579

 

[bobbler]

This was a mixed-methods service evaluation project that used existing observational, cross-sectional data to meet the research objectives.

Participants were a cohort of patients attending their first appointment at a specialist ME/CFS service in an NHS Trust. This service provides assessment by a specialist clinician to diagnose ME/CFS where patients’ experiences met both the Fukuda and the 2007 NICE Guideline criteria [Citation18,Citation19], ensuring that post-exertional malaise was a required element of the diagnostic criteria.

I'm sorry but how dare they use such a poor patient cohort and claim it representative. Not only should they be seeking those who are severe and representative - not just those who might be able to attend a service - BUT this is an old service that was offering treatment that more than likely caused harm to anyone who DID have PEM so was likely 'inadvertently' filtering out people who actually had ME/CFS. Certainly who had anything less mild.

EDITED (to explain more specifically) To then use this cohort as if you can then ask them 'what activities are affected' is shocking because the 'firebreak' that the hopefully soon, change if/when it happens from the old style clinics means such data really isn't either representative or reliable prior to this. I can understand the wish to 'move on' and get data or research going, but without text that really truly acknowledges the extent of this lack of validity for anything more than testing concept/could this work in theory it flags all sorts of worries for me re: precedents for other research. I just do not think historical data from that time should be used without its interpretation being very limited.

Ethics approvals
Ethics approval to commence data processing was obtained from the Research & Development team at the relevant NHS Trust as a service evaluation project (ID: 3821) and ratified by Newman University (project number 2016-08-30-1404324/2903). It is important to note that this work was completed in 2016, prior to the introduction of General Data Protection Regulation (GDPR) legislation. This meant that it was possible to use the data collected for the purpose of clinical assessment for evaluation purposes with appropriate research approvals. However, because of the timing of the research, the raw data is no longer available to comply with GDPR legislation.

So much wrong with this. Made worse by insisting on including it despite really not being GDPR compliant and then using the GDPR legislation to make it non-transparent and non-checkable for fraud given the PACE debacle is I think something for ME/CFS should no longer be allowed.

I also think that given the ongoing abuse of ethics and use of data and lack of trustworthiness in abilities of those running gaslighting clincis, which they were (and many still are) by virtue of doing 'false beliefs CBT' and thereby their clinicians 'believing the patients were deluded [when they were]' and their clinicians 'seeing things that weren't there' THIS should also be banned. In fact such notes should be burned other than patients having copies required to be given to them in order that they have them for their own future needs.

And if it is Newman Universiy, which I think is Birmingham, and that indicates which clinic they perhaps chose then it would be a full-on psychiatric stance clinic which makes the whole thing even less accurate. Anyone by that point who had ME/CFS to any severity who was able to ask advice, or went along and sniffed a rat would run a mile (because of the potential for harm from old GET guidelines and it probably being more obviously it was being 'treated as if it was psychiatric' when in such a location).

Writing a paper suggesting those with ME/CFS are most affected on day trips and holidays - well they are but hey that is called switch-and-bait distracting the reader when some can't get to the loo and most will have serious payback if they shower [and sadly we've all seen such things being twisted and used that way by too many in the area of ME/CFS so that needs to be a validly accepted concern]. And the issue is the cohort you have used then the method by which you've listed in order of 'priority' - you would have thought would be a lot more careful about how they wrote it if they weren't looking to create misinformation and minimisation and basically have done everythign possible to mis-sell a condition by picking the greater reaches of extremes in cohort selection to almost certainly include mostly the misdiagnsed 'chronic fatigues' from clinics that didn't diagnose well or specifically as they were focused on thinking their harmful treatment had merit and worked, then focusing only on the things 'the most people were affected by'.

This level of 'risk' or 'potential' could have been gone into detail of - although I akncowedlge the difficulty of not wanting to 'get into politics' of having to say it, that is the truth and needs to be alongside it when talking about 'what/who is this data' probably so that it doesn't get used in such ways.

I'm astounded someone would publish this in 2023 without massive caveats noting all of these changes, and without a large discussion about how you can't just use old data under old regimes 'you've grabbed' - it just shows an utter misunderstanding of data and its context/reliability based on how it would have been collected at that time, their topic area and the condition to the point I have serious doubts such people could be involved in research or in creating or adapting treatment systems (ie unless under the management of others in order to teach them these things) because they cannot critically evaluate 'contexts'?

 

[MSEsperanza]

Any person that thinks writing a paper suggesting those with ME/CFS are most affecting on day trips and holidays - well they are but hey that is called switch-and-bait distracting the reader when some can't get to the loo and most will have serious payback if they shower -

Apologies, only read the abstract and thought the paper's main finding was that not all reductions in functioning relevant to people with ME/CFS are covered by the International Classification of Functioning, Disability and Health (ICF) [edit: and other scales/ questionnaires commonly used by clinics.]

Thus it's not sufficient to use the ICF, SF-36 etc. for assessing disability / limited functioning for pwME.

I think that's a useful finding?

Agree with your point that it's a limitation of the study that it didn't include more severely affected pwME, but assume that those things like going to the loo and taking a shower are covered by the ICF and the other already existing scales so no need to highlight them?

Edit: clarity

 

[bobbler]

Apologies, only read the abstract and thought the paper's main finding was that not all reductions in functioning relevant to people with ME/CFS are covered by the International Classification of Functioning, Disability and Health (ICF). Thus it's not sufficient to use the ICF for assessing disability / limited functioning for pwME.

I think that's a useful finding?

Agree with your point that it's a limitation of the study that it didn't include more severely affected pwME, but assume that those things like going to the loo and taking a shower are covered by the ICF so no need to highlight them?

Thanks that is an interesting point, and you've flagged that I've missed taking it from that angle - and one which might well have merit given that one of the things pwme can feel they fall foul of is because of the nature being 'energy limiting' they can lose everything and then still be prioritising the most essential so not have the huge impact understood in a way that if e.g. specific functions were 'off/on' ie just couldn't do x ever.

I've now read further into the discussion and noted the following which adds a different contextual angle (eg to the 'sustaining activity into the evening'):

Another commonly mentioned activity which could not be linked to an ICF code was sustaining activity into the evening. This was also not captured in the existing ME/CFS core set [Citation14]. Given the specific requirement of ‘sustainable activity’ to determine a person’s eligibility for the Personal Independence Payment criteria [Citation1] capturing the difficulties that this can cause is crucial for getting people the support they need. Coding activities around spending time with children was also difficult. Participants referred to activities such as childcare, however it was often unclear whether activities should be coded as parent–child relationships d7600, assisting others d660, or play d9200. Further research into the impact of ME/CFS on parenting would add clarity on how this is impacted.

So I feel a bit better as to the intentions behind this and how it might be used etc.

It is a tricky one to imagine what the best way to 'quantify' disability for ME - whether it is the % and disability scale that we might be more used to and I see that maybe this is trying to link a system set up for certain things with how does that 'envelope' crowbar in. I guess the 'showing what drops off first' e.g.evening or extra trips is part of ballparking what this means if you lose x% of function.

This provides an opportunity to benchmark the PROMs used by many ME/CFS services nationally to evaluate the extent to which they capture patients’ experiences of disability. As a precedent, this benchmarking has been done with other measures of participation used for a wide range of other health conditions [Citation13]. PROMs that accurately capture patients’ experiences of functional disability will also help to evaluate effective treatment strategies for the condition. Additionally, this knowledge can help services to make informed choices around service provision and treatment efficacy. Because many ME/CFS services use the same PROMs, the findings of this study have implications for wider clinical assessment across ME/CFS services in the U.K.

I think one thing that is throwing me is using the term 'treatment' or 'treatment strategies' for something that doesn't treat the condition but is actually at best Occupational Health/Occupational Therapy adjustments to allow someone to 'live with/survive with' 'better' (which sadly for ME just means that what they've been left with often, not a 'good/better life').

Is it common that for conditions e.g. RA or arthritis or CP or conditions where there may be 'treatments' but these won't always fix the illness/people will still have disabiliies, illness, limitations that these PROMs are used for the bit that is the 'adjustments part' and such bits are sometimes called 'treatments' also?

Am I needing to try and get my head around the term 'functional disability' and it meaning something like 'training or adjustments or tools or support that mean someone can function beyond said disability if they did have those e.g. like the wheelchair idea meaning they can wheel xm or can do a shower if they have adapted equipment vs not'?

Because I'm getting quite confused at what might be useful 'it matters to patients' measures if it were a new actual biomed treatment that might mean people can name how it makes a difference to them because now they can x which is what we'd each choose if we suddenly got that little bit more energy. eg that might be finally beign able to shower more for some.

Vs the difficult nature of how you could capture something that is more about a pot that only stretches so far, and only some activities are adjustable or reducable e.g. on the way down/no change you reduce your number of showers because that is the one that has most 'bang for buck' in enabling you to say cook and eat, but you really don't like being able to only do it say once a week.

 

[Kitty]

Am I needing to try and get my head around the term 'functional disability' and it meaning something like 'training or adjustments or tools or support that mean someone can function beyond said disability if they did not have those e.g. like the wheelchair idea meaning they can wheel xm or can do a shower if they have adapted equipment vs not'?

Usually the approach seems to be to start with the assumption that patients have access to the aids they normally use. It doesn't matter whether these are small gadgets or highly specialised equipment, it sets a baseline. It also gets around the fact that not all the aids people use are necessarily related to the condition that's being studied; someone could be a lifelong wheelchair user because of an impairment they were born with, but only become chronically ill in their 50s.

Vs the difficult nature of how you could capture something that is more about a pot that only stretches so far, and only some activities are adjustable or reducable e.g. on the way down/no change you reduce your number of showers because that is the one that has most 'bang for buck' in enabling you to say cook and eat, but you really don't like being able to only do it say once a week.

One way to get around this might be to agree something like a list of basic functions that reflect what each person actually wants and needs to do. Arguably, it doesn't matter whether one person's No 6 is doing a part time job and someone else's is sitting up in bed, what matters is how often and how reliably they can do it without having to sacrifice Nos 11 – 15 to enable it. If a treatment works, an approach like this would show whether there are improvements in capacity, and to what extent they are maintained over time (and as we know, "time" probably needs to mean at least two years).

 

[Trish]

I think we need to respond to this in relation specifically to the purpose of the research, ie to look at whether the standard questionnaires used by ME clinics to assess functional capacity and therefore to some extent illness severity and effectiveness of treatments, actually capture adequately the limitations of function experienced by pwME.

To do that they compared the areas of activity covered in the standard PROMS often used by clinics (SF-36 PF and WSA the work and social adjustment scale) with data from a more open ended questionnnaire about things patients missed being ablet to do. In order to compare the two sets of data the classified the activities internationally recognised coding system called ICF codes.

That all seems OK to me. They are trying to find out what information about functional limitations clinics are collecting about patients presumably in order to help them diagnose, advise and assess change for each patient, ie to use as part of getting a picture of a patient's difficulties that they may be able to help with, and to evaluate whether the service is helping patients.

What they found, using data from new patients attending a clinic was that while the SF-36 and WSA capture some information, they miss out on giving a full picture of activities people can't do, like going to work, or going on holiday, as well as missing out on the effect of fatiguability that means people may start the day being able to do stuff but run out of energy by evening.

The actual activities looked at in this study seem to only have relevance to the highest functioning pwME and they only touch the tip of the iceberg of fatiguability, ability to repeat activities and PEM, and the ongoing effects of always feeling ill even when doing activities. and the fluctuating nature of ME so that we can't tell from one day to the next whether we can, for example have a shower, and what its impact will be in the following days.

I hope the new study funded by the MEA will dig much deeper.

 

[Simbindi]

always feeling ill even when doing activities

Yes, I think this is one of the main aspects that a11 the functiona1 disabi1ity sca1es seem to miss. I've never had a minute over the 1ast 30 years where I haven't fe1t i11. But sometimes I am ab1e to 'push through' the sickness and achieve a functiona1 activity. That's very different to a hea1thy person's experience of comp1eting activities.

A1so there is the other aspect of whether you can do something in a 'reasonab1e time' (so for examp1e, not breaking cooking a mea1 up into parts that spread the 'microparts' over severa1 hours). Or being ab1e to do it 'safe1y' or without forgetting what you are in the midd1e of doing.

 

[Simbindi]

It probab1y is important in terms of eva1uating ME services to have a comp1ete understanding of the activities a person has had to curtai1 or stop doing at the start, during and end of their attendance period, because if you aren't seeing this (i.e. making it visab1e and exp1icit) then any 'improvements' seen on functiona1 sca1es can be masking the fact that what the service is doing is simp1y he1ping peop1e manage their condition better by changing their 'activity' priorities or spreading them over a 1onger period.

That is, these services are not actua11y offering any treatments for the under1ying condition, rather simp1y he1ping PWME manage day to day 1ife better (or not, as the case may be, given the time 1imited nature of these c1inics and the fact they on1y accept mi1d and/or new patients).

 

[JemPD]

Yes, I think this is one of the main aspects that a11 the functiona1 disabi1ity sca1es seem to miss. I've never had a minute over the 1ast 30 years where I haven't fe1t i11. But sometimes I am ab1e to 'push through' the sickness and achieve a functiona1 activity. That's very different to a hea1thy person's experience of comp1eting activities.

A1so there is the other aspect of whether you can do something in a 'reasonab1e time' (so for examp1e, not breaking cooking a mea1 up into parts that spread the 'microparts' over severa1 hours). Or being ab1e to do it 'safe1y' or without forgetting what you are in the midd1e of doing.

very, very good points

 

[Mithriel]

That is a very important point and articulates something I have struggled to put into words. So important we should make sure that everyone who works with people with ME understand it. It may be as important in defining ME as PEM.

Unlike other illnesses, ME lets you keep going until the damage is severe. The body has systems in place for when we need emergency energy though they are all meant to be short term and we rely on them. That is why we can do a thing once but not again or do any random two of the processes involved in getting up but can't manage another one.

When I was moderate and had children before I was diagnosed I felt I did everything other mothers did but it was all through a fog of exhaustion and pain. It is another thing where you talk to people with other illnesses and they say they get it too but you realise they are not talking about the same thing at all. ME is so hard to describe.

Even now, I like to get dressed in the mornings as it means I can go into the garden later, things like that. I lie in bed feeling like someone who knows they are going to have to do a hardy days work and are reluctant to start. I force myself because it feels worth the effort though I have made it as simple as possible. To say I get dressed every day does not describe my abilities.

Anyway, for ME it is not enough to list what someone does it is vital to note how they feel when they are doing it. Or they could ask what we find as easy to do now as before we were ill or compared to someone healthy.

Sure would be more accurate than the SF36

 

[JemPD]

That is a very important point and articulates something I have struggled to put into words. So important we should make sure that everyone who works with people with ME understand it. It may be as important in defining ME as PEM.

Unlike other illnesses, ME lets you keep going until the damage is severe. The body has systems in place for when we need emergency energy though they are all meant to be short term and we rely on them. ...

It is another thing where you talk to people with other illnesses and they say they get it too but you realise they are not talking about the same thing at all. ME is so hard to describe.

I force myself because it feels worth the effort though I have made it as simple as possible. To say I get dressed every day does not describe my abilities. ...

Anyway, for ME it is not enough to list what someone does it is vital to note how they feel when they are doing it. Or they could ask what we find as easy to do now as before we were ill or compared to someone healthy.
Sure would be more accurate than the SF36

(my bolding)
Sooo true. Particularly when we have PwME whose energy envelope/budget allows them to feel well enough to enjoy going on a 5 mile hike, & dont have many symptoms until they push too far & end up in PEM. It not much good if they are the people whose experience is being extrapolated for us all. I too havent had a single day where i didnt feel ill for the last 20yrs. And im not particularly severe when compared with others

Not tired or fatigued, but ill.