Patient led measure of outcomes

[bobbler]

Thinking about how we can measure if treatments work. A lot of the questionnaires and scales seem to try to compare across subjects, which is really difficult for many of us, they don’t fit our experiences or severity. So people try to capture a range of experiences and we end up with huge long questionnaires. So how do we measure outcomes?

How about this
- Pick and describe in your own words 5 activities that you feel define your current limitations, include 2 things things you can only occasionally do
Examples: get to the toilet in the morning, sit up comfortably throughout the day, have a 5 minute conversation with someone , have a shower, walk to the car (I don’t know I haven’t done those last two for years but you get the idea, the usual sort of things we see on questionnaires, but defined by the patient)
- Count how many days you can do these per month, before and after interventions, without negative impact, record weekly
- Maybe add a measure of how many days are ‘good’ ‘average’ or ‘bad’ for you, record daily

This would be person specific but capture the changes which are relevant and/or important to them and how their ME/CFS affects them. It would be quicker than most things to record but I think would allow measurement of if an intervention has actually worked.

Probably needs some refinements but…thoughts?

I've just re-read this. And have picked up on the bit in bold in this:

Count how many days you can do these per month, before and after interventions, without negative impact, record weekly

And I think that is an issue. Because almost everything has a negative impact, I just have to make some wicked automatic mental calculation of pros vs cons that sort of combines in with everything else to become eg how much I can brush teeth, shower etc. almost certainly the same with time lying on back or not vs boredom.

Is the negative impact bit important?


I think I'd really struggle with assigning how many days were good, ave or bad or labelling most days that aren't terrible as something.

 

[NelliePledge]

yeah I agree there are 2 aspects 1- managing to do the activity at all, and 2-=the consequences of doing the activity.
But I think the frequency is a rough proxy for the consequences because if they were less I would go out more often.

I think without negative impact doesn’t quite work because when I go out there’s always negative impact just less or more.

 

[Kitty]

Is the negative impact bit important?

Yes it's crucial, but if we do a good enough job of coming up with revealing activity categories, and recording them objectively—i.e., things we actually did, not thought we could probably do—it should show up in long term patterns.

It won't show that we were feeling really crap when we did something, but that crapness will influence what we chose to do in the following days, the following week.

 

[hotblack]

I wonder if the people at visible would be willing to engage with this @hotblack might be worth asking

That’s an interesting idea

Is the negative impact bit important?

Good question and I understand the mental gymnastics and difficulty of capturing this. I’m not sure what the best way is, hence proposing some ideas and seeing if they stand up or can be improved.

I think I phrased it better with ‘significant negative impact’ but maybe ‘more negative impact than usual’ could work? And of course this is all subjective, so what these phrases mean is up to an individual.

My thinking was the difference between something like getting to the loo when you manage it and when it is like hell (see the examples I gave here). So in this way we could capture when an activity you regularly do or have to do becomes harder. Because that could be important and people often say capturing it is.

Maybe you’re right and it’s not important. And just are you doing the thing is what we want to capture. It will depend upon what activities people choose I guess?

Kitty puts this bit well

It won't show that we were feeling really crap when we did something, but that crapness will influence what we chose to do in the following days, the following week.

 

[Suffolkres]

'A simple tool like the disability scale we use being confirmed so if someone is more severe it’s calibratable to eg other serious illnesses..
In 2018, we put some suggestions leaflets etc for our Suffolk GP AGM and exhibition.

An A4 symptom scoring chart was part of this and has proved useful for us at home, to monitor progress, fluctuations and symptom shift over time.

Our banner was taken to RCGP AGM in Glasgow by Nigel Speight for lecture before 250 zGPs, on behalf of 25% Group Tymes Trust and Suffolk Youth & Parent Support Group.

Used by NI Groups, Joan and Tom modified, and Helen in Scotland.

We did it as a pack to raise awareness and streamline process of;

GP as first post of call, referral and diagnosis, severely and symptom Identification and monitoring over time.

It was endorsed and authorised by the CCG to assist with commissioning and direction of future service fulfillment and patient expectations plus Joint Norfolk and Suffolk Health Overview and Scrutiny recommendations.

It was the basis for our journey to where we are now, or will be on 1st August hopefully.

Bearing in mind it was pre NICE NG206.
But we never had that CBT GET problem in Suffolk who embraced CIC .....

We had Jo Edwards give material the once over too!
I will make the pack material available.

It might be worth resurrecting some of it?

 

[Kitty]

An A4 symptom scoring chart was part of this and has proved useful for us at home, to monitor progress, fluctuations and symptom shift over time.

I agree it would be useful, and if the symptoms monitored are tailored to the individual it would make record-keeping quite nimble. I wonder if it could be got down to a list of three or four? Not necessarily the most severe or troublesome symptoms, but the most revealing of how you're functioning.

It could fit in with the suggestions I've been making for monitoring of the effects of a treatment on activity. The idea is to come up with as short a list as possible of ordinary daily activities, which will show trends over time rather than day-to-day change.

It might go something like this:

• Three things you do at least once a week, even in a crash. (Regular ticks indicate no change, no ticks indicate substantial worsening)
  
  
• Two things you usually do at least once a week, but can't do in a crash. (No ticks indicate worsening)
  
  
• Two things you need to avoid doing more than once a month. (Frequent ticks indicate some improvement)
  
  
• Two things you can't even attempt but would love to be able to do. (Any tick indicates substantial improvement)

The person just checks boxes to say how many of them they did this week. The less able they are, the less box ticking they have to do. Ideally, monitoring starts six months or more before the trial.

There might be something in it. It hasn't been explored much in ME/CFS, partly because the BPS theory assumes people avoid doing things they could manage if they tried. This assumes the opposite, that people do things even though they know they'll feel awful afterwards. That they'll squeeze the pips out of even the smallest gain.

It'd probably only work for a treatment that has a significant effect or none at all. The trends might be too faint to measure marginal changes, especially as they'll be confounded by factors like weather, cold viruses, etc. It'd need to be supported by a step counter, a heart rate monitor, and a simple symptom score.

 

[hotblack]

The person just checks boxes to say how many of them they did this week. The less able they are, the less box ticking they have to do. Ideally, monitoring starts six months or more before the trial.

Great ideas! I can picture how this works, more of a weekly tally?

 

[Kitty]

I can picture how this works, more of a weekly tally?

I think so. The challenge is in picking the right activities to demonstrate your "normal", but recording whether or not you did them each week should be as easy and take as little time as possible.

 

[hotblack]

I think so. The challenge is in picking the right activities to demonstrate your "normal", but recording whether or not you did them each week should be as easy and take as little time as possible.

I guess we may not even need the number of times you did them if picking the right activities and having clearly delineated groups as you outlined in your example? It may lose some resolution/detail but could be easier to fill in and still capture any changes.

 

[Kitty]

I guess we may not even need the number of times you did them

That's what I'm trying to get away from.

When you're asking whether a treatment offers meaningful improvement, it doesn't really matter how many times a week participants can do small, ordinary things.

It's whether they've moved up a whole level, and whether they can sustain it.

Whether they eventually go up two levels as their stamina improves, and whether they can sustain that.

Or whether they go down a level, of course. That's likely to happen even in a trial with a positive result, either because people really want to be responders but eventually realise they're not, or because they do too much too soon.

 

[Trish]

When you're asking whether a treatment offers meaningful improvement, it doesn't really matter how many times a week participants can do small, ordinary things.

I'm not sure about that. It makes a big difference to me whether I can shower daily or only once a week, for example. And whether I can prepare a meal from scratch with fresh ingredients every day, or about once a month.

 

[Kitty]

I'm not sure about that. It makes a big difference to me whether I can shower daily or only once a week, for example.

Yes, of course it matters to each individual. But while it would be brilliant to find a drug that makes the difference between one and three showers a week, or being propped up in bed several hours a day instead of just one, it isn't really what we're looking for. If you're still having to spend most of the time lying down it hasn't modified the underlying process.

I'd like to anticipate trialling drugs that do act on it. If they work at all they should block, break or interrupt whatever doom loop our immune systems are stuck in. It's difficult to envisage the long term result being partial gains; presumably the faulty process is either running or not, and if it's not the handbrake should be released. (I've experienced that twice, and it's extraordinary.)

To make the case for further trials or treatment being rolled out, the research team would need to be able to show that in a convincing way. Analysing what people actually did* seems much more satisfactory than asking how they felt (it's a proxy for that anyway), and it would be backed up by data from wearables.

ETA: * I should have said asking what people did in a manageable, accessible way that they'll be willing to do for a long time!

 

[Trish]

I think this illustrates the problem in ME/CFS of the cumulative effect of activities. Being able to shower or cook every day for someone who can currently only do them once a week or a month, if it implies all other daily activities are also able to be increased by that amount too, indicates a very significant overall improvement.

 

[bobbler]

Yes, of course it matters to each individual. But while it would be brilliant to find a drug that makes the difference between one and three showers a week, or being propped up in bed several hours a day instead of just one, it isn't really what we're looking for. If you're still having to spend most of the time lying down it hasn't modified the underlying process.

I'd like to anticipate trialling drugs that do act on it. If they work at all they should block, break or interrupt whatever doom loop our immune systems are stuck in. It's difficult to envisage the long term result being partial gains; presumably the faulty process is either running or not, and if it's not the handbrake should be released. (I've experienced that twice, and it's extraordinary.)

To make the case for further trials or treatment being rolled out, the research team would need to be able to show that in a convincing way. Analysing what people actually did* seems much more satisfactory than asking how they felt (it's a proxy for that anyway), and it would be backed up by data from wearables.

ETA: * I should have said asking what people did in a manageable, accessible way that they'll be willing to do for a long time!

This is where we start to diverge a bit - I really 'got' the last comment on the idea of having a set of criteria that represented a 'stepping up'

I wonder if part of the difference might be the caveat I have in my mind that the really important bit is that anything is followed up long-term and those long-term measures become the primary outcome over eg something 6months or less (because it is the excitement of doing things too much pre- this timing with something that stimulates x ability but doesn't have behind it the 'delivery of y' that makes it not damaging that would be something which one could either term a 'false flag' or actually might turn out to be something important short-term for people in desperate situations, depending on what it actually does).

If for example we are really looking at the 12month, 18month, 24month and focusing on objective activity, but including measures such as time horizontal and the tasks are indeed things like showering then I'm more OK with this

But a real downside of the history of those we get surrounded by staff-wise with ME is coercion (even including of the flavour hidden as 'motivating' and getting us to self-gaslight). I'm not sure because of this deliberate, and sometiems inadvertent by those deluded themselves that they are helping whether 'how we feel' actually compensates for that anyway in the short-medium because I know how thinking the pain=gain stuff can affect what goes on in our heads nevermind on a form etc. So agree with envisaging coming up with something that moves away from this being a necessary balancer.

PLus of course from my experience with things that genuinely made me feel better short term there was the deterioration which I'm convinced is part of what we might need to call a 'cycle of PEM' or something better descriptive to show that PEM isn't just some short term thing, but the accumulation or thinking we are managing to cheat it by riding the edges of it can mean we then take a massive dip in function say 6months in. So it is an illness of these nasty after-effects.

And we might have genuinely 'felt good'. And even done more. BUt perhaps the fact we couldn't also requisitely up the rest to go with that (because there are only 24hrs in a day and if 1hr activity needs 23hrs rest and we start feeling good doing 2hrs, is it 36hrs res or 24hrs rest we needed but the sums didn't add up over that period) caught up with us. To me I think that would probably mostly show in our function and activity rather than 'feel' still, because having a setback after having a 'holiday' comes with all sorts of complicated 'feelings' (does this term include how our body parts 'feel' without the mind getting involved is an interesting question having had different doses of steroids poses to me, because many things can change pain and feeling of strength? which can lull us into the behaviour that only causes the deterioration much later on eg many months).

But all of this absolutely hinges on this caveat of the longer-term, and that the pattern shows consistency (hence why it being measured up to 3yrs starts to make sense because if we get a virus at 2yrs that knocks us back then there is time to say we were good at 12, 18, 30, 36months)

I also think that we need to be aware that something could be 'part of the solution' and genuinely working on ME/CFS. But only part. And whilst we assume that no damage is done that will persist for all, it might. So we need to barter with oureslves a bit I guess at what the level of improvement is that says 'wouldn't want this taken away' vs 'could be tricked' and of course that level needs to be clearly labelled so that we don't end up with people again getting away with suggesting something is a cure and we should be fixed and 'acting normally' when it merely gives us back a massive chunk of quality of life eg being able to shower once a week vs once a month, sitting up for 3hrs vs 1hr is huge.

 

[bobbler]

I think this illustrates the problem in ME/CFS of the cumulative effect of activities. Being able to shower or cook every day for someone who can currently only do them once a week or a month, if it implies all other daily activities are also able to be increased by that amount too, indicates a very significant overall improvement.

Indeed.

I think that if we are measuring at 12, 18, 24, 30, 36months as the ones that count as something is providing something 'sustainable' ie acts on what ever the underlying thing that actually makes 'doing more' supported so it happens without longer term damage, rather than 'possible' but then becomes added up to deterioration in the longer term...

then 10% is enough. To be life-changing enough to be warranted. Particularly if year on year sort of builds on that as it has tipped the threshold making us less vulnerable to the things we can't control like noise and obligations like forms etc. being in themselves already putting us over that.

As long as the amount of variables chosen is appropriate for the level of severity.

3 seems to work for me thinking about being solidly into the severe range and most of my time is in bed.

If I think of showering and going from a struggle of getting it done within 2 weeks to weekly then that is just huge on the grime and effects side of thing health wise.

And then toothbrushing is important too. And noone wants to leave their teeth unbrushed so it comes massively high up the priorities.

Adding in if there were equipment that genuinely measured well the time horizontal and how horizontal (because lying on my back and bending my knees so my legs aren't flat makes a difference, just like putting my legs up on the sofa does if my body isn't flat).

These work because I'm not going to add in something that involves seeing people or leaving house when I haven't brushed my teeth or showered in 10 days in order to do it. But when I was moderate and working or could do the odd social thing then the difference between showering each day vs every other day becomes about planning and a different play-off.

I wouldn't use the toilet one or steps simply because PEM and other things can mean I need to wee more when I'm actually iller. And same issue re: I'm in more pain, move more in bed etc.

I do worry that something could be really life-transforming (in our small way, of being less undermined by reliance on others for stuff that we have to get done like eg booking appt, prescriptions, doing appts, forms, communications with others) by meaning even if I'm stuck as I am I don't get as exhausted from conversations or thinking or doing admin that as we all know we get buried in. And these items wouldn't pick up on that unless the medication also affected the aspects that would impact these activities.

I would care less about things tackling 'different symptoms' if it was properly making the underlying issue better (like B12 does for me), and it takes a few different things that tackle different upstream, downstream and head-on bits and all give clues to that. But it is just that this is used as an excuse to offer damaging treatments based on how medicine claims it can tackle said symptoms in otherwise healthy people or those with completely different illnesses so do the opposite of what is needed. Like sleep hygiene might make people sleep only at the right time, but it doesn't mean they get more good sleep and might mean they have no sleep but those aren't being measured. Along with all the rest of the collapsing body and life caused by those ideas. And is based on a bigotry some have about those who don't sleep at normal times only basically I assume. So someone is a gibbering wreck who can't function and whose body is eating itself in no real rest function but they think it's better than the crime of sleeping 20hrs a day or at the wrong times and at least getting a few hours where they can have something each day.

So I'd be keen to make sure that the measures are broad enough that whatever 'specific' something is claiming to do isn't actually at the expense of the overall health. An example being if some treatment meant someone who had been stuck in the dark or unable to look at screens could suddenly watch tv or internet 12hrs a day and their 'key items' didn't drop then it isn't 'robbing peter to pay paul' training from an OT chasing a KPI.

 

[Kitty]

This is where we start to diverge a bit - I really 'got' the last comment on the idea of having a set of criteria that represented a 'stepping up'

I wonder if part of the difference might be the caveat I have in my mind that the really important bit is that anything is followed up long-term and those long-term measures become the primary outcome

Yes. Participants would need to be followed up for at least two years because we need to show as clearly as we can that the level they reach after treatment is a sustainable gain in function, not just a peak on the graph driven by adrenaline and blind optimism.

Also, the potential for healthcare professionals to try staging "interventions" is part of the reason the measures need to be ordinary activities. Nobody's likely to focus rehab efforts on getting people to listen to music or mash the tea themselves so it's just the way they like it, yet those things might be on the wish list of someone who can't currently do them. They're also things they likely would do every week if they improved, whereas formalised rehab activity (e.g. riding a stationary bike) isn't necessarily.

We could also think about the possibility of follow up for an additional three years if a good proportion of a trial cohort saw significant improvement. Maybe not weekly; four times a year might be enough to show whether they've maintained the gains. That's where a hospital clinic* might take over, gathering data at regular review and feeding it into the trial.

* I mean a real clinic, with consultants, residents, nurse specialists and properly arsey receptionists, not lavender-clad therapists doing an amateur version in front of stage flats painted to look like walls.

 

[poetinsf]

So if you were reading about a drug trial of PwME, and you read that 100% had 'meaningful improvement', would it really be that informative?

Such statements would be meaningful only if the meaningful improvement means objectively measurable improvement, like the average number of steps taken daily or TSLD. The problem with funcap and such is you don't know what 20% improvement in the score means. Depending on how the questions are structured, it could mean being able to take a few more steps or twice as many steps. With 20% more steps, on the other hand, you know what you could do. All other functionings like showering are likely to improve with the improved walking ability as well.

I think walking is a singularly important activity for most of the ME/CFS patients. The inability to walk without resulting in or fearing PEM is really what makes ME/CFS a prison. For an outcome measure of trials, the average number of steps is also easy to measure, though I'd prefer TSLD for a daily measure because the number of steps can vary widely independent of fatigue from day to day.

 

[Peter T]

I think this illustrates the problem in ME/CFS of the cumulative effect of activities. Being able to shower or cook every day for someone who can currently only do them once a week or a month, if it implies all other daily activities are also able to be increased by that amount too, indicates a very significant overall improvement.

I am concerned that by focusing on a small number of individual activities we introduce bias.

I suspect it is a common response in individuals undertaking GET in a clinic setting, to reduce overall activity in order to focus on the specific clinician endorsed activities that they believe will make them better. Further those around them are likely to rise to the occasion supporting the patient through this ‘important’ process. This can create an illusion of improvement even if the patient in reality has reduced their total activity level, which only becomes apparent once the active intervention has ended and ‘normal life’ reasserts itself.

Obviously daily living activities have a hierarchy of importance, with eating, toileting and personal hygiene perhaps topping the list. When at my worst there was perhaps less conscious choices around planning activities; going to the toilet came first, then eating, even if it was just eating cold baked beans straight from the tin, but then adding in a shower was perhaps only achievable once every four to six weeks, with eating maybe falling off the list that day. However once you move off just simplest activities necessary to sustain life, the number of daily activity choices required increases exponentially. Just eating and preparing food can vary across a significant range of activity levels.

Once we decide that say three specific activities are the best marker of what we can do, it gives those activities an importance over other activities. Brushing my teeth is a good example for me. I have periods of focusing on dental hygiene which I can maybe sustain for several weeks, but that means other things disappear, perhaps the quality of my diet might deteriorate.

Having had a number of years of getting almost identical online grocery orders, I can now manage to go with my carer to a supermarket, but the time/energy that takes up means any cleaning drops out of my and my carer’s schedule.

I am not saying that monitoring specific activities is not a useful tool, but which activities selected is highly significant as the act of selecting them will give them an increased importance. Further I think this would only work in conjunction with a more global measure of activity, such as the amount of time spent lying down.

 

[Trish]

What is TSLD?

 

[Chandelier]

What is TSLD?

Time spent laying down