It would require testing each LLM is different and its not something I've tried but generally I've found summarization really good - but on technical topics - and I'm only using small language models (~3 to 8b params) that I can run on my laptop.
Patient led measure of outcomes
- Thread starter hotblack
- Start date
hotblack
Senior Member (Voting Rights)
Me too. But I’ve not got any experience with deeper analysis. Although know its used in quite a few areas.
Definitely interesting and could be good when there is lots of data from different sources to analyse too. It would be a lot more work to administer than what I was thinking, which was going for absolute minimum data and effort by everyone involved. Two different but potentially very valid approaches though.
Kitty
Senior Member (Voting Rights)
I'd say the usefulness of personalised descriptors is that they're things you actually do, not things you have to imagine whether or not you could do.
Things you're motivated to do, either because they're a bodily need, or you do them every day because they've become a habit or you really enjoy them. Or things that need doing, and if you don't do them they either get added to the load of someone else in the household, or the omission impacts your dignity or quality of life (e.g. not changing bedding as often you want to).
So you start with things the person does every day unless something unusual happens. Every day is important, because it's an indication of their base level of functioning. How many days this week did they do them?
Then a couple of things they'd only expect to be able to do once a week or so because they trigger PEM. How often did they do them?
Then things they'd really like to be able to do, but can't without making themselves really unwell. Perhaps a couple of pre-specified things, and one option left blank for something they hadn't thought of. Have they done any of them?
They should all be the things of ordinary life: eating, toileting, catching up with online news/social media, listening to music, washing hair, dealing with laundry. The ambitions need to be ordinary as well, e.g. staying out of bed most of the day, meeting up with someone outside the house, doing a trip away from home.
It might be challenging to explain all this, but it's important to get away from the theoretical ability to do things as much as possible. If trial participants really want a treatment to be working, they're more likely to overestimate their capacity to do theoretical activities. The focus needs to be on things they actually did and how often they did them, because that will show whether they had more, less, or roughly the same capacity.
Last edited:
I'm thinking with AI lots of tasks which could have proved difficult before could be doable now and automatable (although some work to automate). But if we are thinking to future trials we should have in mind these new capabilities in a design and it could lead to new ways of working.
Five seems like quite a lot - would three be good enough?
I think it should just be 'did you do them?' That cuts out the interpretation. I think any study still needs to be long enough to pick up deterioration due to overexertion. So, that covers part of the negative knock ons. I think the measure needs to be completed daily - that way there is an aspect of frequency, so that also covers part of the negative knock ons.
I like the idea of picking activities from Funcap.
Jonathan Edwards
Senior Member (Voting Rights)
Three is probably enough.
I assume this idea of each individual picking 3 activiies as an outcome measure would be for testing a treatment in a trial over a relatively short term, say up to a year. I guess it would be OK for that.
I think for longitiudinal studies of fluctuations of the illness over time and in relation to acitivity levels it would be better to use the whole 27 item version of FUNCAP.
I think for longitiudinal studies of fluctuations of the illness over time and in relation to acitivity levels it would be better to use the whole 27 item version of FUNCAP.
hotblack
Senior Member (Voting Rights)
Lots of really useful and insightful feedback. Thank you everyone.
So maybe something like…
- Pick 3 activity descriptors that you feel best describe your current limitations and level of activity, you can choose from FUNCAP55 or write your own.
Try to pick a range which represents you best, with one you can usually do without much difficulty one less often and one you’re rarely able to do
- Each day (or week) record how many times you do each activity (without significant difficulty?)
- Each day (or week) record if you consider it a good, average or bad day/week for you
In this way you will get both weekly and monthly totals of ‘activities’ and ‘good/average/bad’ periods
You can use a spreadsheet, a piece of paper, or whatever method works for you (I/we could provide some ideas and templates to use, copy or print)
So maybe something like…
- Pick 3 activity descriptors that you feel best describe your current limitations and level of activity, you can choose from FUNCAP55 or write your own.
Try to pick a range which represents you best, with one you can usually do without much difficulty one less often and one you’re rarely able to do
- Each day (or week) record how many times you do each activity (without significant difficulty?)
- Each day (or week) record if you consider it a good, average or bad day/week for you
In this way you will get both weekly and monthly totals of ‘activities’ and ‘good/average/bad’ periods
You can use a spreadsheet, a piece of paper, or whatever method works for you (I/we could provide some ideas and templates to use, copy or print)
Last edited:
hotblack
Senior Member (Voting Rights)
Yes, very much that. Is there a statistically significant benefit to patients of the intervention.
All the other complexities in these questionnaires seem to try to get fine grained or subtle detail or discern if changes are present in different areas or in different groups of people occur or compare us with other conditions.
There may of course be objective or clinical measures which can be done independently, but the idea is a lightweight patient led way of answering ‘does this help me?’
Utsikt
Senior Member (Voting Rights)
How would you get around the issue of redistributing activities in order to do the things you measure more often? Or the opposite: not being able to do the things you measure because life happened and you had to use your energy on something else?
Maybe combining it with some kind of overall measurement of activity to see if you do more or less overall?
hotblack
Senior Member (Voting Rights)
I think that gets too complicated and difficult to measure. That’s one problem with these questionnaires, complexity as a result of trying to design for every possible ‘what if’. How would you measure activity? Most methods seem equally subjective and open to issues. Open to suggestions though!
I’m not sure. Maybe we can’t account for every eventuality? Or maybe those things mentioned all balance out over time or enough payients? If a treatment/intervention has a meaningful, significant difference would it be picked up across a decent sized cohort and outweigh these effects?
Maybe we need to try it and find out.
hotblack
Senior Member (Voting Rights)
I’m planning on making a template and giving this a go for a few weeks to see what works and what doesn’t. This probably won’t be for a month or so, but I’ll share any templates I have then so others who want to can try themselves and also give feedback on what works or doesn’t for them.
If anyone has any more thoughts or feedback beforehand they’d be great to hear!
If anyone has any more thoughts or feedback beforehand they’d be great to hear!
MrMagoo
Senior Member (Voting Rights)
Could people just wear a wearable and the data be fed into a bot/python script?
I’m just thinking about lessening the burden whilst maximising the data.
Personally I’ve been struggling lately with the difference between being active but in pain and forcing myself, vs not being active but in pain. Because both ways I’m in pain. And my mood can be low or high in either.
I’m just thinking about lessening the burden whilst maximising the data.
Personally I’ve been struggling lately with the difference between being active but in pain and forcing myself, vs not being active but in pain. Because both ways I’m in pain. And my mood can be low or high in either.
Sasha
Senior Member (Voting Rights)
There doesn't seem to be anyway to access that paper without paying. The only otherwise available info about the personalised outcome measures is in the abstract:
Paper said:
Is there any way for us to see a bit more about the development and use of the personalised outcomes?
Sasha
Senior Member (Voting Rights)
I'd be happy to clean that up if you'd be willing to send it to me. I'm very curious to see it!
Sasha
Senior Member (Voting Rights)
Why do you think it would only be suitable as a short-term approach?
Utsikt
Senior Member (Voting Rights)
You can download the pdf from here:
A double blind controlled trial of methylprednisolone infusions in systemic lupus erythematosus using individualised outcome assessment - PMC
Twenty one patients with severe systemic lupus erythematosus (SLE) were treated with three daily infusions of either 100 mg or 1 g of methylprednisolone on a randomised double blind basis. Nine patients with unsatisfactory outcome subsequently ...
pmc.ncbi.nlm.nih.gov
JemPD
Senior Member (Voting Rights)
Sorry too ill to read whole thread but just wanted to say that these types of things are problematic because many of us don’t have lives that allow for doing the same every wk month etc.
I always find funcap so etc almost impossible to fill in, because the answer to whether I can bathe or not depends on whether I done the other things.
So for example the answer to “can you do x without crashing?” For many activities is “I don’t know how many times have I had ton get up to go to the loo today”, or “ I don’t know did I havesomeone bring me a meal or did I have to grab snacks from kitchen on way back from bathroom”
These types of scales work well for tracking improvements deterioration when lives allow routine. Many of us don’t have that. Unfortunately.
As an example, I said to carer last wk, I appear to be deteriorating but I hope it’s just that I had to do more unavoidable stuff like medical appointments etc the last couple yrs. plus amount of care support has fluctuated hugely & outside of my control. Leading to lots of crashes unavoidable rolling PEM.
It may be that candidates participants who have less predictable & regular/routine, those who are forced into push crash frequently would not be good candidates for a trial without objective measures.
Sorry don’t know the answer to the problem but just throwing it into the pot
I always find funcap so etc almost impossible to fill in, because the answer to whether I can bathe or not depends on whether I done the other things.
So for example the answer to “can you do x without crashing?” For many activities is “I don’t know how many times have I had ton get up to go to the loo today”, or “ I don’t know did I havesomeone bring me a meal or did I have to grab snacks from kitchen on way back from bathroom”
These types of scales work well for tracking improvements deterioration when lives allow routine. Many of us don’t have that. Unfortunately.
As an example, I said to carer last wk, I appear to be deteriorating but I hope it’s just that I had to do more unavoidable stuff like medical appointments etc the last couple yrs. plus amount of care support has fluctuated hugely & outside of my control. Leading to lots of crashes unavoidable rolling PEM.
It may be that candidates participants who have less predictable & regular/routine, those who are forced into push crash frequently would not be good candidates for a trial without objective measures.
Sorry don’t know the answer to the problem but just throwing it into the pot
I'm trying to have a think on actual examples of things I have tried. Exempting lots of 'no idea really if they do anything' stuff. I'm a cynical so and so and don't try things lightly too so this is a curated bunch.
B12 injections work for me. But I couldn't claim in more than certain symptoms ie not curing the ME/CFS or whatnot, although unlike nonsense like sleep hygiene or other therapy-based where they are getting you to just practice the task to be tested at the expense of everything else or something caffeine-like my kick me in the bum at another point I think there might be something it is fixing in me.
That is easily 'tested' if people believe me. Because it wears off really clearly at a certain point. And at a point say a week or two after that it's even more clear. It affects things like me being able to look at screens particularly TV etc. along with symptoms that are more sensations that would accumulate over longer times. The effect isn't always exact of how much or how great I feel each time I have one (I suspect I've 'used up' whatever before or have more to clear up whatever it does) but I'd know within x weeks if I'd been given a placebo.
That doesn't mean they work for everyone else either in the same symptoms or to the same extent. But what proof or test would I pick to find a group like me that others would 'OK' other than 'I can pick with constant accuracy when I've been on the placebo'?
I know ibuprofen works for certain injuries I have because if I leave it too long between doses then it is utterly stiff. Maybe that is measurable, certainly again I'm sure I'd notice a placebo.
But the question posed elsewhere about whether it helps PEM or ME/CFS I have no idea on even though theoretically I've done an experiment over years before and years after, but as we all know it is so situational re: how much PEM we have and remembering how bad something felt even last time vs this time is hard (points of reference, get used to pain, do other things to help, Noise + pain is worse) nevermind the timespans I'd have to be covering. And that's before I factor in adjustments I might have made and of course over this time my illness getting worse (but meaning I can do less etc).
I've had different doses of steroids which is really hard to report back on. Because it wasn't for ME, but would affect that and other things that add to my burden too. I know how much better I felt when I took a dose once. Obviously better. I also know how clear it was I needed something when coming off that dose, and then doing it even more slowly and it being clear there was an absolute minimum needed there. But the 'right' one is harder and more long-term.
When I had doses that might have been a level you could say were putting other things aside and just looking at some left-over for the ME/CFS I felt great. I'm not sure how much more I actually did, I still got myself stuck in silly situations flat on my back in awkward places but I also pulled off the little windows aping seeming normal a lot more convincingly and enjoyed that. I'm not sure I could have measured and compared how much I payed for it vs before. I probably felt stronger and had better posture when I was at my very good days. But also paid for them requisitely. I was just relieved to get that holiday of an hour here or there of 'less bad' or 'feeling normal doing x' (which any other person wouldn't think twice on like sitting in garden).
I think if you'd asked me a few months in I'd have said it was great, but 6months in perhaps it wasn't the right dose. I had a more debilitated number of months and had a virus of some sort which whilst I was flat on my back didn't feel as scarily ill as I would have done, but it was clear how fragile I was if I just chatted or scraped to the loo and had to sit and lean against a wall at certain points feeling white.
But again a lot of other things that were also significant changed at that point too. Including changes from major noise issues which nothing measures yet and dominated my health for years turning rest into exertion.
Last edited:
I think the approach of each patient picking 3 activities they currently can't do and want to be able to do may be OK for a trial of a treatment, with the follow up running for, say, a year, might be OK to give a clear indication of whether the treatment has had a clinically significant effect. Also for pwME, effects of activities are cumulative, and PEM is delayed, so that needs to be factored in.
I think for something like a long term study of pwME's fluctuations over time that would not be useful. It doesn't take into account worsening, and, depending on the activities picked, may turn out to be to limited, with further improvements or worsening not registering. Something like FUNCAP which covers a wider range of activities across all severity level would be better for that.