Thesis System and methods to determine ME/CFS & Long Covid disease severity using wearable sensor & survey data, 2023, Sun

I note that this is a thesis done as part of a Bachelor's degree. It's a really worthwhile investigation. Well done to the student and their supervisors and clinical and patient collaborators.

3 measures tested:

Hours of upright activity (HUA) - standing, or sitting with feet on the floor - self-reported


UpTime - time in the day spent with lower legs in an upright position - measured with the sensor


Daily steps - number of steps (would be a cheaper measure than using a specialist sensor but may be more unreliable)

They used MbientLab’s MetaMotionS (MMS), link - small, light weight, worn on the ankle of one leg, accurate, compatibility with various programming languages
The battery lasts 7 days

51 participants including ME/CFS (30), Long Covid (15) and healthy (6) cohorts - study of 1 week
Data from 3 participants had to be discarded

Collaboration with the Bateman Horne Center. Staff there were trained to brief the participants, and download the data and analyse it, adding metadata (e.g. date, person)
They used some software to automate the download process, reducing user error.
Cloud storage of data. There's lot of computer-y details.


Results for Uptime percentage (sensor)

The ME/CFS and Long Covid groups overlap; both are significantly different to the Controls.

Results for Hours of Upright Activity (self report)
chart looks similar to the one for Uptime

Results for Step count
ME/CFS cohort statistically different from Healthy controls, but Long Covid cohort isn't. There's a big overlap between the three cohorts. It suggests that step count may not be ideal as an ME/CFS outcome.

(I'm not 100% sure the charts (Fig 3.3 and 3.4) are labelled and captioned right.)

 

Accuracy of Hours of Upright Activity (self report)

There certainly is a positive relationship between the self reported measure of hours of upright activity and the sensor-measured percentage of time with legs upright, but it's far from a perfect match. This isn't surprising, as self-reporting activity all day, every day is a demanding thing. The differences may not be all the fault of the self-reporting patients. For example sitting, as I currently am, with one leg bent with the foot on the floor and the other leg crossed over it with that shin resting on the knee of the upright bent leg, would result in a record of me being horizontal, when in fact my torso is upright.

Anyway, this study suggests that using technology to generate an objective outcome is entirely feasible. Hopefully all research done at the Bateman Horne Center will use activity sensors. Unfortunately, we don't hear anything about how the patients found wearing the sensor.

I note that the daily hours of upright activity reduced substantially over time, even within that one week. Any measurement of activity must be undertaken for a long time - we have been saying 'at least 3 months', to account for initial enthusiasm eventually being replaced with the reality of disease constraints.

 

I'm not sure that's true. I would agree that it's no use as a diagnostic sign, but we already know that some people with mild ME/CFS would overlap with sedentary healthy people in number of steps. But it could still be a useful part of an objective outcome measure for a clinical trial, since it's the change over time that's relevant, not the starting value. I does however indicate a ceiling effect, since someone with mild ME who recovers to healthy sedentary wouldn't necessarily show significant improvement.

I do agree that the hours of upright activity is a better measure for clinical trials, and with the importance of measuring for significant time, or better still throughout a trial. It's also a splendid demonstration that subjective estimates of activity levels don't correlate particularly well with objective data, and that people tend to over estimate. That could be very significant in a clinical trial.

 

Good point.

As I said, I haven't yet understood the difference between the two charts of step counts, but here is Fig 3.3. You can see that the mean and standard error ranges for the ME/CFS and Long Covid participants fall almost entirely within those of the healthy participants. I'm not even sure that the healthy participants were matched for age or gender - and there was only six of them, so it's not a great sample. But I think that reported result probably is not atypical when the ME/CFS participants have mild ME/CFS.


I think the main trouble with step counts is indeed that ceiling effect. People with mild enough ME/CFS that they have the bandwidth to participate in a trial do seem to record a number of steps that, on average, falls within the bounds of normality. We saw that I think with Julia Newton's 2011 study, and most recently for Long Covid in that Knoop Netherlands Recover study, where they found that a large proportion of their participants did not have abnormal activity levels (I haven't checked either of those assertions, my memory might be wrong). Because, unless people are severe enough to not be participating in typical trials that require leaving home, they usually have to do a certain amount of steps to live, and so that essential activity is prioritised, at the expense of, well, everything else.

People with mild ME/CFS who experience an improvement will not necessarily increase their step count. They might walk faster and with a bigger stride, they might do more cognitive activity, they might take on a sedentary job.

I agree that step counts aren't useless, I only said that they aren't ideal. I think they probably are most useful for people with more severe ME/CFS, where their baseline step count really is very low. Perhaps distance walked would be a better measure obtainable from common consumer activity monitors than step counts?

 

As a simple non scientist I think that whatever movement the step counter reacts to for that individual if a given intervention results in them moving say 10% 25% or 50% more in months following than in months before the intervention then that’s evidence of them being able to do more.

 

Good point, @Kitty, I get restless legs sometimes at night. That would probably register on a leg worn step monitor as me getting up and going for a long walk. Mind you, I'm likely to do a lot less then next day as it's is still exercise beyond my limit.

 

Thanks for your post. I think though that this study illustrates clearly that there is technology available that does a good job of distinguishing people with ME/CFS from healthy controls and that it is not at all problematic in the research or clinical setting. Yes, we need more research to fine tune the measures used here, and to investigate others such as acceleration and gait.

This study also illustrates very clearly that patient reporting, even when data is collected daily and even when the reporting is of an objective thing like time spent upright, is inaccurate.

I understand that @sarahtyson has a particular skillset that she wants to apply to the problem of quantifying patient outcomes, and perhaps some good may come of it. It is a shame though that she seems to have chosen to dismiss what was said here and push on with a project that is not as useful as it could be. It is a shame that the UK MEA isn't better informed when it comes to allocating scarce research dollars.

We need a lot more wearable technology and a lot less patient reporting of activity levels and fatigue, both as outcomes in research, and in clinics for monitoring disease progress.

 

I guess this is the difference between our viewpoints. I see wearables as currently adequate technologies that can be incorporated into research and clinical monitoring right now. They may not be perfect, but they are better than subjective questionnaires where people are asked to summarise their level of functioning or fatigue over the past two weeks/months. Members here routinely monitor their illness in a useful way with commercially available wearables.

It's not like this is a new area. The fact that an undergraduate thesis can make a useful contribution illustrates the maturity of the technology with the various tools that are at hand to make things work.

It's great if Sarah has previously tried to use wearables, and a desperate shame if her efforts have been frustrated. I am certainly not dismissive of the institutional obstacles that non-BPS ME/CFS researchers have faced. But clearly her experiences continue to colour her viewpoint now. Her assessment of wearables as 'promising but not suitable for use right now' is objectively wrong.

See, for example this 2017 paper (5 years ago, there have been huge advances since then) about wearables in multiple sclerosis.
Wearable biosensors to monitor disability in multiple sclerosis

The problem of the Hawthorne effect is solved by having a long assessment time. And it's hard to see how the Hawthorne effect from wearables would be worse than that applying to retrospective participant reporting.

 

I think there is also the practical problem of getting patients to buy and use consistently wearables that can be expensive and inconvenient for people already struggling to keep their heads above water financially and in all aspects of life.

As I understand it the primary purpose of Sarah Tyson's project is for patient record keeping to enable some consistency over time and to alert doctors to the range and severity of symptoms and levels of disability so they can give appropriate care. In that context, wearables, particularly in an NHS short of funding, are a bit of a luxury. They would be good to include, but are not so likely to be the core of patient data for some time to come in the NHS.

On the other hand for research, I think the current and future generation of clinical trials should see them as an essential part of outcome measuring.

 

The cost of a basic consumer fitness tracker is pretty minimal (less than the cost to the government of an hour of individual CBT, I expect), and they often can run for two weeks without charging. They are much less hassle than trying to write down how much activity is done.

I think they could be given to people with ME/CFS during the diagnosis process - wear the fitness tracker for a month, keep an activity diary and then the doctor has some good information to work out if you have PEM or sleep issues or if your heart rate is spiking at odd times, and the nurse has some good information to work out how to help you optimise activity levels.

If the NHS really is that poor that it can't give people with ME/CFS a fitness tracker, they could just give people a strap, and get the tracker itself returned. But, like I said, the trackers don't cost much, and the NHS could negotiate a good bulk deal. And really? They can waste money on ineffectual CBT and mindfulness courses but they can't provide a fairly inexpensive tool that helps the person manage their illness? I would have thought that the NICE ME/CFS guideline shows that energy management is pretty much all that we have, in terms of ways to reduce symptoms. People with other diseases get thousands of dollars of medicines every year. I'm sure that there are mechanisms in the welfare system for disabled people to get financial assistance for equipment that helps them function.

Maybe there are issues, but I think that working on addressing them is more useful than work that props up a monitoring system in clinics that relies entirely on subjective patient reporting.

 

I agree with you, but I doubt it works that way in the NHS. We'd need the NICE guideline to recommend them as a key part of management.