I have chosen to reply specifically to Sarah's comment above because I think a reply on her 'you don't get that this activity survey is just about activity' spiel is really important to 'debunk' as a distraction. And then on top of that there are separate, valid and accurate issues regarding 'avoid' stitching up those filling it in etc.
The issue is that, even if it was a good way in the mix of what she has done, the way she has done it has and will I think distort the measure of activity.
Which, given there has been mention of this being used as a dataset ie collecting data for prognosis for the implementation plan (and obviously there long-term outcomes ie 2, 5, 10yr should be the primary outcome), which would cause big problems for all if inaccurate data was being used to inform what % get worse or better over time.
I note that the history of this is that it is effectively replacing the SF-36 physical function survey? Which is the quality of life arm of their measures that calibrates disability against those with other conditions - and obviously 'how adjusted' ceasing to be quantified actually does also compromise 'how much activity/exertion' even each individual task could be. For example if that 'bus/car ride' or 'chat/visit' activity's adjustment has moved from: doing over an hour to doing less than 5mins.
I'll note that the SF-36 physical function survey explicitly focuses on the 'ness'/amount of limitation, whilst still having the small number of options Sarah claims is necessary:
But worse, ALL ITEMS that relate to disability on that SF-36 need to have been accommodated - and I think there is a separate thing to be done fishing that up (again) to see what has 'disappeared'. These exclusions cannot be justified by a Rasch analysis that does not check if what is actually being measured is the concept claimed. If this survey apparently describes 'disability/physical function' (which should be related to activity whichever side-step is being claimed by the team here) then a test needs to be done to see if those coming low on SF-36 PF are showing as such on this.
Anyway, back to the questions on this - how much of a problem the inbuilt ceiling and floor effect are, the lack of 'ness' in the options and combined with the choice of activities, when believing it measures 'how much activity has been done'.
The choices of activity she has defined as the envelope, combined with her choice to not stratify adjustment, avoid into impact or frequency create huge ceiling/floor effects
as well as not at all representing the full envelope for any person, particularly a pwme.
It cannot measure anyone either getting worse or being severe, as 'adjustment is adjustment' for example if there were a question on toileting then using this, even if someone declines from a level needing a raised toilet seat to a commode or bedpan for example it will rank the same).
Because those core 'unavoidable' activities that most pwme will find make up either more than or most of their 'envelope' are 'avoided' by her survey and method.
In simple terms for anyone other than mild that extra activity (work, social, medical/admin) that will 'count' as 'more' will be compensated for by 'less' eating, cooking, getting to the toilet, showering - no 'social life' or grocery trip to lose first. But because it can't be 'none' and will already have been
limited in some way then the options stitch up almost all with ME/CFS from showing that impact - adjustment is adjustment, avoid is avoid.
AND for the claims it will somehow 'help clinics' then it won't because by hiding that
impact then it makes it look-like the activity that caused it - noting there wasn't a 'but it will leave me in bed for a week' option - was OK. It's the PACE-style doing the 6min walk training at the sacrifice of showering and grocery shopping again isn't it?
The very activities where
any impact will be shown are either entirely not present, not wholly accounted for (by frequency eg of shower moving from 1 week to 1month), or hidden by only allowing boxes that effectively 'anchor' the answer someone can give to being unmoveable or not stratifying different severities.
Effectively for
most pwme,
most of the time, the very areas and manner in which significant change in envelope ie
amount of doing (which this survey claims it is measuring) is being hidden/missed.
or indeed whether people do well or worse from clinic treatment (
only measuring 'delivery' and if a clinic's 'treatment' is making pwme get better or
worse), is a problem if someone can get significantly worse and the survey cannot register this due to the way it is using amount of boxes of a select group of activities ticked, and not including the aspects that 'drop-off' in compensation necessarily either by stratifying adjustments or having a complete enough list one could see that a medical appointment meant a significant knock-on effect in frequency and adjustment for personal hygiene type tasks ie the overall envelope didn't stretch but went down.
It is the 'disappearing the harms' again. People at a certain level can
only improve 'amount of activity' according to what counts on this, or stay the same with these boxes and choices?
I used the toilet example myself because if a pwme had to crowbar in a compulsory activity such as an appointment then it would tend to 'show' activity-wise, particularly in those who are not mild ie moderate-very severe,
specifically in those core activities being
compromised further because with activities such as eating or personal hygiene they can
only be further adapted and reduced in frequency - which this does not allow to be measured.
PS a Rasch analysis (mentioned elsewhere in the thread by Sarah) would not cover checking this 'completeness of envelope' issue at all, indeed does
not check for 'is it measuring the concept it claims to be measuring' just internal consistency as a tool,
and ceiling/floor effects ie that it can differentiate moderate ME/CFS from more severe, would not be checked by it unless:
- it was being tested on those more severe or moving between levels of severity (Sarah herself acknowledges this - instead saying those would 'need a separate survey' - which does not answer this point as it is the same as discharging people when they get worse and calling it recovered if that isn't fed into the outcomes but hidden under drop-outs or not providing a box for it to exist but instead putting that box on a survey that doesn't count)
- using a different tool that would pick up on this 'gap' to test this tool against ie a comparator that would identify someone getting worse and seeing if this survey was also picking them up?