Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Re the question on why there has been so much focus on the subjective outcome +unblinded trial issue:

Of course there are lots of ways that a study can be unreliable. But I'm not aware that the person leading a new review of GET for ME/CFS disputes that it's important that studies select cohorts actually with ME/CFS (and therefore with the core symptom of PEM). But @Hilda Bastian does seem to be open to the idea that subjective outcomes can be reliable measures of the utility of an intervention in an unblinded trial. So that seems worth discussing, to understand what we think about that idea and why we hold the views we do.

 

There's a good reply from Physios for ME on the twitter thread, thanks Physios for ME for being on the ball. Also good replies from others including Millions Missing France noting that the Cochrane Exercise Therapy review is under review.

 

@Joan Crawford, thank you very much for your ongoing engagement here. I agree with much of what you write but I'd like to discuss a couple of points that I think are relevant to the Cochrane review and ME/CFS research design.

The outcome measure 'ability to work (or equivalent) is quite a difficult one. I've given the example of my son returning to school after a year of home schooling before. So, if my son had been asked in January what his ability to attend school was, he would have said '100%'. He started school at the beginning of February and was attending well at the end of that month. If you had asked him then or had an objective measure based on school attendance, the rating might have been 90%. But by May, he was bed-bound and having difficulty even being awake to answer questions, let alone getting to school - so the rating was 0%. The point I am making is that it is very difficult to work out what a sustainable level of work or school is until you have been doing it for sometime. Any forecasting by the patient at the end of an intervention about what they might be able to sustain is likely to be optimistic. Particularly in the case of work, it takes time to apply for and obtain a job. And any objective measure of work hours or schooling needs to be made after enough time has passed for any cumulative impact to show itself. Enough time is probably at least three months after the increase in activity.

For the same reason (non-sustainability), a week or two is much too short a time to assess real changes in physical activity. As I've said before, a week or two of activity monitoring is really a subjective measure. Someone with ME who is motivated will often be able to push themselves to achieve higher levels of activity for that length of time and probably longer. A decent period of time for continuous monitoring is probably at least three months. That can be remeasured in a follow-up if desired, but the most important thing is to make the period of measurement long enough.

Thank you.

(I'm sure you meant 'having secondary subjective outcome measures alongside objective ones'.) We've seen that secondary measures are often biased. If they are included, lots of thought needs to be given to ways to reduce the bias. Apps collecting real time assessments (e.g. daily) rather than a question asking how things were over the last month are likely to be better.

Many ME/CFS research proposals I've seen recently seem to want to collect huge amounts of data about mood and emotional wellbeing and the like, even though the intervention isn't aiming to change mood. I commented recently that answering long surveys about our emotional status often seems to be the price we have to pay for biomedical research. I'd like to see a lot more thought given to what data is really needed for a study. The less irrelevant (and probably highly biased subjective) data that is collected, the fewer the opportunities for cherry picking (e.g. our intervention didn't change activity but participants felt a bit happier!) or for later studies to torture the data to fit preconceived ideas.

 

Written over a couple of days, so some points have since been made by others, but anyway...

Well said.

I am firmly on the side of subjective outcome measures requiring adequate blinding, and/or being used alongside objective measures (which are given at least equal weight to the subjective measures). Anything less is not scientific, and is potentially very dangerous.

I don't object to subjective outcome measures, in fact I want them used. They can provide valuable information, in particular from the correlations between subjective and objective measures. But they cannot be used on their own and without blinding, especially in a trial of a treatment whose whole purpose and means is to alter patients' subjective self-perception.

That just becomes circular nonsense revolving entirely around patients' questionnaire scoring behaviour, independent of any practical real world changes or benefit. Changes in actual perceptions and cognitions must have measurable external agency and consequences well beyond mere questionnaire scoring behaviour, otherwise what is the point?

Science works by allowing us to discriminate, and quantify the difference, between subjective and objective elements in our perceptions and cognitions. Unblinded subjective measures on their own don't allow that discrimination and quantification, that necessary element of control (as in Randomised Controlled Trial). It is not possible in such trials to distinguish between genuine therapeutic benefit, and potential confounders and artefacts of that methodology, such as placebo effect, wanting to please (or avoid displeasing) the therapist, etc.

At most unblinded subjective measures can only tell us there is an effect. They don't, on their own, tell us what the effect is due to (and hence if it is genuinely therapeutic). That is why we need blinding and/or objective measures as well, to help tease out causal pathways.

What the BPS school has done is construct an unfalsifiable 'methodology' that tries to maximise the effect of various confounders and artefacts, and arbitrarily relabels them as a therapeutic benefit.

This whole shitshow is going to come down to this technical issue. If the BPS school are not allowed to rely on unblinded subjective measures then they have nothing, and they know it. And so do their critics.

Wessely and Chalder:

Change in activity level is objectively measurable. So there are no excuses for not measuring it.

Measuring patients' acceptability of a treatment might be legit, for example.

This excuse from them really gets up my nose. There is a minimum methodological standard to meet for any study wishing to claim scientific status. Trials that do not meet that minimum standard are not merely a weaker form of evidence, they are non-evidence to start with. They lack the necessary rigour and clarity to be interpreted and applied safely. No amount of hand waving and sophistry can change that.

It's like trying to build a house on a foundation of sand. No matter how well constructed the house is, it is still built on sand.