Trial Report Cost-Effectiveness of Online CBT for Children with CFS/ME Compared to Online Activity Management: FITNET-NHS Trial Findings, 2023, Crawley

Columnist and political commentator at the Guardian newspaper. She has an edge like a scalpel, and is hilarious with it.

 

https://twitter.com/user/status/1741160924344414712

 

Two questions - a) what was the non-CBT intervention? Was it a glorified version of GET/CBT? b) as someone who hasn't done stats in years, is an 0.532 improvement in QALY a statistically significant improvement? Is the study purporting that both interventions are roughly equally effective, or does 0.532/0.530 mean that they're both essentially useless? Sorry for my scientific/statistical ignorance!

 

I am sure this has been uploaded before?

'Full title: Investigating the effectiveness and cost-effectiveness of using FITNET-NHS
(Fatigue In Teenagers on the interNET in the NHS) compared to Activity Management to
treat paediatric Chronic Fatigue Syndrome (CFS)/Myalgic Encephalomyelitis (ME) in the
United Kingdom: A randomised controlled trial (FITNET-NHS)

Short title: How effective is FITNET-NHS for children and young adults with CFS/ME
Sponsor Details:
Sponsor ref: 2571

Research and Enterprise Development,
University of Bristol,
Bristol, BS8 1TH.
Tel: 0117 331 7709

Study Reference Details:
R&D ref: RBB498
REC ref: 16/SW/0268
HTA ref: 14/192/109
ISRCTN ref: ISRCTN18020851
IRAS ref: 211202

Co applicant (no longer employed on
project):
Mrs Mary-Jane Willows
Lead for patient and public involvement'
........... AFME?

 

Hard to know exactly what they are claiming from the poster, but it looks like they are just looking at one year, in which case a 0.53 QALY gain is huge. That's essentially going from quite sick (somewhere around 0.4-0.5) to perfectly fine. But I think the scale they are using here actually allows negative QALYs and we don't know what the starting point was. Of course, there is no control group either. Without any details, it's pretty hard to know what's going on here and of course this crew is likely to try to goose the QALY numbers.

 

Thanks for explaining! I wonder how they'll spin this into yet another CBT triumph

 

I didn't read it as a 0.53 QALY gain but rather felt more likely that those were the final QALYs.
ETA: I'm beginning to think that perhaps your interpretation rather than the one I was leaning towards may be more likely. Time will tell with the full paper.

 

THe issue is that both are pushing harmful treatment - just via different delivery mechanisms, but hey suits them to never unbundle that 'CBT' as they use it is just a 'delivery mechanism' or 'communication style'.

I wondered whether they'd be callous enough to use this as an excuse to PR the misnomer that instead of both being bad they were really trying to trojan horse the lie "treatment in line with guidelines" for treatment that is against the guidelines and pushing upward increments.

 

I actually wasn't quite sure what they meant. But the relevant point for the poster was that the score for the intervention was only 0.002 QALYs better than for the comparison group of activity management--which was at that time the minimalist treatment recommended in the NICE guidelines (after CBT and GET). I'm also not sure why they chose to use a quality-of-life measure for cost-effectiveness--is that standard, does anyone know? What if they now report the "effectiveness" data--that is, the SF-36 results--and report that it's "effective"? Or if it's not cost-effective, does it make the effectiveness data essentially irrelevant.

 

You might be right. They say they adjusted for "baseline utility for QALYs", but that could also mean they adjusted between the groups to balance them. I'd be surprised if they even enrolled patients with that low of a quality of life on average though, let alone after the inevitable regression to the mean. 0.53 is moderate ME and they tend to focus more on the milder side.

 

NICE says an intervention should be below £20,000 - £30,000 per QALY gained to be cost effective. So their approach is very much aimed at getting NICE to accept FITNET, which of course they missed by a mile here.

Edit: Here's a link about evaluating digital health products, which seems quite relevant.

 

Highly pertinent question actually. They said on the poster "The EQ-5D-5Y was collected at baseline, 3, 6 and 12 months using an online questionnaire completed by the child. Quality-adjusted life years (QALYs) for each child were calculated from utility scores using the area under the curve approach."

I think* that I've seen the QALY (quality of life years) used in Nice section where they calculate whether it is 'cost effective'. It is supposed to of course account for e.g. where you might have a treatment for something like cancer or cystic fibrosis that "QALYs assess the effect of a given treatment on how long a patient will live multiplied by their quality of life in those remaining years with that treatment."

From the Nice glossary: Glossary | NICE
"A measure of the state of health of a person or group in which the benefits, in terms of length of life, are adjusted to reflect the quality of life. One quality-adjusted life year (QALY) is equal to 1 year of life in perfect health."

So their audience is the budget holders. And unorthodox usage of terms designed to mean something quite specific in the context they were developed for being then misunderstood I suspect when used in the same way the BPS uses the term 'evidence' to mean something they create and cherry pick to support their arguments vs actual science.

There is something to be said for when this research is claiming the 'Y' part but has no scientific data regarding children in particular on mortality or 'health' (which you'd assume would include things like disability) etc and indeed certain schools in particular have contributed to behaviour that has made such data collection more impossible (lumping and dumping and re-diagnosing with mental illnesses like pervasive avoidance made-up stuff when they are too ill for treatments).

ANd if you are claiming 'in perfect health' as a QALY then how on earth are they getting away with not measuring health objectively.

And hilariously, particularly when you look at the FOI from RPsych where they wanted the Nice evidence gerrymandered to get rid of any even medium-term measures like 135weeks being included in analysis, given children have more years to potentially live are they really trying to claim their short-term measures somehow can be multiplied across more than a few weeks after treatment and therefore coercion involved with feeling they have to give the right answers ends?

When in actual fact they have compared 2 different modes for re-education in dodgy beliefs and behaviours that probably cause relapses, just that happen to hit at about 12months (or at that point is when you realise it actually isn't 'just a bad day' because you finally get acceptance and break free from the brainwashing of the 'CBT'). And, if they have dared to instead of proposing measuring people's actual quality of life three years on, have decided that taking their dodgy results just after and multiplying it by 'years' is somehow valid?

I believe of course that such things should actually be two-stage processes, where if something is useless it doesn't matter if it is 'cheap' and get to compete with things that might have some use.

I also believe that things that don't measure harms shouldn't be being included. It's surely part of the 'informed consent' that these people should be required to have independently monitored both physical and psychological long-term implications of their 're-education' and 'behavioural modification'. So even if someone does want to pay for it in those who do want to risk it, both do so in full knowledge. The idea that isn't the case is even more outrageous for being with children, even more outrageous for being psycho-brainwashing-reeducation, and even more outrageous given how vulnerable those with the illness both are from illness and made situationally from the PR from these people.

All of this reads like a 'consequences doesn't matter' manifesto and people need to begin stating clearly that those who believe that, and who are neither accountable, responsible or interested in consequences of those they 'treat' or 'get money for' cannot have good intentions. It mightn't prove 'bad intentions' but it is callous disregard for what actually will occur to those subject to it if you've never measured or allowed it to be discussed or said by those subject to it and when you engage in behaviour that can only be described as censoring the truth of those you are doing things 'to' the pretence or defence of ignorance is also thin.

It also reads like a distraction where noone is asking whether the material is illogical or harmful that is being delivered by either 'mode', or asking why given they collected some of the data for their 'costs' people aren't using it to ask whether people are objectively any better and recovered ie whether both don't work. I don't see why a curve is needed when these could be collected beyond 12months.

"Costs: Treatment delivery and secondary care costs were captured via electronic health records. Primary and community care, prescribed medications, productivity loss, parent/carer out-ofpocket costs and education impacts were captured via an online resource use questionnaire completed by parents at 3-, 6- and 12-months."

The whole thing is such a strange way of doing something if you cared about the patient outcome, because it avoids asking that question at all stages, that I find it hard to believe there is a good motive behind this.

 

JUst reading through the following paper on use of QALY and methodological issues, with a particular focus on fluctuating conditions: Measuring Quality-Adjusted Life-Years When Health Fluctuates - ScienceDirect

had to just fling this one quote from it out here, which is an 'aside' to the main issues, but worth noting:

"When health is fluctuating and health-related quality of life data are missing, assumptions of missing at random or missing completely at random are less likely to hold than for missing data in “steady” conditions.

Missing not at random may be more likely to occur as data may be missing at certain time points because the patient was experiencing ‘bad days.’

Simple imputation methods, which can produce bias in any circumstance, such as last observation carried forward or next observation carried backward, can be even more prone to bias if the number of missing observations is high or sequential in fluctuating states."

 

This paper quotes the following as a reference: Using Patient-Reported Outcomes for Economic Evaluation: Getting the Timing Right - ScienceDirect

I've put the abstract below

Which is interesting as an example of how timing of PROMs needs to be correct if assumptions are to be made on them. The Sanghera and Coast (2020) paper in the post above ( Measuring Quality-Adjusted Life-Years When Health Fluctuates - ScienceDirect ) notes this is knee arthroscopy and not a fluctuating condition, but demonstrates still how timing of measurements e.g. after this operation is pretty key if you are going to base assumptions like QALY on them.

I'm thinking that this is a very pertinent issue that needs to be resolved with ME/CFS (along with the measures themselves that they choose given the biases) because the very nature of a relapse occuring probably just at the point where the measurements end is so the opposite to the assumption being used of 'area under the curve' or things that assume 'progress will be maintained' or worse the line will continue upwards.

I'm also aware there is research currently going on discussing PROMs for ME/CFS and so find this pretty interesting to understand some of the assumptions and potential issues to watch out for (and beginning with a straightforward knee op seems a good way of building up how these things work to imagining the different fluctuations and issues involved for something more complicated/layered)

Using Patient-Reported Outcomes for Economic Evaluation: Getting the Timing Right
Schilling et all (2016)
Abstract
Background
Patient-reported outcome measures (PROMs) are becoming increasingly popular in orthopedic surgery. Preoperative and postoperative follow-up often elicit PROMs in the form of generic quality-of-life instruments (e.g., Short Form health survey SF-12 [SF-12]) that can be used in economic evaluation to estimate quality-adjusted life-years (QALYs). However, the timing of postoperative measurement is still under debate.

Objectives
To explore the timing of postoperative PROMs collection and the implications for bias in QALY estimation for economic evaluation.

Methods
We compared the accuracy of QALY estimation on the basis of utilities derived from the SF-12 at one of 6 weeks, 3 months, 6 months, and 12 months after total knee arthroplasty, under different methods of interpolation between points. Five years of follow-up data were extracted from the St. Vincent’s Melbourne Arthroplasty Outcomes (SMART) registry (n = 484). The SMART registry collects follow-up PROMs annually and obtained more frequent outcomes on subset of patients (n = 133).

Results
Postoperative PROM collection at 6 weeks, 6 months, or 12 months biased the estimation of QALY gain from total knee arthroplasty by −41% (95% confidence interval [CI] −59% to −22%), 18% (95% CI 4%–32%), and −8% (95% CI −18% to −2%), respectively. This bias was minimized by collecting PROMs at 3 months postoperatively (6% error; 95% CI −9% to 21%).

Conclusions
The timing of PROM collection and the interpolation assumptions between measurements can bias economic evaluation. In the case of total knee arthroplasty, we recommend a postoperative measurement at 3 months with linear interpolation between preoperative and postoperative measures. The design of economic evaluations should consider timing and interpolation issues.

 

Crawley takes an inordinate amount of time to publish her trials. I feel like we were discussing FITNET and MAGENTA before this forum even existed.

 

In paediatric “CFS/ME”, if you don’t say you’re better, they start legal proceedings against your parents.