Anomalies in the review process and interpretation of the evidence in the NICE guideline for (CFS & ME), 2023, White et al

In this domain they seem to serve mostly an automatic-approval function.

 

Not sure about that. I think they do understand it, it's just that they reject the validity of the outcome when it goes against them. They very much like the NICE guidelines when they go their way. In fact for years clinicians pretended to be bound by them when they recommended CBT/GET, and now that they don't, they simply say that guidelines are only advisory. Heads they win, tails they win, don't toss they win again. NICE has produced other guidelines where the same quality of evidence went the other way, there is really no consistency here.

The quacks are like politicians who only accept the validity of an election if they won, and keep insisting that they won years after they have lost, as evidenced by them not being in office anymore. Except in the case of psychosomatic ideologues, they are still very much in office, still hold every bit of power they ever had, even though they lost the 'election' on the basis of their evidence having always been nothing but a bunch of hot air.

Like most institutions, science has no protections against corruption from within, when the people who hold the levers of power simply reject the responsibility they are expected to uphold and simply do as they wish, confident, and right at that, that they can simply clear themselves of any wrongdoing by 'investigating' themselves later on.

Parts of the system are built around 'self-oversight'. No system can function legitimately like this, but it's how it's built.

 

A proper process should not be heavily reliant upon which end of the scale of the mixed bag an ethics committee happens to be or not; a proper process would minimize the variability due to this. If this were the case in aircraft/vehicle/pharmaceuticals safety engineering we would be seeing no end of system failures. No process is perfect of course, as Boeing have managed to stunningly illustrate in recent times. But in safety engineering (which I have only had a glancing contact with in my career, but enough to be confident of what I am saying here), there are specific aspects that have to be worked through and documented, and a failure in any one means going back and reviewing the design, until everything passes or acceptable mitigations are put in place. It is not always an exact science of course (e.g. estimating the human and environmental collateral harms of an aircraft springing a fuel leak in flight).

A safety engineering process is not there to insult the integrity of the people involved. It is there to minimise consequences of failings due to normal human fallibility, and also minimise consequences of less excusable but sometimes inevitable human failings, such as laziness, incompetence, over-confidence, etc. I see no reason why clinical trial ethics committees should be above such oversight. It's no good relying on the fact people have been taught all these things in the past, because it will still go horribly wrong without a safety process.

An ethics committee implicitly includes the responsibilities of a safety committee surely?

 

Perhaps I should have said "lack of understanding, or willingness to understand", which would still be applicable to their approach to research.

 

It does, except that members of an ethics committee are not held responsible through any mechanism. Ethics committees were set up on the basis that they were a good idea. They became compulsory but I don't think members have ever had to commit themselves to behaving ethically themselves, or upholding specified principles. Maybe we did but I have no recollection of it.

But I suppose that if members of an ethics committee took on responsibility they could be expected to be paid for their services. Nobody is going to lay themselves open to litigation for nothing.

 

I'd not realised an ethics committee was so informal and unregulated ... and unpaid. If all involved are properly competent and suitably qualified, then it doubtless works fine. But it is of course, as our well known researchers have proven down the years only too well, wide open to abuse, incompetence, etc. I'd think it an area in urgent need of reform.

In safety engineering a safety committee is typically headed by a qualified safety engineer (probably mandatory but not sure). Given how human safety features so strongly in medical trials, I'm astonished this hasn't caught up with the management of them.

Is the British system typical of other countries? Or do some countries have more formalised rigour?

 

The situation the UK may now be similar on these points. But I doubt hospital and university ethics committees get paid. It isn't part of the job but having done a stint helps to get bonus pay.

But even if they are paid I am not clear how they can be realistically accountable. If five years later there is fuss about a trial being poorly designed do the ethics committee members get fined or sent to prison? Presumably not. The chairman might get a chastising report but very likely he would be fed up with doing it by then anyway.

 

Hmm. If you have train as a doctor you get used to the haunting at night from personal mistakes - or quit. I still wake thinking of some from 40 years ago. I am not sure I would wake up realising I had stupidly approved a badly designed trial with uninterpretable results.

 

If my memory services me correctly there was a study looking at outcomes for the British ME/CFS clinics at a time when GET/CBT was their standard treatment, and they found that following intervention from these services patients were likely to work fewer hours and claim more benefits.

This is a clear inditement of these services as treatment centres, but I would argue for people with a long term currently untreatable medical condition this is a positive outcome.

I don’t think the BPS research repeated use of these metrics, though we still see school attendance being reported in paediatric studies (see current thread on Magenta https://www.s4me.info/threads/grade...l-2024-gaunt-crawley-et-al.37488/#post-518833 , where school attendance improved in the no treatment control, but not the treatment arms. Again perhaps not what the researchers wanted.)

 

And also stop good money being thrown after bad should for example it start to become clear that the objective measures a trial was funded to do aren't showing what they want.

Either: if there is a methodological issue they come across that means they aren't working then that information needs to be written up as the result, and surely provides an opportunity for the next person applying for funding to benefit from that learning - which is as important in itself of 'finding an outcome' for something where new methods and honing how things are done are probably important for progress.

Or: if it is a case of the particular measure 'working fine' but it just isn't showing what was anticipated or wanted then that finding is as important as anything else. It's a rejection of the hypothesis and if that isn't 'protected' then science isn't science anymore because you can never actually confirm a hypothesis 100% but you can 'reject it' which is the point of having them, that the more times something is run and not rejected the more 'sure' you are.

If it basically can't ever be rejected then if it also can't ever show any positive effect you have the money-wasting loop we all see? where noone gets to the bottom of what the issue is

 

there is that. Although when power differences come into play it can serve as a different purpose/instruction/suggestion/warning/party line or what not

 

I think this is an interesting point, because some of the tactics used are only going to be picked up on by people who are looking at these things day-in-day-out vs shipped in as a subject expert to do one every so often. SO you need the expertise of those who look at ethics, and expertise of those who are experienced in administrating/dealing with the applications as much as the subject knowledge.

There might be issues that are more specifically arising in certain either niches or sections - but if you then have people who are doing these things regularly you'd surely have that professional circle where such 'latest problems encountered' are getting communicated and flagged to those other areas that aren't yet seeing them so it stops proliferation.

 

Agreed that it is important to warn of this - that measures such as those aren't quite right in a culture where the temptation is likely to be performance reviews (based on are these improving) and no treatment and plenty of other complications of the condition (which is why we need long-term outcomes not short). The whole thing becomes revoltingly circular.

Doesn't cancer use 5yr outcomes rather than 1yr for any treatment packages there. I don't know on other conditions that are taken seriously and treated clinically and am only quoting that because you hear these things in the news every so often.

There is of course also the issue with those measures that there is a floor effect. Once you are ill enough to no longer be able to work or go to school the measure no longer differentiates further deterioration.

I also think that we can now spot simply from the uttering of the phrase 'secondary benefits' (which they aren't) who is conflicted/has interests in their reason for their being involved/behaviour. Because one could argue that is more 'the message/aim' they are looking to convey than anything else contained in whatever article or speech or anything. Which gives me a sense from who and how many say it how wise it probably is to stay well away from these as measures. After all there are mediating variables in between this.

But of course these days things have become so argument-based anything that is based on what others can term a behaviour seems to be twistable. We really could have done with those long-term stats from any of these objective variables from said trials.

But how do you separate out eg the person who 'overdoes it' who has a high actimeter reading for 6months and then ends up incredibly sick long-term from it later on due to it causing a crash from those who had a high actimeter reading because they were less unwell than the first person, when the criteria were so poor? and of course even now do we and can we have fine enough definitions to really capture the differences as many of us might go downhill from top-end moderate to low end-moderate rather than moderate to severe. And of course said changes might be in one particular aspect more than others or before other aspects.

It makes me realise that without a few really important findings and pieces of research I think we'd be in a right sticky wicket - the Workwell 2-day CPET being one of the primary things to show it isn't cart-before-horse or being unfit etc.

 

ME Research has highlighted the response
https://www.meresearch.org.uk/nice-responds-to-claims-of-a-controversial-u-turn-on-get-and-cbt/

 

Right, that was PACE. And Mark Vink and I wrote this paper last year about how their own data shows no benefits overall in occupational status: https://content.iospress.com/articles/work/wor220569

 

The whole thing's arse-about-face anyway. They shouldn't be focusing entirely on improvements, at least not in anyone who's been ill for more than two to three years.

Real improvement is quite infrequent, so the aim and the measurements should first and foremost be about stability. Any improvements that do occur are often in the form of better QoL, once people have established what level of pacing they need. Their ME isn't really any better, but they're having less pain, less PEM, fewer gut symptoms, etc, because they're not exceeding their capacity.

But I suppose that would require researchers with a passing knowledge of ME.