Can we influence trial and review methodology, eg open-label trials with subjective primary outcomes?

Yes a good idea.

One thing we could do to move things forward, even if only a little, would be to utilise the factsheet project.

When reading the opening post & just glancing through the rest (sorry not been able to properly read it so apologies if i off base here) but while reading i was immediately remnded of what i was saying the other day on another thread (quoted below - I've greyed-out bits of my suggestion that arent directly relevant & possibly off topic), perhaps these 2 ideas could be amalgamated or connected in some way....

 

I can see this being sensible if applied well, but am I wrong or doesn't the whole concept depend on what kind of patients you're recruiting and willing to recruit? How does one ensure that patients are indeed educated on these matters and don't just follow whatever nonsense is currently trending on Twitter or whatever someone once told them? A lot of patients on S4ME seem quite well educated on these matters but what if you just end up getting patients that think the current processes surrounding the "lightening processes" are the culmination of methodological rigor? If someone wants to test the "lightening process" in a riducolously bad study setup, wouldn't they just choose a groups of patients who repeat their gospel and ignore any concerns about how a study should be conducted?

I know there's been a lot of talk in the US about trialling certain drugs for Long-Covid and certainly what was trialled without patient input wasn't very sensible at all, but I haven't seen good ideas by patient groups either that would go beyond things currently trending on social media.

 

Yes absolutely. Was planning to stick to whatever trial questions were set by James Lind Priority setting partnerships. The methodological rigour would be provided by scientists without any conflict of interest and who understand that relying on subjectively reported outcomes in unblindable trials renders those trials useless

 

That's a good thought. 'Clinical trial breaches national research standards' is a much bigger news story that something about Cochrane, who only one in ten thousand people will have heard of.

Solid guidelines should also make it much easier to influence ethics committees at the stage where bad trials are trying to get passed.

In fact, attempting to influence the consultation process could be a great story in itself: 'Patients fight to get basic standards for clinical trials'.

@Jonathan Edwards, do you have any thoughts on this? Do you know when a UK guidelines review might be coming up?

 

I like this idea too. It might seem a bit technical but, as several members have said, once you get your head around the reasons they aren't too difficult to follow. I think it would be good to have on display a simple explanation of why so many trials are uninterpretable, written for the benefit of those who might be exploited by them but plain for everyone else to see.

The BPS people cannot argue against this stuff without making themselves appear even more stupid (e.g. "we only know how to do bad trials of this so you have to allow us to pretend they are good trials"). Maybe even people like Sonya and Charles will come to see that patients are not 'being negative'. They are keeping their eye on the ball.

 

Do you mean guidelines for trial practice?
I am doubtful that any attempt to target protocols and guidelines will have any impact. Protocols and guidelines are inherently flawed, incomplete and bendable. "Because it says so" is never a good argument. I think it is much better just to lay out the arguments for as many people as possible to see. I don't need to read a declaration of Helsinki to know what I think is ethical. If what I think is ethical is different from a declaration then either the declaration is wrong or someone needs to explain some more subtle arguments to me.

 

Yes, such as, 'Don't use subjective measures as primary outcomes in open-label trials'. That could have saved us a whole lot of trouble.

Maybe not, but if there's a review coming up, it's a chance to try.

Only to an extent, surely? Can't you think of maybe ten rules that, if applied to every trial, would raise medical research up out of all recognition? I'm thinking, 'Don't do open-label trials with subjective primary measures. Always publish your results according to your original analysis protocol, even if you provide additional analyses. Always publish all your planned analyses together in a single paper... etc.' (I'm basically taking PACE as my template for how to breach ethics.)

"Because it says so" is never a good argument.[/quote]

You don't need to read a declaration because you're an ethical person but the huge mess that is the clinical trial literature is full of unethical rubbish that could have been stopped in its tracks by ethical boards actually applying ethical principles, and apparently they're not capable of doing it without a checklist.

A checklist may not be perfect but isn't it better than the alternative? (Like democracy.)

 

The basic problem is that if ethics committees are not applying ethical principles it is either because they do not understand or because they, or their chairman, does not want to understand because it is politically inconvenient. In that situation it will make no difference how many checklists you have. The committee will either not understand how to apply them or not want to understand.

The issue of subjective outcomes in unblinded trials is something taught to all junior doctors. Just like they get taught that the thyroid gland is in the neck. Nobody needs it in a checklist. And the problem is that under the right circumstances unblinded trials with subjective outcomes can be entirely satisfactory - because the issue relates to the way human nature affects medical interactions and there are neat ways to offset it. You could do a dose response study in which there was no identifiable expectation for a best dose. So no bias. So if you get a consistent peak at one dose you can believe it.

Every trial has to be taken on its own merits from first principles.

The biggest problem with ethics committees is that chairman are deliberately chosen by the local physician community as those who will not rock the boat and cause difficulties. If the community is psychiatrists that means sticking to the status quo.

Remember the old adage.
You can always tell a Bart's man.
But you can never tell him anything.

I have spent my life pointing out bad practice. I have ended up about a million pounds poorer in terms of salary and pension increments for being a good chap. That is the reality.

 

Though if researchers commit to abiding by the declaration of Helsinki, but don’t, this provides support in arguing the research is unethical and leverage for those arguing for better quality research.

 

It might do. But I have now given up offering to review grant applications, unless I think they are really important ME/CFS ones, because I cannot bear the tedium and meaninglessness of wading through pages of stuff about conforming to Helsinki, PIP, diversity, and trying to answer why these have been satisfied on an online form that usually rubs itself out every half hour. The bureaucratic work I have to do, on top of the hoops that the applicants have to go through has made the whole process impossible for me. If the applicants simply presented their proposal and addressed the important ethical issues, as they used to, it would be fine.

If it gets any worse I will stop doing ME/CFS grants too!

 

Thank you for your service.

Noooooooooo!

 

Frankly, if I had had to think about all that when I was on an ethics committee for ten years I would have resigned overnight. This is the problem I was talking about. Being buried in issues that should be sorted by simple default legal rules.

 

yes thats what i was meaning. I'm not doing well at explaining myself atm.

 

this is my experience so far. They refuse to see that a badly designed study is just as unethical as a study which harms and/or inconveniences study participants during their participation. Because the harm comes later when the study results are manipulated and mis-used used to justify poor practice in the future which harms all patients ad infinitum. This, apparently, is not what we should be concerned about...it has been disappointing so far that we are constantly being told to neither look at the nitty gritty of study design (eg the suitability of outcomes and outcome measures), or the bigger picture.

 

I can think of one rule that would make a big difference: Apply the same standards to research of therapist delivered interventions (particularly in psychological medicine) that are applied to other types of medical research (particularly pharmaceutical interventions).

I’m reminded of the reply from Barry et al to the JNNP Anomalies article, where they pointed out that Wessely was calling for the opposite of what he had been lobbing for in another context. There was also a paper that Peter White had written on some aspect of trial design which was completely disregarded in PACE. (Sorry, I can’t remember the details without looking them up but I’m sure someone will.) My impression is that in many cases it’s not that people don’t know how to do things properly, it’s more that doing things properly is not expedient.

 

This is one of the examples I was thinking of:

From X: “Simon Wessely co-authored article criticising NICE decision to prioritise long term outcomes in assessing treatments for ME/CFS. In response Barry et al cite statement by Wessely, Geralda et al which calls on NICE to prioritise longterm outcomes in assessing treatments.”

 

Something I was thinking about the other day, and how relevant it is to the growing calls to make mental and physical health equal. Of course no one who says that means it that way, that mental health care and research should follow the high standards of biomedical medicine, but of course it absolutely 100% should.

And although some would disagree, without much of a reason for it, but mental health care and research are massively, ridiculously lower quality, efficacy and reliability than, let's go with biomedical, medicine, and this is likely the main reason.

Especially since every single one of the breakthroughs on mental health have come out of biomedicine (or through social and technological progress) anyway, since the standards in psychosomatic care don't even allow to produce anything competent. Lower standards breed mediocrity, and sure enough we observe from the current standards not much above base mediocrity. Entirely as expected.

But I expect this to be rejected with lots of reasons that stomp around the low standards bush, because the goal isn't to implement parity between both, but to overrule the higher standards using the far lower ones from mental health. It's plainly obvious that the real idea behind it is to basically erase most disability support, to make it more restrictive than ever, as a short-term cost-reduction policy. One that is massively more expensive long term, but sadly it doesn't seem that more than a handful of people in the industry understand this, or at the very least almost no one who does understand it dares say it plainly.