Guidance for best practice for clinical trials, World Health Organisation, 2024

Thank you @Trish for wading through this document, do let us known when you find the caveat ‘except when such trials relate to ME/CFS, then all of these necessary methodological design principles can be thrown out of the window’.

 

Reposted here:

If blinding of an allocated trial intervention is not feasible (for example in trials of different types of patient management or surgical procedures), blinded or masked outcome assessment should be pursued for objectively determined outcomes, for example through use of a prospective randomized open-label blinded endpoint (PROBE) design (see also Section 2.1.9 ascertainment of outcomes).

This suggests some confusion. If an endpoint is truly objective then there is no need to blind or mask. I have never heard of PROBE but it sounds dubious. Usually these acronyms are there to weasel through dodgy practices by making them sound respectable.

Recommending GRADE isn't good.

All the right words seem to be there, but the impression given is that as long as everyone follows a mindless recipe all will be fine (and you can cut a few corners if you really have to).

 

So the new WHO policy is just turning the handle on half-understood concepts of method and no application of common sense. Par for the course.

 

That's not what objective means. Doing this does not making an outcome objective, this is ridiculous. It can slightly lower the bias involved, but even then it only can do that, it does not mean that it will, and only slightly.

Just the same way, let's imagine measuring the length of something. Let's say we don't have a standard measuring device and instead use a sample of guesstimators whose guesses are assessed by a blinded assessor. This does not make the guesstimates objective. Everyone involved in making the quoted statement above understands this. And yet they put this ridiculous notion that they can do that out of questionnaires that have layers and layers of biases and uncertainty baked into the process.

And in behavioral and psychological trials the literal aim of the interventions is to influence the participants' responses. So this completely negates any and all attempts at mitigating the influence of bias, because it consists of targeted bias. It's like building a submarine that can reach the Mariana trench, but has a single porthole with a mosquito net. It doesn't matter if the porthole is small.

There is also the usual problem where they talk of randomized clinical trials, used the acronym RCT, where the C is typically meant to stand for controlled. This only confuses the issue further, the ambiguity is everywhere. And they do this in what is supposed to be a clarifying document. Mercy.

As we know, and they know, the entire body of evidence-based medicine in clinical psychology abuses this. It's made of nothing but that, because the primary objectives always fail, and they always fall back onto secondary assessments. This is something that Cochrane abuses the hell out of. The entire body of evidence for psychosomatic medicine depends wholly on this, so this is common practice. They can write it down in black and white all they want, they exempt the contexts in which it's needed the most. Fake rules made from fake standards leading to fake results.

Another very problematic idea since many trials don't even reach statistical significance and the interventions are still recommended, this is common in psychological interventions, basically the norm. They can write it down all they want, they cannot not know that this is standard practice and that they won't be doing anything to change that.

It's laughable to have this while organizations like Cochrane simply insist on ignoring exactly that. For years all we've heard from the trialists is that they don't see any harm in their trials and clinics and that's final, any data outside of this context is invalid. Write it down all you want, those are fake rules because they are exempted in exactly the cases where it's needed the most.

None of this matters if the rules are not enforced as intended. But this invalidates all psychosomatic ideology and most evidence-based medicine so all they do is write it down and pretend like they're following any of this. This entire way of doing things is invalid, completely unreliable. It's like children play-acting being at work.

Nothing will change in the future, and none of the past errors will be recontextualized. This is a completely empty set of fantasy rules. Like a non-binding code of ethics.

 

I can't find any other way to read this but "the standard in clinical practice is very low, so let's have the same standard in research". Of course it lowers costs. And of course it makes results closer to real-world, where a scientific approach is not possible. This is exactly what you don't want. This makes as much sense as arguing to build semiconductors in standard rooms because they will not be used in clean rooms. Good grief they are abandoning all sense and reason.

One of the most basic standards of scientific experiments is for "all other things being equal", that you make sure that you are testing one and only one thing influences the outcome. But here they basically prefer the alternative standard of "you know what? let's just not bother with that and go with what we prefer the outcomes to be".

The death of expertise and reason rolled into one. It's not even novel either. It's literally abandoning the only standard that has ever reliably worked: the scientific method. They are literally advising to just not bother with any of that. Just wing it.

 

I wonder what's Cochrane's stance on this. "Just putting out there to look good, we don't intend and never meant to follow any of this"? It's for others? Even though basically no one will actually respect this, neither spirit nor word? What is good for, then? Just virtue signaling?

Would it be worth writing to Cochrane and Bastian about all the points they are explicitly ignoring with the exercise review? I know it wouldn't make any difference, but it would expose the hypocrisy and futility of this organization and their pretend standards. Just to put it out there in writing.

 

the main thing with blinding of course is if you have 50-50 controls-treated then there is no incentive to bumping scores (even if done 'unconsciously') because really the important thing should be treatment increase - [minus] controls increase, so bumping up the score is just as likely to be 'subtracted' from your outcome as added to it if you don't know who is who.

I'm not quite sure how you can claim this level of blindness however when there are likely to be other 'hallmarks' like a surgical scar or for those who've done robotic CBT it'll be really blinking obvious (oh I shouldn't whinge, phrase positively, certain programmed phrases). And of course the biggie is how incestuous some areas are and how dominant the hierarchy is - so how independent can people really be if:

- they are in the same clinic
- they are in the same hospital
- there is overlap of other staff
- they are at conferences or other small meetings which might be regular with other people from the same niche
- they all think the same way/have the same bias (hence the objective measures issue for certain areas because some seem to confuse 'saying the right thing' with 'being better')
- and then you add in the above and issues with perceived threats for the patient (these people will be reporting back and writing letters that will affect my employers and GP and access to things)... I think being in a visibly different place, with it being made clear there is no way things will be fed back to those delivering not just the trial but potentially their 'care' in such a way that would indeed 100% reassure (does that exist? given the risk = impact vs likelihood, and you can maybe try and reassure on likelihood but that impact bit... without big change to culture and not just laws/rules but being able to make sure they are implemented..)

 

Crikey @Trish what a herculean task! you're definitely not pacing. I downloaded it from the Cochrane thread, saw it was yet another single line space 70 odd page document to plough through and thought - hell no! Safety, safety, safety. The WHO have been highlighting this since 2004.

 

Please feel free to move this. I'm having a Grok day.

Grok, what are some of the worst mistakes made in clinical trials that reduce the reliability and rigor of results? [Not in any order.]

"The list is on point, but daunting all together. Experienced academics keep these in mind when reviewing grant applications and research for publication, as well as when designing studies. But it's a reminder, and good to know for those trying to understand how to assess the validity of results..."

-Inadequate sample size
-Poor randomization
-Lack of blinding (single, double, or triple)
-Selection bias in participant recruitment
-Failure to define clear inclusion/exclusion criteria
-Insufficient statistical power
-Improper control group selection
-Non-standardized intervention protocols
-Inconsistent data collection methods
-Missing or incomplete data
-Failure to account for dropouts or withdrawals
-Inadequate follow-up duration
-Overreliance on surrogate endpoints
-Cherry-picking outcome measures
-Post-hoc changes to study design or hypotheses
-Lack of pre-registration of the trial
-Publication bias (not reporting negative results)
-Data dredging or p-hacking
-Inappropriate statistical analysis
-Confounding variables not controlled
-Lack of replication or validation studies
-Poorly defined primary endpoints
-Inadequate training of study personnel
-Non-compliance with ethical standards
-Falsification or fabrication of data -Insufficient monitoring or oversight
-Failure to adjust for multiple comparisons -Overgeneralization of results -Ignoring adverse events or side effects
-Inadequate reporting of methodology
-Conflict of interest not disclosed
-Use of unreliable or unvalidated measurement tools
-Improper handling of outliers
-Lack of transparency in data sharing
-Misinterpretation of correlation as causation
-Inconsistent dosing or treatment schedules
-Failure to account for placebo effects -Inadequate baseline assessments
-Ignoring patient heterogeneity
-Premature termination of the trial"

 

This is one exam on which no one should want a passing grade.

Our psychobehavioral overlords easily get top grades. May in fact often get perfect marks.

And we're wrong for pointing it out. Ugh. I hate being early on the right side of history so much. It just sucks.