Ideas for a Declaration to raise standards in evidence-based medicine

[rvallee]

Moved from
Have there been any high-profile critiques of open-label, subjective-measures BPS randomised trials? If not, why not?

I was thinking about this the other day and I think that if something like this is to happen, we'll have to lead the way. Or at least get the ball rolling. There are too many disincentives for professionals to get into anything this controversial, this is not a discipline that is capable of dealing with anything of this nature. Too weak and incapable of criticism. And, really, it's actually embarrassing to see just how weak this discipline is, every single bit of mild criticism is met with whines about "personal attacks" and so on. It's all style, no substance. About rhetoric, not science.

The best format I came up with is a Declaration, roughly similar in make-up to the Oslo Declaration where they say... not much. Same idea, just, you know, not clownish and wrong. Where we make a stand and demand that standards be raised significantly in evidence-based medicine, taking Long Covid as the prime example of how this set of methods is entirely useless on its own, having not only produced nothing useful in 5 years, but actually rehashing the same failed nonsense that we know doesn't work, a fact that bothers almost no one.

The crux of the matter is that this gravy train has been looping around on its track for decades, and has not produced a single useful thing. There is no sign of this changing, so there will be many more of those, even though we know they are completely useless. On conditions where this is the only method of knowledge, there has been complete stagnation. It basically adds nothing, precisely because the standards are simply far too low. The arguments are pretty straightforward, and even in the very best case, proponents of this failed system can't deny that their claims are highly controversial.

I doubt we'd get many academics and clinicians to sign on, but then it would be interesting to have this dynamic where a coalition of mostly patients demands high standards, while professionals petulantly insist that their crap standards are just fine, even though they produce absolutely nothing useful, represent textbook waste of funding.

Most importantly, we are right. They are wrong. It's as black and white as it gets. Worst case it eventually serves as major embarrassment to the profession once a few breakthroughs do to other conditions what happened for peptic ulcers, when all the psychobehavioral nonsense became obsolete, and has not been mentioned ever since.

Ah to hell with it, I'll put the rough draft I have so far below, see where it goes. My brain only works occasionally so I have a lot of work tweaking the text, but it roughly has most of the main ideas so at least it's a starting point.

 

[rvallee]

Very rough early draft of a Declaration to raise standards in evidence-based medicine:



We, a coalition of patients, health consumers, clinicians, researchers, scientists, patient associations and others, have for too long witnessed a major problem in how many health care decisions are made, particularly through the use of poor quality trials and biased models of illnesses and treatments.

The crisis of replication is long-known in psychology, but evidence-based medicine suffers from those flaws just the same. Thousands of very low quality and highly biased studies have already been funded, their findings published into papers. All for nothing. No good has come out of this process.

The worst of those flaws are in the pragmatic trial space, where the standards of high quality pharmaceutical trials are swapped for the worst combinations of the lowest methodological quality alongside excessive levels of bias.

More trials continue to be funded and published into papers, serving the interest of absolutely no one. Those trials do not give useful answers to the problems they claim to address, and not only is their quality not improving, over time it is actually getting worse, as the main lesson from decades of roundabout nothingness has clearly been the perfection of methods to achieve a particular answer in what can only be described as a race to the bottom.

This is all too evident in medicine’s growing failure to provide solutions to Long Covid. 5 years into an illness disabling tens of millions, not a single useful solution has been found, in large part because this method of assessing treatments simply does not work outside the narrow confines of drug trials, which can be both double-blinded and measured objectively.

Dozens of trials of very low quality and high levels of bias have been published so far, most of which feature treatments that have been put into widespread use from day one of Long Covid, because they have been mainstays of clinical practice for many years, despite lacking entirely in reliable evidence.

So it is not a surprise that we find ourselves, five years later, with widespread acknowledgement that there are no treatments for Long Covid, a statement that includes the aforementioned common treatments, while patients are routinely advised and referred to such treatments, despite a complete lack of evidence for their efficacy.

Such trials lack any feature that makes pharmaceutical trials reliable. They cannot be blinded. They only use subjective outcomes. The treatments themselves consist largely of trying to convince the patient they are actually healthy. The trials are generally run by the same people who made the content that gets delivered to patients.

This is clinical theater. It serves no purpose to anyone, and most of the blame lays on the evidence-based industry for producing meaningless recommendations out of largely, or even completely, unreliable evidence. Bad evidence cannot be polished into evidence, only in bad outcomes.

This problem has been growing for decades, especially in conditions where evidence-based methods are the exclusive method of knowledge. The main problem here is widespread and intransigent acceptance for very low quality work whose cumulative contribution is simply nil. It is evident that it’s not possible to produce good solutions to real life problems in those conditions, and this fact is evident by the complete absence of effective treatments in all medical conditions where evidence-based medicine is the main source of clinical knowledge.

Which spells very clearly that in itself, this way of producing clinical knowledge is entirely useless. It produces nothing useful on its own. All the usefulness out of evidence-based medicine comes from high quality studies, not from its review and analyses processes.

And yet they form the entirety of the evidence base for treating some conditions, are even used to retroactively infer causality, which pragmatic trials are not capable of doing. This situation is simply not acceptable, and it should be surprising to no one that medicine has not made any progress whatsoever in reliably delivering solutions to people suffering from those conditions.

This stalemate is the outcome of the poor standards in evidence-based medicine, and can no longer be tolerated. Poor quality trials with high levels of bias need to no longer be funded, they simply cannot be accepted any longer. The floor needs to be raised significantly higher in all aspects.

Clinical trials, whether they test pharmaceutical or non-pharmaceutical treatments alike, should be of the highest standards at all time. The outcome of accepting lower standards in non-pharmaceutical interventions has been a massive failure. We demand that those standards be raised significantly, so that low quality trials be retired for good.

 

[rvallee]

It would be helpful to then have a list of specific demands, those require a lot more thinking, but roughly:

1. Low quality clinical trials should no longer be funded
2. Trials should not involve the producers/inventors of what is being tried
3. Patient reports should be published unfiltered as supplementary information on any paper derived from studies they participated in, submitted pre-intervention, post-intervention, and in response to the paper itself
4. Harms should be reported unfiltered as well
5. Trials must involve unique combinations with relevant differences, ending the cycle of entirely redundant trials
6. Only outcomes relevant to the problem should be utilized

Ideas #3 and #4 about having patient reports is to end the standard where what patients say is completely distorted. Over the years we have seen numerous 'studies' where some academics talk to patients, or analyze their writing, and come up with themes and sentiment that mostly ignore what the patients actually say, and instead are simply used as voices to push the biased opinions of the researchers.

So papers about a trial or study would be obligated to publish participants' raw comments, unfiltered, and not available on request but submitted with the paper itself. Not prompted or responding to any questionnaire, just the patients' words as is. I'm pretty sure that if we have a good number of such unfiltered comments from participants of most psychobehavioral trials, they'd paint a very different picture of the fiction the authors wrote about.

 

[MeSci]

Just a couple of points, with the same underlying reason:

"3. Patient reports should be published unfiltered as supplementary information on any paper derived from studies they participated in, submitted pre-intervention, post-intervention, and in response to the paper itself
4. Harms should be reported unfiltered as well."

It is essential that participants are able to refuse to be identified in such reports.

 

[bobbler]

It would be helpful to then have a list of specific demands, those require a lot more thinking, but roughly:

1. Low quality clinical trials should no longer be funded
2. Trials should not involve the producers/inventors of what is being tried
3. Patient reports should be published unfiltered as supplementary information on any paper derived from studies they participated in, submitted pre-intervention, post-intervention, and in response to the paper itself
4. Harms should be reported unfiltered as well
5. Trials must involve unique combinations with relevant differences, ending the cycle of entirely redundant trials
6. Only outcomes relevant to the problem should be utilized

Ideas #3 and #4 about having patient reports is to end the standard where what patients say is completely distorted. Over the years we have seen numerous 'studies' where some academics talk to patients, or analyze their writing, and come up with themes and sentiment that mostly ignore what the patients actually say, and instead are simply used as voices to push the biased opinions of the researchers.

So papers about a trial or study would be obligated to publish participants' raw comments, unfiltered, and not available on request but submitted with the paper itself. Not prompted or responding to any questionnaire, just the patients' words as is. I'm pretty sure that if we have a good number of such unfiltered comments from participants of most psychobehavioral trials, they'd paint a very different picture of the fiction the authors wrote about.

Retrospectives that assume consent or can’t involve voluntary informed consent because they use people who were patients and are ‘blanket’ ie not detailing which research or methods or how they might just select certain people to report on. And that’s if they get the option to say no and still get treatment if they were even in treatment voluntarily.

I think there is an issue that if there are inpatient units that are doing ‘research’ or reporting then there should be proper external involvement. It feels like there’s a lot of cherrypicking from past notes.

I also worry about the rediagnosing of eg children (but perhaps slso adults) who get worse with ‘new conditions’ like pervasive refusal syndrome which sounds like a cover for more severe ME/CFS not being the ‘outcome’

but even if it weren’t then I think it’s a collateral damage approach where treatment that only could ever work on those misdiagnosed who didn’t have me/cfs being used as a pretend treatment for me/cfs knowing /thinking that ‘at least the ones who could be helped are’ and writing off it harming those with the actual illness they claim it treats as ‘well there’s no treatment anyway/lost cause’ or in other words so disposable that if they are made unnecessarily more severe it’s worth it for the milder ones they claim felt 10% better (when their freedom depended on them writing that on their form, treatment involved reframing their symptoms more positively etc)

there is a major issue with ‘the only way to escape more of this harmful treatment/this place/keep what you need to survive is to please us and say you are better/recovered’ coercion that is unfortunately implicit due to the way things are run rather unusually vs other illnesses with control over the entire system of medics and other services/school/employers rather than these being independent reporters of whether ‘the treatment’ has worked …

its a hellish situation where when a treatment doesn’t work/harms patients feel threatened with MORE of what harmed them . I don’t think it needs to be me/cfs or you need to have that condition to realise how that distorts facts and is a human rights issue.

 

[bobbler]

Just a couple of points, with the same underlying reason:

"3. Patient reports should be published unfiltered as supplementary information on any paper derived from studies they participated in, submitted pre-intervention, post-intervention, and in response to the paper itself
4. Harms should be reported unfiltered as well."

It is essential that participants are able to refuse to be identified in such reports.

I agree. I think the history of coercion and perceived threats means it would be impossible for many to report harms to those who caused them for obvious reasons unless a system of proper anonymity is agreed.

I think ideally somewhere under all this we have an issue that in biomedical medicine any specialist department must report their outcomes which include long-term ones.

and no part of those are either coerced or Pooh poohed as down to patient effort (although I don’t know in mental health in the Uk I suspect this is a department where this might be exempting itself or doing this differently)

we don’t currently have ANY clinicians, nevermind as would be normal clinicians (biomedical ones) making a regular assessment long-term responsible for people.

it’s simply not normal for the therapists or inventors and beneficiaries of a therapy to be ‘in charge of’ anything that will affect that patients future wellbeing, support, reputation etc

instead we’ve got these two things bunged together for me/cfs which isn’t far off if regs were changed so an inventor of a drug was allowed to be in charge of writing reports that affected a patients freedom, finance, employment, school attendance issues, GP’s perception often being used to deliver ‘trials’ or other types of research. And no one else checking any objective measures of whether the drug made them better or worse , just ‘what they answer in a non anonymised questionnaire’

there would be no triangulation in such an instance (because it would have what me/cfs has where each of those you joints are told ‘how that patient is’ by the person at the top of that tree) , rather than each of those points being points where data could show an issue ?

 

[Ravn]

In the current environment I fear any declaration or similar coming from us or from our usual advocates would make little impact. Most likely it would be brushed off as just another example of deluded activism and then simply ignored into oblivion

Maybe I'm too cynical but I think right now a more effective strategy might be to try and make the updated Declaration of Helsinki work for us

Some possible avenues:

Publish papers investigating how well research in the ME field complies with Helsinki (keeping it neutral, the facts speak for themselves), e.g.

• qualitative analysis of examples of good practice (like DecodeME), partially compliant examples (many in the biomedical line probably fit here) and examples of poor practice (most in the behavioural line)
• quantitative analysis, would show how flooded the field is with poor practice, to a degree that impedes the usual progress of science where the influence of poor papers is expected to fade as better evidence becomes available
• if none exists yet, develop & publish a tool to screen for Helsinki compliance (a screen to quickly identify where deeper questions need to be asked, not a simplistic add-up-a-score type tool)

Monitor for non-compliant papers being published and respond by highlighting the non-compliance, e.g.

• write to the publishing editor (in whatever format the journal allows)
• write to the funders (an accepted compliance screening tool could lend useful credibility here)
• write to the ethics board (ditto for screening tool)

Identify other groups interested in monitoring Helsinki compliance and work with them, make it broader than about ME because the problem is much more widespread

I'll copy this post across to the Helsinki declaration thread