Contrary to common belief, randomised controlled trials inevitably produce biased results

[Sly Saint]

"Much of the social and medical sciences depend on randomised control trials. But while this may be considered the foundational experimental method, a certain degree of bias inevitably arises in any trial; whether this is sample bias, selection bias, or measurement bias. This is important as the level of validity of a trial’s causal claims can be a matter of life or death. To Alexander Krauss, the scientific process is a complex human process, involving many actors required to take many unique decisions at many different stages, and so some degree of bias is unavoidable. This has implications for the reproducibility crisis, as variation between study outcomes becomes the norm, and one-to-one replication is not possible."

"Taken together, researchers, practitioners, and policymakers need to become better aware of the broader range of biases facing trials. Journals need to begin to require researchers to outline in detail the assumptions, biases, and limitations in their studies. If researchers do not report this crucial information, practitioners and citizens will have to just rely on information and warning labels provided by policymakers, biopharmaceutical companies, and the like implementing the tested policies and selling the tested treatments."

http://blogs.lse.ac.uk/impactofsoci...led-trials-inevitably-produce-biased-results/

"This blog post originally appeared under a different title on the Institute for New Economic Thinking blog. It is based on the author’s article, “Why all randomised controlled trials produce biased results”, published in the Annals of Medicine (DOI: 10.1080/07853890.2018.1453233)."

 

[Melanie]

a certain degree of bias inevitably arises in any trial; whether this is sample bias, selection bias, or measurement bias.

Fibromyalgia drugs are a good example of this. Cymbalta is only helping about 6% of Fibromyalgia patients, many have MORE pain using it, and is causing horrific withdraw resulting in lawsuits. How this drug ever became approved and marketed for Fibromyalgia is beyond me.

 

[Esther12]

Coyne does not like this author:

I think we discussed the paper this blog was based on, and the consensus was that there were problems with it. Can't remember the details now.

 

[duncan]

But while this may be considered the foundational experimental method, a certain degree of bias inevitably arises in any trial;

I cannot speak to whether bias is inevitable in RCTs, but I believe it does happen and is easy to do. All one has to do is manipulate or define a single variable that determines outcome. Three famous (notorious?) RCTs helped decide what abx therapy is appropriate for Lyme going on close to two decades now, and the variable was treatment duration. If that variable had been say, six months vs say, 90 days, the outcome might have been very different.

 

[Jonathan Edwards]

I think this guy is attacking a straw man. Of course bias will slip in anywhere it can but you get less bias if you control, less if you randomise, less if you blind and even less if you double blind. Every trainee should know all this. I forget what alternatives he regurgitates but they are guaranteed to be less good.