Statistical power in clinical trials of interventions for mood, anxiety, and psychotic disorders, 2021, Bastiaansen et al

Full title: Statistical power in clinical trials of interventions for mood, anxiety, and psychotic disorders
Authors: Ymkje Anna de Vries, Robert A Schoevers, Julian P T Higgins, Marcus R Munafò, Jojanneke A Bastiaansen
Published: Psychological medicine, May 2022
Link: https://pubmed.ncbi.nlm.nih.gov/35588241/
Open access: https://www.cambridge.org/core/jour...ic-disorders/71E0669ABAA95D800BED81CA0CED4955


Background: Previous research has suggested that statistical power is suboptimal in many biomedical disciplines, but it is unclear whether power is better in trials for particular interventions, disorders, or outcome types. We therefore performed a detailed examination of power in trials of psychotherapy, pharmacotherapy, and complementary and alternative medicine (CAM) for mood, anxiety, and psychotic disorders.

Methods: We extracted data from the Cochrane Database of Systematic Reviews (Mental Health). We focused on continuous efficacy outcomes and estimated power to detect predetermined effect sizes (standardized mean difference [SMD] = 0.20-0.80, primary SMD = 0.40) and meta-analytic effect sizes (ESMA). We performed meta-regression to estimate the influence of including underpowered studies in meta-analyses.

Results: We included 256 reviews with 10 686 meta-analyses and 47 384 studies. Statistical power for continuous efficacy outcomes was very low across intervention and disorder types (overall median [IQR] power for SMD = 0.40: 0.32 [0.19-0.54]; for ESMA: 0.23 [0.09-0.58]), only reaching conventionally acceptable levels (80%) for SMD = 0.80. Median power to detect the ESMA was higher in treatment-as-usual (TAU)/waitlist-controlled (0.49-0.63) or placebo-controlled (0.12-0.38) trials than in trials comparing active treatments (0.07-0.13). Adequately-powered studies produced smaller effect sizes than underpowered studies (B = -0.06, p ⩽ 0.001).

Conclusions: Power to detect both predetermined and meta-analytic effect sizes in psychiatric trials was low across all interventions and disorders examined. Consistent with the presence of reporting bias, underpowered studies produced larger effect sizes than adequately-powered studies. These results emphasize the need to increase sample sizes and to reduce reporting bias against studies reporting null results to improve the reliability of the published literature.

 

I'm generally suspicious of systematic reviews reviewing systematic reviews since, you know, systematic reviews are unreliable. But this is a general trend that is being confirmed by multiple reviews, is in line with what the NICE committee found, as the psychosocial evidence base on ME is built from this formulaic model.

Basically everything is underpowered, effect sizes are directly related to poor methodology, there is massive amount of bias and frankly none of this research should ever be used to make decisions for another human being, let alone coercively imposed onto people who reject consent.

I'm trying to look if it used ME-related studies but it's a bit hard to find the data. It only used Cochrane as source so it should be easy to check.