Larger Sample Size Needed to Increase Reproducibility in Neuroscience Studies

[Indigophoton]

Summary: Researchers say typical neuroimaging studies generally utilize small sample sizes, making them a challenge for other neuroscientists to replicate.

Small sample sizes in studies using functional MRI to investigate brain connectivity and function are common in neuroscience, despite years of warnings that such studies likely lack sufficient statistical power. A new analysis reveals that task-based fMRI experiments involving typical sample sizes of about 30 participants are only modestly replicable. This means that independent efforts to repeat the experiments are as likely to challenge as to confirm the original results.

The study, reported in the journal Nature Communications Biology, also finds that task-based fMRI studies with sample sizes of up to 100 also fall short of being perfectly replicable.

Task-based fMRI studies track changes in blood oxygen levels in the brain while study subjects are engaged in cognitive tasks. The technique allows researchers to see which brain regions are recruited to perform specific tasks.

But those in the field of cognitive neuroscience have not agreed on specific standards for the design of task-based fMRI studies – in particular, how many study subjects are needed to ensure reliable findings. The new research aims to address this shortfall, researchers said.

The article, https://neurosciencenews.com/neuroscience-study-size-replication-9282

Abstract

Despite a growing body of research suggesting that task-based functional magnetic resonance imaging (fMRI) studies often suffer from a lack of statistical power due to too-small samples, the proliferation of such underpowered studies continues unabated. Using large independent samples across eleven tasks, we demonstrate the impact of sample size on replicability, assessed at different levels of analysis relevant to fMRI researchers. We find that the degree of replicability for typical sample sizes is modest and that sample sizes much larger than typical (e.g., N = 100) produce results that fall well short of perfectly replicable. Thus, our results join the existing line of work advocating for larger sample sizes. Moreover, because we test sample sizes over a fairly large range and use intuitive metrics of replicability, our hope is that our results are more understandable and convincing to researchers who may have found previous results advocating for larger samples inaccessible.

The paper, https://www.nature.com/articles/s42003-018-0073-z