Blog: Neuroskeptic: Psychology, Neuroscience: Lacking in Individuality?

[Andy]

Psychology, Neuroscience: Lacking in Individuality?

In research on people, scientists are typically interested in the group data – the mean, median, and variance of a sample of people. But according to a provocative new paper out in PNAS, the statistics of a group can obscure the variability within individuals, over time.

The paper, from Aaron J. Fisher, John D. Medaglia, and Bertus F. Jeronimus, isn’t really making a new point. The pitfalls of generalizing from the group to the individual level have long been known – but these issues are typically discussed in the form of hypothetical scenarios or contrived examples. Fisher et al. show how these issues apply to real-world data.

http://blogs.discovermagazine.com/n...logy-neuroscience-individuality/#.Wy5jCKkh0i9

 

[alktipping]

stating the obvious their methods of gathering and analysing data have always been flawed . It makes me wonder why it takes four authors for this paper is it because of the equally flawed system that demands that academics publish papers regardless of the quality to keep income streams . damn I have only just realised it is a blog .

 

[TiredSam]

The comments are entertaining.

 

[Trish]

I tried to read the article, but I get about 2 paragraphs down and my screen suddenly fills with a big thing saying Science for the Curious, Space Medicine Environment. Every time I try it this happens. I don't think they want me to read it!

 

[Arnie Pye]

I tried to read the article, but I get about 2 paragraphs down and my screen suddenly fills with a big thing saying Science for the Curious, Space Medicine Environment. Every time I try it this happens. I don't think they want me to read it!

There is an x to close the picture with, top right.

 

[Trish]

There is an x to close the picture with, top right.

Thanks, @Arnie Pye. I'd forgotten I had zoomed in to make the print bigger and easier to read, so the x was off my screen. Once I zoomed it out again the x re-appeared like magic.

 

[Trish]

Here's the paper referred to in the blog.
http://www.pnas.org/content/early/2018/06/15/1711978115.long

Lack of group-to-individual generalizability is a threat to human subjects research
Aaron J. Fisher, John D. Medaglia, and Bertus F. Jeronimus
PNAS June 18, 2018. 201711978; published ahead of print June 18, 2018.

Significance
The current study quantified the degree to which group data are able to describe individual participants. We utilized intensive repeated-measures data—data that have been collected many times, across many individuals—to compare the distributions of bivariate correlations calculated within subjects vs. those calculated between subjects. Because the vast majority of social and medical science research aggregates across subjects, we aimed to assess how closely such aggregations reflect their constituent individuals.

We provide evidence that conclusions drawn from aggregated data may be worryingly imprecise. Specifically, the variance in individuals is up to four times larger than in groups. These data call for a focus on idiography and open science that may substantially alter best-practice guidelines in the medical and behavioral sciences.

Abstract
Only for ergodic processes will inferences based on group-level data generalize to individual experience or behavior. Because human social and psychological processes typically have an individually variable and time-varying nature, they are unlikely to be ergodic.

In this paper, six studies with a repeated-measure design were used for symmetric comparisons of interindividual and intraindividual variation.

Our results delineate the potential scope and impact of nonergodic data in human subjects research. Analyses across six samples (with 87–94 participants and an equal number of assessments per participant) showed some degree of agreement in central tendency estimates (mean) between groups and individuals across constructs and data collection paradigms. However, the variance around the expected value was two to four times larger within individuals than within groups.

This suggests that literatures in social and medical sciences may overestimate the accuracy of aggregated statistical estimates. This observation could have serious consequences for how we understand the consistency between group and individual correlations, and the generalizability of conclusions between domains.

Researchers should explicitly test for equivalence of processes at the individual and group level across the social and medical sciences.

I've split it into short paragraphs for easier reading.

The rest of the paper is paywalled.