Construct Validation of Experimental Manipulations in Social Psychology: Current Practices and Recommendations for the Future (2020) Chester et al.

Cheshire · May 2, 2020

Abstract
Experimental manipulations in social psychologymust exhibit construct validity by influencing their intended psychological constructs. Yet how do experimenters in social psychology attempt to establish the construct validity of their manipulations?
Following a preregistered plan, we coded 348 experimental manipulations from the 2017 issues of the Journal of Personality and Social Psychology.
Representing a reliance upon ‘on the fly’ experimentation, the vast majority of these manipulations were created ad hoc for a given study and not previously validated prior to implementation.
A minority of manipulations had their construct validity evaluated by pilot testing prior to implementation or via a manipulation check. Of the manipulation checks administered, most were face-valid, single item self-reports and only a few met criteria for ‘true’ validation.
In aggregate, roughly two-fifths of manipulations relied solely on face validity. To the extent that they are representative of the field, these results suggest that best practices for validating
manipulations are not commonplace — a potential contributor to replicability issues.
These issues can be remedied by validating manipulations prior to implementation, using validated manipulation checks, standardizing manipulation protocols, estimating the size and duration of manipulations’ effects, and estimating each manipulation’s effects on multiple constructs within the target nomological network
Click to expand...

https://psyarxiv.com/t7ev9

Snowdrop · May 2, 2020

After all I've read at this point-- I'd also be concerned about the validity of the 'true validation'
to be honest.

Although throwing some doubt on validity of findings in this field is at least a start.

Log in or Sign up

Construct Validation of Experimental Manipulations in Social Psychology: Current Practices and Recommendations for the Future (2020) Chester et al.

Cheshire Moderator Staff Member

Snowdrop Senior Member (Voting Rights)

Share This Page

Log in or Sign up

Construct Validation of Experimental Manipulations in Social Psychology: Current Practices and Recommendations for the Future (2020) Chester et al.

Cheshire Moderator Staff Member

Snowdrop Senior Member (Voting Rights)

Share This Page

Useful Searches