Estimating prevalence of transparency & reproducibility-related research practices in psychology (2014-2017) (2020 preprint) Hardwicke Ioannadis et al

J

JohnTheJack

Moderator
Staff member
Abstract
Psychological science is navigating an unprecedented period of introspection about the credibility and utility of its research. A number of reform initiatives aimed at increasing adoption of transparency and reproducibility-related research practices appear to have been effective in specific contexts; however, their broader, collective impact amidst a wider discussion about research credibility and reproducibility is largely unknown.

In the present study, we estimated the prevalence of several transparency and reproducibility-related indicators in the psychology literature published between 2014-2017 by manually assessing these indicators in a random sample of 250 articles.

Over half of the articles we examined were publicly available (154/237, 65% [95% confidence interval, 59% to 71%]). However, sharing of important research resources such as materials (26/183, 14% [10% to 19%]), study protocols (0/188, 0% [0% to 1%]), raw data (4/188, 2% [1% to 4%]), and analysis scripts (1/188, 1% [0% to 1%]) was rare. Pre-registration was also uncommon (5/188, 3% [1% to 5%]).

Although many articles included a funding disclosure statement (142/228, 62% [56% to 69%]), conflict of interest disclosure statements were less common (88/228, 39% [32% to 45%]). Replication studies were rare (10/188, 5% [3% to 8%]) and few studies were included in systematic reviews (21/183, 11% [8% to 16%]) or meta-analyses (12/183, 7% [4% to 10%]).

Overall, the findings suggest that transparent and reproducibility-related research practices are far from routine in psychological science. Future studies can use the present findings as a baseline to assess progress towards increasing the credibility and utility of psychology research.
Click to expand...


Link: https://osf.io/preprints/metaarxiv/9sz2y/
DOI: 10.31222/osf.io/9sz2y
 
Psychological science is navigating an unprecedented period of introspection about the credibility and utility of its research
Click to expand...
If only that were true. There are pockets of sanity but overall the field is going at full speed away from any introspection. Which itself kinda makes the point, it's a bit lacking in introspection to pretend there is any meaningful introspection happening. It would be great if there were, just as it would be great if psychology actually behaved like a science, but that's aspirational, like most of psychological medicine.

The problem is even deeper than that, quite frankly, with stuff like FND and the weird goop of mind-body quackery that is encroaching every nook and cranny of medicine. It isn't just lack of transparency and reproducibility, there is outright quackery being promoted and essentially given as much effective power as the most reliable medical evidence with absolutely no accountability and the bar keeps getting lower to keep the pendulum of sanity from swinging back, to keep the escalation of commitment over absurd ideologies.

Real introspection would look at what the BPS model and its peripheral ideologies are doing and would call most of it straight quackery. When that happens there may be a shift from aspirational to practical introspection, but until then the field lacks the introspection necessary to even begin introspection into how well it's doing. Which, yeah, pretty ironic for the field that studies things like escalation of commitment, Dunning-Kruger and biases. Right now, recognition of the problem is about as significant as calling climate change a bit of a worry. The scale of the problem is much larger than even the few pockets of sanity recognize.

We are the canaries in the coal mine. And we are singing. And dying. But no one's listening. Not even those who say "watch for the canaries". Or most of them anyway. Introspect much further, please, our lives depend on it.
 
You must log in or register to reply here.
Back
Top Bottom