No raw data, no science: another possible source of the reproducibility crisis, 2020, Miyakawa

[Andy]

A reproducibility crisis is a situation where many scientific studies cannot be reproduced. Inappropriate practices of science, such as HARKing, p-hacking, and selective reporting of positive results, have been suggested as causes of irreproducibility. In this editorial, I propose that a lack of raw data or data fabrication is another possible cause of irreproducibility.

As an Editor-in-Chief of Molecular Brain, I have handled 180 manuscripts since early 2017 and have made 41 editorial decisions categorized as “Revise before review,” requesting that the authors provide raw data. Surprisingly, among those 41 manuscripts, 21 were withdrawn without providing raw data, indicating that requiring raw data drove away more than half of the manuscripts. I rejected 19 out of the remaining 20 manuscripts because of insufficient raw data. Thus, more than 97% of the 41 manuscripts did not present the raw data supporting their results when requested by an editor, suggesting a possibility that the raw data did not exist from the beginning, at least in some portions of these cases.

Considering that any scientific study should be based on raw data, and that data storage space should no longer be a challenge, journals, in principle, should try to have their authors publicize raw data in a public database or journal site upon the publication of the paper to increase reproducibility of the published results and to increase public trust in science.

https://molecularbrain.biomedcentral.com/articles/10.1186/s13041-020-0552-2

 

[Daisymay]

Blimey, that is really staggering, shocking.

 

[Jonathan Edwards]

The message here is fair enough but the author may be a bit guilty of biased information too.

The author claims to be editor in chief of a journal that has had 180 manuscripts to review since early 2017 - roughly one a week. An established quality journal would probably get a minimum of 10 manuscripts a week. A premier journal might get 100+. So it looks rather as if this journal, which I expect nobody much has heard of, is tending towards the last chance saloon category for researchers.

If so it is not too surprising that the editor has been receiving a lot of non-existent studies. This may be unfair but the editorial would be a bit more weighty if it had come from Science or Blood or the Annals of Rheumatic Disease.

 

[Forbin]

On the other hand, "Molecular Brain" currently has an "impact factor 4.051 (2018-2019), which, according to this chart by the "Journal Citation Reports" (JCR) puts "Molecular Brain" in the top 11.4% of 12,298 journals evaluated, in terms of citations. It's highest impact factor was 4.902 in 2014, which put it in roughly the top 7% of the journals evaluated at the time. In the last three years, the number of articles they have published has decreased, while their total citations have increased, which might suggest greater selectivity.
https://www.bioxbio.com/journal/MOL-BRAIN

They may not get many submissions, but those they publish seem to get cited a fair amount.