Data is not available upon request Hussey

Data is not available upon request
Ian Hussey

Abstract

Many journals now require data sharing and require articles to include a Data Availability Statement. However, several studies over the past two decades have shown that promissory notes about data sharing are rarely abided by and that data is generally not available upon request. This has negative consequences for many essential aspects of scientific knowledge production, including independent verification of results, efficient secondary use of data, and knowledge synthesis. I assessed the prevalence of data sharing upon request in articles employing the Implicit Relational Assessment Procedure published within the last 5 years. Of 52 articles, 42% contained a Data Availability Statement, most of which stated that data was available upon request. This rose from 0% in 2018 to 100% in 2022, indicating a change in journals’ policies. However, only 27% of articles’ authors actually shared data. Among articles stating that data was available upon request, only 17% shared data upon request. The presence of Data Availability Statements was not associated with higher rates of data sharing (p = .55), indicating a lack of adherence with journals’ policies. Results replicate those found elsewhere: data is generally not available upon request, and promissory Data Availability Statements are typically not adhered to. Issues, causes, and implications are considered.

https://osf.io/preprints/psyarxiv/jbu9r_v1

 

Seems about right. How can it be fixed?

 

If I was a funder, and if (this is a big if) I was funding research with the goal of advancing societal knowledge about something (instead of, you know, legitimising my actions through dodgy research agreeing with me)

I would require every study I fund to open source anonymysed version of the data. In many studies, the raw data is often far more valuable than the conclusions in the manuscript.

 

Well, then, clearly they aren't actually required then. Because if they were actually required, then they couldn't not be available. As required. Ah screw it let's go to the dictionary:

OK, we have to go with compulsory, can probably skip over 'officially':

Oh, hum. We have a circular definition here. It's required because it's compulsory, and compulsory means it's required. But obligatory makes it pretty clear regardless.

OK this almost feels like the definition of recursive, but it remains very clear that it is not a choice, not an option.

Still, since they aren't compulsory, they clearly aren't required. Thus those requirements are fake, a lie. They are just paper requirements, giving the appearance of a rule, but unbothered by what the rule requires. Like we've seen with Cochrane, who can simply investigate themselves about following their own rules, and can just as well decide that it doesn't matter, because no one can force them to follow their own rules.

This is a common problem in academia, where the meaning of words is supposed to matter, and yet they twist it and frame things according to the meaning of words that are almost never actually in effect, like in any other place, just like in politics.

Plus there's the added twist here where it can be 'compulsory', but as authors see fit, as the PACE authors have done. They are technically 'required' to provide the data, but they can decide not to provide them if asked, based on their own preferences. So, in effect: not required, zero enforcement here.

Rules are worthless in themselves, they do not enforce themselves. Enforcement of rules matters 100x more than the word or spirit of those rules. In fact, rules are pretty much meaningless unless they are enforced, and that means, requires, consequences and compulsion, enforcement. Actually it shouldn't even be up to the authors themselves to decide that, that's just a silly system built on virtue signaling more than anything.

 

Also I might add that this is a problem that open source software has long fully solved. All of it. The legal frameworks, terms and conditions, the technical details, the security and safety mechanisms, the tools and protocols, all of it.

This problem has been fully solved on a technical basis. What remains is human factors: writing rules that make them appear good and professional, but with no intention of ever applying those rules to themselves.

 

I guess they want citations for the strong claims in their abstract, not have people look through the data and debunk their work.

 

It's a pity the manuscript doesn't discuss FOI or Alem's effort at all.

 

That would have been a great case study to highlight the cost of not sharing the data..

 

Erm. Yes and no. Visibility is way ahead of other areas but the amount of community and legal tangles and disputes that crop up in the world of open source software and licenses is not insignificant.

I guess that comes under the people part of what you say rather than the technical though! And I agree with the sentiment, just don’t want people who aren’t familiar to believe there is a magical panacea.