An idea for improving research quality

Creekside

Senior Member (Voting Rights)
The present system seems to be about winning points for publishing something that will make other people's papers look better by citing it, regardless of how bad the study actually is. Could a system be set up in which scientists win (desirable, career-benefiting) points for pointing out flaws or weaknesses in published studies? Reward people for actually reading papers and finding which ones are flawed.

Just an idea from an outsider to the research system.
 
I think you’d just find ways to abuse it. Because who’s going to check and determine if the correction is correct?
Well currently journal editors already determine that. Maybe the difference could be that the person who reported something to a journal which eventually led to a correction or retraction gets this linked to their name.

Although probably a good way to be ostracized by some in scientific community if you are known for causing these kinds of headaches to authors.

Edit: It incentivizes a type of adversarial landscape in science which doesn't sound too appealing, with authors knowing there are going to be people poring meticulously through every word looking for any possible error.
 
I think most of alternative science publication and rating models, of which tremendously many exist with different scientists having different preferences for alternative systems, often end up suffering from what some would call "Goodhart's law".

Reward people for actually reading papers and finding which ones are flawed.
I think this goes against the somewhat general consensus that there's no need to point out the flaws of what others have done in forms of publications, you should just do things better yourself (and flaws are typically discussed elsewhere, such as seminars). In your system you'd just end up having people write studies on nonsense studies instead of doing actual scientific work. Maybe people will just start publishing more nonsense collectively as it allows them to "earn points twice".

The problem about the current publication model is that people publish a tremendous amount of stuff but only very little has any influence on the scientific understanding of problems and that would only be worsened by the above system. You publish to publish. I think to some extent this has very much always been the case. A majority of scientists probably write nothing that needs publishing and good ones write a handful of papers that contribute something, whilst only a handful really have more than 20 different meaningful contributions.

I think it would be perfectly fine if every scientist was only able to publish 5 papers per year. The rest of their work can still be upload to repository servers or be published under pseudonyms as others have done in the past, so nothing would be lost.

I also don't think it makes sense to just equate all of science. There seems to be large differences between disciplines when it comes to the above topics with some having far lower standards than others and the implications for the disciplines being very different.
 
Back
Top Bottom