“How to Write a Peer Review: 12 things you need to know”

https://publons.com/blog/how-to-write-a-peer-review-12-things-you-need-to-know/

 

If only that was how it is done, considering some of the appalling methodology that gets through peer review I think it can be more like...

1. Check if the conclusion in line with your personal allegiances
2. Consider if it would be beneficial to be/stay on good terms with the author
3. Might the author one day review a paper or yours
4. Does the paper make any of your own research look bad
5. Is there any inconvenient data that the author didn't properly bury or obfuscate
6. Could the conclusion be sensationalised to more strongly support preexisting beliefs
7. If it generally serves your needs go ahead and rubber stamp your friends work

 

Possibly if peer review work was paid, more people might take more opportunities to do it. Currently there is an incentive to be inclined to do it if it might help you while turning down papers to review if the paper isn't likely to help you and your views/own research interests.

 

There were a few papers posted recently here that had peer review comments. For clinical psychology papers anyway, it doesn't appear that there is more than 15-30 minutes spent on the review. Some of the comments are borderline praise for the authors, clearly biased. A recent and particularly bad trial (the CBT music one I think) even had a comment saying the review didn't read beyond the abstract and it was 1 of 2 reviewers.

Beyond typos and a few vocabulary terms, it looks more like a cursory look over style, with little spent on substance, sometimes basically none. I doubt anyone other than the PACE authors actually read their papers in full, and I include the Lancet peer reviewers in that. It's just a skim over egregious language, basically zero attention to the underlying science or "science".

Maybe in serious fields there is a more genuine effort, but the voluntary peer review process is clearly inadequate and needs massive reform. It's obviously hard to judge the science of what are essentially opinion papers, but that's another flaw in science altogether.