An expert criticism on post-publication peer review platforms: the case of pubpeer, 2025, Tsatsakis et al

[forestglip]

An expert criticism on post-publication peer review platforms: the case of pubpeer

Spoiler: Authors

Tsatsakis, Aristidis; Aschner, Michael; Sarigiannis, Dimosthenis; Docea, Anca Oana; Rezaee, Ramin; Daghighi, Seyed Mojtaba; Svistunov, Andrey A.; Domingo, José L.; Abdollahi, Mohammad

Abstract
While traditional peer review offers advantages in academic publishing, it is often hampered by significant weaknesses, leading to frustration among many authors.

Scientific discoveries after publication depend on thorough discussions and critiques, making post-publication peer review (PPPR) an essential tool for identifying errors and encouraging authors to make necessary corrections. PPPR is defined as a critical, ongoing, and public review conducted by the broader scientific community once research findings are formally published. Its goal is to enable more academic experts to continuously examine, question, and validate the work, identifying potential flaws or strengths that might have been missed during the initial review. This ongoing dialogue promotes transparency and motivates authors to make necessary corrections.

Although the goal of PPPR is to enhance scientific integrity, the open nature of PPPR platforms makes them vulnerable to misuse. It can also be exploited to undermine colleagues, suppress differing viewpoints, or further personal or organizational interests. We also observe an increase in “hyper-skepticism,” which differs from constructive criticism, reflecting an overly critical mindset that focuses on doubt rather than fostering understanding.

To fully realize the benefits of PPPR and prevent misuse, the scientific community must build a more equitable and more responsible framework. Addressing these challenges requires a thoughtful strategy that integrates technological advancements, strengthens editorial policies, enhances transparency measures, and provides robust protections for good-faith scientific debate.

Web | DOI | PDF | DARU Journal of Pharmaceutical Sciences | Paywall

 

[forestglip]

I only read the abstract, but thought it might be kind of interesting since I would say S4ME's discussions often seem to fit the "post-publication peer review" idea.

 

[Jonathan Edwards]

To fully realize the benefits of PPPR and prevent misuse, the scientific community must build a more equitable and more responsible framework. Addressing these challenges requires a thoughtful strategy that integrates technological advancements, strengthens editorial policies, enhances transparency measures, and provides robust protections for good-faith scientific debate.

This is the same old utopian view. Thought police are bad, we need better thought police.
Get over it and let people say what they are going to say - but yes, in public, with names.
There are. no robust protections in a dog eat dog world.

 

[Utsikt]

We also observe an increase in “hyper-skepticism,” which differs from constructive criticism, reflecting an overly critical mindset that focuses on doubt rather than fostering understanding.

If the authors left room for doubt - that’s on them. Try to do better next time.

 

[forestglip]

If the authors left room for doubt - that’s on them. Try to do better next time.

While I'm all for free speech and being able to freely point out issues with papers, I just want to at least acknowledge that PubPeer can at least theoretically be weaponized against authors.

Moderators on PubPeer don't thoroughly check every comment for accuracy. So if someone had a grudge or something, it would not be too hard to give an author a major headache, picking apart tiny details of all their papers, all from different anonymous accounts. Even if the authors took the time to debunk every single point posted, the discussions still have the potential to bias casual readers who don't fully understand the arguments into thinking that maybe there is something to all the criticisms.

And I imagine that if you fed nearly any paper into an AI and asked it to find a bunch of possible issues with the paper, it would give you plenty of accurate, even if very minor, little criticisms one could post just to instill a bit of doubt about the paper.

This all might be prevented if PubPeer wasn't anonymous, as Jonathan suggested. [Edit: But then should any criticism posted on S4ME only be done from non-anonymous accounts as well?]

Whether anything like the above actually even happens, I don't know. But it seems easy enough to do for someone that wants to discredit a scientist, so I think it's worth keeping in mind.

 

[Jonathan Edwards]

[Edit: But then should any criticism posted on S4ME only be done from non-anonymous accounts as well?]

Probably.

 

[Yann04]

And I imagine that if you fed nearly any paper into an AI and asked it to find a bunch of possible issues with the paper, it would give you plenty of accurate, even if very minor, little criticisms one could post just to instill a bit of doubt about the paper.

I see the problem with harassment or idk using this for vaccine scepticism or something.

But also I think the sciences are way too far on the side of not enough scepticism-epistemic humility.

So I think extra could be a good thing.

 

[forestglip]

Probably.

The thing is, unless governments make laws saying you can't criticize a study on the internet unless you have uploaded an ID, which sounds ridiculous to me, it can't practically be achieved.

If S4ME and Pubpeer take the initiative and somehow require your real name, then people will just start a new website where you can post anonymously again, since many people care about privacy.

So it seems the goal should not be trying to eliminate anonymous speech, but making sure that the places that do allow it have moderation that is competent and has the resources to try to prevent blatant smear campaigns.

Even while I think the limitations I mentioned should be considered as a potential and significant cost, I think having ways to criticize harmful research as easily as possible wins out.

 

[Jonathan Edwards]

If S4ME and Pubpeer take the initiative and somehow require your real name, then people will just start a new website where you can post anonymously again, since many people care about privacy.

I don't know anything about Pubpeer. Maybe I should look it up. But I think there is a difference between making comments on S4ME, which are not directly appended to a publication, and post publication peer review spaces that are.

I would like to see all publications come from institutions, not journals. That way the institution has more interest in ensuring quality at home and nobody makes a profit out of research. I think open comment sites should be attached but that commenters should give their real name. If people are overcritical the authors can then respond by saying so, and score points in the process. If peer reviewers had to give their naes they would not be able to get away with the bullshit they do now. Third parties could also chip in. This happened for the ritux phase 2 from F & M. Somebody put up a rubbish comment and I pointed out that it was rubbish. Everybody now knows that person makes ill-informed comments.

I can understand people here not wanting to give real names but I am a bit ambivalent even here about being critical without saying who you are. I never use a pseudonym. I find it very confusing when I get emails or see public statements from people who I also know as some avatar or other. Fortunately, quite a few of you tell me who you are in private.

 

[Jonathan Edwards]

Pubpeer seems to be more about manipulated images and fraud. I think it is reasonable to have an open site where concerns like that can be raised anonymously. But authors would have full right of reply. If an image was not manipulated you can simply say "the image was not manipulated".

 

[forestglip]

I don't know anything about Pubpeer. Maybe I should look it up. But I think there is a difference between making comments on S4ME, which are not directly appended to a publication, and post publication peer review spaces that are.

I think Pubpeer is kind of an in-between. It's considered fairly reputable because a large amount of comments about image manipulation and other issues have been posted about on there and led to retractions. But it's not officially attached to any journal, just a non-profit.

While it might be most known for things like image manipulation, any types of comments about a paper, even positive ones, are allowed (within some boundaries like only claims based on facts, no allegations of misconduct).

I think a comment I posted there played some part in the retraction of a paper we discussed here. Though I also emailed the journal, but I suspect the potential negative publicity of PubPeer motivates them more than an email.

People can install an extension on their browser so that whenever they're on the PubMed or journal page of a paper, it adds a box saying that this paper has however many comments on PubPeer.

 

[forestglip]

But authors would have full right of reply. If an image was not manipulated you can simply say "the image was not manipulated".

Absolutely. Authors often reply, and sometimes get into spirited debates.

 

[forestglip]

I would like to see all publications come from institutions, not journals. That way the institution has more interest in ensuring quality at home and nobody makes a profit out of research.

It's a very strange system we currently have. The authors doing the science have to pay to publish. And the people reading the science have to pay. The peer-reviewers don't get paid for their help making money for the publisher.

I don't know what value a publisher adds that can't be provided by a public university that is more interested in the goal of making scientific knowledge accessible.

 

[Utsikt]

While I'm all for free speech and being able to freely point out issues with papers, I just want to at least acknowledge that PubPeer can at least theoretically be weaponized against authors.

Sure, but that’s true of any system open to the public. And as long as the publication system allows so much garbage being published, even in the so-called high quality journals, they are hardly in a position to complain about garbage comments.

I’m no for any abuse of the system, but I’m also not very sympathetic to the people in charge of it.

[Edit: But then should any criticism posted on S4ME only be done from non-anonymous accounts as well?]

Probably.

I think this could be problematic for patients that receive as much abuse as pwME/CFS. I can envision a world where someone gets denied benefits based on their online activity.

A Norwegian patient was thrown out of Landmark’s LP study after being accepted initially, based om having been critical about the study and the intervention publicly. She was committed to trying the method and doing her best.

According to some involved in the study, she had been active on social media over time "criticizing a biopsychosocial understanding of ME and of professionals with such an understanding," wrote head of department and research manager Magne Arve Flaten.

He also pointed out that she had been critical of the study. And for such a treatment to be effective, motivation and hope were necessary.

In a letter to the research committee REK-Midt, which approved the study, Flaten wrote, among other things, that she "has opposed the study on social media."

If she joined the study, this would, among other things, "challenge" the quality of the work of the PhD candidate in the project, her study environment, and the safety of the other participants, Flaten wrote.

In the article in the University Newspaper, Lena Kjempengren-Vold asks, among other things, whether the participants are informed in advance that "they are being monitored and assessed with regard to their opinions about the study method, on social media and/or in other contexts?"

Magne Arve Flaten writes that it was not a requirement that one not be critical of the project. Regarding the claim that the participants' opinions were surveyed, he writes:

"It is unknown to me."

ME-pasient ble kastet ut av studie: — Har motarbeidet studien i sosiale medier

ME-syke Lena Kjempengren-Vold skulle egentlig være med i en kontroversiell ME-studie. Hun ble ekskludert blant annet fordi hun har ytret seg kritisk om studien.

www.khrono.no