A Columbia Surgeon’s Study Was Pulled. He Kept Publishing Flawed Data.

Good article—features Sholto David.

https://www.nytimes.com/2024/02/15/...e_code=1.Vk0.jI3c.bWPZPrIWtWE5&smid=url-share

 

Ha! If only they did. As we've seen, many won't even correct obvious mistakes once pointed out, because it's only when they correct that it makes them look bad. In cases like Cochrane, they will even double down on publishing something harmful again, renewing its influence, then work to stall even after they admit that what they published is flawed. This is far far below the bare minimum, it's dysfunctional to the point where the system simply cannot be trusted as it is.

AIs capable of doing reviews of scientific papers are coming soon. Maybe as soon as this year. Once ready, it will be trivial to basically review the entire academic literature. All of it. The results will be shocking. Even more shocking will be how obvious many of the flaws are and how they are commonly defended even after they're caught.

In a way this is kind of to be expected. The idea of peer review is nonsensical when you consider that all important human behavior requires oversight. Peer review is explicitly not oversight. Such a system can work out, in software development we commonly do code reviews, but the way it's done is explicitly broken in academic peer review. And that's on top of the for-profit paywalled system.