Reliability of psychiatric diagnoses in the 21st century, 2026, Boberg et al.

[Chandelier]

Reliability of psychiatric diagnoses in the 21st century

Spoiler: Authors

Boberg, Mateo; Henriksen, Mads Gram; Berge, Jonas; Fascendini, Nikolas; Jandl, Martin; Karakuła-Juchnowicz, Hanna; Madeira, Luís; Rossi Grauenfels, Daniele; Soloey-Nilsen, Hedda; Giurgiuca, Ana; Studart, Igor; Messas, Guilherme; Ojeda Uribe, Tamara; Varela, Luis; Corral, Ricardo; Cermolacce, Michel; Gozé, Tudi; Franzen, Léon; Borgwardt, Stefan; Huarcaya-Victoria, Jeff; Karkala, Konstantina; Mazaliauskiene, Ramune; Eberhard, Jonas; Schmidt, Troels; Micoulaud-Franchi, Jean-Arthur; Skodlar, Borut; Nordgaard, Julie

Abstract​

Background:
Reliable diagnosis is a cornerstone of medical practice, yet concerns about diagnostic reliability have long haunted psychiatry.
In the 1970s, the US–UK Diagnostic Project revealed striking international discrepancies in diagnostic practice.
Although operationalized diagnostic criteria were introduced to improve reliability, contemporary evidence indicates that substantial problems persist.
To investigate this, we conducted a large-scale international study of diagnostic reliability among medical doctors using standardized clinical case vignettes.

Material and methods:
In this cross−sectional study, medical doctors working in adult psychiatry across 19 countries from two continents assigned ICD−10 diagnoses to written cases.
Each doctor was randomly assigned two of nine available cases. Inter−rater reliability among medical doctors was assessed using Krippendorff’s α.
Diagnostic variability at case-level was quantified using Shannon entropy.
We also calculated diagnostic accuracy as concordance with predefined best-estimate reference diagnoses.

Results:
A total of 1,038 medical doctors provided 1,902 diagnostic assessments across nine cases.
Inter−rater reliability among medical doctors was modest (Krippendorff’s α = 0.48), indicating that diagnostic agreement between two doctors occurred in roughly 55% of cases.
Overall diagnostic accuracy was 66%, with the lowest accuracy observed for cases depicting schizophrenia spectrum disorders.
Diagnostic disagreements followed systematic patterns, with schizophrenia cases frequently misclassified as OCD, personality disorder, or bipolar disorder.

Conclusions:
Diagnostic inter−rater reliability among medical doctors remains limited, even under standardized conditions.
The observed patterns of misclassification suggest that the variability in diagnostic assessments reflects systematic differences in clinical interpretation rather than random error, pointing to challenges in assessing psychopathology as well as ambiguities in psychiatric classification.


Web | DOI | PMC | PDF | Frontiers in Psychiatry | Open Access

 

[Chandelier]

New study: Psychiatrists agree on a diagnosis in only 55% of cases

When psychiatrists assess the same patient, they agree on the diagnosis in only about half of cases, according to a new study from the University of Copenhagen. The findings raise concerns about potential misdiagnosis, inappropriate treatment, and errors in psychiatric research.

news.ku.dk

---

New study: Psychiatrists agree on a diagnosis in only 55% of cases​

When psychiatrists assess the same patient, they agree on the diagnosis in only about half of cases, according to a new study from the University of Copenhagen.​

The findings raise concerns about potential misdiagnosis, inappropriate treatment, and errors in psychiatric research.​

---

 

[rvallee]

Overall diagnostic accuracy was 66%

Funny that they still think this is a valid thing to say considering what the paper is about. It's like reporting votes total from an election that was acknowledged to have been rigged.

We also calculated diagnostic accuracy as concordance with predefined best-estimate reference diagnoses.

This is the paper I mentioned was discussed on reddit this week, where several clinical psychologists and psychiatrists pitched in saying how the DSM is mostly a vague playbook where almost everything overlaps and that they can never really be accurate anyway, the best they can do is wing it and work very long term with the patients. In fact many said that diagnosis here is mostly meaningless.

They included anxiety, depression and somatoform diagnoses in those, which are the most generic and error-prone concepts in all the professions. So even the assessment of the accuracy is itself inaccurate. How ironic. It makes a total mockery of the idea of academic being self-correcting. It actually resists self-correction, as much as corporate monopolies do, if not more, because they don't even have to use the power of the state to do it, they are effectively the state in this context. They are the authority, unchecked, unaccountable and insulated from all consequences, but they never feel the power they hold, so they wield it recklessly without even knowing it. Here it's actually darkly funny that the fact that they agree is actually a problem, because the concepts are so flimsy that it makes them even more suspicious. They all agree that the emperor's robe has a furry trim, somehow.

This is what makes psychosomatic ideology criminal. It's as criminal as looking at decades of millions of people being sentenced to prison, some to death, based on polygraphs, despite knowing that there is no such thing as a lie detector. Fortunately this did not happen, but forensic pseudoscience has been used this way to sentence many innocent people, just not millions.

It's all known to be very inaccurate at best, yet the certainty of the conversion disorder is held as absolute enough to decide the fate of hundreds of millions with zero quality control, oversight or accountability. A totally illegitimate system for this basis alone, because no legitimate system would ever do such an evil thing.

 

[Sean]

Degree of diagnostic agreement (i.e. consensus) measures nothing of value if the validity of the diagnosis in not also factored in.

A 100% rate of agreement is not just useless if the underlying diagnosis is false, it is extremely dangerous.