Microsoft's AI based diagnosis system

[Adrian]

https://microsoft.ai/new/the-path-to-medical-superintelligence/

The Microsoft AI team shares research that demonstrates how AI can sequentially investigate and solve medicine’s most complex diagnostic challenges—cases that expert physicians struggle to answer.

Benchmarked against real-world case records published each week in the New England Journal of Medicine, we show that the Microsoft AI Diagnostic Orchestrator (MAI-DxO) correctly diagnoses up to 85% of NEJM case proceedings, a rate more than four times higher than a group of experienced physicians. MAI-DxO also gets to the correct diagnosis more cost-effectively than physicians.

Paper: https://arxiv.org/abs/2506.22405

Interesting article from Microsoft around AI based diagnosis claiming very good results in diagnosing and reducing the number of tests needed.

 

[Jesse]

Very interesting. Honestly it doesn't even surprise me based on my experiences with doctors.. they always seem incredibly biased, authoritarian and rushed.

Then again I do also wonder about the limitations, if any. They do write the following for example:

Of course, our research has important limitations. Although MAI-DxO excels at tackling the most complex diagnostic challenges, further testing is needed to assess its performance on more common, everyday presentations. Clinicians in our study worked without access to colleagues, textbooks, or even generative AI, which may feature in their normal clinical practice.  This was done to enable a fair comparison to raw human performance

 

[Adrian]

Very interesting. Honestly it doesn't even surprise me based on my experiences with doctors.. they always seem incredibly biased, authoritarian and rushed.

Then again I do also wonder about the limitations, if any. They do write the following for example:

Yes I would be very cautious about the details of claims in terms of performance but the results do suggest a good capability. I also wonder how such a capability could be used in healthcare. Some doctor just aren't very good and authoritarian so AI could provide protections - but such doctors wouldn't accept there use.

I also wonder if AI could be used in trials to help select patients (say diagnose that they are likely to have the target disease) and in this way help scale trials to more participants. (Or even assess harms during trials).

 

[alex3619]

https://microsoft.ai/new/the-path-to-medical-superintelligence/

Paper: https://arxiv.org/abs/2506.22405

Interesting article from Microsoft around AI based diagnosis claiming very good results in diagnosing and reducing the number of tests needed.

Pay attention to the cost savings. I have been predicting for years that AI will transform the medical system, most docs can be replaced by nurse practitioners at a fraction of the cost, though I think new qualifications will be created. This is in part corporatized medicine, but also insurance coverage. Up till now there have been too many unsettled questions regarding AI in medicine, but there will now be a push to resolve the issues and replace doctors with less trained people backed by AI. This might even be a good thing for ME, as the AI might be more willing to recognize issues. However it might also make things worse for us because it learns from literature that is already biased.