Microsoft's AI diagnostic system correctly diagnosed 83% of complex cases from the New England Journal of Medicine, against 36% for generalist doctors with reference material and 22% for those without. The results are published in Science.
The system, MAI-DxO (Microsoft AI Diagnostic Orchestrator), coordinates several AI agents that simulate a panel of physicians. The process is sequential: it starts from the main symptom and must decide on its own which questions to ask and which tests to order, without receiving all symptoms, history and reports at once. The agents have distinct roles: they propose diagnoses, choose tests, challenge hypotheses and weigh costs.
Image: The-Swarm-Corporation — https://github.com/The-Swarm-Corporation/Open-MAI-Dx-Orchestrator
To measure this kind of reasoning, Microsoft built its own benchmark, SDBench, whose code is open source, while the case data is available under license and usage limits. The 2025 preliminary version had 304 cases.
On emergency department cases, drawn from a large database of admissions at Boston's Beth Israel Deaconess Medical Center, the system reached 80% against a doctor average of 47%. Cost, however, depends on configuration: in the most accurate version the estimated average cost is $5,431 per case, above the $4,725 of doctors. Capping spending cuts costs to roughly half or a little over a quarter, with slightly lower accuracy. These are estimated costs on simulated cases, not real healthcare spending.
The model matters more than the orchestration
The main results were obtained with OpenAI's GPT-5. The data analysis shows the orchestration's contribution is more limited than the headline suggests: with GPT-5 alone and a simple prompt, accuracy on complex cases is 78.9%, at an estimated cost of about $8,400 per case. MAI-DxO cuts cost by about $3,000 and raises accuracy by about four points. On emergency cases the difference between the model alone and the full system is minimal.
Orchestration helps more with weaker models, but the gap between weak and strong models remains wider than the advantage the system brings.
The comparison excludes specialists and is not a clinical trial
The comparison does not include specialists, who usually handle cases of this complexity. The system has only been tested on past cases, and it is unclear how it would perform in a real clinical setting. "Clinical validation is needed," admits Mustafa Suleyman, CEO of Microsoft AI. "It's a leap, but this is an important step along the path."
Microsoft will now evaluate MAI-DxO at Beth Israel Deaconess Medical Center, in a newly launched collaboration. Patients will receive standard care, and those who consent will also interact with the system in parallel, with its recommendations compared to those of doctors. The trial is expected to start "within a couple of months," but no precise date has been set.
Medical superintelligence
The study coincides with a Microsoft position paper on medical superintelligence, defined as the ability to outperform human-led healthcare systems in medical reasoning and coordination across the whole care pathway. Suleyman expects performance indistinguishable from humans in this area within roughly two years. Microsoft is also training a healthcare foundation model with the Mayo Clinic, to be released next year.
No comments yet. Be the first!