A groundbreaking Harvard study has found that artificial intelligence systems outperformed human doctors in high-pressure emergency medicine triage, diagnosing more accurately in potentially life-and-death moments when patients are first rushed to hospital. The results, published in the journal Science, were described by independent experts as showing a genuine step forward in the clinical reasoning of AIs.
Study Details and Results
One experiment focused on 76 patients arriving at the emergency room of a Boston hospital. An AI and two human doctors each received the same standard electronic health record, including vital signs, demographic information, and a brief nurse note. The AI identified the exact or very close diagnosis in 67% of cases, beating the human doctors, who were correct only 50% to 55% of the time. The AI's advantage was particularly pronounced in triage situations requiring rapid decisions with minimal information.
When more detail was available, the AI's accuracy rose to 82%, compared with 70% to 79% for expert humans, though this difference was not statistically significant. Additionally, the AI outperformed a larger cohort of human doctors when asked to provide longer-term treatment plans, such as antibiotic regimes or end-of-life processes. The AI and 46 doctors examined five clinical case studies, with the computer scoring 89% compared with 34% for humans using conventional resources.
Expert Commentary
Dr. Adam Rodman, a lead author and physician at Boston's Beth Israel Deaconess Medical Center, said AI large language models are among the most impactful technologies in decades. Over the next decade, AI will not replace physicians but join them in a new triadic care model involving the doctor, the patient, and an artificial intelligence system. Dr. Arjun Manrai, another lead author, emphasized that the findings do not mean AI replaces doctors but signify a profound change in technology that will reshape medicine.
In one case, a patient presented with a blood clot to the lungs and worsening symptoms. Human doctors thought anticoagulants were failing, but the AI noticed the patient's history of lupus might be causing lung inflammation, which proved correct.
Current Usage and Concerns
Nearly one in five U.S. physicians already use AI to assist diagnosis, according to research published last month. In the UK, 16% of doctors use AI daily and 15% weekly, with clinical decision-making being a common application. However, concerns remain about AI error and liability risks. Dr. Rodman noted there is no formal framework for accountability, and patients ultimately want human guidance for life-or-death decisions.
Prof. Ewen Harrison, co-director of the University of Edinburgh's centre for medical informatics, said the study shows these systems are no longer just passing medical exams but are becoming useful second-opinion tools for clinicians. Dr. Wei Xing, an assistant professor at the University of Sheffield, cautioned that doctors may unconsciously defer to AI answers, a tendency that could grow with routine AI use. He also highlighted the lack of information about which patients the AI struggled with, such as elderly or non-English speakers, and stressed that the study does not demonstrate AI is safe for routine clinical use.
