Background Emergency department (ED) crowding strains patient care and drives up costs. Early decisions on the need for patient hospital admissions can allow for better planning and potentially improve throughput and alleviate crowding. We sought to prospectively compare nurse predictions with a machine learning (ML) model for hospital admissions and to evaluate whether adding the nurse prediction to ML outputs enhances predictive performance.

Methods In this prospective, observational study at six hospitals in a large mixed quarternary/community ED system (annual ED census ∼500,000), triage nurses recorded a binary admission prediction for adult patients. These predictions were compared with an ensemble ML model (XGBoost + Bio-Clinical BERT) trained on structured data (demographics, vital signs, medical history) and triage text. Nurse predictions were similarly analyzed and then integrated with the ML output to assess for improvement in predictive accuracy.

Results The ensemble ML model (XGBoost + Bio-Clinical BERT) was trained on 1.8 million historical ED visits (January 2019–December 2023). It was then tested on 46,912 prospective ED visits with recorded nurse predictions (September to October 2024). In the prospective arm, nurse predictions yielded an accuracy of 81.6% (95% CI, 81.3–81.9), sensitivity of 64.8% (63.7–65.8), and specificity of 85.7% (85.3–86.0). At a 0.30 probability threshold, the ML model attained an accuracy of 85.4% (85.0–85.7) and sensitivity of 70.8% (69.8–71.7). Combining nurse predictions with the ML output did not improve accuracy beyond the model alone.

Conclusions Machine learning–based predictions outperformed triage nurse estimates for hospital admissions. These findings suggest that an admission prediction system anchored by ML can perform reliably using data available at triage.

The authors have declared no competing interest.

https://icahn.mssm.edu/about/departments-offices/ai-human-health

This work was supported in part through the computational and data resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai and supported by the Clinical and Translational Science Awards (CTSA) grant UL1TR004419 from the National Center for Advancing Translational Sciences. Research reported in this publication was also supported by the Office of Research Infrastructure of the National Institutes of Health under award number S10OD026880 and S10OD030463. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funders played no role in study design, data collection, analysis and interpretation of data, or the writing of this manuscript.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The Icahn School of Medicine of Mount Sinai Institutional Review Board gave ethical approval for this work. (IRB-18-00573-MODCR001)

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes