A machine-learning model that used only information from the first 24 hours after intensive care admission retained a useful ability to sort adult neurocritical-care patients by their risk of dying in hospital when it was tested in a separate database. But the warning was far from a verdict: in the external test, the model's positive predictive value was 30.3%, and about 70% of patients it flagged as high risk survived to discharge.
The LightGBM model's AUROC was 0.801 in the eICU-CRD external cohort. AUROC is a measure of how well a model ranks patients who die above those who survive; the 95% confidence interval, a statistical range around the estimate, was 0.779 to 0.824. At the prespecified cutoff, sensitivity was 72.8% and specificity was 73.9%. Positive predictive value is the share of high-risk flags followed by death, and that share was 30.3%.
That combination makes the model better suited to prompting a closer look than to delivering a final prognosis. The analysis was retrospective, and its decision-curve results were exploratory rather than evidence that using the score improves clinical outcomes. A high-risk label therefore needs to be read as a risk signal, not as a stand-alone prognosis.
A narrow window onto a complex illness
The researchers included adults aged 18 or older who were having their first ICU stay, had an ICD-defined neurocritical-care diagnosis, stayed in the ICU for at least 24 hours, and had a known hospital outcome. The target was in-hospital mortality after ICU admission.
Only information from the opening 24 hours could enter the model. The candidate inputs included demographics, comorbidities, vital signs, laboratory results, neurological status, severity scores and early ICU therapies. The treatment variables represented the patient's condition and decisions before the prediction; they were not treated as causal explanations for death.
Before modeling, the researchers harmonized variables across the two databases and treated implausible values as missing. They fitted multivariate imputation on the MIMIC-IV training data only, then applied the resulting approach without refitting to the internal test set and the external cohort.
Built and tested in two stages
MIMIC-IV included 6,842 patients and 1,014 in-hospital deaths, an event rate of 14.8%. An 8:2 stratified split put 5,474 patients in development, with approximately 811 deaths, and 1,368 in the internal test set, with 203 deaths. eICU-CRD supplied 5,236 patients and 686 deaths for external validation, an event rate of 13.1%.
After cross-database harmonization, 64 candidate variables were retained and LASSO selected 27 predictors in the development cohort. Six candidate algorithms were compared, hyperparameters were tuned by cross-validation, and class weighting addressed the imbalance between deaths and survivors. The operating threshold was fixed in development and applied unchanged to both test cohorts.
The internal test was held out from model development within MIMIC-IV. The external test used eICU-CRD with the same predictor definitions and preprocessing wherever possible, without refitting. That design asks a more demanding question than whether the model can perform on the data used to build it: does it still separate risk in another database?
Performance held up, with an important catch
In the held-out MIMIC-IV test cohort, LightGBM had an AUROC of 0.841, with a 95% confidence interval of 0.821 to 0.861. Its AUPRC, a measure that focuses on how often positive flags are correct, was 0.518, with a 95% confidence interval of 0.486 to 0.552. The study reported this as the strongest discrimination among the candidate models in that internal test.
In eICU-CRD, the model's AUROC was 0.801 and its AUPRC was 0.413, with confidence intervals of 0.779 to 0.824 and 0.380 to 0.449. At the fixed threshold, its negative predictive value was 94.7%, while its positive predictive value was 30.3%; its F1 score was 0.428. The external result suggests the model was more reliable at identifying lower-risk patients than at confirming which high-risk patients would die.
What drove the predictions
To make the system easier to inspect, the researchers used SHAP, an attribution method that estimates which inputs most influenced a model's predictions. The leading contributors included total GCS score, invasive mechanical ventilation, age, lactate, SOFA score, vasopressor use, creatinine, respiratory rate, mean arterial pressure and oxygen saturation. Together, these signals represented neurological severity, respiratory support, age, metabolic stress, organ dysfunction and circulatory support.
Those features should not be read as independent causes of death. In particular, SOFA overlaps with several component measures, so the SHAP contribution assigned to SOFA and related variables reflects how the model used correlated information, not a separate causal effect for each variable.
Why a validation result is not a clinical outcome
External calibration was also assessed. Calibration asks whether predicted risks line up with observed outcomes; the external Brier score was 0.101, the calibration intercept was 0.050 and the slope was 0.84. Decision-curve analysis showed positive net benefit across an approximate threshold range of 0.11 to 0.51, but the curves were exploratory and did not establish clinical benefit.
Several features of the data limit how far the result can be taken. The study was retrospective, and the neurocritical-care diagnoses were identified from ICD codes. The primary outcome also could not consistently distinguish deaths after withdrawal or limitation of life-sustaining treatment or comfort-care transitions from deaths despite continued life-sustaining therapy.
The model was limited to a fixed first-day snapshot, so it did not incorporate later changes in the ICU course. It measured in-hospital mortality, not long-term neurological, cognitive, psychiatric or functional outcomes. The study therefore established database-level prediction performance, not that acting on the score improves outcomes.
For now, the result supports a narrower conclusion: the model can rank early mortality risk across two ICU databases, but its 30.3% positive predictive value means a high-risk flag still needs clinical context rather than being treated as a definitive prognosis.
Paper data and sources
Original title: Development and external validation of an interpretable machine learning model for early prediction of in-hospital mortality in neurocritical care
Authors: Cheng X, Ye J, Wei H
Journal/Repository: Frontiers in neurology
Status: Peer-reviewed
First online: 2026-10-09
DOI: Not available
Original paper · Full text