Skip to main content
Top
Published in: Respiratory Research 1/2022

Open Access 01-12-2022 | Influenza | Research

Machine learning-derived prediction of in-hospital mortality in patients with severe acute respiratory infection: analysis of claims data from the German-wide Helios hospital network

Authors: Johannes Leiner, Vincent Pellissier, Sebastian König, Sven Hohenstein, Laura Ueberham, Irit Nachtigall, Andreas Meier-Hellmann, Ralf Kuhlen, Gerhard Hindricks, Andreas Bollmann

Published in: Respiratory Research | Issue 1/2022

Login to get access

Abstract

Background

Severe acute respiratory infections (SARI) are the most common infectious causes of death. Previous work regarding mortality prediction models for SARI using machine learning (ML) algorithms that can be useful for both individual risk stratification and quality of care assessment is scarce. We aimed to develop reliable models for mortality prediction in SARI patients utilizing ML algorithms and compare its performances with a classic regression analysis approach.

Methods

Administrative data (dataset randomly split 75%/25% for model training/testing) from years 2016–2019 of 86 German Helios hospitals was retrospectively analyzed. Inpatient SARI cases were defined by ICD-codes J09-J22. Three ML algorithms were evaluated and its performance compared to generalized linear models (GLM) by computing receiver operating characteristic area under the curve (AUC) and area under the precision-recall curve (AUPRC).

Results

The dataset contained 241,988 inpatient SARI cases (75 years or older: 49%; male 56.2%). In-hospital mortality was 11.6%. AUC and AUPRC in the testing dataset were 0.83 and 0.372 for GLM, 0.831 and 0.384 for random forest (RF), 0.834 and 0.382 for single layer neural network (NNET) and 0.834 and 0.389 for extreme gradient boosting (XGBoost). Statistical comparison of ROC AUCs revealed a better performance of NNET and XGBoost as compared to GLM.

Conclusion

ML algorithms for predicting in-hospital mortality were trained and tested on a large real-world administrative dataset of SARI patients and showed good discriminatory performances. Broad application of our models in clinical routine practice can contribute to patients’ risk assessment and quality management.
Appendix
Available only for authorised users
Literature
1.
go back to reference SPRINT-SARI-Investigators: For the SPRINT-SARI-Investigators: using research to prepare for outbreaks of severe acute respiratory infection. BMJ Global Health 2019, 4:e001061. SPRINT-SARI-Investigators: For the SPRINT-SARI-Investigators: using research to prepare for outbreaks of severe acute respiratory infection. BMJ Global Health 2019, 4:e001061.
2.
go back to reference Fitzner J, Qasmieh S, Mounts AW, Alexander B, Besselaar T, Briand S, Brown C, Clark S, Dueger E, Gross D, et al. Revision of clinical case definitions: influenza-like illness and severe acute respiratory infection. Bull World Health Organ. 2018;96:122–8.CrossRef Fitzner J, Qasmieh S, Mounts AW, Alexander B, Besselaar T, Briand S, Brown C, Clark S, Dueger E, Gross D, et al. Revision of clinical case definitions: influenza-like illness and severe acute respiratory infection. Bull World Health Organ. 2018;96:122–8.CrossRef
3.
go back to reference Kumar A. Critically Ill patients with 2009 influenza A(H1N1) infection in Canada. JAMA. 1872;2009:302. Kumar A. Critically Ill patients with 2009 influenza A(H1N1) infection in Canada. JAMA. 1872;2009:302.
4.
go back to reference Martirosyan L, Paget WJ, Jorgensen P, Brown CS, Meerhoff TJ, Pereyaslov D, Mott JA. The community impact of the 2009 influenza pandemic in the WHO European Region: a comparison with historical seasonal data from 28 countries. BMC Infect Dis. 2012;12:36.CrossRef Martirosyan L, Paget WJ, Jorgensen P, Brown CS, Meerhoff TJ, Pereyaslov D, Mott JA. The community impact of the 2009 influenza pandemic in the WHO European Region: a comparison with historical seasonal data from 28 countries. BMC Infect Dis. 2012;12:36.CrossRef
5.
go back to reference Troeger C, Forouzanfar M, Rao PC, Khalil I, Brown A, Swartz S, Fullman N, Mosser J, Thompson RL, Reiner RC, et al. Estimates of the global, regional, and national morbidity, mortality, and aetiologies of lower respiratory tract infections in 195 countries: a systematic analysis for the Global Burden of Disease Study 2015. Lancet Infect Dis. 2017;17:1133–61.CrossRef Troeger C, Forouzanfar M, Rao PC, Khalil I, Brown A, Swartz S, Fullman N, Mosser J, Thompson RL, Reiner RC, et al. Estimates of the global, regional, and national morbidity, mortality, and aetiologies of lower respiratory tract infections in 195 countries: a systematic analysis for the Global Burden of Disease Study 2015. Lancet Infect Dis. 2017;17:1133–61.CrossRef
6.
go back to reference Murthy S, Archambault PM, Atique A, Carrier FM, Cheng MP, Codan C, Daneman N, Dechert W, Douglas S, Fiest KM, et al. Characteristics and outcomes of patients with COVID-19 admitted to hospital and intensive care in the first phase of the pandemic in Canada: a national cohort study. CMAJ Open. 2021;9:E181–8.CrossRef Murthy S, Archambault PM, Atique A, Carrier FM, Cheng MP, Codan C, Daneman N, Dechert W, Douglas S, Fiest KM, et al. Characteristics and outcomes of patients with COVID-19 admitted to hospital and intensive care in the first phase of the pandemic in Canada: a national cohort study. CMAJ Open. 2021;9:E181–8.CrossRef
7.
go back to reference Sakr Y, Ferrer R, Reinhart K, Beale R, Rhodes A, Moreno R, Timsit JF, Brochard L, Thompson BT, Rezende E, Chiche JD. The Intensive Care Global Study on Severe Acute Respiratory Infection (IC-GLOSSARI): a multicenter, multinational, 14-day inception cohort study. Intensive Care Med. 2016;42:817–28.CrossRef Sakr Y, Ferrer R, Reinhart K, Beale R, Rhodes A, Moreno R, Timsit JF, Brochard L, Thompson BT, Rezende E, Chiche JD. The Intensive Care Global Study on Severe Acute Respiratory Infection (IC-GLOSSARI): a multicenter, multinational, 14-day inception cohort study. Intensive Care Med. 2016;42:817–28.CrossRef
9.
go back to reference Shamout F, Zhu T, Clifton DA. Machine learning for clinical outcome prediction. IEEE Rev Biomed Eng. 2021;14:116–26.CrossRef Shamout F, Zhu T, Clifton DA. Machine learning for clinical outcome prediction. IEEE Rev Biomed Eng. 2021;14:116–26.CrossRef
11.
go back to reference Ning W, Lei S, Yang J, Cao Y, Jiang P, Yang Q, Zhang J, Wang X, Chen F, Geng Z, et al. Open resource of clinical data from patients with pneumonia for the prediction of COVID-19 outcomes via deep learning. Nat Biomed Eng. 2020;4:1197–207.CrossRef Ning W, Lei S, Yang J, Cao Y, Jiang P, Yang Q, Zhang J, Wang X, Chen F, Geng Z, et al. Open resource of clinical data from patients with pneumonia for the prediction of COVID-19 outcomes via deep learning. Nat Biomed Eng. 2020;4:1197–207.CrossRef
13.
go back to reference Cooper GF, Abraham V, Aliferis CF, Aronis JM, Buchanan BG, Caruana R, Fine MJ, Janosky JE, Livingston G, Mitchell T, et al. Predicting dire outcomes of patients with community acquired pneumonia. J Biomed Inform. 2005;38:347–66.CrossRef Cooper GF, Abraham V, Aliferis CF, Aronis JM, Buchanan BG, Caruana R, Fine MJ, Janosky JE, Livingston G, Mitchell T, et al. Predicting dire outcomes of patients with community acquired pneumonia. J Biomed Inform. 2005;38:347–66.CrossRef
15.
go back to reference Hu C-A, Chen C-M, Fang Y-C, Liang S-J, Wang H-C, Fang W-F, Sheu C-C, Perng W-C, Yang K-Y, Kao K-C, et al. Using a machine learning approach to predict mortality in critically ill influenza patients: a cross-sectional retrospective multicentre study in Taiwan. BMJ Open. 2020;10: e033898.CrossRef Hu C-A, Chen C-M, Fang Y-C, Liang S-J, Wang H-C, Fang W-F, Sheu C-C, Perng W-C, Yang K-Y, Kao K-C, et al. Using a machine learning approach to predict mortality in critically ill influenza patients: a cross-sectional retrospective multicentre study in Taiwan. BMJ Open. 2020;10: e033898.CrossRef
16.
go back to reference Wu C, Rosenfeld R, Clermont G. Using data-driven rules to predict mortality in severe community acquired pneumonia. PLoS ONE. 2014;9: e89053.CrossRef Wu C, Rosenfeld R, Clermont G. Using data-driven rules to predict mortality in severe community acquired pneumonia. PLoS ONE. 2014;9: e89053.CrossRef
17.
go back to reference Bratzler DW, Normand S-LT, Wang Y, O'Donnell WJ, Metersky M, Han LF, Rapp MT, Krumholz HM: An Administrative Claims Model for Profiling Hospital 30-Day Mortality Rates for Pneumonia Patients. PLoS ONE 2011, 6:e17401. Bratzler DW, Normand S-LT, Wang Y, O'Donnell WJ, Metersky M, Han LF, Rapp MT, Krumholz HM: An Administrative Claims Model for Profiling Hospital 30-Day Mortality Rates for Pneumonia Patients. PLoS ONE 2011, 6:e17401.
18.
go back to reference Uematsu H, Kunisawa S, Sasaki N, Ikai H, Imanaka Y. Development of a risk-adjusted in-hospital mortality prediction model for community-acquired pneumonia: a retrospective analysis using a Japanese administrative database. BMC Pulm Med. 2014;14:203.CrossRef Uematsu H, Kunisawa S, Sasaki N, Ikai H, Imanaka Y. Development of a risk-adjusted in-hospital mortality prediction model for community-acquired pneumonia: a retrospective analysis using a Japanese administrative database. BMC Pulm Med. 2014;14:203.CrossRef
19.
go back to reference Lim WS. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58:377–82.CrossRef Lim WS. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58:377–82.CrossRef
20.
go back to reference Bauer TT, Ewig S, Marre R, Suttorp N, Welte T. CRB-65 predicts death from community-acquired pneumonia*. J Intern Med. 2006;260:93–101.CrossRef Bauer TT, Ewig S, Marre R, Suttorp N, Welte T. CRB-65 predicts death from community-acquired pneumonia*. J Intern Med. 2006;260:93–101.CrossRef
21.
go back to reference Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, Coley CM, Marrie TJ, Kapoor WN. A Prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336:243–50.CrossRef Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, Coley CM, Marrie TJ, Kapoor WN. A Prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336:243–50.CrossRef
22.
go back to reference Aujesky D, Fine MJ. The pneumonia severity index: a decade after the initial derivation and validation. Clin Infect Dis. 2008;47(Suppl 3):S133-139.CrossRef Aujesky D, Fine MJ. The pneumonia severity index: a decade after the initial derivation and validation. Clin Infect Dis. 2008;47(Suppl 3):S133-139.CrossRef
23.
go back to reference Di Tanna GL, Wirtz H, Burrows KL, Globe G. Evaluating risk prediction models for adults with heart failure: A systematic literature review. PLoS ONE. 2020;15: e0224135.CrossRef Di Tanna GL, Wirtz H, Burrows KL, Globe G. Evaluating risk prediction models for adults with heart failure: A systematic literature review. PLoS ONE. 2020;15: e0224135.CrossRef
24.
go back to reference Rahimi K, Bennett D, Conrad N, Williams TM, Basu J, Dwight J, Woodward M, Patel A, McMurray J, MacMahon S. Risk prediction in patients with heart failure: a systematic review and analysis. JACC Heart Fail. 2014;2:440–6.CrossRef Rahimi K, Bennett D, Conrad N, Williams TM, Basu J, Dwight J, Woodward M, Patel A, McMurray J, MacMahon S. Risk prediction in patients with heart failure: a systematic review and analysis. JACC Heart Fail. 2014;2:440–6.CrossRef
25.
go back to reference Adler ED, Voors AA, Klein L, Macheret F, Braun OO, Urey MA, Zhu W, Sama I, Tadel M, Campagnari C, et al. Improving risk prediction in heart failure using machine learning. Eur J Heart Fail. 2020;22:139–47.CrossRef Adler ED, Voors AA, Klein L, Macheret F, Braun OO, Urey MA, Zhu W, Sama I, Tadel M, Campagnari C, et al. Improving risk prediction in heart failure using machine learning. Eur J Heart Fail. 2020;22:139–47.CrossRef
28.
go back to reference Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, Saunders LD, Beck CA, Feasby TE, Ghali WA. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43:1130–9.CrossRef Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, Saunders LD, Beck CA, Feasby TE, Ghali WA. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43:1130–9.CrossRef
29.
go back to reference Moore BJ, White S, Washington R, Coenen N, Elixhauser A. Identifying increased risk of readmission and in-hospital mortality using hospital administrative data: the AHRQ Elixhauser comorbidity index. Med Care. 2017;55:698–705.CrossRef Moore BJ, White S, Washington R, Coenen N, Elixhauser A. Identifying increased risk of readmission and in-hospital mortality using hospital administrative data: the AHRQ Elixhauser comorbidity index. Med Care. 2017;55:698–705.CrossRef
30.
go back to reference Lundberg SM, Lee S-I: A unified approach to interpreting model predictions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. pp. 4768-4777. Long Beach, California, USA: Curran Associates Inc.; 2017:4768-4777. Lundberg SM, Lee S-I: A unified approach to interpreting model predictions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. pp. 4768-4777. Long Beach, California, USA: Curran Associates Inc.; 2017:4768-4777.
31.
go back to reference Huang Y, Li W, Macheret F, Gabriel RA, Ohno-Machado L. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc. 2020;27:621–33.CrossRef Huang Y, Li W, Macheret F, Gabriel RA, Ohno-Machado L. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc. 2020;27:621–33.CrossRef
32.
go back to reference DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45.CrossRef DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45.CrossRef
33.
go back to reference Tolksdorf KBS, Schuler E, Wieler LH, Haas W. Eine höhere Letalität und lange Beatmungsdauer unterscheiden COVID-19 von schwer verlaufenden Atemwegsinfektionen in Grippewellen. Epid Bull. 2020;2020(41):3–10. Tolksdorf KBS, Schuler E, Wieler LH, Haas W. Eine höhere Letalität und lange Beatmungsdauer unterscheiden COVID-19 von schwer verlaufenden Atemwegsinfektionen in Grippewellen. Epid Bull. 2020;2020(41):3–10.
34.
go back to reference Kompaniyets L, Goodman AB, Belay B, Freedman DS, Sucosky MS, Lange SJ, Gundlapalli AV, Boehmer TK, Blanck HM. Body mass index and risk for COVID-19-related hospitalization, intensive care unit admission, invasive mechanical ventilation, and death—United States, March-December 2020. Morb Mortal Wkly Rep. 2021;70:355–61.CrossRef Kompaniyets L, Goodman AB, Belay B, Freedman DS, Sucosky MS, Lange SJ, Gundlapalli AV, Boehmer TK, Blanck HM. Body mass index and risk for COVID-19-related hospitalization, intensive care unit admission, invasive mechanical ventilation, and death—United States, March-December 2020. Morb Mortal Wkly Rep. 2021;70:355–61.CrossRef
35.
go back to reference Sakr Y, Alhussami I, Nanchal R, Wunderink RG, Pellis T, Wittebole X, Martin-Loeches I, François B, Leone M, Vincent JL. Being overweight is associated with greater survival in ICU patients: results from the intensive care over nations audit. Crit Care Med. 2015;43:2623–32.CrossRef Sakr Y, Alhussami I, Nanchal R, Wunderink RG, Pellis T, Wittebole X, Martin-Loeches I, François B, Leone M, Vincent JL. Being overweight is associated with greater survival in ICU patients: results from the intensive care over nations audit. Crit Care Med. 2015;43:2623–32.CrossRef
37.
go back to reference Krumholz HM, Wang Y, Mattera JA, Wang Y, Han LF, Ingber MJ, Roman S, Normand S-LT. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction. Circulation. 2006;113:1683–92.CrossRef Krumholz HM, Wang Y, Mattera JA, Wang Y, Han LF, Ingber MJ, Roman S, Normand S-LT. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction. Circulation. 2006;113:1683–92.CrossRef
38.
go back to reference Krumholz HM, Wang Y, Mattera JA, Wang Y, Han LF, Ingber MJ, Roman S, Normand S-LT. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure. Circulation. 2006;113:1693–701.CrossRef Krumholz HM, Wang Y, Mattera JA, Wang Y, Han LF, Ingber MJ, Roman S, Normand S-LT. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure. Circulation. 2006;113:1693–701.CrossRef
39.
go back to reference Khera R, Krumholz HM. With great power comes great responsibility. Circulation. 2017;10:e003846.PubMed Khera R, Krumholz HM. With great power comes great responsibility. Circulation. 2017;10:e003846.PubMed
40.
go back to reference Konig S, Ueberham L, Schuler E, Wiedemann M, Reithmann C, Seyfarth M, Sause A, Tebbenjohanns J, Schade A, Shin DI, et al. In-hospital mortality of patients with atrial arrhythmias: insights from the German-wide Helios hospital network of 161,502 patients and 34,025 arrhythmia-related procedures. Eur Heart J. 2018;39:3947–57.CrossRef Konig S, Ueberham L, Schuler E, Wiedemann M, Reithmann C, Seyfarth M, Sause A, Tebbenjohanns J, Schade A, Shin DI, et al. In-hospital mortality of patients with atrial arrhythmias: insights from the German-wide Helios hospital network of 161,502 patients and 34,025 arrhythmia-related procedures. Eur Heart J. 2018;39:3947–57.CrossRef
42.
go back to reference Davis J, Goadrich M. The relationship between Precision-Recall and ROC curves. In: Proceedings of the 23rd international conference on Machine learning. Pittsburgh, Pennsylvania, USA: Association for Computing Machinery; 2006; pp. 233-240. Davis J, Goadrich M. The relationship between Precision-Recall and ROC curves. In: Proceedings of the 23rd international conference on Machine learning. Pittsburgh, Pennsylvania, USA: Association for Computing Machinery; 2006; pp. 233-240.
43.
go back to reference Mandell LA, Wunderink RG, Anzueto A, Bartlett JG, Campbell GD, Dean NC, Dowell SF, File TM, Musher DM, Niederman MS, et al. Infectious diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. 2007;44:S27–72.CrossRef Mandell LA, Wunderink RG, Anzueto A, Bartlett JG, Campbell GD, Dean NC, Dowell SF, File TM, Musher DM, Niederman MS, et al. Infectious diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. 2007;44:S27–72.CrossRef
44.
go back to reference Johnson EK, Nelson CP. Values and pitfalls of the use of administrative databases for outcomes assessment. J Urol. 2013;190:17–8.CrossRef Johnson EK, Nelson CP. Values and pitfalls of the use of administrative databases for outcomes assessment. J Urol. 2013;190:17–8.CrossRef
Metadata
Title
Machine learning-derived prediction of in-hospital mortality in patients with severe acute respiratory infection: analysis of claims data from the German-wide Helios hospital network
Authors
Johannes Leiner
Vincent Pellissier
Sebastian König
Sven Hohenstein
Laura Ueberham
Irit Nachtigall
Andreas Meier-Hellmann
Ralf Kuhlen
Gerhard Hindricks
Andreas Bollmann
Publication date
01-12-2022
Publisher
BioMed Central
Keyword
Influenza
Published in
Respiratory Research / Issue 1/2022
Electronic ISSN: 1465-993X
DOI
https://doi.org/10.1186/s12931-022-02180-w

Other articles of this Issue 1/2022

Respiratory Research 1/2022 Go to the issue
Live Webinar | 27-06-2024 | 18:00 (CEST)

Keynote webinar | Spotlight on medication adherence

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

WHO estimates that half of all patients worldwide are non-adherent to their prescribed medication. The consequences of poor adherence can be catastrophic, on both the individual and population level.

Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.

Prof. Kevin Dolgin
Prof. Florian Limbourg
Prof. Anoop Chauhan
Developed by: Springer Medicine
Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine