Top

BMC Medical Informatics and Decision Making

Published in:

Open Access 01-12-2023 | Artificial Intelligence | Research

The accuracy of artificial intelligence in predicting COVID-19 patient mortality: a systematic review and meta-analysis

Authors: Yu Xin, Hongxu Li, Yuxin Zhou, Qing Yang, Wenjing Mu, Han Xiao, Zipeng Zhuo, Hongyu Liu, Hongying Wang, Xutong Qu, Changsong Wang, Haitao Liu, Kaijiang Yu

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Abstract

Background

The purpose of this paper was to systematically evaluate the application value of artificial intelligence in predicting mortality among COVID-19 patients.

Methods

The PubMed, Embase, Web of Science, CNKI, Wanfang, China Biomedical Literature, and VIP databases were systematically searched from inception to October 2022 to identify studies that evaluated the predictive effects of artificial intelligence on mortality among COVID-19 patients. The retrieved literature was screened according to the inclusion and exclusion criteria. The quality of the included studies was assessed using the QUADAS-2 tools. Statistical analysis of the included studies was performed using Review Manager 5.3, Stata 16.0, and Meta-DiSc 1.4 statistical software. This meta-analysis was registered in PROSPERO (CRD42022315158).

Findings

Of 2193 studies, 23 studies involving a total of 25 AI models met the inclusion criteria. Among them, 18 studies explicitly mentioned training and test sets, and 5 studies did not explicitly mention grouping. In the training set, the pooled sensitivity was 0.93 [0.87, 0.96], the pooled specificity was 0.94 [0.87, 0.97], and the area under the ROC curve was 0.98 [0.96, 0.99]. In the validation set, the pooled sensitivity was 0.84 [0.78, 0.88], the pooled specificity was 0.89 [0.85, 0.92], and the area under the ROC curve was 0.93 [1.00, 0.00]. In the subgroup analysis, the areas under the summary receiver operating characteristic (SROC) curves of the artificial intelligence models KNN, SVM, ANN, RF and XGBoost were 0.98, 0.98, 0.94, 0.92, and 0.91, respectively. The Deeks funnel plot indicated that there was no significant publication bias in this study (P > 0.05).

Interpretation

Artificial intelligence models have high accuracy in predicting mortality among COVID-19 patients and have high prognostic value. Among them, the KNN, SVM, ANN, RF, XGBoost, and other models have the highest levels of accuracy.

Available only for authorised users

WHO. WHO Coronavirus disease (COVID-19) pandemic. 2022. https://www.who.int/emergencies/diseases/novel-coronavirus-2019. Accessed December 21, 2022, 9, 2022.

Weiss P, Murdoch DR. Clinical course and mortality risk of severe COVID-19. Lancet. 2020;395(10229):1014–5.CrossRefPubMedPubMedCentral

Toniolo M, Negri F, Antonutti M, Masè M, Facchin D. Unpredictable fall of severe Emergent Cardiovascular Diseases Hospital admissions during the COVID-19 pandemic: experience of a single large Center in Northern Italy. J Am Heart Assoc. 2020;9(13):e017122.CrossRefPubMedPubMedCentral

Ngiam KY, Khor IW. Big data and machine learning algorithms for health-care delivery. Lancet Oncol. 2019;20(5):e262–73.CrossRefPubMed

Schwalbe N, Wahl B. Artificial intelligence and the future of global health. Lancet. 2020;395(10236):1579–86.CrossRefPubMedPubMedCentral

Khan IU, Aslam N, Aljabri M, Aljameel SS, Kamaleldin MMA, Alshamrani FM, Chrouf SMB. Computational intelligence-based model for Mortality Rate Prediction in COVID-19 patients. Int J Environ Res Public Health. 2021;18(12).

Vaid A, Somani S, Russak AJ, De Freitas JK, Chaudhry FF, Paranjpe I, Johnson KW, Lee SJ, Miotto R, Richter F, et al. Machine learning to Predict Mortality and critical events in a cohort of patients with COVID-19 in New York City: Model Development and Validation. J Med Internet Res. 2020;22(11):e24018.CrossRefPubMedPubMedCentral

Schiaffino S, Codari M, Cozzi A, Albano D, Alì M, Arioli R, Avola E, Bnà C, Cariati M, Carriero S, et al. Machine learning to predict in-hospital mortality in covid-19 patients using computed tomography-derived pulmonary and vascular features. J Personalized Med. 2021;11(6).

Bae J, Kapse S, Singh G, Gattu R, Ali S, Shah N, Marshall C, Pierce J, Phatak T, Gupta A, et al. Predicting mechanical ventilation and mortality in covid-19 using radiomics and deep learning on chest radiographs: a multi-institutional study. Diagnostics. 2021;11(10).

10.

Lee SW, Koo MJ. PRISMA 2020 statement and guidelines for systematic review and meta-analysis articles, and their underlying mathematics: Life Cycle Committee Recommendations. Life Cycle. 2022;2:e9.CrossRef

11.

Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–36.CrossRefPubMed

12.

Das AK, Mishra S, Saraswathy Gopalan S. Predicting CoVID-19 community mortality risk using machine learning and development of an online prognostic tool. PeerJ. 2020;8:e10083.CrossRefPubMedPubMedCentral

13.

Yu L, Halalau A, Dalal B, Abbas AE, Ivascu F, Amin M, Nair GB. Machine learning methods to predict mechanical ventilation and mortality in patients with COVID-19. PLoS ONE. Apr 2021;116(4):e0249285.CrossRef

14.

Tezza F, Lorenzoni G, Azzolina D, Barbar S, Leone LAC, Gregori D. Predicting in-hospital mortality of patients with covid-19 using machine learning techniques. J Personalized Med. 2021;11(5).

15.

Santos-Lozano A, Calvo-Boyero F, López-Jiménez A, Cueto-Felgueroso C, Castillo-García A, Valenzuela PL, Arenas J, Lucia A, Martín MA. Can routine laboratory variables predict survival in COVID-19? An artificial neural network-based approach. Clin Chem Lab Med. 2020;58(12):e299–e302.CrossRefPubMed

16.

Parchure P, Joshi H, Dharmarajan K, Freeman R, Reich DL, Mazumdar M, Timsina P, Kia A. Development and validation of a machine learning-based prediction model for near-term in-hospital mortality among patients with COVID-19. BMJ Support Palliat Care. 2020.

17.

Murri R, Lenkowicz J, Masciocchi C, Iacomini C, Fantoni M, Damiani A, Marchetti A, Sergi PDA, Arcuri G, Cesario A, et al. A machine-learning parsimonious multivariable predictive model of mortality risk in patients with Covid-19. Sci Rep. 2021;11(1):21136.CrossRefPubMedPubMedCentral

18.

Li X, Ge P, Zhu J, Li H, Graham J, Singer A, Richman PS, Duong TQ. Deep learning prediction of likelihood of ICU admission and mortality in COVID-19 patients using clinical variables. PeerJ. 2020;8:e10337.

19.

Kar S, Chawla R, Haranath SP, Ramasubban S, Ramakrishnan N, Vaishya R, Sibal A, Reddy S. Multivariable mortality risk prediction using machine learning for COVID-19 patients at admission (AICOVID). Sci Rep. 2021;11(1):12801.CrossRefPubMedPubMedCentral

20.

Booth AL, Abels E, McCaffrey P. Development of a prognostic model for mortality in COVID-19 infection using machine learning. Mod Pathol. 2021;34(3):522–31.CrossRefPubMedPubMedCentral

21.

Garrafa E, Vezzoli M, Ravanelli M, Farina D, Borghesi A, Calza S, Maroldi R. Early prediction of in-hospital death of COVID-19 patients: a machine-learning model based on age, blood analyses, and chest x-ray score. Elife. 2021;10:e70640.CrossRefPubMedPubMedCentral

22.

Banoei MM, Dinparastisaleh R, Zadeh AV, Mirsaeidi M. Machine-learning-based COVID-19 mortality prediction model and identification of patients at low and high risk of dying. Crit Care. 2021;25(1):328.CrossRefPubMedPubMedCentral

23.

Ko H, Chung H, Kang WS, Park C, Kim DW, Kim SE, Chung CR, Ko RE, Lee H, Seo JH, et al. An artificial intelligence model to predict the mortality of COVID-19 patients at hospital admission time using routine blood samples: development and validation of an ensemble model. J Med Internet Res. 2020;22(12).

24.

Lin JK, Chien TW, Wang LY, Chou W. An artificial neural network model to predict the mortality of COVID-19 patients using routine blood samples at the time of hospital admission: development and validation study. Med (Baltim). 2021;100(28):e26532.CrossRef

25.

Abdulaal A, Patel A, Charani E, Denny S, Mughal N, Moore L. Prognostic modeling of COVID-19 using Artificial Intelligence in the United Kingdom: Model Development and Validation. J Med Internet Res. 2020;22(8):e20259.CrossRefPubMedPubMedCentral

26.

Moghaddam-Tabrizi F, Omidi T, Mahdi-Akhgar M, Bahadori R, Valizadeh R, Farrokh-Eslamlou H. Identification of risk factors associated with mortality among patients with covid-19 using random forest model: a historical cohort study. Acta Medica Iranica. 2021;59(8):457–65.

27.

Aljouie AF, Almazroa A, Bokhari Y, Alawad M, Mahmoud E, Alawad E, Alsehawi A, Rashid M, Alomair L, Almozaai S, et al. Early Prediction of COVID-19 Ventilation requirement and mortality from routinely collected baseline chest radiographs, Laboratory, and Clinical Data with Machine Learning. J Multidisciplinary Healthc. 2021;14:2017–33.CrossRef

28.

Naseem M, Arshad H, Hashmi SA, Irfan F, Ahmed FS. Predicting mortality in SARS-COV-2 (COVID-19) positive patients in the inpatient setting using a novel deep neural network. Int J Med Informatics. 2021;154.

29.

Moulaei K, Ghasemian F, Bahaadin-Beigy K, Sarbi RE, Taghiabad ZM. Predicting mortality of covid-19 patients based on data mining techniques. J Biomedical Phys Eng. 2021;11(5):653–62.CrossRef

30.

Ryan L, Lam C, Mataraso S, Allen A, Green-Saxena A, Pellegrini E, Hoffman J, Barton C, McCoy A, Das R. Mortality prediction model for the triage of COVID-19, pneumonia, and mechanically ventilated ICU patients: a retrospective study. Annals of Medicine and Surgery. 2020;59:207–16.CrossRefPubMedPubMedCentral

31.

Ikemura K, Bellin E, Yagi Y, Billett H, Saada M, Simone K, Stahl L, Szymanski J, Goldstein DY, Gil MR. Using automated machine learning to predict the mortality of patients with COVID-19: prediction model development study. J Med Internet Res. 2021;23(2).

32.

Chi PENG, Ge-yao QI, Chen-xu ZHANG, Yu-feng GUO. JIN Zhi-chaoPrediction of in-hospital clinical outcomes of coronavirus disease 2019 patients based on machine learning algorithms. Acad J Sec Mil Med Univ. 2021;42(10):1115–23.

33.

Munera N, Garcia-Gallo E, Gonzalez Á, Zea J, Fuentes YV, Serrano C, Ruiz-Cuartas A, Rodriguez A, Reyes LF. A novel model to predict severe COVID-19 and mortality using an artificial intelligence algorithm to interpret chest radiographs and clinical variables. ERJ Open Res. 2022;8(2):00010–2022.CrossRefPubMedPubMedCentral

34.

Peng H, Hu C, Deng W, Huang L, Zhang Y, Luo B, Wang X, Long X, Huang X. Incubation period, clinical and lung CT features for early prediction of COVID-19 deterioration: development and internal verification of a risk model. BMC Pulm Med. 2022;22(1):188.CrossRefPubMedPubMedCentral

35.

Subbe CP, Slater A, Menon D, Gemmell L. Validation of physiological scoring systems in the accident and emergency department. Emerg Med J. 2006;23(11):841–5.CrossRefPubMedPubMedCentral

36.

Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):1619–36.CrossRefPubMed

37.

Le Gall JR, Lemeshow S, Saulnier F. A new simplified Acute Physiology score (SAPS II) based on a European/North american multicenter study. JAMA. 1993;270(24):2957–63.CrossRefPubMed

38.

Ferreira FL, Bota DP, Bross A, Mélot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286(14):1754–8.CrossRefPubMed

39.

Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, et al. The Third International Consensus Definitions for Sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–10.CrossRefPubMedPubMedCentral

40.

Pourhomayoun M, Shakibi M. Predicting mortality risk in patients with COVID-19 using machine learning to help medical decision-making. Smart Health (Amst). 2021;20:100178.CrossRefPubMed

41.

42.

Barzi F, & Woodward M. Imputations of missing values in practice: results from imputations of serum cholesterol in 28 cohort studies. Am J Epidemiol. 2004;160(1):34–45.

43.

An C, Lim H, Kim DW, Chang JH, Choi YJ, Kim SW. Machine learning prediction for mortality of patients diagnosed with COVID-19: a nationwide korean cohort study. Sci Rep. 2020;10(1):18716.CrossRefPubMedPubMedCentral

44.

Sun L, Song F, Shi N, Liu F, Li S, Li P, Zhang W, Jiang X, Zhang Y, Sun L, et al. Combination of four clinical indicators predicts the severe/critical symptom of patients infected COVID-19. J Clin Virol. 2020;128:104431.CrossRefPubMedPubMedCentral

45.

Clinical characteristics. day-90 outcomes of 4244 critically ill adults with COVID-19: a prospective cohort study. Intensive Care Med. 2021;47(1):60–73.CrossRef

46.

Fajnzylber J, Regan J, Coxen K, Corry H, Wong C, Rosenthal A, Worrall D, Giguel F, Piechocka-Trocha A, Atyeo C, et al. SARS-CoV-2 viral load is associated with increased disease severity and mortality. Nat Commun. 2020;11(1):5493.CrossRefPubMedPubMedCentral

47.

Becerra-Muñoz VM, Núñez-Gil IJ, Eid CM, García Aguado M, Romero R, Huang J, Mulet A, Ugo F, Rametta F, Liebetrau C, et al. Clinical profile and predictors of in-hospital mortality among older patients hospitalized for COVID-19. Age Aging. 2021;50(2):326–34.CrossRef

48.

Escobar GJ, Adams AS, Liu VX, Soltesz L, Chen YI, Parodi SM, Ray GT, Myers LC, Ramaprasad CM, Dlott R, et al. Racial disparities in COVID-19 testing and outcomes: Retrospective Cohort Study in an Integrated Health System. Ann Intern Med. 2021;174(6):786–93.CrossRefPubMed

Title: The accuracy of artificial intelligence in predicting COVID-19 patient mortality: a systematic review and meta-analysis
Authors: Yu Xin
Hongxu Li
Yuxin Zhou
Qing Yang
Wenjing Mu
Han Xiao
Zipeng Zhuo
Hongyu Liu
Hongying Wang
Xutong Qu
Changsong Wang
Haitao Liu
Kaijiang Yu
Publication date: 01-12-2023
Publisher: BioMed Central
Keywords: Artificial Intelligence
COVID-19
Published in: BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI: https://doi.org/10.1186/s12911-023-02256-7

Keynote webinar | Spotlight on medication adherence

Springer Medicine

The accuracy of artificial intelligence in predicting COVID-19 patient mortality: a systematic review and meta-analysis

Abstract

Background

Methods

Findings

Interpretation

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Findings

Interpretation

Please log in to get access to this content

Other articles of this Issue 1/2023

Digital health literacy and digital engagement for people with severe mental ill health across the course of the COVID-19 pandemic in England

Correction: The impact of data from remote measurement technology on the clinical practice of healthcare professionals in depression, epilepsy and multiple sclerosis: survey

Smartwatches in healthcare medicine: assistance and monitoring; a scoping review

Quest markup for developing FAIR questionnaire modules for epidemiologic studies

KAMPNet: multi-source medical knowledge augmented medication prediction network with multi-level graph contrastive learning

A hybrid stacked ensemble and Kernel SHAP-based model for intelligent cardiotocography classification and interpretability