Skip to main content
Top
Published in: Digestive Diseases and Sciences 8/2019

01-08-2019 | Gastrointestinal Bleeding | Review

Machine Learning to Predict Outcomes in Patients with Acute Gastrointestinal Bleeding: A Systematic Review

Authors: Dennis Shung, Michael Simonov, Mark Gentry, Benjamin Au, Loren Laine

Published in: Digestive Diseases and Sciences | Issue 8/2019

Login to get access

Abstract

Risk stratification of patients with gastrointestinal bleeding (GIB) is recommended, but current risk assessment tools have variable performance. Machine learning (ML) has promise to improve risk assessment. We performed a systematic review to evaluate studies utilizing ML techniques for GIB. Bibliographic databases and conference abstracts were searched for studies with a population of overt GIB that used an ML algorithm with outcomes of mortality, rebleeding, hemostatic intervention, and/or hospital stay. Two independent reviewers screened titles and abstracts, reviewed full-text studies, and extracted data from included studies. Risk of bias was assessed with an adapted Quality in Prognosis Studies tool. Area under receiver operating characteristic curves (AUCs) were the primary assessment of performance with AUC ≥ 0.80 predefined as an acceptable threshold of good performance. Fourteen studies with 30 assessments of ML models met inclusion criteria. No study had low risk of bias. Median AUC reported in validation datasets for predefined outcomes of mortality, intervention, or rebleeding was 0.84 (range 0.40–0.98). AUCs were higher with artificial neural networks (median 0.93, range 0.78–0.98) than other ML models (0.81, range 0.40–0.92). ML performed better than clinical risk scores (Glasgow-Blatchford, Rockall, Child–Pugh, MELD) for mortality in upper GIB. Limitations include heterogeneity of ML models, inconsistent comparisons of ML models with clinical risk scores, and high risk of bias. ML generally provided good–excellent prognostic performance in patients with GIB, and artificial neural networks tended to outperform other ML models. ML was better than clinical risk scores for mortality in upper GIB.
Appendix
Available only for authorised users
Literature
1.
go back to reference Peery AF, Crockett SD, Barritt AS, et al. Burden of gastrointestinal, liver, and pancreatic diseases in the United States. Gastroenterology. 2015;149:e1733. Peery AF, Crockett SD, Barritt AS, et al. Burden of gastrointestinal, liver, and pancreatic diseases in the United States. Gastroenterology. 2015;149:e1733.
2.
go back to reference Barkun AN, Bardou M, Kuipers EJ, et al. International consensus upper gastrointestinal bleeding conference G: International consensus recommendations on the management of patients with nonvariceal upper gastrointestinal bleeding. Ann Intern Med. 2010;152:101–113.CrossRefPubMed Barkun AN, Bardou M, Kuipers EJ, et al. International consensus upper gastrointestinal bleeding conference G: International consensus recommendations on the management of patients with nonvariceal upper gastrointestinal bleeding. Ann Intern Med. 2010;152:101–113.CrossRefPubMed
3.
go back to reference Laine L, Jensen DM. Management of patients with ulcer bleeding. Am J Gastroenterol. 2012;107:345–360.CrossRefPubMed Laine L, Jensen DM. Management of patients with ulcer bleeding. Am J Gastroenterol. 2012;107:345–360.CrossRefPubMed
4.
go back to reference Gralnek IM, Dumonceau JM, Kuipers EJ, et al. Diagnosis and management of nonvariceal upper gastrointestinal hemorrhage: European society of gastrointestinal endoscopy (esge) guideline. Endoscopy. 2015;47:a1–46.CrossRefPubMed Gralnek IM, Dumonceau JM, Kuipers EJ, et al. Diagnosis and management of nonvariceal upper gastrointestinal hemorrhage: European society of gastrointestinal endoscopy (esge) guideline. Endoscopy. 2015;47:a1–46.CrossRefPubMed
5.
go back to reference Strate LL, Gralnek IM. Acg clinical guideline: Management of patients with acute lower gastrointestinal bleeding. Am J Gastroenterol. 2016;111:755.CrossRefPubMed Strate LL, Gralnek IM. Acg clinical guideline: Management of patients with acute lower gastrointestinal bleeding. Am J Gastroenterol. 2016;111:755.CrossRefPubMed
7.
go back to reference Blatchford O, Murray WR, Blatchford M. A risk score to predict need for treatment for upper-gastrointestinal haemorrhage. Lancet. 2000;356:1318–1321.CrossRefPubMed Blatchford O, Murray WR, Blatchford M. A risk score to predict need for treatment for upper-gastrointestinal haemorrhage. Lancet. 2000;356:1318–1321.CrossRefPubMed
8.
go back to reference Saltzman JR, Tabak YP, Hyett BH, Sun X, Travis AC, Johannes RS. A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper gi bleeding. Gastrointest Endosc. 2011;74:1215–1224.CrossRefPubMed Saltzman JR, Tabak YP, Hyett BH, Sun X, Travis AC, Johannes RS. A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper gi bleeding. Gastrointest Endosc. 2011;74:1215–1224.CrossRefPubMed
9.
go back to reference Strate LL, Saltzman JR, Ookubo R, Mutinga ML, Syngal S. Validation of a clinical prediction rule for severe acute lower intestinal bleeding. Am J Gastroenterol. 2005;100:1821–1827.CrossRefPubMed Strate LL, Saltzman JR, Ookubo R, Mutinga ML, Syngal S. Validation of a clinical prediction rule for severe acute lower intestinal bleeding. Am J Gastroenterol. 2005;100:1821–1827.CrossRefPubMed
10.
go back to reference Velayos FS, Williamson A, Sousa KH, et al. Early predictors of severe lower gastrointestinal bleeding and adverse outcomes: A prospective study. Clin Gastroenterol Hepatol. 2004;2:485–490.CrossRefPubMed Velayos FS, Williamson A, Sousa KH, et al. Early predictors of severe lower gastrointestinal bleeding and adverse outcomes: A prospective study. Clin Gastroenterol Hepatol. 2004;2:485–490.CrossRefPubMed
11.
go back to reference Newman J, Fitzgerald JE, Gupta S, von Roon AC, Sigurdsson HH, Allen-Mersh TG. Outcome predictors in acute surgical admissions for lower gastrointestinal bleeding. Colorectal Dis. 2012;14:1020–1026.CrossRefPubMed Newman J, Fitzgerald JE, Gupta S, von Roon AC, Sigurdsson HH, Allen-Mersh TG. Outcome predictors in acute surgical admissions for lower gastrointestinal bleeding. Colorectal Dis. 2012;14:1020–1026.CrossRefPubMed
12.
go back to reference Aoki T, Nagata N, Shimbo T, et al. Development and validation of a risk scoring system for severe acute lower gastrointestinal bleeding. Clin Gastroenterol Hepatol. 2016;14:e1562.CrossRef Aoki T, Nagata N, Shimbo T, et al. Development and validation of a risk scoring system for severe acute lower gastrointestinal bleeding. Clin Gastroenterol Hepatol. 2016;14:e1562.CrossRef
13.
go back to reference Stanley AJ, Ashley D, Dalton HR, et al. Outpatient management of patients with low-risk upper-gastrointestinal haemorrhage: Multicentre validation and prospective evaluation. Lancet. 2009;373:42–47.CrossRefPubMed Stanley AJ, Ashley D, Dalton HR, et al. Outpatient management of patients with low-risk upper-gastrointestinal haemorrhage: Multicentre validation and prospective evaluation. Lancet. 2009;373:42–47.CrossRefPubMed
14.
go back to reference Abu-Mostafa YS, Magdon-Ismail M, Lin H. Learning from data: A short course. United States of America; 2012. Abu-Mostafa YS, Magdon-Ismail M, Lin H. Learning from data: A short course. United States of America; 2012.
15.
go back to reference Mitchell TM. Machine Learning. New York: McGraw-Hill; 1997. Mitchell TM. Machine Learning. New York: McGraw-Hill; 1997.
16.
go back to reference Wilson FP, Shashaty M, Testani J, et al. Automated, electronic alerts for acute kidney injury: A single-blind, parallel-group, randomised controlled trial. Lancet. 2015;385:1966–1974.CrossRefPubMedPubMedCentral Wilson FP, Shashaty M, Testani J, et al. Automated, electronic alerts for acute kidney injury: A single-blind, parallel-group, randomised controlled trial. Lancet. 2015;385:1966–1974.CrossRefPubMedPubMedCentral
17.
go back to reference Andrew W, Albert TY, April SL, Ralph G, Vanja CD, Dexter H. Development and validation of an electronic health record—based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment. JAMA Netw Open. 2018;1(4):e181018.CrossRef Andrew W, Albert TY, April SL, Ralph G, Vanja CD, Dexter H. Development and validation of an electronic health record—based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment. JAMA Netw Open. 2018;1(4):e181018.CrossRef
18.
go back to reference Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: A randomised clinical trial. BMJ Open Respir Res. 2017;4:e000234.CrossRefPubMedPubMedCentral Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: A randomised clinical trial. BMJ Open Respir Res. 2017;4:e000234.CrossRefPubMedPubMedCentral
19.
go back to reference Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: The prisma statement. PLoS Med. 2009;6:e1000097.CrossRefPubMedPubMedCentral Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: The prisma statement. PLoS Med. 2009;6:e1000097.CrossRefPubMedPubMedCentral
20.
go back to reference Hayden JA, van der Windt DA, Cartwright JL, Cote P, Bombardier C. Assessing bias in studies of prognostic factors. Ann Intern Med. 2013;158:280–286.CrossRefPubMed Hayden JA, van der Windt DA, Cartwright JL, Cote P, Bombardier C. Assessing bias in studies of prognostic factors. Ann Intern Med. 2013;158:280–286.CrossRefPubMed
21.
go back to reference Moons KG, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: The charms checklist. PLoS Med. 2014;11:e1001744.CrossRefPubMedPubMedCentral Moons KG, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: The charms checklist. PLoS Med. 2014;11:e1001744.CrossRefPubMedPubMedCentral
22.
go back to reference Zarshenas S, Tam L, Colantonio A, Alavinia SM, Cullen N. Predictors of discharge destination from acute care in patients with traumatic brain injury. BMJ Open. 2017;7:e016694.CrossRefPubMedPubMedCentral Zarshenas S, Tam L, Colantonio A, Alavinia SM, Cullen N. Predictors of discharge destination from acute care in patients with traumatic brain injury. BMJ Open. 2017;7:e016694.CrossRefPubMedPubMedCentral
23.
go back to reference Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol. 2010;5:1315–1316.CrossRefPubMed Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol. 2010;5:1315–1316.CrossRefPubMed
24.
go back to reference Ali A, Swingland J, Choi CH, et al. OC-143 artificial neural network for the risk stratification of acute upper gastrointestinal bleeding: Multicentre comparative analysis vs the Glasgow Blatchford and Rockall scores. Gut. 2012;61:A62–A62.CrossRef Ali A, Swingland J, Choi CH, et al. OC-143 artificial neural network for the risk stratification of acute upper gastrointestinal bleeding: Multicentre comparative analysis vs the Glasgow Blatchford and Rockall scores. Gut. 2012;61:A62–A62.CrossRef
25.
go back to reference Augustin S, Muntaner L, Altamirano JT, et al. Predicting early mortality after acute variceal hemorrhage based on classification and regression tree analysis. Clin Gastroenterol Hepatol. 2009;7:1347–1354.CrossRefPubMed Augustin S, Muntaner L, Altamirano JT, et al. Predicting early mortality after acute variceal hemorrhage based on classification and regression tree analysis. Clin Gastroenterol Hepatol. 2009;7:1347–1354.CrossRefPubMed
26.
go back to reference Ayaru L, Ypsilantis PP, Nanapragasam A, et al. Prediction of outcome in acute lower gastrointestinal bleeding using gradient boosting. PLoS One. 2015;10:e0132485.CrossRefPubMedPubMedCentral Ayaru L, Ypsilantis PP, Nanapragasam A, et al. Prediction of outcome in acute lower gastrointestinal bleeding using gradient boosting. PLoS One. 2015;10:e0132485.CrossRefPubMedPubMedCentral
27.
go back to reference Choi CH, Swingland J, Ali A, Bose S, Ayaru L. Assessing risk of adverse outcome in acute lower gastrointestinal bleeding: Artificial neural network vs sign guidelines and bleed score. Gut. 2012;61:A156–A157.CrossRef Choi CH, Swingland J, Ali A, Bose S, Ayaru L. Assessing risk of adverse outcome in acute lower gastrointestinal bleeding: Artificial neural network vs sign guidelines and bleed score. Gut. 2012;61:A156–A157.CrossRef
28.
go back to reference Chu A, Ahn H, Halwan B, et al. A decision support system to facilitate management of patients with acute gastrointestinal bleeding. Artif Intell Med. 2008;42:247–259.CrossRefPubMed Chu A, Ahn H, Halwan B, et al. A decision support system to facilitate management of patients with acute gastrointestinal bleeding. Artif Intell Med. 2008;42:247–259.CrossRefPubMed
29.
go back to reference Das A, Ben-Menachem T, Cooper GS, et al. Prediction of outcome in acute lower-gastrointestinal haemorrhage based on an artificial neural network: Internal and external validation of a predictive model. Lancet (London, England). 2003;362:1261–1266.CrossRef Das A, Ben-Menachem T, Cooper GS, et al. Prediction of outcome in acute lower-gastrointestinal haemorrhage based on an artificial neural network: Internal and external validation of a predictive model. Lancet (London, England). 2003;362:1261–1266.CrossRef
30.
go back to reference Das A, Ben-Menachem T, Farooq FT, et al. Artificial neural network as a predictive instrument in patients with acute nonvariceal upper gastrointestinal hemorrhage. Gastroenterology. 2008;134:65–74.CrossRefPubMed Das A, Ben-Menachem T, Farooq FT, et al. Artificial neural network as a predictive instrument in patients with acute nonvariceal upper gastrointestinal hemorrhage. Gastroenterology. 2008;134:65–74.CrossRefPubMed
31.
go back to reference Grossi E, Marmo R, Intraligi M, Buscema M. Artificial neural networks for early prediction of mortality in patients with non variceal upper gi bleeding (ugib). Biomed Inf Insights. 2008;1:7–19. Grossi E, Marmo R, Intraligi M, Buscema M. Artificial neural networks for early prediction of mortality in patients with non variceal upper gi bleeding (ugib). Biomed Inf Insights. 2008;1:7–19.
32.
go back to reference Loftus TJ, Brakenridge SC, Croft CA, et al. Neural network prediction of severe lower intestinal bleeding and the need for surgical intervention. J Surg Res. 2017;212:42–47.CrossRefPubMed Loftus TJ, Brakenridge SC, Croft CA, et al. Neural network prediction of severe lower intestinal bleeding and the need for surgical intervention. J Surg Res. 2017;212:42–47.CrossRefPubMed
33.
go back to reference Lyles T, Elliott A, Rockey DC. A risk scoring system to predict in-hospital mortality in patients with cirrhosis presenting with upper gastrointestinal bleeding. J Clin Gastroenterol. 2014;48:712–720.CrossRefPubMed Lyles T, Elliott A, Rockey DC. A risk scoring system to predict in-hospital mortality in patients with cirrhosis presenting with upper gastrointestinal bleeding. J Clin Gastroenterol. 2014;48:712–720.CrossRefPubMed
34.
go back to reference Rotondano G, Cipolletta L, Grossi E, et al. Italian registry on upper gastrointestinal B: Artificial neural networks accurately predict mortality in patients with nonvariceal upper gi bleeding. Gastrointest Endosc. 2011;73:226.e211.CrossRef Rotondano G, Cipolletta L, Grossi E, et al. Italian registry on upper gastrointestinal B: Artificial neural networks accurately predict mortality in patients with nonvariceal upper gi bleeding. Gastrointest Endosc. 2011;73:226.e211.CrossRef
35.
go back to reference Thon K, Stoltzing H, Ohmann C, Lorenz W, Roher HD. Decision-making and clinical problem solving in upper gastrointestinal bleeding. Theor Surg. 1988;2:185–198. Thon K, Stoltzing H, Ohmann C, Lorenz W, Roher HD. Decision-making and clinical problem solving in upper gastrointestinal bleeding. Theor Surg. 1988;2:185–198.
36.
go back to reference Lee HH, Park JM, Han S, et al. A simplified prognostic model to predict mortality in patients with acute variceal bleeding. Dig Liver Dis. 2018;50:247–253.CrossRefPubMed Lee HH, Park JM, Han S, et al. A simplified prognostic model to predict mortality in patients with acute variceal bleeding. Dig Liver Dis. 2018;50:247–253.CrossRefPubMed
37.
go back to reference D’Amico G, De Franchis R. Upper digestive bleeding in cirrhosis. Post-therapeutic outcome and prognostic indicators. Hepatology. 2003;38:599–612.CrossRefPubMed D’Amico G, De Franchis R. Upper digestive bleeding in cirrhosis. Post-therapeutic outcome and prognostic indicators. Hepatology. 2003;38:599–612.CrossRefPubMed
38.
go back to reference Stanley AJ, Laine L, Dalton HR, et al. International gastrointestinal bleeding C: Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: International multicentre prospective study. BMJ. 2017;356:i6432.CrossRefPubMedPubMedCentral Stanley AJ, Laine L, Dalton HR, et al. International gastrointestinal bleeding C: Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: International multicentre prospective study. BMJ. 2017;356:i6432.CrossRefPubMedPubMedCentral
39.
go back to reference Laursen SB, Hansen JM. Schaffalitzky de Muckadell OB: The glasgow blatchford score is the most accurate assessment of patients with upper gastrointestinal hemorrhage. Clin Gastroenterol Hepatol. 2012;10:e1131.CrossRef Laursen SB, Hansen JM. Schaffalitzky de Muckadell OB: The glasgow blatchford score is the most accurate assessment of patients with upper gastrointestinal hemorrhage. Clin Gastroenterol Hepatol. 2012;10:e1131.CrossRef
40.
go back to reference Rajkomar A, Oren E, Chen K, et al. Scalable and accurate deep learning with electronic health records. npj Dig Med. 2018;1:18.CrossRef Rajkomar A, Oren E, Chen K, et al. Scalable and accurate deep learning with electronic health records. npj Dig Med. 2018;1:18.CrossRef
42.
go back to reference Goldstein BA, Navar AM, Pencina MJ, Ioannidis JP. Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review. J Am Med Inform Assoc. 2017;24:198–208.CrossRefPubMed Goldstein BA, Navar AM, Pencina MJ, Ioannidis JP. Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review. J Am Med Inform Assoc. 2017;24:198–208.CrossRefPubMed
43.
go back to reference Waljee AK, Joyce JC, Wang S, et al. Algorithms outperform metabolite tests in predicting response of patients with inflammatory bowel disease to thiopurines. Clin Gastroenterol Hepatol. 2010;8:143–150.CrossRefPubMed Waljee AK, Joyce JC, Wang S, et al. Algorithms outperform metabolite tests in predicting response of patients with inflammatory bowel disease to thiopurines. Clin Gastroenterol Hepatol. 2010;8:143–150.CrossRefPubMed
44.
go back to reference Shah ND, Steyerberg EW, Kent DM. Big data and predictive analytics: Recalibrating expectations. JAMA. 2018;320:27–28.CrossRefPubMed Shah ND, Steyerberg EW, Kent DM. Big data and predictive analytics: Recalibrating expectations. JAMA. 2018;320:27–28.CrossRefPubMed
Metadata
Title
Machine Learning to Predict Outcomes in Patients with Acute Gastrointestinal Bleeding: A Systematic Review
Authors
Dennis Shung
Michael Simonov
Mark Gentry
Benjamin Au
Loren Laine
Publication date
01-08-2019
Publisher
Springer US
Published in
Digestive Diseases and Sciences / Issue 8/2019
Print ISSN: 0163-2116
Electronic ISSN: 1573-2568
DOI
https://doi.org/10.1007/s10620-019-05645-z

Other articles of this Issue 8/2019

Digestive Diseases and Sciences 8/2019 Go to the issue
Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine

Highlights from the ACC 2024 Congress

Year in Review: Pediatric cardiology

Watch Dr. Anne Marie Valente present the last year's highlights in pediatric and congenital heart disease in the official ACC.24 Year in Review session.

Year in Review: Pulmonary vascular disease

The last year's highlights in pulmonary vascular disease are presented by Dr. Jane Leopold in this official video from ACC.24.

Year in Review: Valvular heart disease

Watch Prof. William Zoghbi present the last year's highlights in valvular heart disease from the official ACC.24 Year in Review session.

Year in Review: Heart failure and cardiomyopathies

Watch this official video from ACC.24. Dr. Biykem Bozkurt discusses last year's major advances in heart failure and cardiomyopathies.