Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2023

Open Access 01-12-2023 | Research

Confidence-based laboratory test reduction recommendation algorithm

Authors: Tongtong Huang, Linda T. Li, Elmer V. Bernstam, Xiaoqian Jiang

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Login to get access

Abstract

Background

We propose a new deep learning model to identify unnecessary hemoglobin (Hgb) tests for patients admitted to the hospital, which can help reduce health risks and healthcare costs.

Methods

We collected internal patient data from a teaching hospital in Houston and external patient data from the MIMIC III database. The study used a conservative definition of unnecessary laboratory tests, which was defined as stable (i.e., stability) and below the lower normal bound (i.e., normality). Considering that machine learning models may yield less reliable results when trained on noisy inputs containing low-quality information, we estimated prediction confidence to assess the reliability of predicted outcomes. We adopted a “select and predict” design philosophy to maximize prediction performance by selectively considering samples with high prediction confidence for recommendations. Our model accommodated irregularly sampled observational data to make full use of variable correlations (i.e., with other laboratory test values) and temporal dependencies (i.e., previous laboratory tests performed within the same encounter) in selecting candidates for training and prediction.

Results

The proposed model demonstrated remarkable Hgb prediction performance, achieving a normality AUC of 95.89% and a Hgb stability AUC of 95.94%, while recommending a reduction of 9.91% of Hgb tests that were deemed unnecessary. Additionally, the model could generalize well to external patients admitted to another hospital.

Conclusions

This study introduces a novel deep learning model with the potential to significantly reduce healthcare costs and improve patient outcomes by identifying unnecessary laboratory tests for hospitalized patients.
Appendix
Available only for authorised users
Literature
1.
go back to reference Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, et al. Evidence for overuse of medical services around the world. Lancet. 2017;390:156–68.CrossRefPubMedPubMedCentral Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, et al. Evidence for overuse of medical services around the world. Lancet. 2017;390:156–68.CrossRefPubMedPubMedCentral
2.
go back to reference Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA. 1998;280:1000–5.CrossRefPubMed Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA. 1998;280:1000–5.CrossRefPubMed
3.
go back to reference Shander A, Corwin HL. A narrative review on hospital-acquired anemia: keeping blood where it belongs. Transfus Med Rev. 2020;34:195–9.CrossRefPubMed Shander A, Corwin HL. A narrative review on hospital-acquired anemia: keeping blood where it belongs. Transfus Med Rev. 2020;34:195–9.CrossRefPubMed
4.
go back to reference Statland BE, Winkel P. Utilization review and management of laboratory testing in the ambulatory setting. Med Clin North Am. 1987;71:719–32.CrossRefPubMed Statland BE, Winkel P. Utilization review and management of laboratory testing in the ambulatory setting. Med Clin North Am. 1987;71:719–32.CrossRefPubMed
5.
go back to reference Roy SK, Hom J, Mackey L, Shah N, Chen JH. Predicting low information laboratory diagnostic tests. AMIA Jt Summits Transl Sci Proc. 2018;2017:217–26.PubMed Roy SK, Hom J, Mackey L, Shah N, Chen JH. Predicting low information laboratory diagnostic tests. AMIA Jt Summits Transl Sci Proc. 2018;2017:217–26.PubMed
6.
go back to reference Nasserinejad K, de Kort W, Baart M, Komárek A, van Rosmalen J, Lesaffre E. Predicting hemoglobin levels in whole blood donors using transition models and mixed effects models. BMC Med Res Methodol. 2013;13:62.CrossRefPubMedPubMedCentral Nasserinejad K, de Kort W, Baart M, Komárek A, van Rosmalen J, Lesaffre E. Predicting hemoglobin levels in whole blood donors using transition models and mixed effects models. BMC Med Res Methodol. 2013;13:62.CrossRefPubMedPubMedCentral
7.
go back to reference Lobo B, Abdel-Rahman E, Brown D, Dunn L, Bowman B. A recurrent neural network approach to predicting hemoglobin trajectories in patients with End-Stage Renal Disease. Artif Intell Med. 2020;104:101823.CrossRefPubMed Lobo B, Abdel-Rahman E, Brown D, Dunn L, Bowman B. A recurrent neural network approach to predicting hemoglobin trajectories in patients with End-Stage Renal Disease. Artif Intell Med. 2020;104:101823.CrossRefPubMed
8.
go back to reference Pellicer-Valero OJ, Cattinelli I, Neri L, Mari F, Martín-Guerrero JD, Barbieri C. Enhanced prediction of hemoglobin concentration in a very large cohort of hemodialysis patients by means of deep recurrent neural networks. Artif Intell Med. 2020;107:101898.CrossRefPubMed Pellicer-Valero OJ, Cattinelli I, Neri L, Mari F, Martín-Guerrero JD, Barbieri C. Enhanced prediction of hemoglobin concentration in a very large cohort of hemodialysis patients by means of deep recurrent neural networks. Artif Intell Med. 2020;107:101898.CrossRefPubMed
9.
go back to reference Cismondi F, Celi LA, Fialho AS, Vieira SM, Reti SR, Sousa JMC, et al. Reducing unnecessary lab testing in the ICU with artificial intelligence. Int J Med Inform. 2013;82:345–58.CrossRefPubMed Cismondi F, Celi LA, Fialho AS, Vieira SM, Reti SR, Sousa JMC, et al. Reducing unnecessary lab testing in the ICU with artificial intelligence. Int J Med Inform. 2013;82:345–58.CrossRefPubMed
10.
go back to reference Aikens RC, Balasubramanian S, Chen JH. A Machine Learning Approach to Predicting the Stability of Inpatient Lab Test Results. AMIA Jt Summits Transl Sci Proc. 2019;2019:515–23.PubMedPubMedCentral Aikens RC, Balasubramanian S, Chen JH. A Machine Learning Approach to Predicting the Stability of Inpatient Lab Test Results. AMIA Jt Summits Transl Sci Proc. 2019;2019:515–23.PubMedPubMedCentral
11.
go back to reference Yu L, Zhang Q, Bernstam EV, Jiang X. Predict or draw blood: An integrated method to reduce lab tests. J Biomed Inform. 2020;104:103394.CrossRefPubMed Yu L, Zhang Q, Bernstam EV, Jiang X. Predict or draw blood: An integrated method to reduce lab tests. J Biomed Inform. 2020;104:103394.CrossRefPubMed
12.
go back to reference Yu L, Li L, Bernstam E, Jiang X. A deep learning solution to recommend laboratory reduction strategies in ICU. Int J Med Inform. 2020;144:104282.CrossRefPubMed Yu L, Li L, Bernstam E, Jiang X. A deep learning solution to recommend laboratory reduction strategies in ICU. Int J Med Inform. 2020;144:104282.CrossRefPubMed
13.
go back to reference Kuragano T, Matsumura O, Matsuda A, Hara T, Kiyomoto H, Murata T, et al. Association between hemoglobin variability, serum ferritin levels, and adverse events/mortality in maintenance hemodialysis patients. Kidney Int. 2014;86:845–54.CrossRefPubMed Kuragano T, Matsumura O, Matsuda A, Hara T, Kiyomoto H, Murata T, et al. Association between hemoglobin variability, serum ferritin levels, and adverse events/mortality in maintenance hemodialysis patients. Kidney Int. 2014;86:845–54.CrossRefPubMed
14.
go back to reference Sabatine MS, Morrow DA, Giugliano RP, Burton PBJ, Murphy SA, McCabe CH, et al. Association of hemoglobin levels with clinical outcomes in acute coronary syndromes. Circulation. 2005;111:2042–9.CrossRefPubMed Sabatine MS, Morrow DA, Giugliano RP, Burton PBJ, Murphy SA, McCabe CH, et al. Association of hemoglobin levels with clinical outcomes in acute coronary syndromes. Circulation. 2005;111:2042–9.CrossRefPubMed
15.
go back to reference Gal Y, Ghahramani Z. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. arXiv [stat.ML]. 2015. Gal Y, Ghahramani Z. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. arXiv [stat.ML]. 2015.
16.
go back to reference Neal RM. Bayesian Learning for Neural Networks. Springer Science & Business Media; 2012. Neal RM. Bayesian Learning for Neural Networks. Springer Science & Business Media; 2012.
17.
go back to reference Geifman Y, El-Yaniv R. SelectiveNet: A deep neural network with an integrated reject option. arXiv [cs.LG]. 09--15 Jun 2019;:2151–9. Geifman Y, El-Yaniv R. SelectiveNet: A deep neural network with an integrated reject option. arXiv [cs.LG]. 09--15 Jun 2019;:2151–9.
18.
go back to reference Lipton ZC, Kale DC, Wetzel R. Modeling missing data in clinical time series with RNNs. arXiv [cs.LG]. 2016. Lipton ZC, Kale DC, Wetzel R. Modeling missing data in clinical time series with RNNs. arXiv [cs.LG]. 2016.
19.
go back to reference Raffel C, Ellis DPW. Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems. arXiv [cs.LG]. 2015. Raffel C, Ellis DPW. Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems. arXiv [cs.LG]. 2015.
20.
go back to reference Liu P, Qiu X, Huang X. Recurrent Neural Network for Text Classification with Multi-Task Learning. arXiv [cs.CL]. 2016. Liu P, Qiu X, Huang X. Recurrent Neural Network for Text Classification with Multi-Task Learning. arXiv [cs.CL]. 2016.
21.
go back to reference Potra FA, Wright SJ. Interior-point methods. J Comput Appl Math. 2000;124:281–302.CrossRef Potra FA, Wright SJ. Interior-point methods. J Comput Appl Math. 2000;124:281–302.CrossRef
22.
go back to reference Cramer JS. Predictive performance of the binary logit model in unbalanced samples. J Royal Statistical Soc D. 1999;48:85–94. Cramer JS. Predictive performance of the binary logit model in unbalanced samples. J Royal Statistical Soc D. 1999;48:85–94.
23.
go back to reference Guerrero SC, Sridhar S, Edmonds C, Solis CF, Zhang J, McPherson DD, et al. Access to routinely collected clinical data for research: a process implemented at an Academic Medical Center. Clin Transl Sci. 2019;12:231–5.CrossRefPubMedPubMedCentral Guerrero SC, Sridhar S, Edmonds C, Solis CF, Zhang J, McPherson DD, et al. Access to routinely collected clinical data for research: a process implemented at an Academic Medical Center. Clin Transl Sci. 2019;12:231–5.CrossRefPubMedPubMedCentral
Metadata
Title
Confidence-based laboratory test reduction recommendation algorithm
Authors
Tongtong Huang
Linda T. Li
Elmer V. Bernstam
Xiaoqian Jiang
Publication date
01-12-2023
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-023-02187-3

Other articles of this Issue 1/2023

BMC Medical Informatics and Decision Making 1/2023 Go to the issue