Top

BMC Medical Informatics and Decision Making

Published in:

Open Access 01-12-2023 | Central Nervous System Trauma | Research

A deep neural network framework to derive interpretable decision rules for accurate traumatic brain injury identification of infants

Authors: Baiming Zou, Xinlei Mi, Elizabeth Stone, Fei Zou

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Abstract

Objective

We aimed to develop a robust framework to model the complex association between clinical features and traumatic brain injury (TBI) risk in children under age two, and identify significant features to derive clinical decision rules for triage decisions.

Methods

In this retrospective study, four frequently used machine learning models, i.e., support vector machine (SVM), random forest (RF), deep neural network (DNN), and XGBoost (XGB), were compared to identify significant clinical features from 24 input features associated with the TBI risk in children under age two under the permutation feature importance test (PermFIT) framework by using the publicly available data set from the Pediatric Emergency Care Applied Research Network (PECARN) study. The prediction accuracy was determined by comparing the predicted TBI status with the computed tomography (CT) scan results since CT scan is the gold standard for diagnosing TBI.

Results

At a significance level of \(p = 0.05\), DNN, RF, XGB, and SVM identified 9, 1, 2, and 4 significant features, respectively. In a comparison of accuracy (Accuracy), the area under the curve (AUC), and the precision-recall area under the curve (PR-AUC), the permutation feature importance test for DNN model was the most powerful framework for identifying significant features and outperformed other methods, i.e., RF, XGB, and SVM, with Accuracy, AUC, and PR-AUC as 0.915, 0.794, and 0.974, respectively.

Conclusion

These results indicate that the PermFIT-DNN framework robustly identifies significant clinical features associated with TBI status and improves prediction performance. The findings could be used to inform the development of clinical decision tools designed to inform triage decisions.

Martin J, Weaver M, Barnato A, Yabes J, Yealy D, Roberts M. Variation in emergency department head computed tomography use for pediatric head trauma. Acad Emerg Med. 2014;21(9):987–95. https://doi.org/10.1111/acem.12458.CrossRef

Martin J, Weaver M, Yealy D, Mannix R. Trends in visits for traumatic brain injury to emergency departments in the United States. JAMA. 2014;311(8):1917–9. https://doi.org/10.1001/jama.2014.3979.CrossRef

Amanullah S, Schlichting L, Linakis S, Steele D, Linakis J. Emergency Department Visits Owing to Intentional and Unintentional Traumatic Brain Injury among Infants in the United States: A Population-Based Assessment. J Pediatr. 2018;203:259–65. https://doi.org/10.1016/j.jpeds.2018.08.023.CrossRefPubMed

Dayan P, Holmes J, et al. Association of traumatic brain injuries with vomiting in children with blunt head trauma. Ann Emerg Med. 2014;63(6):657–65. https://doi.org/10.1016/j.annemergmed.2014.01.009.CrossRefPubMed

Greenes D, Schutzman S. Clinical indicators of intracranial injury in head-injured infants. Pediatrics. 1999;104(4):861–7. https://doi.org/10.1542/peds.104.4.861.CrossRefPubMed

Kuppermann N, Holmes J, Dayan P, et al. Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet. 2009;374:1160–70. https://doi.org/10.1016/S0140-6736(09)61558-0.CrossRefPubMed

Shlosberg D, Benifla M, Kaufer D, Friedman A. Blood-brain barrier breakdown as a therapeutic target in traumatic brain injury. Nat Rev Neurol. 2010;6(7):393–403. https://doi.org/10.1038/nrneurol.2010.74.CrossRefPubMedPubMedCentral

Stone E, Davis L, McCoy T, Travers D, Van Horn E, Krowchuk H. A secondary analysis to inform a clinical decision rule for predicting skull fracture and intracranial injury in children under age 2. Res Nurs Health. 2020;43(1):28–39. https://doi.org/10.1002/nur.21993.CrossRefPubMed

Griffin E, Lippmann S, Travers D, Woodard E. A matched-cohort study of pediatric head injuries: collecting data to inform an evidence-based triage assessment. J Emerg Nurs. 2014;40(1):98–104. https://doi.org/10.1016/j.jen.2013.07.001.CrossRefPubMed

10.

Atabaki S, Hoyle JJ, Schunk J, Monroe D, Alpern E, Quayle K, et al. Comparison of Prediction Rules and Clinician Suspicion for Identifying Children With Clinically Important Brain Injuries After Blunt Head Trauma. Acad Emerg Med. 2016;23(5):566–75. https://doi.org/10.1111/acem.12923.CrossRefPubMed

11.

Dayan P, Holmes J, et al. Risk of traumatic brain injuries in children younger than 24 months with isolated scalp hematomas. Ann Emerg Med. 2014;64(2):153–62. https://doi.org/10.1016/j.annemergmed.2014.02.003.CrossRefPubMed

12.

Lee L, Monroe D, et al. Isolated loss of consciousness in children with minor blunt head trauma. JAMA Pediatr. 2014;168(9):837–43. https://doi.org/10.1001/jamapediatrics.2014.361.CrossRefPubMed

13.

Dietrich A, Bowman M, Ginn-Pease M, Kosnik E, King D. Pediatric head injuries: can clinical factors reliably predict an abnormality on computed tomography? Ann Emerg Med. 1993;22(10):1535–40. https://doi.org/10.1016/s0196-0644(05)81254-5.CrossRefPubMed

14.

Osmond M, Klassen T, Wells G, et al. Pediatric Emergency Research Canada (PERC) Head Injury Study Group. CATCH: A clinical decision rule for the use of computed tomography in children with minor head injury. CMAJ. 2010;182(4):341–348. https://doi.org/10.1503/cmaj.091421.

15.

Dunning J, Daly J, Lomas J, et al. Children’s Head Injury Algorithm. Derivation of the children’s head injury algorithm for the prediction of important clinical events decision rule for head injury in children. Arch Dis Child. 2006;91(11):885–891. https://doi.org/10.1136/adc.2005.083980.

16.

Harrell F, Lee K, Mark D. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–7. https://doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4.CrossRefPubMed

17.

McNett M. A review of the predictive ability of Glasgow Coma Scale scores in head-injured patients. J Neurosci Nurs. 2007;39(2):68–75. https://doi.org/10.1097/01376517-200704000-00002.CrossRefPubMed

18.

Jacobs B, Beems T, Stulemeijer M, van Vugt A, van der Vliet T, Borm G, et al. Outcome prediction in mild traumatic brain injury: age and clinical variables are stronger predictors than CT abnormalities. J Neurotrauma. 2010;27(4):665–8. https://doi.org/10.1089/neu.2009.1059.CrossRef

19.

Easter J, Bakes K, Dhaliwal J, Miller M, Caruso E, Haukoos J. Comparison of PECARN, CATCH, and CHALICE rules for children with minor head injury: a prospective cohort study. Ann Emerg Med. 2014;64(2):145–52. https://doi.org/10.1016/j.annemergmed.2014.01.030.CrossRefPubMedPubMedCentral

20.

Cowley L, Farewell D, Maguire S, Kemp A. Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature. Diagn Progn Res. 2019;3(16):837–43. https://doi.org/10.1186/s41512-019-0060-y.CrossRef

21.

Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer-Verlag; 2001.CrossRef

22.

Bertsimas D, Dunn J, Steele D, Trikalinos T, Wang Y. Comparison of Machine Learning Optimal Classification Trees With the Pediatric Emergency Care Applied Research Network Head Trauma Decision Rules. JAMA Pediatr. 2019;173(7):648–56. https://doi.org/10.1001/jamapediatrics.2019.1068.CrossRefPubMedPubMedCentral

23.

Leung M, et al. Deep learning of the tissue-regulated splicing code. Bioinformatics. 2014;30:i121–9. https://doi.org/10.1093/bioinformatics/btu277.CrossRefPubMedPubMedCentral

24.

Goto T, Camargo C, et al. Machine Learning-Based Prediction of Clinical Outcomes for Children During Emergency Department Triage. JAMA Netw Open. 2019;2(1):e186937. https://doi.org/10.1001/jamanetworkopen.2018.6937.CrossRefPubMedPubMedCentral

25.

Mi X, Zou F, Zhu R. Bagging and deep learning in optimal individualized treatment rules. Biometrics. 2019;75:674–84. https://doi.org/10.1111/biom.12990.CrossRefPubMed

26.

Mi X, Tighe P, Zou F, Zou B. A Deep Learning Semiparametric Regression for Adjusting Complex Confounding Structures. Ann Appl Stat. 2021;15(3):1086–100. https://doi.org/10.1214/21-AOAS1481.CrossRef

27.

Joseph J, James M, Lalit B. Machine learning at the clinical bedside-the ghost in the machine. JAMA Pediatr. 2019;173(7):622–4. https://doi.org/10.1001/jamapediatrics.2019.1075.CrossRef

28.

Mi X, Zou B, Zou F, Hu J. Permutation-based Identification of Important Biomarkers for Complex Diseases via Machine Learning Models. Nat Commun. 2021;12(1):3008. https://doi.org/10.1038/s41467-021-22756-2.CrossRefPubMedPubMedCentral

29.

Altmann A, Toloşi L, Sander O, Lengauer T. Permutation importance: a corrected feature importance measure. Bioinformatics. 2010;26(10):1340–7. https://doi.org/10.1093/bioinformatics/btq134.CrossRefPubMed

30.

Bengio Y. Learning Deep Architectures for AI. Found Trends Mach Learn. 2009;2(1):1–127. https://doi.org/10.1561/2200000006.CrossRef

31.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436. https://doi.org/10.1038/nature14539.CrossRefPubMed

32.

Amit Y, Geman D. Shape quantization and recognition with randomized trees. Neural Comput. 1997;9(7):1545–88. https://doi.org/10.1162/neco.1997.9.7.1545.CrossRef

33.

Breiman L. Random forests. Mach Learn. 2001;45:5–32. https://doi.org/10.1023/A:1010933404324.CrossRef

34.

Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. New York: Association for Computing Machinery; 2016. p. 785–94.

35.

Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–97. https://doi.org/10.1007/BF00994018.CrossRef

36.

Drucker H, Burges C, Kaufman L, Smola A, Vapnik V. Support Vector Regression Machines. In: Advances in Neural Information Processing Systems. 1997. p. 155-161. https://doi.org/10.5555/2998981.2999003.

37.

Cybenko G. Approximations by superpositions of sigmoidal functions. Math Control Signals Syst. 1989;2(4):303–314. https://doi.org/10.1007/BF02551274.

38.

Byrd RH, Chin GM, Nocedal J, Wu Y. Sample size selection in optimization methods for machine learning. Math Program. 2012;134(1):127–55. https://doi.org/10.1007/s10107-012-0572-5.CrossRef

39.

Mei S. A mean field view of the landscape of two-layer neural networks. Proc Natl Acad Sci. 2018;115(33):E7665–71. https://doi.org/10.1073/pnas.1806579115.CrossRefPubMedPubMedCentral

40.

Kinga D, Adam JB. A method for stochastic optimization. In: International Conference on Learning Representations (ICLR), vol. 5. Ithaca; 2015. p. 1–13. https://arXiv.org.

41.

Breiman L. Bagging predictors. Mach Learn. 1996;24(2):123–140. https://doi.org/10.1007/BF00058655.

42.

Hansen LK, Salamon P. Neural network ensembles. IEEE Trans Pattern Anal Mach Intell. 1990;12(10):993–1001. https://doi.org/10.1109/34.58871.

43.

Zhou Z, Wu J, Tang W. Ensembling neural networks: Many could be better than all. Artif Intell. 2002;137:239–63. https://doi.org/10.1016/S0004-3702(02)00190-X.CrossRef

44.

Putin E, Mamoshina P, Aliper A, Korzinkin M, Moskalev A, Kolosov A, et al. Deep biomarkers of human aging: application of deep neural networks to biomarker development. Aging (Albany NY). 2016;8(5):1021–1033. https://doi.org/10.18632/aging.100968.

Title: A deep neural network framework to derive interpretable decision rules for accurate traumatic brain injury identification of infants
Authors: Baiming Zou
Xinlei Mi
Elizabeth Stone
Fei Zou
Publication date: 01-12-2023
Publisher: BioMed Central
Keywords: Central Nervous System Trauma
Hematoma
Published in: BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI: https://doi.org/10.1186/s12911-023-02155-x

Keynote webinar | Spotlight on medication adherence

Springer Medicine

A deep neural network framework to derive interpretable decision rules for accurate traumatic brain injury identification of infants

Abstract

Objective

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Objective

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2023

Opportunities and challenges of virtual reality-based interventions for patients with breast cancer: a systematic review

Construction of a knowledge graph for breast cancer diagnosis based on Chinese electronic medical records: development and usability study

Health care needs, eHealth literacy, use of mobile phone functionalities, and intention to use it for self-management purposes by informal caregivers of children with burns: a survey study

An intelligent decision support system for acute postoperative endophthalmitis: design, development and evaluation of a smartphone application

Development and usability evaluation of a mHealth application for albinism self-management

De-identified Bayesian personal identity matching for privacy-preserving record linkage despite errors: development and validation