Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2020

01-12-2020 | Septicemia | Research article

How good is our diagnostic intuition? Clinician prediction of bacteremia in critically ill children

Authors: Katherine E. M. Hoops, James C. Fackler, Anne King, Elizabeth Colantuoni, Aaron M. Milstone, Charlotte Woods-Hill

Published in: BMC Medical Informatics and Decision Making | Issue 1/2020

Login to get access

Abstract

Background

Clinical intuition and nonanalytic reasoning play a major role in clinical hypothesis generation; however, clinicians’ intuition about whether a critically ill child is bacteremic has not been explored. We endeavored to assess pediatric critical care clinicians’ ability to predict bacteremia and to evaluate what affected the accuracy of those predictions.

Methods

We conducted a retrospective review of clinicians’ responses to a sepsis screening tool (“Early Sepsis Detection Tool” or “ESDT”) over 6 months. The ESDT was completed during the initial evaluation of a possible sepsis episode. If a culture was ordered, they were asked to predict if the culture would be positive or negative. Culture results were compared to predictions for each episode as well as vital signs and laboratory data from the preceding 24 h.

Results

From January to July 2017, 266 ESDTs were completed. Of the 135 blood culture episodes, 15% of cultures were positive. Clinicians correctly predicted patients with bacteremia in 82% of cases, but the positive predictive value was just 28% as there was a tendency to overestimate the presence of bacteremia. The negative predictive value was 96%. The presence of bandemia, thrombocytopenia, and abnormal CRP were associated with increased likelihood of correct positive prediction.

Conclusions

Clinicians are accurate in predicting critically ill children whose blood cultures, obtained for symptoms of sepsis, will be negative. Clinicians frequently overestimate the presence of bacteremia. The combination of evidence-based practice guidelines and bedside judgment should be leveraged to optimize diagnosis of bacteremia.
Appendix
Available only for authorised users
Literature
1.
go back to reference Noorbakhsh-Sabet N, Zand R, Zhang Y, Abedi V. Artificial intelligence transforms the future of health care. Am J Med. 2019;132(7):795–801.CrossRef Noorbakhsh-Sabet N, Zand R, Zhang Y, Abedi V. Artificial intelligence transforms the future of health care. Am J Med. 2019;132(7):795–801.CrossRef
2.
go back to reference Fackler JC, Watts C, Grome A, Miller T, et al. Critical care physician cognitive task analysis: an exploratory study. Crit Care. 2009;13(2):R33.CrossRef Fackler JC, Watts C, Grome A, Miller T, et al. Critical care physician cognitive task analysis: an exploratory study. Crit Care. 2009;13(2):R33.CrossRef
3.
go back to reference Tomasev N, Glorot X, Rae JW, Zielinski M, et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature. 2019;572(7767):116–9.CrossRef Tomasev N, Glorot X, Rae JW, Zielinski M, et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature. 2019;572(7767):116–9.CrossRef
4.
go back to reference Larkin J, McDermott J, Simon DP, Simon HA. Expert and novice performance in solving physics problems. Science. 1980;208(4450):1335–42.CrossRef Larkin J, McDermott J, Simon DP, Simon HA. Expert and novice performance in solving physics problems. Science. 1980;208(4450):1335–42.CrossRef
6.
go back to reference Rhee C, Kadri SS, Danner RL, Suffredini AF, et al. Diagnosing sepsis is subjective and highly variable: a survey of intensivists using case vignettes. Crit Care. 2016;20:89.CrossRef Rhee C, Kadri SS, Danner RL, Suffredini AF, et al. Diagnosing sepsis is subjective and highly variable: a survey of intensivists using case vignettes. Crit Care. 2016;20:89.CrossRef
7.
go back to reference Weiss SL, Fitzgerald JC, Maffei FA, Kane JM, et al. Discordant identification of pediatric severe sepsis by research and clinical definitions in the SPROUT international point prevalence study. Crit Care. 2015;19:325.CrossRef Weiss SL, Fitzgerald JC, Maffei FA, Kane JM, et al. Discordant identification of pediatric severe sepsis by research and clinical definitions in the SPROUT international point prevalence study. Crit Care. 2015;19:325.CrossRef
9.
go back to reference Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, et al. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017;4(1):e000234.CrossRef Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, et al. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017;4(1):e000234.CrossRef
10.
go back to reference Liu R, Greenstein JL, Granite SJ, Fackler JC, et al. Data-driven discovery of a novel sepsis pre-shock state predicts impending septic shock in the ICU. Sci Rep. 2019;9(1):6145.CrossRef Liu R, Greenstein JL, Granite SJ, Fackler JC, et al. Data-driven discovery of a novel sepsis pre-shock state predicts impending septic shock in the ICU. Sci Rep. 2019;9(1):6145.CrossRef
11.
go back to reference Brush JE Jr, Sherbino J, Norman GR. How expert clinicians intuitively recognize a medical diagnosis. Am J Med. 2017;130(6):629–34.CrossRef Brush JE Jr, Sherbino J, Norman GR. How expert clinicians intuitively recognize a medical diagnosis. Am J Med. 2017;130(6):629–34.CrossRef
12.
go back to reference Price A, Zulkosky K, White K, Pretz J. Accuracy of intuition in clinical decision-making among novice clinicians. J Adv Nurs. 2017;73(5):1147–57.CrossRef Price A, Zulkosky K, White K, Pretz J. Accuracy of intuition in clinical decision-making among novice clinicians. J Adv Nurs. 2017;73(5):1147–57.CrossRef
13.
go back to reference Snelson E, Ramlakhan S. Which observed behaviours may reassure physicians that a child is not septic? An international Delphi study. Arch Dis Child. 2018;103(9):864–7.CrossRef Snelson E, Ramlakhan S. Which observed behaviours may reassure physicians that a child is not septic? An international Delphi study. Arch Dis Child. 2018;103(9):864–7.CrossRef
14.
go back to reference Friedman CP. A “fundamental theorem” of biomedical informatics. J Am Med Inform Assoc. 2009;16(2):169–70.CrossRef Friedman CP. A “fundamental theorem” of biomedical informatics. J Am Med Inform Assoc. 2009;16(2):169–70.CrossRef
15.
go back to reference Shafazand S, Weinacker AB. Blood cultures in the critical care unit: improving utilization and yield. Chest. 2002;122(5):1727–36.CrossRef Shafazand S, Weinacker AB. Blood cultures in the critical care unit: improving utilization and yield. Chest. 2002;122(5):1727–36.CrossRef
16.
go back to reference Balogh EP, Miller BT, Ball JR, Committee on Diagnostic Error in Health Care; Board on Health Care Services; Institute of Medicine; Improving Diagnosis in Health Care. Washington (DC): National Academies Press (US); 2015. Balogh EP, Miller BT, Ball JR, Committee on Diagnostic Error in Health Care; Board on Health Care Services; Institute of Medicine; Improving Diagnosis in Health Care. Washington (DC): National Academies Press (US); 2015.
17.
go back to reference Alahmadi YM, Aldeyab MA, McElnay JC, Scott MG, et al. Clinical and economic impact of contaminated blood cultures within the hospital setting. J Hosp Infect. 2011;77(3):233–6.CrossRef Alahmadi YM, Aldeyab MA, McElnay JC, Scott MG, et al. Clinical and economic impact of contaminated blood cultures within the hospital setting. J Hosp Infect. 2011;77(3):233–6.CrossRef
18.
go back to reference Goff DA, Kullar R, Goldstein EJC, Gilchrist M, et al. A global call from five countries to collaborate in antibiotic stewardship: united we succeed, divided we might fail. Lancet Infect Dis. 2017;17(2):e56–63.CrossRef Goff DA, Kullar R, Goldstein EJC, Gilchrist M, et al. A global call from five countries to collaborate in antibiotic stewardship: united we succeed, divided we might fail. Lancet Infect Dis. 2017;17(2):e56–63.CrossRef
19.
go back to reference Woods-Hill CZ, Fackler J, Nelson McMillan K, Ascenzi J, et al. Association of a Clinical Practice Guideline with Blood Culture use in critically ill children. JAMA Pediatr. 2017;171(2):157–64.CrossRef Woods-Hill CZ, Fackler J, Nelson McMillan K, Ascenzi J, et al. Association of a Clinical Practice Guideline with Blood Culture use in critically ill children. JAMA Pediatr. 2017;171(2):157–64.CrossRef
20.
go back to reference Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. 2012;307(14):1513–6.CrossRef Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. 2012;307(14):1513–6.CrossRef
21.
go back to reference Esserman LJ, Thompson IM Jr, Reid B. Overdiagnosis and overtreatment in cancer: an opportunity for improvement. JAMA. 2013;310(8):797–8.CrossRef Esserman LJ, Thompson IM Jr, Reid B. Overdiagnosis and overtreatment in cancer: an opportunity for improvement. JAMA. 2013;310(8):797–8.CrossRef
22.
go back to reference Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, et al. The implications of regional variations in Medicare spending. Part 2: health outcomes and satisfaction with care. Ann Intern Med. 2003;138(4):288–98.CrossRef Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, et al. The implications of regional variations in Medicare spending. Part 2: health outcomes and satisfaction with care. Ann Intern Med. 2003;138(4):288–98.CrossRef
23.
go back to reference Garner S, Littlejohns P. Disinvestment from low value clinical interventions: NICEly done? BMJ. 2011;343:d4519.CrossRef Garner S, Littlejohns P. Disinvestment from low value clinical interventions: NICEly done? BMJ. 2011;343:d4519.CrossRef
24.
go back to reference Elshaug AG, Watt AM, Mundy L, Willis CD. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2012;197(10):556–60.CrossRef Elshaug AG, Watt AM, Mundy L, Willis CD. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2012;197(10):556–60.CrossRef
25.
go back to reference Joshi NPSF, Noseworthy TW. Reassessment of health technologies: obsolescence and waste. Canadian Assoc Drugs Technol Health. 2009. Joshi NPSF, Noseworthy TW. Reassessment of health technologies: obsolescence and waste. Canadian Assoc Drugs Technol Health. 2009.
26.
go back to reference Levinson W, Born K, Wolfson D. Choosing wisely campaigns: a work in Progress. JAMA. 2018;319(19):1975–6.CrossRef Levinson W, Born K, Wolfson D. Choosing wisely campaigns: a work in Progress. JAMA. 2018;319(19):1975–6.CrossRef
27.
go back to reference Lautenbach E, Localio R, Nachamkin I. Clinicians required very high sensitivity of a bacteremia prediction rule. J Clin Epidemiol. 2004;57(10):1104–6.CrossRef Lautenbach E, Localio R, Nachamkin I. Clinicians required very high sensitivity of a bacteremia prediction rule. J Clin Epidemiol. 2004;57(10):1104–6.CrossRef
28.
go back to reference Evans IVR, Phillips GS, Alpern ER, Angus DC, et al. Association between the New York Sepsis care mandate and in-hospital mortality for pediatric Sepsis. JAMA. 2018;320(4):358–67.CrossRef Evans IVR, Phillips GS, Alpern ER, Angus DC, et al. Association between the New York Sepsis care mandate and in-hospital mortality for pediatric Sepsis. JAMA. 2018;320(4):358–67.CrossRef
29.
go back to reference Lautz AJ, Dziorny AC, Denson AR, O'Connor KA, et al. Value of Procalcitonin measurement for early evidence of severe bacterial infections in the pediatric intensive care unit. J Pediatr. 2016;179:74–81 e72.CrossRef Lautz AJ, Dziorny AC, Denson AR, O'Connor KA, et al. Value of Procalcitonin measurement for early evidence of severe bacterial infections in the pediatric intensive care unit. J Pediatr. 2016;179:74–81 e72.CrossRef
30.
go back to reference Morgan DJ, Malani P, Diekema DJ. Diagnostic stewardship-leveraging the laboratory to improve antimicrobial use. JAMA. 2017;318(7):607–8.CrossRef Morgan DJ, Malani P, Diekema DJ. Diagnostic stewardship-leveraging the laboratory to improve antimicrobial use. JAMA. 2017;318(7):607–8.CrossRef
31.
go back to reference Woods-Hill CZ, Lee L, Xie A, King AF, et al. Dissemination of a novel framework to improve blood culture use in pediatric critical care. Pediatr Qual Saf. 2018;3(5):e112.CrossRef Woods-Hill CZ, Lee L, Xie A, King AF, et al. Dissemination of a novel framework to improve blood culture use in pediatric critical care. Pediatr Qual Saf. 2018;3(5):e112.CrossRef
Metadata
Title
How good is our diagnostic intuition? Clinician prediction of bacteremia in critically ill children
Authors
Katherine E. M. Hoops
James C. Fackler
Anne King
Elizabeth Colantuoni
Aaron M. Milstone
Charlotte Woods-Hill
Publication date
01-12-2020
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2020
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-020-01165-3

Other articles of this Issue 1/2020

BMC Medical Informatics and Decision Making 1/2020 Go to the issue