Skip to main content
Top
Published in: Journal of General Internal Medicine 1/2024

18-09-2023 | Original Research

Derivation and External Validation of the Ottawa Bloodstream Infection Model for Acutely Ill Adults

Authors: Brett N. Hryciw, MD, Stefan Rodic, MD, Shehab Selim, MD, Chuqi Wang, MD, Melissa-Fay Lepage, MD, Long Hoai Nguyen, MD, Vineet Goyal, MD, Carl van Walraven, MD, MSc

Published in: Journal of General Internal Medicine | Issue 1/2024

Login to get access

Abstract

Background

Knowing the probability that patients have a bloodstream infection (BSI) could influence the ordering of blood cultures and interpretation of their preliminary results. Many previous BSI probability models have limited applicability and accuracy. This study used currently recommended modeling techniques and a large sample to derive and validate the Ottawa BSI Model.

Methods

At a tertiary care teaching hospital, we retrieved a random sample of 4180 adults having blood cultures in our emergency department or during the initial 48 h of the encounter. Variable selection was based on clinical experience and a systematic review of previous model performance. Model performance was measured in a temporal external validation group of 4680 patients.

Results

A total of 327 derivation patients had a BSI (8.0%). BSI risk increased with increased number of culture sets (2 sets: adjusted odds ratio [aOR] 1.52 [1.10–2.11]; 3 sets: 1.99 [0.86–4.58]); with indwelling catheter (aOR 2.07 [1.34–3.20); with increasing temperature, heart rate, and neutrophil–lymphocyte ratio; and with decreasing systolic blood pressure, platelet count, urea-creatinine ratio, and estimated glomerular filtration rate. In the temporal external validation group, model discrimination was good (c-statistic 0.71 [0.69–0.74]) and calibration was very good (integrated calibration index .016 [.010–.024]). Exclusion of validation patients with acute SARS-CoV-2 infection improved discrimination slightly (c-statistic 0.73 [0.69–0.76]).

Conclusions

The Ottawa BSI Model uses commonly available data to return an expected BSI probability for acutely ill patients. However, it cannot exclude BSI and its complexity requires computational assistance to use.
Appendix
Available only for authorised users
Literature
1.
go back to reference Bates DW, Goldman L, Lee TH. Contaminant blood cultures and resource utilization. The true consequences of false-positive results. JAMA 1991;265(3):365–369. Bates DW, Goldman L, Lee TH. Contaminant blood cultures and resource utilization. The true consequences of false-positive results. JAMA 1991;265(3):365–369.
2.
go back to reference Eliakim-Raz N, Bates DW, Leibovici L. Predicting bacteraemia in validated models - a systematic review. Clinical Microbiology and Infection 2015;21(4):295-301.CrossRefPubMed Eliakim-Raz N, Bates DW, Leibovici L. Predicting bacteraemia in validated models - a systematic review. Clinical Microbiology and Infection 2015;21(4):295-301.CrossRefPubMed
3.
go back to reference Rodic S, Hryciw BN, Selim S et al. Concurrent external validation of bloodstream infection probability models. Clinical Microbiology and Infection. In press. Rodic S, Hryciw BN, Selim S et al. Concurrent external validation of bloodstream infection probability models. Clinical Microbiology and Infection. In press.
4.
go back to reference Wolff RF, Moons KGM, Riley RD et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 2019;170(1):51-58.CrossRefPubMed Wolff RF, Moons KGM, Riley RD et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 2019;170(1):51-58.CrossRefPubMed
5.
go back to reference Riley RD, Ensor J, Snell KIE et al. Calculating the sample size required for developing a clinical prediction model. Br Med J 2020;368:m441.CrossRef Riley RD, Ensor J, Snell KIE et al. Calculating the sample size required for developing a clinical prediction model. Br Med J 2020;368:m441.CrossRef
6.
go back to reference Institute for Quality Management in Healthcare. Consensus practice recommendations - BACT - Blood cultures. Toronto: 2012 Institute for Quality Management in Healthcare. Consensus practice recommendations - BACT - Blood cultures. Toronto: 2012
7.
go back to reference Steyerberg EW. Overfitting and optimism in prediction models. Clinical Prediction Models: a practical approach to development, validation, and updating. 2 ed. New York: Springer; 2019:95–112. Steyerberg EW. Overfitting and optimism in prediction models. Clinical Prediction Models: a practical approach to development, validation, and updating. 2 ed. New York: Springer; 2019:95–112.
8.
go back to reference Cockerill FR, III, Wilson JW, Vetter EA et al. Optimal testing parameters for blood cultures. Clinical Infectious Diseases 2004;38(12):1724-30.CrossRefPubMed Cockerill FR, III, Wilson JW, Vetter EA et al. Optimal testing parameters for blood cultures. Clinical Infectious Diseases 2004;38(12):1724-30.CrossRefPubMed
9.
go back to reference Jiang J, Liu R, Yu X et al. The neutrophil-lymphocyte count ratio as a diagnostic marker for bacteraemia: a systematic review and meta-analysis. The American Journal of Emergency Medicine 2019;37(8):1482-1489.CrossRefPubMed Jiang J, Liu R, Yu X et al. The neutrophil-lymphocyte count ratio as a diagnostic marker for bacteraemia: a systematic review and meta-analysis. The American Journal of Emergency Medicine 2019;37(8):1482-1489.CrossRefPubMed
10.
go back to reference van Walraven C, Tuna M. The Network Relative Model Accuracy (NeRMA) score can quantify the relative accuracy of prediction models in concurrent external validations. Journal of Evaluation in Clinical Practice. In press. van Walraven C, Tuna M. The Network Relative Model Accuracy (NeRMA) score can quantify the relative accuracy of prediction models in concurrent external validations. Journal of Evaluation in Clinical Practice. In press.
11.
go back to reference Leibovici L, Greenshtain S, Cohen O, Mor F, Wysenbeek AJ. Bacteremia in febrile patients: a clinical model for diagnosis. Arch Intern Med 1991;151(9):1801-1806.CrossRefPubMed Leibovici L, Greenshtain S, Cohen O, Mor F, Wysenbeek AJ. Bacteremia in febrile patients: a clinical model for diagnosis. Arch Intern Med 1991;151(9):1801-1806.CrossRefPubMed
12.
go back to reference Levey AS, Coresh J, Greene T et al. Using standardized serum creatinine values in the modification of diet in renal disease study equation for estimating glomerular filtration rate. Ann Intern Med 2006;145(4):247-254.CrossRefPubMed Levey AS, Coresh J, Greene T et al. Using standardized serum creatinine values in the modification of diet in renal disease study equation for estimating glomerular filtration rate. Ann Intern Med 2006;145(4):247-254.CrossRefPubMed
13.
go back to reference van Walraven C, McCudden C, Austin PC. Laboratory test results differ significantly when they are not ordered: implications for imputing missing lab data. J Clin Epidemiol. In press. van Walraven C, McCudden C, Austin PC. Laboratory test results differ significantly when they are not ordered: implications for imputing missing lab data. J Clin Epidemiol. In press.
14.
go back to reference Sauerbrei W, Meier-Hirmer C, Benner A, Royston P. Multivariable regression model building by using fractional polynomials: description of SAS, STATA and R programs. Computational Statistics and Data Analysis 2006;50(12):3464-3485.CrossRef Sauerbrei W, Meier-Hirmer C, Benner A, Royston P. Multivariable regression model building by using fractional polynomials: description of SAS, STATA and R programs. Computational Statistics and Data Analysis 2006;50(12):3464-3485.CrossRef
15.
go back to reference Steyerberg EW. Coding of categorical and continuous variables. Clinical Prediction Models: a practical approach to development, validation, and updating. 2 ed. New York: Springer; 2019:175–190. Steyerberg EW. Coding of categorical and continuous variables. Clinical Prediction Models: a practical approach to development, validation, and updating. 2 ed. New York: Springer; 2019:175–190.
16.
go back to reference Sullivan LM, Massaro JM, D’Agostino RB, Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med 2004;23(10):1631-1660.CrossRefPubMed Sullivan LM, Massaro JM, D’Agostino RB, Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med 2004;23(10):1631-1660.CrossRefPubMed
17.
go back to reference Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med 2014;33(3):517-535.CrossRefPubMed Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med 2014;33(3):517-535.CrossRefPubMed
18.
go back to reference Efron B, Tibshirani RJ. Confidence intervals based on bootstrap percentiles. An introduction to the bootstrap. New York: Chapman&Hall; 1994:168–177. Efron B, Tibshirani RJ. Confidence intervals based on bootstrap percentiles. An introduction to the bootstrap. New York: Chapman&Hall; 1994:168–177.
19.
go back to reference Riley RD, Debray TPA, Collins GS et al. Minimum sample size for external validation of a clinical prediction model with a binary outcome. Stat Med 2021;40(19):4230-4251.CrossRefPubMed Riley RD, Debray TPA, Collins GS et al. Minimum sample size for external validation of a clinical prediction model with a binary outcome. Stat Med 2021;40(19):4230-4251.CrossRefPubMed
20.
go back to reference Karvanen J. The statistical basis of laboratory data normalization. Drug information journal : DIJ / Drug Information Association 2003;37(1):101-107.CrossRef Karvanen J. The statistical basis of laboratory data normalization. Drug information journal : DIJ / Drug Information Association 2003;37(1):101-107.CrossRef
21.
go back to reference Colgan R, Nicolle LE, McGlone A, Hooton TM. Asymptomatic bacteriuria in adults. American Family Physician 2006;74(6):985-990.PubMed Colgan R, Nicolle LE, McGlone A, Hooton TM. Asymptomatic bacteriuria in adults. American Family Physician 2006;74(6):985-990.PubMed
22.
go back to reference Mozes B, Milatiner D, Block C, Blumstein Z, Halkin H. Inconsistency of a model aimed at predicting bacteremia in hospitalized patients. J Clin Epidemiol 1993;46(9):1035-1040.CrossRefPubMed Mozes B, Milatiner D, Block C, Blumstein Z, Halkin H. Inconsistency of a model aimed at predicting bacteremia in hospitalized patients. J Clin Epidemiol 1993;46(9):1035-1040.CrossRefPubMed
23.
go back to reference Shapiro NI, Wolfe REM, Wright SB, Moore R, Bates DW. Who needs a blood culture? A prospectively derived and validated prediction rule. The Journal of Emergency Medicine 2008;35(3):255-264.CrossRefPubMed Shapiro NI, Wolfe REM, Wright SB, Moore R, Bates DW. Who needs a blood culture? A prospectively derived and validated prediction rule. The Journal of Emergency Medicine 2008;35(3):255-264.CrossRefPubMed
24.
go back to reference Jessen MK, Mackenhauer J, Hvass AM et al. Prediction of bacteremia in the emergency department: an external validation of a clinical decision rule. Eur J Emerg Med 2016;23(1):44-49.CrossRefPubMed Jessen MK, Mackenhauer J, Hvass AM et al. Prediction of bacteremia in the emergency department: an external validation of a clinical decision rule. Eur J Emerg Med 2016;23(1):44-49.CrossRefPubMed
Metadata
Title
Derivation and External Validation of the Ottawa Bloodstream Infection Model for Acutely Ill Adults
Authors
Brett N. Hryciw, MD
Stefan Rodic, MD
Shehab Selim, MD
Chuqi Wang, MD
Melissa-Fay Lepage, MD
Long Hoai Nguyen, MD
Vineet Goyal, MD
Carl van Walraven, MD, MSc
Publication date
18-09-2023
Publisher
Springer International Publishing
Published in
Journal of General Internal Medicine / Issue 1/2024
Print ISSN: 0884-8734
Electronic ISSN: 1525-1497
DOI
https://doi.org/10.1007/s11606-023-08407-w

Other articles of this Issue 1/2024

Journal of General Internal Medicine 1/2024 Go to the issue
Live Webinar | 27-06-2024 | 18:00 (CEST)

Keynote webinar | Spotlight on medication adherence

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

WHO estimates that half of all patients worldwide are non-adherent to their prescribed medication. The consequences of poor adherence can be catastrophic, on both the individual and population level.

Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.

Prof. Kevin Dolgin
Prof. Florian Limbourg
Prof. Anoop Chauhan
Developed by: Springer Medicine
Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine