Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2019

Open Access 01-12-2019 | Research article

Using decision fusion methods to improve outbreak detection in disease surveillance

Authors: Gaëtan Texier, Rodrigue S. Allodji, Loty Diop, Jean-Baptiste Meynard, Liliane Pellegrin, Hervé Chaudet

Published in: BMC Medical Informatics and Decision Making | Issue 1/2019

Login to get access

Abstract

Background

When outbreak detection algorithms (ODAs) are considered individually, the task of outbreak detection can be seen as a classification problem and the ODA as a sensor providing a binary decision (outbreak yes or no) for each day of surveillance. When they are considered jointly (in cases where several ODAs analyze the same surveillance signal), the outbreak detection problem should be treated as a decision fusion (DF) problem of multiple sensors.

Methods

This study evaluated the benefit for a decisions support system of using DF methods (fusing multiple ODA decisions) compared to using a single method of outbreak detection. For each day, we merged the decisions of six ODAs using 5 DF methods (two voting methods, logistic regression, CART and Bayesian network - BN). Classical metrics of accuracy, prediction and timelines were used during the evaluation steps.

Results

In our results, we observed the greatest gain (77%) in positive predictive value compared to the best ODA if we used DF methods with a learning step (BN, logistic regression, and CART).

Conclusions

To identify disease outbreaks in systems using several ODAs to analyze surveillance data, we recommend using a DF method based on a Bayesian network. This method is at least equivalent to the best of the algorithms considered, regardless of the situation faced by the system. For those less familiar with this kind of technique, we propose that logistic regression be used when a training dataset is available.
Appendix
Available only for authorised users
Literature
1.
go back to reference Texier G, Buisson Y. From outbreak detection to anticipation. Rev Epidemiol Sante Publique. 2010;58(6):425–33.CrossRef Texier G, Buisson Y. From outbreak detection to anticipation. Rev Epidemiol Sante Publique. 2010;58(6):425–33.CrossRef
2.
go back to reference Texier G. Evaluation methods for temporal outbreak dectection algorithms in early warning surveillance. PhD. Marseille: Aix-Marseille University; 2016. Texier G. Evaluation methods for temporal outbreak dectection algorithms in early warning surveillance. PhD. Marseille: Aix-Marseille University; 2016.
3.
go back to reference Bravata DM, McDonald KM, Smith WM, Rydzak C, Szeto H, Buckeridge DL, Haberland C, Owens DK. Systematic review: surveillance systems for early detection of bioterrorism-related diseases. Ann Intern Med. 2004;140(11):910–22.CrossRef Bravata DM, McDonald KM, Smith WM, Rydzak C, Szeto H, Buckeridge DL, Haberland C, Owens DK. Systematic review: surveillance systems for early detection of bioterrorism-related diseases. Ann Intern Med. 2004;140(11):910–22.CrossRef
4.
go back to reference Jackson ML, Baer A, Painter I, Duchin J. A simulation study comparing aberration detection algorithms for syndromic surveillance. BMC Med Informat Decis Making. 2007;7:6.CrossRef Jackson ML, Baer A, Painter I, Duchin J. A simulation study comparing aberration detection algorithms for syndromic surveillance. BMC Med Informat Decis Making. 2007;7:6.CrossRef
5.
go back to reference Buckeridge DL. Outbreak detection through automated surveillance: a review of the determinants of detection. J Biomed Inform. 2007;40(4):370–9.CrossRef Buckeridge DL. Outbreak detection through automated surveillance: a review of the determinants of detection. J Biomed Inform. 2007;40(4):370–9.CrossRef
6.
go back to reference Lombardo JS, Buckeridge DL. Disease surveillance: a public health informatics approach. Hoboken: Wiley; 2007.CrossRef Lombardo JS, Buckeridge DL. Disease surveillance: a public health informatics approach. Hoboken: Wiley; 2007.CrossRef
7.
go back to reference Texier G, Farouh M, Pellegrin L, Jackson ML, Meynard JB, Deparis X, Chaudet H. Outbreak definition by change point analysis: a tool for public health decision? BMC Med Inform Decis Making. 2016;16:33.CrossRef Texier G, Farouh M, Pellegrin L, Jackson ML, Meynard JB, Deparis X, Chaudet H. Outbreak definition by change point analysis: a tool for public health decision? BMC Med Inform Decis Making. 2016;16:33.CrossRef
8.
go back to reference Chen H, Zeng D, Yan P. Public health syndromic surveillance systems. In: Infectious disease informatics: syndromic surveillance for public health and BioDefense. Boston: Springer US; 2010. p. 9–31.CrossRef Chen H, Zeng D, Yan P. Public health syndromic surveillance systems. In: Infectious disease informatics: syndromic surveillance for public health and BioDefense. Boston: Springer US; 2010. p. 9–31.CrossRef
9.
go back to reference Fourati H, editor. Multisensor Data Fusion: From Algorithms and Architectural Design to Applications (Book). United States: Series: Devices, Circuits, and Systems, CRC Press, Taylor & Francis Group LLC; 2015. Fourati H, editor. Multisensor Data Fusion: From Algorithms and Architectural Design to Applications (Book). United States: Series: Devices, Circuits, and Systems, CRC Press, Taylor & Francis Group LLC; 2015.
10.
go back to reference Khaleghi B, Khamis A, Karray FO, Razavi SN. Multisensor data fusion: a review of the state-of-the-art. Information Fusion. 2013;14(1):28–44.CrossRef Khaleghi B, Khamis A, Karray FO, Razavi SN. Multisensor data fusion: a review of the state-of-the-art. Information Fusion. 2013;14(1):28–44.CrossRef
11.
go back to reference Li SZ. Encyclopedia of Biometrics: I-Z, vol. 1. New York: Springer Science & Business Media; 2009. Li SZ. Encyclopedia of Biometrics: I-Z, vol. 1. New York: Springer Science & Business Media; 2009.
12.
go back to reference Rolka H, Burkom H, Cooper GF, Kulldorff M, Madigan D, Wong WK. Issues in applied statistics for public health bioterrorism surveillance using multiple data streams: research needs. Stat Med. 2007;26(8):1834–56.CrossRef Rolka H, Burkom H, Cooper GF, Kulldorff M, Madigan D, Wong WK. Issues in applied statistics for public health bioterrorism surveillance using multiple data streams: research needs. Stat Med. 2007;26(8):1834–56.CrossRef
13.
go back to reference Burkom H, Loschen W, Mnatsakanyan Z, Lombardo J. Tradeoffs driving policy and research decisions in biosurveillance. Johns Hopkins APL Tech Dig. 2008;27(4):299–312. Burkom H, Loschen W, Mnatsakanyan Z, Lombardo J. Tradeoffs driving policy and research decisions in biosurveillance. Johns Hopkins APL Tech Dig. 2008;27(4):299–312.
14.
go back to reference Burkom HS, Ramac-Thomas L, Babin S, Holtry R, Mnatsakanyan Z, Yund C. An integrated approach for fusion of environmental and human health data for disease surveillance. Stat Med. 2011;30(5):470–9.CrossRef Burkom HS, Ramac-Thomas L, Babin S, Holtry R, Mnatsakanyan Z, Yund C. An integrated approach for fusion of environmental and human health data for disease surveillance. Stat Med. 2011;30(5):470–9.CrossRef
15.
go back to reference Mnatsakanyan ZR, Burkom HS, Coberly JS, Lombardo JS. Bayesian information fusion networks for biosurveillance applications. J Am Med Inform Assoc. 2009;16(6):855–63.CrossRef Mnatsakanyan ZR, Burkom HS, Coberly JS, Lombardo JS. Bayesian information fusion networks for biosurveillance applications. J Am Med Inform Assoc. 2009;16(6):855–63.CrossRef
16.
go back to reference Najmi AH, Magruder SF. An adaptive prediction and detection algorithm for multistream syndromic surveillance. BMC Med Inform Decis Making. 2005;5:33.CrossRef Najmi AH, Magruder SF. An adaptive prediction and detection algorithm for multistream syndromic surveillance. BMC Med Inform Decis Making. 2005;5:33.CrossRef
17.
go back to reference Lau EH, Cowling BJ, Ho LM, Leung GM. Optimizing use of multistream influenza sentinel surveillance data. Emerg Infect Dis. 2008;14(7):1154–7.CrossRef Lau EH, Cowling BJ, Ho LM, Leung GM. Optimizing use of multistream influenza sentinel surveillance data. Emerg Infect Dis. 2008;14(7):1154–7.CrossRef
18.
go back to reference Jafarpour N, Precup D, Izadi M, Buckeridge D. Using hierarchical mixture of experts model for fusion of outbreak detection methods. AMIA Annu Symp Proc. 2013;2013:663–9.PubMedPubMedCentral Jafarpour N, Precup D, Izadi M, Buckeridge D. Using hierarchical mixture of experts model for fusion of outbreak detection methods. AMIA Annu Symp Proc. 2013;2013:663–9.PubMedPubMedCentral
19.
go back to reference Dietterich TG. Ensemble Methods in Machine Learning. In: Multiple Classifier Systems: First International Workshop, MCS 2000 Cagliari, Italy, June 21–23, 2000 Proceedings. Berlin, Heidelberg: Springer Berlin Heidelberg; 2000. p. 1–15. Dietterich TG. Ensemble Methods in Machine Learning. In: Multiple Classifier Systems: First International Workshop, MCS 2000 Cagliari, Italy, June 21–23, 2000 Proceedings. Berlin, Heidelberg: Springer Berlin Heidelberg; 2000. p. 1–15.
20.
go back to reference Texier G, Jackson ML, Siwe L, Meynard JB, Deparis X, Chaudet H. Building test data from real outbreaks for evaluating detection algorithms. PLoS One. 2017;12(9):e0183992.CrossRef Texier G, Jackson ML, Siwe L, Meynard JB, Deparis X, Chaudet H. Building test data from real outbreaks for evaluating detection algorithms. PLoS One. 2017;12(9):e0183992.CrossRef
21.
go back to reference Centers for Disease C, Prevention. Outbreaks of gastroenteritis associated with noroviruses on cruise ships--United States, 2002. MMWR Morb Mortal Wkly Rep. 2002;51(49):1112–5. Centers for Disease C, Prevention. Outbreaks of gastroenteritis associated with noroviruses on cruise ships--United States, 2002. MMWR Morb Mortal Wkly Rep. 2002;51(49):1112–5.
22.
go back to reference Jafarpour N, Izadi M, Precup D, Buckeridge DL. Quantifying the determinants of outbreak detection performance through simulation and machine learning. J Biomed Inform. 2015;53:180–7.CrossRef Jafarpour N, Izadi M, Precup D, Buckeridge DL. Quantifying the determinants of outbreak detection performance through simulation and machine learning. J Biomed Inform. 2015;53:180–7.CrossRef
24.
go back to reference Rossi G, Lampugnani L, Marchi M. An approximate CUSUM procedure for surveillance of health events. Stat Med. 1999;18(16):2111–22.CrossRef Rossi G, Lampugnani L, Marchi M. An approximate CUSUM procedure for surveillance of health events. Stat Med. 1999;18(16):2111–22.CrossRef
25.
go back to reference Hutwagner L, Thompson W, Seeman GM, Treadwell T. The bioterrorism preparedness and response early aberration reporting system (EARS). J Urban Health. 2003;80(2 Suppl 1):i89–96.PubMedPubMedCentral Hutwagner L, Thompson W, Seeman GM, Treadwell T. The bioterrorism preparedness and response early aberration reporting system (EARS). J Urban Health. 2003;80(2 Suppl 1):i89–96.PubMedPubMedCentral
26.
go back to reference Farrington CP, Andrews NJ, Beale AD, Catchpole MA. A statistical algorithm for the early detection of outbreaks of infectious disease. J R Stat Soc Ser A. 1996;159(3):547.CrossRef Farrington CP, Andrews NJ, Beale AD, Catchpole MA. A statistical algorithm for the early detection of outbreaks of infectious disease. J R Stat Soc Ser A. 1996;159(3):547.CrossRef
27.
go back to reference Dasarathy BV. Sensor fusion potential exploitation-innovative architectures and illustrative applications. Proc IEEE. 1997;85(1):24–38.CrossRef Dasarathy BV. Sensor fusion potential exploitation-innovative architectures and illustrative applications. Proc IEEE. 1997;85(1):24–38.CrossRef
28.
go back to reference Ruta D, Gabrys B. An overview of classifier fusion methods. Comput Inf Syst. 2000;7(1):1–10. Ruta D, Gabrys B. An overview of classifier fusion methods. Comput Inf Syst. 2000;7(1):1–10.
29.
go back to reference Xu L, Krzyzak A, Suen C. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans Syst Man Cybernet. 1992;22(3):418–35.CrossRef Xu L, Krzyzak A, Suen C. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans Syst Man Cybernet. 1992;22(3):418–35.CrossRef
30.
go back to reference Sinha A, Chen H, Danu DG, Kirubarajan T, Farooq M. Estimation and decision fusion: a survey. Neurocomputing. 2008;71(13–15):2650–6.CrossRef Sinha A, Chen H, Danu DG, Kirubarajan T, Farooq M. Estimation and decision fusion: a survey. Neurocomputing. 2008;71(13–15):2650–6.CrossRef
31.
go back to reference Jordan MI, Jacobs RA. Hierarchical mixtures of experts and the EM algorithm. Neural Comput. 1994;6(2):181–214.CrossRef Jordan MI, Jacobs RA. Hierarchical mixtures of experts and the EM algorithm. Neural Comput. 1994;6(2):181–214.CrossRef
32.
go back to reference Rahman AFR, Alam H, Fairhurst MC. Multiple classifier combination for character recognition: revisiting the majority voting system and its variations. In: Document analysis systems V: 5th international workshop, vol. 2002. Berlin, Heidelberg: Springer Berlin Heidelberg; 2002. p. 167–78.CrossRef Rahman AFR, Alam H, Fairhurst MC. Multiple classifier combination for character recognition: revisiting the majority voting system and its variations. In: Document analysis systems V: 5th international workshop, vol. 2002. Berlin, Heidelberg: Springer Berlin Heidelberg; 2002. p. 167–78.CrossRef
33.
go back to reference Hosmer DW, Lemeshow S, Sturdivant RX. Applied logistic regression. New York: Wiley; 2013. Hosmer DW, Lemeshow S, Sturdivant RX. Applied logistic regression. New York: Wiley; 2013.
34.
go back to reference Harper PR. A review and comparison of classification algorithms for medical decision making. Health Policy. 2005;71(3):315–31.CrossRef Harper PR. A review and comparison of classification algorithms for medical decision making. Health Policy. 2005;71(3):315–31.CrossRef
35.
go back to reference Bishop CM. Pattern recognition and machine learning. Information science and statistics. New York: Springer-Verlag; 2006. Bishop CM. Pattern recognition and machine learning. Information science and statistics. New York: Springer-Verlag; 2006.
36.
go back to reference Breiman L. Classification and regression trees. Belmont: Wadsworth International Group; 1984. Breiman L. Classification and regression trees. Belmont: Wadsworth International Group; 1984.
38.
go back to reference Scutari M. Learning Bayesian networks with the bnlearn R package. J Stat Softw. 2010;35(3):22.CrossRef Scutari M. Learning Bayesian networks with the bnlearn R package. J Stat Softw. 2010;35(3):22.CrossRef
39.
go back to reference Nagarajan R, Scutari M, Lbre S. Bayesian Networks in R: with Applications in Systems Biology. New York: Springer-Verlag; 2013. Nagarajan R, Scutari M, Lbre S. Bayesian Networks in R: with Applications in Systems Biology. New York: Springer-Verlag; 2013.
41.
go back to reference Kleinman KP, Abrams AM. Assessing surveillance using sensitivity, specificity and timeliness. Stat Methods Med Res. 2006;15(5):445–64.CrossRef Kleinman KP, Abrams AM. Assessing surveillance using sensitivity, specificity and timeliness. Stat Methods Med Res. 2006;15(5):445–64.CrossRef
42.
go back to reference Buckeridge DL, Burkom H, Campbell M, Hogan WR, Moore AW. Algorithms for rapid outbreak detection: a research synthesis. J Biomed Inform. 2005;38(2):99–113.CrossRef Buckeridge DL, Burkom H, Campbell M, Hogan WR, Moore AW. Algorithms for rapid outbreak detection: a research synthesis. J Biomed Inform. 2005;38(2):99–113.CrossRef
43.
go back to reference Lam L, Suen SY. Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans Syst Man Cybern Syst Hum. 1997;27(5):553–68.CrossRef Lam L, Suen SY. Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans Syst Man Cybern Syst Hum. 1997;27(5):553–68.CrossRef
44.
45.
go back to reference Verlinde P, Druyts P, Cholet G, Acheroy M. Applying Bayes based classifiers for Decision fusion in a multimodal identity verification system. In: International symposium on pattern recognition February 1999; Brussels, Belgium. 1999. Verlinde P, Druyts P, Cholet G, Acheroy M. Applying Bayes based classifiers for Decision fusion in a multimodal identity verification system. In: International symposium on pattern recognition February 1999; Brussels, Belgium. 1999.
46.
go back to reference Ho TK, Hull JJ, Srihari SN. Decision combination in multiple classifier systems. IEEE Trans Pattern Anal Mach Intell. 1994;16(1):66–75.CrossRef Ho TK, Hull JJ, Srihari SN. Decision combination in multiple classifier systems. IEEE Trans Pattern Anal Mach Intell. 1994;16(1):66–75.CrossRef
47.
go back to reference Altmann A, Rosen-Zvi M, Prosperi M, Aharoni E, Neuvirth H, Schulter E, Buch J, Struck D, Peres Y, Incardona F, et al. Comparison of classifier fusion methods for predicting response to anti HIV-1 therapy. PLoS One. 2008;3(10):e3470.CrossRef Altmann A, Rosen-Zvi M, Prosperi M, Aharoni E, Neuvirth H, Schulter E, Buch J, Struck D, Peres Y, Incardona F, et al. Comparison of classifier fusion methods for predicting response to anti HIV-1 therapy. PLoS One. 2008;3(10):e3470.CrossRef
48.
go back to reference Hastie T, Tibshirani R, Friedman JH. The elements of statistical learning: data mining, inference, and prediction. 2nd ed. New York: Springer-Verlag; 2009. Hastie T, Tibshirani R, Friedman JH. The elements of statistical learning: data mining, inference, and prediction. 2nd ed. New York: Springer-Verlag; 2009.
49.
go back to reference McNeish D. On using Bayesian methods to address small sample problems. Struct Equ Model Multidiscip J. 2016;23(5):750–73.CrossRef McNeish D. On using Bayesian methods to address small sample problems. Struct Equ Model Multidiscip J. 2016;23(5):750–73.CrossRef
50.
go back to reference Ducher M, Kalbacher E, Combarnous F, Finaz de Vilaine J, McGregor B, Fouque D, Fauvel JP. Comparison of a Bayesian network with a logistic regression model to forecast IgA nephropathy. Biomed Res Int. 2013;2013:686150.CrossRef Ducher M, Kalbacher E, Combarnous F, Finaz de Vilaine J, McGregor B, Fouque D, Fauvel JP. Comparison of a Bayesian network with a logistic regression model to forecast IgA nephropathy. Biomed Res Int. 2013;2013:686150.CrossRef
51.
go back to reference Giabbanelli PJ, Peters JG. An algebraic approach to combining classifiers. Procedia Comput Sci. 2015;51(C):1545–54.CrossRef Giabbanelli PJ, Peters JG. An algebraic approach to combining classifiers. Procedia Comput Sci. 2015;51(C):1545–54.CrossRef
52.
go back to reference Texier G, Pellegrin L, Vignal C, Meynard JB, Deparis X, Chaudet H. Dealing with uncertainty when using a surveillance system. Int J Med Inform. 2017;104:65–73.CrossRef Texier G, Pellegrin L, Vignal C, Meynard JB, Deparis X, Chaudet H. Dealing with uncertainty when using a surveillance system. Int J Med Inform. 2017;104:65–73.CrossRef
53.
go back to reference Salas E, Klein G. Linking expertise and naturalistic decision making. Mahwah: Lawrence Erlbaum Associates Publishers; 2001.CrossRef Salas E, Klein G. Linking expertise and naturalistic decision making. Mahwah: Lawrence Erlbaum Associates Publishers; 2001.CrossRef
54.
go back to reference Chaudet H, Pellegrin L, Bonnardel N. Special issue on the 11th conference on naturalistic decision making. Cogn Tech Work. 2015;17(3):315–8.CrossRef Chaudet H, Pellegrin L, Bonnardel N. Special issue on the 11th conference on naturalistic decision making. Cogn Tech Work. 2015;17(3):315–8.CrossRef
Metadata
Title
Using decision fusion methods to improve outbreak detection in disease surveillance
Authors
Gaëtan Texier
Rodrigue S. Allodji
Loty Diop
Jean-Baptiste Meynard
Liliane Pellegrin
Hervé Chaudet
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2019
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-019-0774-3

Other articles of this Issue 1/2019

BMC Medical Informatics and Decision Making 1/2019 Go to the issue