Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2012

Open Access 01-12-2012 | Technical advance

Spatial cluster detection using dynamic programming

Authors: Yuriy Sverchkov, Xia Jiang, Gregory F Cooper

Published in: BMC Medical Informatics and Decision Making | Issue 1/2012

Login to get access

Abstract

Background

The task of spatial cluster detection involves finding spatial regions where some property deviates from the norm or the expected value. In a probabilistic setting this task can be expressed as finding a region where some event is significantly more likely than usual. Spatial cluster detection is of interest in fields such as biosurveillance, mining of astronomical data, military surveillance, and analysis of fMRI images. In almost all such applications we are interested both in the question of whether a cluster exists in the data, and if it exists, we are interested in finding the most accurate characterization of the cluster.

Methods

We present a general dynamic programming algorithm for grid-based spatial cluster detection. The algorithm can be used for both Bayesian maximum a-posteriori (MAP) estimation of the most likely spatial distribution of clusters and Bayesian model averaging over a large space of spatial cluster distributions to compute the posterior probability of an unusual spatial clustering. The algorithm is explained and evaluated in the context of a biosurveillance application, specifically the detection and identification of Influenza outbreaks based on emergency department visits. A relatively simple underlying model is constructed for the purpose of evaluating the algorithm, and the algorithm is evaluated using the model and semi-synthetic test data.

Results

When compared to baseline methods, tests indicate that the new algorithm can improve MAP estimates under certain conditions: the greedy algorithm we compared our method to was found to be more sensitive to smaller outbreaks, while as the size of the outbreaks increases, in terms of area affected and proportion of individuals affected, our method overtakes the greedy algorithm in spatial precision and recall. The new algorithm performs on-par with baseline methods in the task of Bayesian model averaging.

Conclusions

We conclude that the dynamic programming algorithm performs on-par with other available methods for spatial cluster detection and point to its low computational cost and extendability as advantages in favor of further research and use of the algorithm.
Appendix
Available only for authorised users
Literature
1.
go back to reference Kulldorff M: Spatial scan statistics: models, calculations, and applications. Scan Statistics and Applications. Edited by: Glaz J, Balakrishnan M, Birkhauser. 1999, 303-322.CrossRef Kulldorff M: Spatial scan statistics: models, calculations, and applications. Scan Statistics and Applications. Edited by: Glaz J, Balakrishnan M, Birkhauser. 1999, 303-322.CrossRef
2.
go back to reference Wang X, Hutchinson R, Mitchell TM: Training fMRI Classifiers to Detect Cognitive States across Multiple Human Subjects. Advances in Neural Information Processing Systems. 2004, 18: 709-716. Wang X, Hutchinson R, Mitchell TM: Training fMRI Classifiers to Detect Cognitive States across Multiple Human Subjects. Advances in Neural Information Processing Systems. 2004, 18: 709-716.
3.
go back to reference Kulldorff M: A Spatial scan statistic. Commun Stat Theory Methods. 1997, 26 (6): 1481-1496. 10.1080/03610929708831995.CrossRef Kulldorff M: A Spatial scan statistic. Commun Stat Theory Methods. 1997, 26 (6): 1481-1496. 10.1080/03610929708831995.CrossRef
4.
go back to reference Kulldorff M, Huang L, Pickle L, Duczmal L: An elliptic spatial scan statistic. Stat Med. 2006, 25: 3929-3943. 10.1002/sim.2490.CrossRefPubMed Kulldorff M, Huang L, Pickle L, Duczmal L: An elliptic spatial scan statistic. Stat Med. 2006, 25: 3929-3943. 10.1002/sim.2490.CrossRefPubMed
5.
go back to reference Kulldorff M, Athas W, Feuer E, Miller B, Key C: Evaluating cluster alarms: A space-time scan statistic and cluster alarms in Los Alamos. Am J Public Health. 1998, 88: 1377-1380. 10.2105/AJPH.88.9.1377.CrossRefPubMedPubMedCentral Kulldorff M, Athas W, Feuer E, Miller B, Key C: Evaluating cluster alarms: A space-time scan statistic and cluster alarms in Los Alamos. Am J Public Health. 1998, 88: 1377-1380. 10.2105/AJPH.88.9.1377.CrossRefPubMedPubMedCentral
6.
go back to reference Kulldorff M: Prospective time-periodic geographical disease surveillance using a scan statistic. J R Stat Soc A. 2001, 164: 61-72. 10.1111/1467-985X.00186.CrossRef Kulldorff M: Prospective time-periodic geographical disease surveillance using a scan statistic. J R Stat Soc A. 2001, 164: 61-72. 10.1111/1467-985X.00186.CrossRef
7.
go back to reference Neill DB, Moore AW, Sabhnani MR: Detecting elongated disease clusters. Morb Mortal Wkly Rep. 54: doi:2005. Supplement on Syndromic Surveillance Neill DB, Moore AW, Sabhnani MR: Detecting elongated disease clusters. Morb Mortal Wkly Rep. 54: doi:2005. Supplement on Syndromic Surveillance
8.
go back to reference Duczmal L, Assunção R: A simulated annealing strategy for the detection of arbitrary shaped spatial clusters. Comput Stat Data Anal. 2004, 45: 269-286. 10.1016/S0167-9473(02)00302-X.CrossRef Duczmal L, Assunção R: A simulated annealing strategy for the detection of arbitrary shaped spatial clusters. Comput Stat Data Anal. 2004, 45: 269-286. 10.1016/S0167-9473(02)00302-X.CrossRef
9.
go back to reference Patil GP, Taillie C: Upper level set scan statistic for detecting arbitrarily shaped hotspots. Environ Ecol Stat. 2004, 11: 183-197.CrossRef Patil GP, Taillie C: Upper level set scan statistic for detecting arbitrarily shaped hotspots. Environ Ecol Stat. 2004, 11: 183-197.CrossRef
10.
11.
go back to reference Neill DB, Moore AW: Rapid Detection of Significant Spatial Clusters. Conference on Knowledge Discovery in Databases (KDD) 2004. Edited by: Guerke J, DuMouchel W. 2004 Neill DB, Moore AW: Rapid Detection of Significant Spatial Clusters. Conference on Knowledge Discovery in Databases (KDD) 2004. Edited by: Guerke J, DuMouchel W. 2004
12.
go back to reference Neill DB, Moore AW, Pereira F, Mitchell T: Detecting Significant Multidimensional Spatial Clusters. Advances in Neural Information Processing Systems. 2004, 17: Neill DB, Moore AW, Pereira F, Mitchell T: Detecting Significant Multidimensional Spatial Clusters. Advances in Neural Information Processing Systems. 2004, 17:
13.
go back to reference Neill D, Moore A, Cooper G: A Bayesian spatial scan statistic. Advances in Neural Information Processing Systems. Edited by: Y Weiss ea. 2005, 18: 1003-1010. Neill D, Moore A, Cooper G: A Bayesian spatial scan statistic. Advances in Neural Information Processing Systems. Edited by: Y Weiss ea. 2005, 18: 1003-1010.
14.
go back to reference Neill DB, Cooper GF: A Multivariate Bayesian Spatial Scan Statistics. Adv Dis Surveill. 2007, 2: 60- Neill DB, Cooper GF: A Multivariate Bayesian Spatial Scan Statistics. Adv Dis Surveill. 2007, 2: 60-
15.
go back to reference Neill DB, Cooper GF, Das K, Jiang X, Schneider J: Bayesian network scan statistics for multivariate pattern detection. Scan Statistics: Methods and Applications. Edited by: Glaz J, Pozdnyakov V, Wallenstein S, Birkhäuser. 2009 Neill DB, Cooper GF, Das K, Jiang X, Schneider J: Bayesian network scan statistics for multivariate pattern detection. Scan Statistics: Methods and Applications. Edited by: Glaz J, Pozdnyakov V, Wallenstein S, Birkhäuser. 2009
16.
go back to reference Cooper GF, Dash DH, Levander JD, Wong WK, Hogan WR, Wagner MM: Bayesian Biosurveillance of Disease Outbreaks. Proceedings of the 20th Annual Conference on Uncertainty in Artificial Intelligence (UAI-04). 2004, AUAI Press, 94-103. Cooper GF, Dash DH, Levander JD, Wong WK, Hogan WR, Wagner MM: Bayesian Biosurveillance of Disease Outbreaks. Proceedings of the 20th Annual Conference on Uncertainty in Artificial Intelligence (UAI-04). 2004, AUAI Press, 94-103.
17.
go back to reference Jiang X, Neill DB, Cooper GF: A Bayesian network model for spatial event surveillance. Int J Approximate Reasoning. 2010, 51 (2): 224-239. 10.1016/j.ijar.2009.01.001. Bayesian Model ViewsCrossRef Jiang X, Neill DB, Cooper GF: A Bayesian network model for spatial event surveillance. Int J Approximate Reasoning. 2010, 51 (2): 224-239. 10.1016/j.ijar.2009.01.001. Bayesian Model ViewsCrossRef
18.
go back to reference Jiang X, Cooper GF: A Recursive Algorithm for Spatial Cluster Detection. Proceedings of the Symposium of the American Medical Informatics Association (AMIA). 2007, 369-373. Jiang X, Cooper GF: A Recursive Algorithm for Spatial Cluster Detection. Proceedings of the Symposium of the American Medical Informatics Association (AMIA). 2007, 369-373.
19.
go back to reference Shen Y, Wong WK, Levander JD, Cooper GF: An Outbreak Detection Algorithm that Efficiently Performs Complete Bayesian Model Averaging Over All Possible Spatial Distributions of Disease. Adv Dis Surveill. 2007, 4: 113- Shen Y, Wong WK, Levander JD, Cooper GF: An Outbreak Detection Algorithm that Efficiently Performs Complete Bayesian Model Averaging Over All Possible Spatial Distributions of Disease. Adv Dis Surveill. 2007, 4: 113-
20.
go back to reference Ester M, peter Kriegel H, S J, Xu X: A density-based algorithm for discovering clusters in large spatial databases with noise. 2nd International Conference on Knowledge Discovery and Data Mining, Portland, OR. 1996, AAAI Press, 226-231. Ester M, peter Kriegel H, S J, Xu X: A density-based algorithm for discovering clusters in large spatial databases with noise. 2nd International Conference on Knowledge Discovery and Data Mining, Portland, OR. 1996, AAAI Press, 226-231.
21.
go back to reference Pei T, Jasra A, Hand DJ, Zhu AX, Zhou C: DECODE: A new method for discovering clusters of different densities in spatial data. Data Min Knowl Discov. 2009, 18 (3): 337-369. 10.1007/s10618-008-0120-3.CrossRef Pei T, Jasra A, Hand DJ, Zhu AX, Zhou C: DECODE: A new method for discovering clusters of different densities in spatial data. Data Min Knowl Discov. 2009, 18 (3): 337-369. 10.1007/s10618-008-0120-3.CrossRef
22.
go back to reference Heckerman D: A tutorial on learning with Bayesian networks. Tech rep, Learning in Graphical Models. 1995 Heckerman D: A tutorial on learning with Bayesian networks. Tech rep, Learning in Graphical Models. 1995
23.
go back to reference Buntine WL: Operations for Learning with Graphical Models. J Artif Intell Res. 1994, 2: 159-225. Buntine WL: Operations for Learning with Graphical Models. J Artif Intell Res. 1994, 2: 159-225.
24.
go back to reference Jiang X: A Bayesian Network Model for Spatio-Temporal Event Surveillance. PhD dissertation. 2008, University of Pittsburgh, Department of Biomedical Informatics Jiang X: A Bayesian Network Model for Spatio-Temporal Event Surveillance. PhD dissertation. 2008, University of Pittsburgh, Department of Biomedical Informatics
25.
go back to reference DeLong ER, DeLong DM, Clarke-Pearson DL: Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics. 1988, 44 (3): 837-845. 10.2307/2531595.CrossRefPubMed DeLong ER, DeLong DM, Clarke-Pearson DL: Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics. 1988, 44 (3): 837-845. 10.2307/2531595.CrossRefPubMed
27.
go back to reference Zhang Z, Assunçãdo R, Kulldorff M: Spatial Scan Statistics Adjusted for Multiple Clusters. Journal of Probability and Statistics. 2010, 2010: Zhang Z, Assunçãdo R, Kulldorff M: Spatial Scan Statistics Adjusted for Multiple Clusters. Journal of Probability and Statistics. 2010, 2010:
Metadata
Title
Spatial cluster detection using dynamic programming
Authors
Yuriy Sverchkov
Xia Jiang
Gregory F Cooper
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2012
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/1472-6947-12-22

Other articles of this Issue 1/2012

BMC Medical Informatics and Decision Making 1/2012 Go to the issue