Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2013

Open Access 01-12-2013 | Research article

Determination of French influenza outbreaks periods between 1985 and 2011 through a web-based Delphi method

Authors: Marion Debin, Cécile Souty, Clément Turbelin, Thierry Blanchon, Pierre-Yves Boëlle, Thomas Hanslik, Gilles Hejblum, Yann Le Strat, Flavien Quintus, Alessandra Falchi, the DelFluWeb study group

Published in: BMC Medical Informatics and Decision Making | Issue 1/2013

Login to get access

Abstract

Background

Assessing the accuracy of influenza epidemic periods determined by statistical models is important to improve the performance of algorithms used in real-time syndromic surveillance systems. This is a difficult problem to address in the absence of a reliable gold standard. The objective of this study is to establish an expert-based determination of the start and the end of influenza epidemics in France.

Methods

A three-round international web-based Delphi survey was proposed to 288 eligible influenza experts. Fifty-seven (20%) experts completed the three-rounds of the study. The experts were invited to indicate the starting and the ending week of influenza epidemics, on 32 time-series graphs of influenza seasons drawn using data from the French Sentinelles Network (Influenza-like illness incidence rates) and virological data from the WHO-FluNet. Twenty-six of 32 time-series graphs proposed corresponded to each of the French influenza seasons observed between 1985 and 2011. Six influenza seasons were proposed twice at each round to measure variation among expert responses.

Results

We obtained consensual results for 88% (23/26) of the epidemic periods. In two or three rounds (depending on the season) answers gathered around modes, and the internal control demonstrated a good reproducibility of the answers. Virological data did not appear to have a significant impact on the answers or the level of consensus, except for a season with a major mismatch between virological and incidence data timings.

Conclusions

Thanks to this international web-based Delphi survey, we obtained reproducible, stable and consensual results for the majority of the French influenza epidemic curves analysed. The detailed curves together with the estimates from the Delphi study could be a helpful tool for assessing the performance of statistical outbreak detection methods, in order to optimize them.
Appendix
Available only for authorised users
Literature
1.
go back to reference Steiner SH, Grant K, Coory M, Kelly HA: Detecting the start of an influenza outbreak using exponentially weighted moving average charts. BMC Med Inform Decis Mak. 2010, 10: 37-10.1186/1472-6947-10-37.CrossRefPubMedPubMedCentral Steiner SH, Grant K, Coory M, Kelly HA: Detecting the start of an influenza outbreak using exponentially weighted moving average charts. BMC Med Inform Decis Mak. 2010, 10: 37-10.1186/1472-6947-10-37.CrossRefPubMedPubMedCentral
2.
go back to reference Kelly H, Carville K, Grant K, Jacoby P, Tran T, Barr I: Estimation of influenza vaccine effectiveness from routine surveillance data. PLoS One. 2009, 4 (3): e5079-10.1371/journal.pone.0005079.CrossRefPubMedPubMedCentral Kelly H, Carville K, Grant K, Jacoby P, Tran T, Barr I: Estimation of influenza vaccine effectiveness from routine surveillance data. PLoS One. 2009, 4 (3): e5079-10.1371/journal.pone.0005079.CrossRefPubMedPubMedCentral
3.
go back to reference Valleron AJ, Bouvet E, Garnerin P, Menares J, Heard I, Letrait S, Lefaucheux J: A computer network for the surveillance of communicable diseases: the French experiment. Am J Public Health. 1986, 76 (11): 1289-1292. 10.2105/AJPH.76.11.1289.CrossRefPubMedPubMedCentral Valleron AJ, Bouvet E, Garnerin P, Menares J, Heard I, Letrait S, Lefaucheux J: A computer network for the surveillance of communicable diseases: the French experiment. Am J Public Health. 1986, 76 (11): 1289-1292. 10.2105/AJPH.76.11.1289.CrossRefPubMedPubMedCentral
4.
go back to reference Flahault A, Blanchon T, Dorleans Y, Toubiana L, Vibert JF, Valleron AJ: Virtual surveillance of communicable diseases: a 20-year experience in France. Stat Methods Med Res. 2006, 15 (5): 413-421.PubMed Flahault A, Blanchon T, Dorleans Y, Toubiana L, Vibert JF, Valleron AJ: Virtual surveillance of communicable diseases: a 20-year experience in France. Stat Methods Med Res. 2006, 15 (5): 413-421.PubMed
5.
go back to reference Widdowson MA, Bosman A, van Straten E, Tinga M, Chaves S, van Eerden L, van Pelt W: Automated, laboratory-based system using the Internet for disease outbreak detection, the Netherlands. Emerg Infect Dis. 2003, 9 (9): 1046-1052. 10.3201/eid0909.020450.CrossRefPubMedPubMedCentral Widdowson MA, Bosman A, van Straten E, Tinga M, Chaves S, van Eerden L, van Pelt W: Automated, laboratory-based system using the Internet for disease outbreak detection, the Netherlands. Emerg Infect Dis. 2003, 9 (9): 1046-1052. 10.3201/eid0909.020450.CrossRefPubMedPubMedCentral
6.
go back to reference Stern L, Lightfoot D: Automated outbreak detection: a quantitative retrospective analysis. Epidemiol Infect. 1999, 122 (1): 103-110. 10.1017/S0950268898001939.CrossRefPubMedPubMedCentral Stern L, Lightfoot D: Automated outbreak detection: a quantitative retrospective analysis. Epidemiol Infect. 1999, 122 (1): 103-110. 10.1017/S0950268898001939.CrossRefPubMedPubMedCentral
7.
go back to reference Hutwagner LC, Maloney EK, Bean NH, Slutsker L, Martin SM: Using laboratory-based surveillance data for prevention: an algorithm for detecting Salmonella outbreaks. Emerg Infect Dis. 1997, 3 (3): 395-400. 10.3201/eid0303.970322.CrossRefPubMedPubMedCentral Hutwagner LC, Maloney EK, Bean NH, Slutsker L, Martin SM: Using laboratory-based surveillance data for prevention: an algorithm for detecting Salmonella outbreaks. Emerg Infect Dis. 1997, 3 (3): 395-400. 10.3201/eid0303.970322.CrossRefPubMedPubMedCentral
8.
9.
go back to reference Cowling BJ, Wong IO, Ho LM, Riley S, Leung GM: Methods for monitoring influenza surveillance data. Int J Epidemiol. 2006, 35 (5): 1314-1321. 10.1093/ije/dyl162.CrossRefPubMed Cowling BJ, Wong IO, Ho LM, Riley S, Leung GM: Methods for monitoring influenza surveillance data. Int J Epidemiol. 2006, 35 (5): 1314-1321. 10.1093/ije/dyl162.CrossRefPubMed
10.
go back to reference Costagliola D, Flahault A, Galinec D, Garnerin P, Menares J, Valleron AJ: A routine tool for detection and assessment of epidemics of influenza-like syndromes in France. Am J Public Health. 1991, 81 (1): 97-99. 10.2105/AJPH.81.1.97.CrossRefPubMedPubMedCentral Costagliola D, Flahault A, Galinec D, Garnerin P, Menares J, Valleron AJ: A routine tool for detection and assessment of epidemics of influenza-like syndromes in France. Am J Public Health. 1991, 81 (1): 97-99. 10.2105/AJPH.81.1.97.CrossRefPubMedPubMedCentral
11.
go back to reference Dalkey N, Helmer O: An experimental application of the Delphi method to the use of experts. Manage Sci. 1963, 9 (3): 458-467. 10.1287/mnsc.9.3.458.CrossRef Dalkey N, Helmer O: An experimental application of the Delphi method to the use of experts. Manage Sci. 1963, 9 (3): 458-467. 10.1287/mnsc.9.3.458.CrossRef
12.
go back to reference Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LJ R, Lazaro P, Van het Loo M, McDonnell J, Vader JP, Kahan JP: The RAND/UCLA Approriateness Method User’s Manual. 2001, Santa Monica: CA: RAND Corporation Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LJ R, Lazaro P, Van het Loo M, McDonnell J, Vader JP, Kahan JP: The RAND/UCLA Approriateness Method User’s Manual. 2001, Santa Monica: CA: RAND Corporation
14.
go back to reference Hejblum G, Ioos V, Vibert JF, Boelle PY, Chalumeau-Lemoine L, Chouaid C, Valleron AJ, Guidet B: A web-based Delphi study on the indications of chest radiographs for patients in ICUs. Chest. 2008, 133 (5): 1107-1112. 10.1378/chest.06-3014.CrossRefPubMed Hejblum G, Ioos V, Vibert JF, Boelle PY, Chalumeau-Lemoine L, Chouaid C, Valleron AJ, Guidet B: A web-based Delphi study on the indications of chest radiographs for patients in ICUs. Chest. 2008, 133 (5): 1107-1112. 10.1378/chest.06-3014.CrossRefPubMed
18.
go back to reference Norder G, Roelen CA, van Rhenen W, Buitenhuis J, Bultmann U, Anema JR: Predictors of recurrent sickness absence due to depressive disorders - a delphi approach involving scientists and physicians. PLoS One. 2012, 7 (12): e51792-10.1371/journal.pone.0051792.CrossRefPubMedPubMedCentral Norder G, Roelen CA, van Rhenen W, Buitenhuis J, Bultmann U, Anema JR: Predictors of recurrent sickness absence due to depressive disorders - a delphi approach involving scientists and physicians. PLoS One. 2012, 7 (12): e51792-10.1371/journal.pone.0051792.CrossRefPubMedPubMedCentral
19.
go back to reference R Development Core Team: R: A language and environment for statistical computing. 2008, Vienna, Austria: R Foundation for Statistical Computing R Development Core Team: R: A language and environment for statistical computing. 2008, Vienna, Austria: R Foundation for Statistical Computing
20.
go back to reference Bond S, Bond J: A Delphi survey of clinical nursing research priorities. J Adv Nurs. 1982, 7 (6): 565-575. 10.1111/j.1365-2648.1982.tb00277.x.CrossRefPubMed Bond S, Bond J: A Delphi survey of clinical nursing research priorities. J Adv Nurs. 1982, 7 (6): 565-575. 10.1111/j.1365-2648.1982.tb00277.x.CrossRefPubMed
21.
go back to reference Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C: Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011, 6 (6): e20476-10.1371/journal.pone.0020476.CrossRefPubMedPubMedCentral Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C: Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011, 6 (6): e20476-10.1371/journal.pone.0020476.CrossRefPubMedPubMedCentral
22.
go back to reference Hasson F, Keeney S, McKenna H: Research guidelines for the Delphi survey technique. J Adv Nurs. 2000, 32 (4): 1008-1015.PubMed Hasson F, Keeney S, McKenna H: Research guidelines for the Delphi survey technique. J Adv Nurs. 2000, 32 (4): 1008-1015.PubMed
23.
go back to reference McKenna HP, Bradley M, Keeney S: Primary Care Nursing: A Study Exploring Key Issues For Future Developments. 2001, Coleraine: University of Ulster McKenna HP, Bradley M, Keeney S: Primary Care Nursing: A Study Exploring Key Issues For Future Developments. 2001, Coleraine: University of Ulster
24.
go back to reference Keeney S, Hasson F, McKenna H: Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006, 53 (2): 205-212. 10.1111/j.1365-2648.2006.03716.x.CrossRefPubMed Keeney S, Hasson F, McKenna H: Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006, 53 (2): 205-212. 10.1111/j.1365-2648.2006.03716.x.CrossRefPubMed
25.
go back to reference McKenna HP: The Delphi technique: a worthwhile research approach for nursing?. J Adv Nurs. 1994, 19 (6): 1221-1225. 10.1111/j.1365-2648.1994.tb01207.x.CrossRefPubMed McKenna HP: The Delphi technique: a worthwhile research approach for nursing?. J Adv Nurs. 1994, 19 (6): 1221-1225. 10.1111/j.1365-2648.1994.tb01207.x.CrossRefPubMed
26.
go back to reference Loughlin KG, Moore LF: Using Delphi to achieve congruent objectives and activities in a pediatrics department. J Med Educ. 1979, 54 (2): 101-106.PubMed Loughlin KG, Moore LF: Using Delphi to achieve congruent objectives and activities in a pediatrics department. J Med Educ. 1979, 54 (2): 101-106.PubMed
27.
go back to reference Bender AD, Strack AE, Ebright GW, von Haunalter G: A Delphic study of the future of medicine. 1969, Philadelphia: Smith, Kline and French Laboratory, Research and Development Division Bender AD, Strack AE, Ebright GW, von Haunalter G: A Delphic study of the future of medicine. 1969, Philadelphia: Smith, Kline and French Laboratory, Research and Development Division
28.
go back to reference Sumsion T: The Delphi technique: an adaptive research tool. Br J Occup Ther. 1998, 61: 153-156.CrossRef Sumsion T: The Delphi technique: an adaptive research tool. Br J Occup Ther. 1998, 61: 153-156.CrossRef
29.
go back to reference Green B, Jones M, Hughes D, Williams A: Applying the Delphi technique in a study of GPs’ information requirements. Health Soc Care Community. 1999, 7 (3): 198-205. 10.1046/j.1365-2524.1999.00176.x.CrossRefPubMed Green B, Jones M, Hughes D, Williams A: Applying the Delphi technique in a study of GPs’ information requirements. Health Soc Care Community. 1999, 7 (3): 198-205. 10.1046/j.1365-2524.1999.00176.x.CrossRefPubMed
30.
go back to reference Speksnijder HT, Mank AP, van Achterberg T: Nursing diagnoses (NANDA-I) in hematology-oncology: a Delphi-study. Int J Nurs Terminol Classif. 2011, 22 (2): 77-91. 10.1111/j.1744-618X.2011.01183.x.CrossRefPubMed Speksnijder HT, Mank AP, van Achterberg T: Nursing diagnoses (NANDA-I) in hematology-oncology: a Delphi-study. Int J Nurs Terminol Classif. 2011, 22 (2): 77-91. 10.1111/j.1744-618X.2011.01183.x.CrossRefPubMed
31.
go back to reference Hill KQ, Fowles J: The methodological worth of Delphi forecasting technique. Technol Forecast Soc Change. 1975, 7: 179-192. 10.1016/0040-1625(75)90057-8.CrossRef Hill KQ, Fowles J: The methodological worth of Delphi forecasting technique. Technol Forecast Soc Change. 1975, 7: 179-192. 10.1016/0040-1625(75)90057-8.CrossRef
32.
go back to reference Greatorex J, Dexter T: An accessible analytical approach for investigating what happens between the rounds of a Delphi study. J Adv Nurs. 2000, 32 (4): 1016-1024.PubMed Greatorex J, Dexter T: An accessible analytical approach for investigating what happens between the rounds of a Delphi study. J Adv Nurs. 2000, 32 (4): 1016-1024.PubMed
Metadata
Title
Determination of French influenza outbreaks periods between 1985 and 2011 through a web-based Delphi method
Authors
Marion Debin
Cécile Souty
Clément Turbelin
Thierry Blanchon
Pierre-Yves Boëlle
Thomas Hanslik
Gilles Hejblum
Yann Le Strat
Flavien Quintus
Alessandra Falchi
the DelFluWeb study group
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2013
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/1472-6947-13-138

Other articles of this Issue 1/2013

BMC Medical Informatics and Decision Making 1/2013 Go to the issue