Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2014

Open Access 01-12-2014 | Research article

Evidence-based decision-making in infectious diseases epidemiology, prevention and control: matching research questions to study designs and quality appraisal tools

Authors: Thomas Harder, Anja Takla, Eva Rehfuess, Alex Sánchez-Vivar, Dorothea Matysiak-Klose, Tim Eckmanns, Gérard Krause, Helena de Carvalho Gomes, Andreas Jansen, Simon Ellis, Frode Forland, Roberta James, Joerg J Meerpohl, Antony Morgan, Holger Schünemann, Teun Zuiderent-Jerak, Ole Wichmann

Published in: BMC Medical Research Methodology | Issue 1/2014

Login to get access

Abstract

Background

The Project on a Framework for Rating Evidence in Public Health (PRECEPT) was initiated and is being funded by the European Centre for Disease Prevention and Control (ECDC) to define a methodology for evaluating and grading evidence and strength of recommendations in the field of public health, with emphasis on infectious disease epidemiology, prevention and control. One of the first steps was to review existing quality appraisal tools (QATs) for individual research studies of various designs relevant to this area, using a question-based approach.

Methods

Through team discussions and expert consultations, we identified 20 relevant types of public health questions, which were grouped into six domains, i.e. characteristics of the pathogen, burden of disease, diagnosis, risk factors, intervention, and implementation of intervention. Previously published systematic reviews were used and supplemented by expert consultation to identify suitable QATs. Finally, a matrix was constructed for matching questions to study designs suitable to address them and respective QATs. Key features of each of the included QATs were then analyzed, in particular in respect to its intended use, types of questions and answers, presence/absence of a quality score, and if a validation was performed.

Results

In total we identified 21 QATs and 26 study designs, and matched them. Four QATs were suitable for experimental quantitative study designs, eleven for observational quantitative studies, two for qualitative studies, three for economic studies, one for diagnostic test accuracy studies, and one for animal studies. Included QATs consisted of six to 28 items. Six of the QATs had a summary quality score. Fourteen QATs had undergone at least one validation procedure.

Conclusions

The results of this methodological study can be used as an inventory of potentially relevant questions, appropriate study designs and QATs for researchers and authorities engaged with evidence-based decision-making in infectious disease epidemiology, prevention and control.
Appendix
Available only for authorised users
Literature
1.
go back to reference Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ. 1996, 312: 71-72. 10.1136/bmj.312.7023.71.CrossRefPubMedPubMedCentral Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ. 1996, 312: 71-72. 10.1136/bmj.312.7023.71.CrossRefPubMedPubMedCentral
3.
go back to reference Latham J, Murajda L, Forland F, Jansen A: Capacities, practices and perceptions of evidence-based public health in Europe. Eur Surveill. 2013, 18: pii=20421- Latham J, Murajda L, Forland F, Jansen A: Capacities, practices and perceptions of evidence-based public health in Europe. Eur Surveill. 2013, 18: pii=20421-
4.
go back to reference Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Pol. 2005, 10 (Suppl 1): 21-34. 10.1258/1355819054308530.CrossRef Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Pol. 2005, 10 (Suppl 1): 21-34. 10.1258/1355819054308530.CrossRef
5.
go back to reference Egan M, Bambra C, Petticrew M, Whitehead M: Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions. J Epidemiol Community Health. 2009, 63: 4-11.CrossRefPubMed Egan M, Bambra C, Petticrew M, Whitehead M: Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions. J Epidemiol Community Health. 2009, 63: 4-11.CrossRefPubMed
6.
go back to reference Kelly M, Morgan A, Ellis S, Younger T, Huntley J, Swann C: Evidence based public health: a review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England. Soc Sci Med. 2010, 71: 1056-1062. 10.1016/j.socscimed.2010.06.032.CrossRefPubMed Kelly M, Morgan A, Ellis S, Younger T, Huntley J, Swann C: Evidence based public health: a review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England. Soc Sci Med. 2010, 71: 1056-1062. 10.1016/j.socscimed.2010.06.032.CrossRefPubMed
7.
go back to reference Victora C, Habicht J-P, Bryce J: Evidence-based public health: moving beyond randomised trials. Am J Publ Health. 2004, 94: 400-405. 10.2105/AJPH.94.3.400.CrossRef Victora C, Habicht J-P, Bryce J: Evidence-based public health: moving beyond randomised trials. Am J Publ Health. 2004, 94: 400-405. 10.2105/AJPH.94.3.400.CrossRef
9.
go back to reference Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008, 336: 924-926. 10.1136/bmj.39489.470347.AD.CrossRefPubMedPubMedCentral Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008, 336: 924-926. 10.1136/bmj.39489.470347.AD.CrossRefPubMedPubMedCentral
10.
go back to reference Matysiak-Klose D, Ahmed F, Duclos P, Falck-Ytter Y, Forland F, Houweling H, Kramarz P, Langley JM, Mertens T, Schünemann H, Senouci K, Temte J, Wichmann O: Report on the 1st international workshop on procedures for the development of evidence-based vaccination recommendations, Berlin, Germany, 22-23 November 2010. Vaccine. 2012, 30: 2399-2404. 10.1016/j.vaccine.2011.12.004.CrossRefPubMed Matysiak-Klose D, Ahmed F, Duclos P, Falck-Ytter Y, Forland F, Houweling H, Kramarz P, Langley JM, Mertens T, Schünemann H, Senouci K, Temte J, Wichmann O: Report on the 1st international workshop on procedures for the development of evidence-based vaccination recommendations, Berlin, Germany, 22-23 November 2010. Vaccine. 2012, 30: 2399-2404. 10.1016/j.vaccine.2011.12.004.CrossRefPubMed
14.
go back to reference European Centre for Disease Prevention and Control: Evidence-based Methodologies for Public Health – How to assess the best available evidence when time is limited and there is lack of sound evidence. 2011, Stockholm: ECDC European Centre for Disease Prevention and Control: Evidence-based Methodologies for Public Health – How to assess the best available evidence when time is limited and there is lack of sound evidence. 2011, Stockholm: ECDC
15.
go back to reference Lomas J, Culyer T, McCutcheon C: Conceptualizing and Combining Evidence for Health System Guidance: Final Report. 2005, Ottawa: Canadian Health Services Research Foundation Lomas J, Culyer T, McCutcheon C: Conceptualizing and Combining Evidence for Health System Guidance: Final Report. 2005, Ottawa: Canadian Health Services Research Foundation
16.
go back to reference West S, King V, Carey TS, Lohr KN, McKoy N, Suton SF, Lux L: Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment No. 47. 2002, Rockville, MD: AHRQ Publication No. 02-E016 West S, King V, Carey TS, Lohr KN, McKoy N, Suton SF, Lux L: Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment No. 47. 2002, Rockville, MD: AHRQ Publication No. 02-E016
17.
go back to reference Bai A, Shukla VK, Bak G, Wells G: Quality Assessment Tools Project Report. 2012, Ottawa: Canadian Agency for Drugs and Technologies in Health Bai A, Shukla VK, Bak G, Wells G: Quality Assessment Tools Project Report. 2012, Ottawa: Canadian Agency for Drugs and Technologies in Health
18.
go back to reference A Dictionary of Epidemiology. Edited by: Porta M. 2008, New York: Oxford University Press A Dictionary of Epidemiology. Edited by: Porta M. 2008, New York: Oxford University Press
19.
go back to reference Sanderson S, Tatt ID, Higgins JPT: Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007, 36: 666-676. 10.1093/ije/dym018.CrossRefPubMed Sanderson S, Tatt ID, Higgins JPT: Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007, 36: 666-676. 10.1093/ije/dym018.CrossRefPubMed
20.
go back to reference Huwiler-Münterer K, Jüni P, Junker C, Egger M: Quality of reporting of randomized trials as a measure of methodological quality. JAMA. 2002, 287: 2801-2804. 10.1001/jama.287.21.2801.CrossRef Huwiler-Münterer K, Jüni P, Junker C, Egger M: Quality of reporting of randomized trials as a measure of methodological quality. JAMA. 2002, 287: 2801-2804. 10.1001/jama.287.21.2801.CrossRef
21.
go back to reference Schulz KF, Altman DG, Moher D: CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials. BMC Med. 2010, 8: 18-10.1186/1741-7015-8-18.CrossRefPubMedPubMedCentral Schulz KF, Altman DG, Moher D: CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials. BMC Med. 2010, 8: 18-10.1186/1741-7015-8-18.CrossRefPubMedPubMedCentral
22.
go back to reference von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP: STROBE Initiative: The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007, 370: 1453-1457. 10.1016/S0140-6736(07)61602-X.CrossRefPubMed von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP: STROBE Initiative: The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007, 370: 1453-1457. 10.1016/S0140-6736(07)61602-X.CrossRefPubMed
24.
go back to reference Greenhalgh T, Peacock R: Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005, 331: 1064-1065. 10.1136/bmj.38636.593461.68.CrossRefPubMedPubMedCentral Greenhalgh T, Peacock R: Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005, 331: 1064-1065. 10.1136/bmj.38636.593461.68.CrossRefPubMedPubMedCentral
25.
go back to reference Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F, Petticrew M, Altman DG: International stroke trial collaborative group; European carotid surgery trial collaborative group: evaluating non-randomized intervention studies. Health Technol Assess. 2003, 7: 1-173.CrossRef Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F, Petticrew M, Altman DG: International stroke trial collaborative group; European carotid surgery trial collaborative group: evaluating non-randomized intervention studies. Health Technol Assess. 2003, 7: 1-173.CrossRef
26.
go back to reference Shamliyan TA, Kane RL, Ansari MT, Raman G, Berkman ND, Grant M, Janes G, Maglione M, Moher D, Nasser M, Robinson KA, Segal JB, Tsouros S: Development of quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists. J Clin Epidemiol. 2001, 64: 637-657.CrossRef Shamliyan TA, Kane RL, Ansari MT, Raman G, Berkman ND, Grant M, Janes G, Maglione M, Moher D, Nasser M, Robinson KA, Segal JB, Tsouros S: Development of quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists. J Clin Epidemiol. 2001, 64: 637-657.CrossRef
27.
go back to reference Jefferson T, Demicheli V, Vale L: Quality of systematic reviews of economic evaluations in health care. JAMA. 2002, 287: 2809-2812. 10.1001/jama.287.21.2809.CrossRefPubMed Jefferson T, Demicheli V, Vale L: Quality of systematic reviews of economic evaluations in health care. JAMA. 2002, 287: 2809-2812. 10.1001/jama.287.21.2809.CrossRefPubMed
28.
go back to reference Katrak P, Bialocerkowski AE, Massy-Westropp N, Kumar VSS, Grimmer KA: A systematic review of the content of critical appraisal tools. BMC Med Res Meth. 2004, 4: 22-10.1186/1471-2288-4-22.CrossRef Katrak P, Bialocerkowski AE, Massy-Westropp N, Kumar VSS, Grimmer KA: A systematic review of the content of critical appraisal tools. BMC Med Res Meth. 2004, 4: 22-10.1186/1471-2288-4-22.CrossRef
29.
go back to reference Shamliyan T, Kane RL, Dickinson S: A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol. 2010, 63: 1061-1070. 10.1016/j.jclinepi.2010.04.014.CrossRefPubMed Shamliyan T, Kane RL, Dickinson S: A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol. 2010, 63: 1061-1070. 10.1016/j.jclinepi.2010.04.014.CrossRefPubMed
30.
go back to reference Cho MK, Bero LA: Instruments for assessing the quality of drug studies published in the medical literature. JAMA. 1994, 272: 101-104. 10.1001/jama.1994.03520020027007.CrossRefPubMed Cho MK, Bero LA: Instruments for assessing the quality of drug studies published in the medical literature. JAMA. 1994, 272: 101-104. 10.1001/jama.1994.03520020027007.CrossRefPubMed
31.
go back to reference Downs SH, Black N: The feasibility of creating a checklist for the assessment of the methodological quality both of randomized and non-randomized studies of health care interventions. J Epidemiol Community Health. 1998, 52: 377-384. 10.1136/jech.52.6.377.CrossRefPubMedPubMedCentral Downs SH, Black N: The feasibility of creating a checklist for the assessment of the methodological quality both of randomized and non-randomized studies of health care interventions. J Epidemiol Community Health. 1998, 52: 377-384. 10.1136/jech.52.6.377.CrossRefPubMedPubMedCentral
32.
go back to reference Loney PL, Chambers LW, Bennett KJ, Roberts JG, Stratford PW: Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chron Dis Can. 1998, 19: 170-177. Loney PL, Chambers LW, Bennett KJ, Roberts JG, Stratford PW: Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chron Dis Can. 1998, 19: 170-177.
33.
go back to reference Al-Jader LN, Newcombe RG, Hayes S, Murray A, Layzell J, Harper PS: Developing a quality scoring system for epidemiological surveys of genetic disorders. Clin Genet. 2002, 62: 230-234. 10.1034/j.1399-0004.2002.620308.x.CrossRefPubMed Al-Jader LN, Newcombe RG, Hayes S, Murray A, Layzell J, Harper PS: Developing a quality scoring system for epidemiological surveys of genetic disorders. Clin Genet. 2002, 62: 230-234. 10.1034/j.1399-0004.2002.620308.x.CrossRefPubMed
34.
go back to reference Ofman JJ, Sullivan SD, Neumann PJ, Chiou CF, Henning JM, Wade SW, Hay JW: Examining the value and quality of health economic analyses: implications of utilizing the QHES. J Manag Care Pharm. 2003, 9: 53-61.PubMed Ofman JJ, Sullivan SD, Neumann PJ, Chiou CF, Henning JM, Wade SW, Hay JW: Examining the value and quality of health economic analyses: implications of utilizing the QHES. J Manag Care Pharm. 2003, 9: 53-61.PubMed
35.
go back to reference Higgins JT, Altman DG, Gotzsche PC, Jüni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA, Cochrane Bias Methods Group: Cochrane statistical methods group: the cochrane collaboration’s tool for assessing risk of bias in randomized trials. BMJ. 2011, 343: d5928-10.1136/bmj.d5928.CrossRefPubMedPubMedCentral Higgins JT, Altman DG, Gotzsche PC, Jüni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA, Cochrane Bias Methods Group: Cochrane statistical methods group: the cochrane collaboration’s tool for assessing risk of bias in randomized trials. BMJ. 2011, 343: d5928-10.1136/bmj.d5928.CrossRefPubMedPubMedCentral
37.
go back to reference Van der Worp HB, Howells DW, Sena ES, Porritt MJ, Rewell S, O’Collins V, MacLeod MR: Can animal models of disease reliably inform human studies?. PLoS Med. 2010, 7: e1000245-10.1371/journal.pmed.1000245.CrossRefPubMedPubMedCentral Van der Worp HB, Howells DW, Sena ES, Porritt MJ, Rewell S, O’Collins V, MacLeod MR: Can animal models of disease reliably inform human studies?. PLoS Med. 2010, 7: e1000245-10.1371/journal.pmed.1000245.CrossRefPubMedPubMedCentral
45.
go back to reference Hoy D, Brooks P, Woolf A, Blyth F, March L, Bain C, Baker P, Smith E, Buchbinder R: Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement. J Clin Epidemiol. 2012, 65: 934-939. 10.1016/j.jclinepi.2011.11.014.CrossRefPubMed Hoy D, Brooks P, Woolf A, Blyth F, March L, Bain C, Baker P, Smith E, Buchbinder R: Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement. J Clin Epidemiol. 2012, 65: 934-939. 10.1016/j.jclinepi.2011.11.014.CrossRefPubMed
46.
47.
go back to reference National Institute for Health and Care Excellence: Quality appraisal checklist: quantitative studies reporting correlations and associations. Methods for the Development of NICE Public Health Guidance. 2012, London: NICE, http://www.nice.org.uk/phmethods, National Institute for Health and Care Excellence: Quality appraisal checklist: quantitative studies reporting correlations and associations. Methods for the Development of NICE Public Health Guidance. 2012, London: NICE, http://​www.​nice.​org.​uk/​phmethods,
50.
go back to reference Cochrane Effective Practice and Organization of Care (EPOC) Group: Suggested risk of bias criteria for EPOC reviews: Risk of bias for interrupted time series (ITS) studies. http://epoc.cochrane.org, Cochrane Effective Practice and Organization of Care (EPOC) Group: Suggested risk of bias criteria for EPOC reviews: Risk of bias for interrupted time series (ITS) studies. http://​epoc.​cochrane.​org,
51.
go back to reference Krauth D, Woodruff TJ, Bero L: Instruments for assessing risk of bias and other methodological criteria of published animal studies: a systematic review. Environ Health Perspect. 2013, 121: 985-992.PubMedPubMedCentral Krauth D, Woodruff TJ, Bero L: Instruments for assessing risk of bias and other methodological criteria of published animal studies: a systematic review. Environ Health Perspect. 2013, 121: 985-992.PubMedPubMedCentral
52.
go back to reference Jüni P, Witschi A, Bloch R, Egger M: The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999, 282: 1054-1060. 10.1001/jama.282.11.1054.CrossRefPubMed Jüni P, Witschi A, Bloch R, Egger M: The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999, 282: 1054-1060. 10.1001/jama.282.11.1054.CrossRefPubMed
53.
go back to reference Voss PH, Rehfuess EA: Quality appraisal in systematic reviews of public health interventions: an empirical study on the impact of choice of tool on meta-analysis. J Epidemiol Community Health. 2013, 67: 98-104. 10.1136/jech-2011-200940.CrossRefPubMed Voss PH, Rehfuess EA: Quality appraisal in systematic reviews of public health interventions: an empirical study on the impact of choice of tool on meta-analysis. J Epidemiol Community Health. 2013, 67: 98-104. 10.1136/jech-2011-200940.CrossRefPubMed
54.
go back to reference Weldeselassie YG, Whitaker HJ, Farrington CP: Use of the self-controlled case-series method in vaccine safety studies: review and recommendations for best practice. Epidemiol Infect. 2011, 139: 1805-1817. 10.1017/S0950268811001531.CrossRefPubMed Weldeselassie YG, Whitaker HJ, Farrington CP: Use of the self-controlled case-series method in vaccine safety studies: review and recommendations for best practice. Epidemiol Infect. 2011, 139: 1805-1817. 10.1017/S0950268811001531.CrossRefPubMed
55.
go back to reference Centers for Disease Control and Prevention: Introduction to Process Evaluation in Tobacco use Prevention and Control. 2008, Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, http://www.cdc.gov/tobacco/publications/index.htm, Centers for Disease Control and Prevention: Introduction to Process Evaluation in Tobacco use Prevention and Control. 2008, Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, http://​www.​cdc.​gov/​tobacco/​publications/​index.​htm,
56.
go back to reference Krippendorff K: Content Analysis: An Introduction to its Methodology. 2012, Thousand Oaks: SAGE Press Krippendorff K: Content Analysis: An Introduction to its Methodology. 2012, Thousand Oaks: SAGE Press
57.
go back to reference Mauskopf JA, Paul JE, Grant DM, Stergachis A: The role of cost-consequence analysis in healthcare decision-making. Pharmacoeconomics. 1998, 13: 277-288. 10.2165/00019053-199813030-00002.CrossRefPubMed Mauskopf JA, Paul JE, Grant DM, Stergachis A: The role of cost-consequence analysis in healthcare decision-making. Pharmacoeconomics. 1998, 13: 277-288. 10.2165/00019053-199813030-00002.CrossRefPubMed
Metadata
Title
Evidence-based decision-making in infectious diseases epidemiology, prevention and control: matching research questions to study designs and quality appraisal tools
Authors
Thomas Harder
Anja Takla
Eva Rehfuess
Alex Sánchez-Vivar
Dorothea Matysiak-Klose
Tim Eckmanns
Gérard Krause
Helena de Carvalho Gomes
Andreas Jansen
Simon Ellis
Frode Forland
Roberta James
Joerg J Meerpohl
Antony Morgan
Holger Schünemann
Teun Zuiderent-Jerak
Ole Wichmann
Publication date
01-12-2014
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2014
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-14-69

Other articles of this Issue 1/2014

BMC Medical Research Methodology 1/2014 Go to the issue