Skip to main content
Top
Published in: Systematic Reviews 1/2016

Open Access 01-12-2016 | Protocol

Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review protocol

Authors: Myriam Cielo Pérez, Nanor Minoyan, Valéry Ridde, Marie-Pierre Sylvestre, Mira Johri

Published in: Systematic Reviews | Issue 1/2016

Login to get access

Abstract

Background

Cluster randomised trials (CRTs) are a key instrument to evaluate public health interventions, particularly in low- and middle-income countries (LMICs). Fidelity assessment examines study processes to gauge whether an intervention was delivered as initially planned. Evaluation of implementation fidelity (IF) is required to establish whether the measured effects of a trial are due to the intervention itself and may be particularly important for CRTs of complex interventions. Current CRT reporting guidelines offer no guidance on IF assessment. We will systematically review the scientific literature to study current practices concerning the assessment of IF in CRTs of public health interventions in LMICs.

Methods

We will include CRTs of public health interventions in LMICs that planned or assessed IF in either the trial protocol or the main trial report (or an associated document). Search strategies use Medical Subject Headings (MESH) and text words related to CRTs, developing countries, and public health interventions. The electronic database search was developed first for MEDLINE and adapted for the following databases: EMBASE, CINAHL, PubMed, and EMB Reviews, to identify CRT reports in English, Spanish, or French published on or after January 1, 2012. To ensure availability of a study protocol, we will include CRTs reporting a registration number in the abstract. For each included study, we will compare planned versus reported assessment of IF, and consider the dimensions of IF studied, and data collection methods used to evaluate each dimension. Data will be synthesised using quantitative and narrative techniques. Risk of bias for individual studies will be assessed using the Cochrane Collaboration Risk of Bias Tool criteria and additional criteria related to CRT methods. We will investigate possible sources of heterogeneity by performing subgroup analysis. This review was not eligible for inclusion in the PROSPERO registry.

Discussion

Fidelity assessment may be a key tool for making studies more reliable, internally valid, and externally generalizable. This review will provide a portrait of current practices related to the assessment of intervention fidelity in CRTs and offer suggestions for improvement. Results will be relevant to researchers, those who finance health interventions, and for decision-makers who seek the best evidence on public health interventions.
Appendix
Available only for authorised users
Literature
3.
go back to reference Isaakidis P, Ioannidis JP. Evaluation of cluster randomized controlled trials in sub-Saharan Africa. Am J Epidemiol. 2003;158(9):921–6.CrossRefPubMed Isaakidis P, Ioannidis JP. Evaluation of cluster randomized controlled trials in sub-Saharan Africa. Am J Epidemiol. 2003;158(9):921–6.CrossRefPubMed
4.
go back to reference Campbell MJ, Donner A, Klar N. Developments in cluster randomized trials and statistics in medicine. Stat Med. 2007;26(1):2–19.CrossRefPubMed Campbell MJ, Donner A, Klar N. Developments in cluster randomized trials and statistics in medicine. Stat Med. 2007;26(1):2–19.CrossRefPubMed
5.
go back to reference Moberg J, Kramer M. A brief history of the cluster randomised trial design. J R Soc Med. 2015;108(5):192–8.CrossRefPubMed Moberg J, Kramer M. A brief history of the cluster randomised trial design. J R Soc Med. 2015;108(5):192–8.CrossRefPubMed
6.
go back to reference Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M. Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract. 2000;17:192–6.CrossRefPubMed Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M. Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract. 2000;17:192–6.CrossRefPubMed
7.
go back to reference Osrin D, Azad K, Fernandez A, Manandhar DS, Mwansambo CW, Tripathy P, Costello AM. Ethical challenges in cluster randomized controlled trials: experiences from public health interventions in Africa and Asia. Bull World Health Org. 2009;87(10):772–9.CrossRefPubMedPubMedCentral Osrin D, Azad K, Fernandez A, Manandhar DS, Mwansambo CW, Tripathy P, Costello AM. Ethical challenges in cluster randomized controlled trials: experiences from public health interventions in Africa and Asia. Bull World Health Org. 2009;87(10):772–9.CrossRefPubMedPubMedCentral
8.
go back to reference Handlos LN, Chakraborty H, Sen PK. Evaluation of cluster‐randomized trials on maternal and child health research in developing countries. Trop Med Int Health. 2009;14(8):947–56.CrossRefPubMed Handlos LN, Chakraborty H, Sen PK. Evaluation of cluster‐randomized trials on maternal and child health research in developing countries. Trop Med Int Health. 2009;14(8):947–56.CrossRefPubMed
9.
go back to reference Richards DA, Hallberg IR. Complex interventions in health: an overview of research methods. London & New York: Routledge; 2015. Richards DA, Hallberg IR. Complex interventions in health: an overview of research methods. London & New York: Routledge; 2015.
10.
go back to reference Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694–6.CrossRefPubMedPubMedCentral Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694–6.CrossRefPubMedPubMedCentral
11.
go back to reference Craig P, Cooper C, Gunnell D, Haw S, Lawson K, Macintyre S, Thompson S. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Community Health. 2012;66(12):1182–6. jech-2011.CrossRefPubMedPubMedCentral Craig P, Cooper C, Gunnell D, Haw S, Lawson K, Macintyre S, Thompson S. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Community Health. 2012;66(12):1182–6. jech-2011.CrossRefPubMedPubMedCentral
12.
go back to reference Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420.CrossRef Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420.CrossRef
13.
go back to reference Sanson-Fisher RW, D’Este CA, Carey ML, Noble N, Paul CL. Evaluation of systems-oriented public health interventions: alternative research designs. Annu Rev Public Health. 2014;35:9–27.CrossRefPubMed Sanson-Fisher RW, D’Este CA, Carey ML, Noble N, Paul CL. Evaluation of systems-oriented public health interventions: alternative research designs. Annu Rev Public Health. 2014;35:9–27.CrossRefPubMed
14.
go back to reference Upshur RE. Principles for the justification of public health intervention. Can J Public Health. 2002;93(2):101–3.PubMed Upshur RE. Principles for the justification of public health intervention. Can J Public Health. 2002;93(2):101–3.PubMed
15.
go back to reference Hayes RJ, Alexander ND, Bennett S, Cousens SN. Design and analysis issues in cluster-randomized trials of interventions against infectious diseases. Stat Methods Med Res. 2000;9(2):95–116.CrossRefPubMed Hayes RJ, Alexander ND, Bennett S, Cousens SN. Design and analysis issues in cluster-randomized trials of interventions against infectious diseases. Stat Methods Med Res. 2000;9(2):95–116.CrossRefPubMed
16.
go back to reference Morris SS, Ranson MK, Sinha T, Mills AJ. Measuring improved targeting of health interventions to the poor in the context of a community-randomised trial in rural India. Contemp Clin Trials. 2007;28(4):382–90.CrossRefPubMed Morris SS, Ranson MK, Sinha T, Mills AJ. Measuring improved targeting of health interventions to the poor in the context of a community-randomised trial in rural India. Contemp Clin Trials. 2007;28(4):382–90.CrossRefPubMed
17.
go back to reference Ranson MK, Sinha T, Morris SS, Mills AJ. CRTs–cluster randomized trials or “courting real troubles”: challenges of running a CRT in rural Gujarat, India. Can J Public Health. 2006;97(1):72.PubMedPubMedCentral Ranson MK, Sinha T, Morris SS, Mills AJ. CRTs–cluster randomized trials or “courting real troubles”: challenges of running a CRT in rural Gujarat, India. Can J Public Health. 2006;97(1):72.PubMedPubMedCentral
18.
go back to reference Grossman J, Mackenzie FJ. The randomized controlled trial: gold standard, or merely standard? Perspect Biol Med. 2005;48(4):516–34.CrossRefPubMed Grossman J, Mackenzie FJ. The randomized controlled trial: gold standard, or merely standard? Perspect Biol Med. 2005;48(4):516–34.CrossRefPubMed
19.
go back to reference Eldridge S, Ashby D, Bennett C, Wakelin M, Feder G. Internal and external validity of cluster randomised trials: systematic review of recent trials. BMJ. 2008;336(7649):876–80.CrossRefPubMedPubMedCentral Eldridge S, Ashby D, Bennett C, Wakelin M, Feder G. Internal and external validity of cluster randomised trials: systematic review of recent trials. BMJ. 2008;336(7649):876–80.CrossRefPubMedPubMedCentral
20.
go back to reference Bonell C, Oakley A, Hargreaves J, Strange V, Rees R. Assessment of generalisability in trials of health interventions: suggested framework and systematic review. Br Med J. 2006;333:346-49. doi:10.1136/bmj.333.7563.346. Bonell C, Oakley A, Hargreaves J, Strange V, Rees R. Assessment of generalisability in trials of health interventions: suggested framework and systematic review. Br Med J. 2006;333:346-49. doi:10.​1136/​bmj.​333.​7563.​346.
21.
go back to reference Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6.CrossRefPubMedPubMedCentral Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6.CrossRefPubMedPubMedCentral
22.
go back to reference Brierley G, Brabyn S, Torgerson D, Watson J. Bias in recruitment to cluster randomized trials: a review of recent publications. J Eval Clin Pract. 2012;18(4):878–86.CrossRefPubMed Brierley G, Brabyn S, Torgerson D, Watson J. Bias in recruitment to cluster randomized trials: a review of recent publications. J Eval Clin Pract. 2012;18(4):878–86.CrossRefPubMed
23.
go back to reference Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ. 2003;327(7418):785–9.CrossRefPubMedPubMedCentral Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ. 2003;327(7418):785–9.CrossRefPubMedPubMedCentral
24.
25.
go back to reference Medical Research Council. Developing and evaluating complex interventions: new guidance. London: Medical Research Council; 2008. Medical Research Council. Developing and evaluating complex interventions: new guidance. London: Medical Research Council; 2008.
26.
go back to reference Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(s1):S52–63.CrossRefPubMedCentral Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(s1):S52–63.CrossRefPubMedCentral
27.
go back to reference Hasson H, Blomberg S, Dunér A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implement Sci. 2012;7(23):1–11. Hasson H, Blomberg S, Dunér A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implement Sci. 2012;7(23):1–11.
28.
29.
go back to reference Spoth RL, Redmond C, Shin C. Randomized trial of brief family interventions for general populations: adolescent substance use outcomes 4 years following baseline. J Consult Clin Psychol. 2001;69(4):627. Spoth RL, Redmond C, Shin C. Randomized trial of brief family interventions for general populations: adolescent substance use outcomes 4 years following baseline. J Consult Clin Psychol. 2001;69(4):627.
30.
go back to reference Bradley F, Wiles R, Kinmonth A-L, Mant D, Gantley M. Development and evaluation of complex interventions in health services research: case study of the Southampton heart integrated care project (SHIP). BMJ. 1999;318(7185):711–5.CrossRefPubMedPubMedCentral Bradley F, Wiles R, Kinmonth A-L, Mant D, Gantley M. Development and evaluation of complex interventions in health services research: case study of the Southampton heart integrated care project (SHIP). BMJ. 1999;318(7185):711–5.CrossRefPubMedPubMedCentral
32.
go back to reference Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.CrossRefPubMed Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.CrossRefPubMed
33.
go back to reference Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: are implementation effects out of control? Clin Psychol Rev. 1998;18(1):23–45.CrossRefPubMed Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: are implementation effects out of control? Clin Psychol Rev. 1998;18(1):23–45.CrossRefPubMed
34.
go back to reference Hawe P. Lessons from complex interventions to improve health. Annu Rev Public Health. 2015;36:307–23.CrossRefPubMed Hawe P. Lessons from complex interventions to improve health. Annu Rev Public Health. 2015;36:307–23.CrossRefPubMed
35.
go back to reference Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Baird J. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.CrossRefPubMedPubMedCentral Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Baird J. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.CrossRefPubMedPubMedCentral
36.
37.
go back to reference Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–e37.CrossRefPubMed Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–e37.CrossRefPubMed
38.
go back to reference Campbell MK, Piaggio G, Elbourne DR, Altman DG. CONSORT 2010 Statement: extension to cluster randomised trials. 2012. Campbell MK, Piaggio G, Elbourne DR, Altman DG. CONSORT 2010 Statement: extension to cluster randomised trials. 2012.
39.
go back to reference Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15(5):291–303.CrossRefPubMed Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15(5):291–303.CrossRefPubMed
40.
go back to reference Thabane L, Mbuagbaw L, Zhang S, Samaan Z, Marcucci M, Ye C, Debono VB. A tutorial on sensitivity analyses in clinical trials: the what, why, when and how. BMC Med Res Methodol. 2013;13(1):92.CrossRefPubMedPubMedCentral Thabane L, Mbuagbaw L, Zhang S, Samaan Z, Marcucci M, Ye C, Debono VB. A tutorial on sensitivity analyses in clinical trials: the what, why, when and how. BMC Med Res Methodol. 2013;13(1):92.CrossRefPubMedPubMedCentral
41.
go back to reference Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Stewart LA. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1.CrossRefPubMedPubMedCentral Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Stewart LA. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1.CrossRefPubMedPubMedCentral
43.
go back to reference Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Moher D. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390.CrossRefPubMedPubMedCentral Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Moher D. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390.CrossRefPubMedPubMedCentral
44.
go back to reference Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutic trials. J Chronic Dis. 20(8):637-48. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutic trials. J Chronic Dis. 20(8):637-48.
46.
go back to reference World Health Organization. Health promotion glossary. 1998. World Health Organization. Health promotion glossary. 1998.
48.
49.
go back to reference Keller LO, Strohschein S, Lia‐Hoagberg B, Schaffer MA. Population‐based public health interventions: practice‐based and evidence‐supported. Part I. Public Health Nurs. 2004;21(5):453–68.CrossRefPubMed Keller LO, Strohschein S, Lia‐Hoagberg B, Schaffer MA. Population‐based public health interventions: practice‐based and evidence‐supported. Part I. Public Health Nurs. 2004;21(5):453–68.CrossRefPubMed
51.
go back to reference Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Sterne JA. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.CrossRefPubMedPubMedCentral Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Sterne JA. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.CrossRefPubMedPubMedCentral
53.
go back to reference Donner A, Klar N. Design and analysis of cluster randomization trials in health research. 2000. p. 6–10. Donner A, Klar N. Design and analysis of cluster randomization trials in health research. 2000. p. 6–10.
54.
go back to reference Guyatt Gordon H, Oxman Andrew D, Vist Gunn E, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924.CrossRefPubMedPubMedCentral Guyatt Gordon H, Oxman Andrew D, Vist Gunn E, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924.CrossRefPubMedPubMedCentral
55.
go back to reference Ibrahim S, Sidani S. Fidelity of intervention implementation: a review of instruments. Health. 2015;7(12):1687.CrossRef Ibrahim S, Sidani S. Fidelity of intervention implementation: a review of instruments. Health. 2015;7(12):1687.CrossRef
56.
go back to reference Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327–50.CrossRefPubMed Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327–50.CrossRefPubMed
57.
go back to reference Duflo E, Glennerster R, Kremer M. Using randomization in development economics research: a toolkit. Handb Dev Econ. 2007;4:3895–962. Duflo E, Glennerster R, Kremer M. Using randomization in development economics research: a toolkit. Handb Dev Econ. 2007;4:3895–962.
58.
go back to reference Banerjee AV, Esther D. Poor economics: a radical rethinking of the way to fight global poverty. New York: Public Affairs; 2011. p. 303. ISBN 978-1-58648-798-0. Banerjee AV, Esther D. Poor economics: a radical rethinking of the way to fight global poverty. New York: Public Affairs; 2011. p. 303. ISBN 978-1-58648-798-0.
59.
go back to reference Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, Czajkowski S. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443.CrossRefPubMed Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, Czajkowski S. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443.CrossRefPubMed
Metadata
Title
Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review protocol
Authors
Myriam Cielo Pérez
Nanor Minoyan
Valéry Ridde
Marie-Pierre Sylvestre
Mira Johri
Publication date
01-12-2016
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2016
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/s13643-016-0351-0

Other articles of this Issue 1/2016

Systematic Reviews 1/2016 Go to the issue