Skip to main content
Top
Published in: Trials 1/2018

Open Access 01-12-2018 | Research

Surveying knowledge, practice and attitudes towards intervention fidelity within trials of complex healthcare interventions

Authors: Daragh McGee, Fabiana Lorencatto, Karen Matvienko-Sikar, Elaine Toomey

Published in: Trials | Issue 1/2018

Login to get access

Abstract

Background

Intervention fidelity is the degree to which interventions have been implemented as intended by their developers. Assessing fidelity is crucial for accurate interpretation of intervention effectiveness, but fidelity is often poorly addressed within trials of complex healthcare interventions. The reasons for this are unclear, and information on the use of methods to enhance and assess fidelity in trials of complex interventions remains insufficient. This study aimed to explore the knowledge, practice and attitudes towards intervention fidelity amongst researchers, triallists and healthcare professionals involved with the design and conduct of trials of complex healthcare interventions.

Methods

An online survey consisting of closed and open-ended questions exploring four sections (Demographics, Fidelity knowledge, Practice and Attitudes) was conducted. This was an opportunistic sample of individuals with experience of direct involvement in trials of complex healthcare interventions (e.g. design/development, conduct, evaluation).

Results

Data from 264 participants representing 15 countries were analysed. The majority (65.9%, n = 174) of participants identified themselves as ‘Researchers’. The majority of participants were familiar with the term “intervention fidelity” (69.7%, n = 184) and indicated that fidelity is important (89.7%, n = 236). Mean self-reported understanding of fidelity was moderate. Although 68% (n = 182) had previously used strategies to assess (e.g. audio/video-recording sessions) and enhance (e.g. training manual) fidelity in trials of complex interventions, only a limited proportion of participants indicated always reporting these strategies in subsequent publications (30.9%, n = 56). Poor knowledge or understanding was the most commonly cited barrier to addressing intervention fidelity in trials (77.4%, n = 202). Over half of respondents (52.1%, n = 137) had never completed specific fidelity training or research, and the vast majority (89.7%, n = 236) would welcome specific training in this area.

Conclusion

Despite good awareness of intervention fidelity and its importance, poor knowledge and understanding appears to be a key factor limiting how intervention fidelity is addressed in trials of complex interventions. Participants identified a need for further training and education in this area. Additionally, clarification of the terminology, definition and components of intervention fidelity would facilitate better understanding of the concept. A discrepancy between participants’ use of fidelity strategies and subsequent reporting raises concerns around inadequate fidelity reporting in the trials literature.
Appendix
Available only for authorised users
Literature
1.
go back to reference Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:1–6. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:1–6.
2.
go back to reference Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40.CrossRef Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40.CrossRef
3.
go back to reference Perepletchikova F, Kazdin AE. Treatment integrity and therapeutic change: issues and research recommendations. Clin Psychol Sci Pract. 2005;12:365–83.CrossRef Perepletchikova F, Kazdin AE. Treatment integrity and therapeutic change: issues and research recommendations. Clin Psychol Sci Pract. 2005;12:365–83.CrossRef
4.
go back to reference Slaughter SE, Hill JN, Snelgrove-Clarke E. What is the extent and quality of documentation and reporting of fidelity to implementation strategies: a scoping review. Implement Sci. 2015;10:129.CrossRef Slaughter SE, Hill JN, Snelgrove-Clarke E. What is the extent and quality of documentation and reporting of fidelity to implementation strategies: a scoping review. Implement Sci. 2015;10:129.CrossRef
5.
go back to reference Toomey E, Matthews J, Guerin S, Hurley DA. Development of a feasible implementation Fidelity protocol within a complex physical therapy–led self-management intervention. Phys Ther. 2016;96:1287–98.CrossRef Toomey E, Matthews J, Guerin S, Hurley DA. Development of a feasible implementation Fidelity protocol within a complex physical therapy–led self-management intervention. Phys Ther. 2016;96:1287–98.CrossRef
6.
go back to reference Dusenbury L, Brannigan R, Falco M, Hansen W. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18:237–56.CrossRef Dusenbury L, Brannigan R, Falco M, Hansen W. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18:237–56.CrossRef
7.
go back to reference Breitenstein SM, Gross D, Garvey C, Hill C, Fogg L, Resnick B. Implementation Fidelity in community-based interventions. Res Nurs Health. 2010;33:164–73.PubMedPubMedCentral Breitenstein SM, Gross D, Garvey C, Hill C, Fogg L, Resnick B. Implementation Fidelity in community-based interventions. Res Nurs Health. 2010;33:164–73.PubMedPubMedCentral
8.
go back to reference O’Shea O, McCormick R, Bradley JM, O’Neill B. Fidelity review: a scoping review of the methods used to evaluate treatment fidelity in behavioural change interventions. Phys Ther Rev. 2016;21:207–14.CrossRef O’Shea O, McCormick R, Bradley JM, O’Neill B. Fidelity review: a scoping review of the methods used to evaluate treatment fidelity in behavioural change interventions. Phys Ther Rev. 2016;21:207–14.CrossRef
9.
go back to reference Borrelli B, Sepinwall D, Ernst D, Bellg AJ, Czajkowski S, Breger R, DeFrancesco C, Levesque C, Sharp DL, Ogedegbe G, et al. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73:852–60.CrossRef Borrelli B, Sepinwall D, Ernst D, Bellg AJ, Czajkowski S, Breger R, DeFrancesco C, Levesque C, Sharp DL, Ogedegbe G, et al. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73:852–60.CrossRef
10.
go back to reference Toomey E, Currie-Murphy L, Matthews J, Hurley DA. Implementation fidelity of physiotherapist-delivered group education and exercise interventions to promote self-management in people with osteoarthritis and chronic low back pain: a rapid review part II. Man Ther. 2015;20:287–94.CrossRef Toomey E, Currie-Murphy L, Matthews J, Hurley DA. Implementation fidelity of physiotherapist-delivered group education and exercise interventions to promote self-management in people with osteoarthritis and chronic low back pain: a rapid review part II. Man Ther. 2015;20:287–94.CrossRef
11.
go back to reference Moncher FJ, Prinz RJ. Treatment fidelity in outcome studies. Clin Psychol Rev. 1991;11:247–66.CrossRef Moncher FJ, Prinz RJ. Treatment fidelity in outcome studies. Clin Psychol Rev. 1991;11:247–66.CrossRef
12.
go back to reference Schinckus L, Broucke S, Housiaux M. Assessment of implementation fidelity in diabetes self-management education programs: a systematic review. Patient Educ Couns. 2014;96:13–21.CrossRef Schinckus L, Broucke S, Housiaux M. Assessment of implementation fidelity in diabetes self-management education programs: a systematic review. Patient Educ Couns. 2014;96:13–21.CrossRef
13.
go back to reference Walton H, Spector A, Tombor I, Michie S. Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: a systematic review of measure quality. Br J Health Psychol. 2017;22:872–903.CrossRef Walton H, Spector A, Tombor I, Michie S. Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: a systematic review of measure quality. Br J Health Psychol. 2017;22:872–903.CrossRef
14.
go back to reference Lambert JD, Greaves CJ, Farrand P, Cross R, Haase AM, Taylor AH. Assessment of fidelity in individual level behaviour change interventions promoting physical activity among adults: a systematic review. BMC Public Health. 2017;17:765.CrossRef Lambert JD, Greaves CJ, Farrand P, Cross R, Haase AM, Taylor AH. Assessment of fidelity in individual level behaviour change interventions promoting physical activity among adults: a systematic review. BMC Public Health. 2017;17:765.CrossRef
15.
go back to reference Bellg A, Borrelli B, Resnick B, Hecht J, Minicucci D, Ory M, Ogedegbe G, Orwig D, Ernst D, Czajkowski S. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23:443–51.CrossRef Bellg A, Borrelli B, Resnick B, Hecht J, Minicucci D, Ory M, Ogedegbe G, Orwig D, Ernst D, Czajkowski S. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23:443–51.CrossRef
16.
go back to reference Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71:S52–63.CrossRef Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71:S52–63.CrossRef
17.
go back to reference Robb SL, Burns DS, Docherty SL, Haase JE. Ensuring treatment fidelity in a multi-site behavioral intervention study: implementing NIH behavior change consortium recommendations in the SMART trial. Psycho-Oncology. 2011;20:1193–201.CrossRef Robb SL, Burns DS, Docherty SL, Haase JE. Ensuring treatment fidelity in a multi-site behavioral intervention study: implementing NIH behavior change consortium recommendations in the SMART trial. Psycho-Oncology. 2011;20:1193–201.CrossRef
18.
go back to reference Spillane V, Byrne MC, Byrne M, Leathem CS, O’Malley M, Cupples ME. Monitoring treatment fidelity in a randomized controlled trial of a complex intervention. J Adv Nurs. 2007;60:343–52.CrossRef Spillane V, Byrne MC, Byrne M, Leathem CS, O’Malley M, Cupples ME. Monitoring treatment fidelity in a randomized controlled trial of a complex intervention. J Adv Nurs. 2007;60:343–52.CrossRef
19.
go back to reference Montgomery P, Underhill K, Gardner F, Operario D, Mayo-Wilson E. The Oxford implementation index: a new tool for incorporating implementation data into systematic reviews and meta-analyses. J Clin Epidemiol. 2013;66:874–82.CrossRef Montgomery P, Underhill K, Gardner F, Operario D, Mayo-Wilson E. The Oxford implementation index: a new tool for incorporating implementation data into systematic reviews and meta-analyses. J Clin Epidemiol. 2013;66:874–82.CrossRef
22.
go back to reference Lichstein KL, Riedel BW, Grieve R. Fair tests of clinical trials: a treatment implementation model. Adv Behav Res Ther. 1994;16:1–29.CrossRef Lichstein KL, Riedel BW, Grieve R. Fair tests of clinical trials: a treatment implementation model. Adv Behav Res Ther. 1994;16:1–29.CrossRef
23.
go back to reference Dane A, Schneider B. Program integrity in primary and early secondary prevention: are implementation effects out of control. Clin Psychol Rev. 1998;18:23–45.CrossRef Dane A, Schneider B. Program integrity in primary and early secondary prevention: are implementation effects out of control. Clin Psychol Rev. 1998;18:23–45.CrossRef
24.
go back to reference Gearing RE, El-Bassel N, Ghesquiere A, Baldwin S, Gillies J, Ngeow E. Major ingredients of fidelity: a review and scientific guide to improving quality of intervention research implementation. Clin Psychol Rev. 2011;31:79–88.CrossRef Gearing RE, El-Bassel N, Ghesquiere A, Baldwin S, Gillies J, Ngeow E. Major ingredients of fidelity: a review and scientific guide to improving quality of intervention research implementation. Clin Psychol Rev. 2011;31:79–88.CrossRef
25.
go back to reference Prowse PT, Nagel T. A meta-evaluation: the role of treatment fidelity within psychosocial interventions during the last decade. J Psychiatry. 2015;18:1–7. Prowse PT, Nagel T. A meta-evaluation: the role of treatment fidelity within psychosocial interventions during the last decade. J Psychiatry. 2015;18:1–7.
26.
go back to reference Parham LD, Cohn ES, Spitzer S, Koomar JA, Miller LJ, Burke JP, Brett-Green B, Mailloux Z, May-Benson TA, Roley SS, et al. Fidelity in sensory integration intervention research. Am J Occup Ther. 2007;61:216–27.CrossRef Parham LD, Cohn ES, Spitzer S, Koomar JA, Miller LJ, Burke JP, Brett-Green B, Mailloux Z, May-Benson TA, Roley SS, et al. Fidelity in sensory integration intervention research. Am J Occup Ther. 2007;61:216–27.CrossRef
27.
go back to reference Perepletchikova F, Hilt LM, Chereji E, Kazdin AE. Barriers to implementing treatment integrity procedures: survey of treatment outcome researchers. J Consult Clin Psychol. 2009;77:212–8.CrossRef Perepletchikova F, Hilt LM, Chereji E, Kazdin AE. Barriers to implementing treatment integrity procedures: survey of treatment outcome researchers. J Consult Clin Psychol. 2009;77:212–8.CrossRef
28.
go back to reference Cochrane WS, Laux JM. Investigating school psychologists’ perceptions of treatment integrity in school-based interventions for children with academic and behavior concerns. Prev Sch Fail. 2007;51:29–34.CrossRef Cochrane WS, Laux JM. Investigating school psychologists’ perceptions of treatment integrity in school-based interventions for children with academic and behavior concerns. Prev Sch Fail. 2007;51:29–34.CrossRef
29.
go back to reference Smith OM, Dale C, Mehta S, Pinto R, Rose L. Nurse research experiences and attitudes toward the conduct of intensive care research: a questionnaire study. Crit Care Med. 2016;44:153–61.CrossRef Smith OM, Dale C, Mehta S, Pinto R, Rose L. Nurse research experiences and attitudes toward the conduct of intensive care research: a questionnaire study. Crit Care Med. 2016;44:153–61.CrossRef
30.
go back to reference O’Donnell CL. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Rev Educ Res. 2008;78:33–84.CrossRef O’Donnell CL. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Rev Educ Res. 2008;78:33–84.CrossRef
31.
go back to reference Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.CrossRef Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.CrossRef
32.
go back to reference Waltz J, Addis ME, Koerner K, Jacobson NS. Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. J Consult Clin Psychol. 1993;61:620–30.CrossRef Waltz J, Addis ME, Koerner K, Jacobson NS. Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. J Consult Clin Psychol. 1993;61:620–30.CrossRef
33.
go back to reference Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Altman D, Barbour V, Macdonald H, Johnston M, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:1–12.CrossRef Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Altman D, Barbour V, Macdonald H, Johnston M, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:1–12.CrossRef
34.
go back to reference Preyde M, Burnham PV. Intervention fidelity in psychosocial oncology. J Evid Based Soc Work. 2011;8:379–96.CrossRef Preyde M, Burnham PV. Intervention fidelity in psychosocial oncology. J Evid Based Soc Work. 2011;8:379–96.CrossRef
35.
go back to reference Johnson-Kozlow M, Hovell MF, Rovniak LS, Sirikulvadhana L, Wahlgren DR, Zakarian JM. Fidelity issues in secondhand smoking interventions for children. Nicotine Tob Res. 2008;10:1677–90.CrossRef Johnson-Kozlow M, Hovell MF, Rovniak LS, Sirikulvadhana L, Wahlgren DR, Zakarian JM. Fidelity issues in secondhand smoking interventions for children. Nicotine Tob Res. 2008;10:1677–90.CrossRef
36.
go back to reference Holliday J, Audrey S, Moore L, Parry-Langdon N, Campbell R. High fidelity? How should we consider variations in the delivery of school-based health promotion interventions? Health Educ J. 2009;68:44–62.CrossRef Holliday J, Audrey S, Moore L, Parry-Langdon N, Campbell R. High fidelity? How should we consider variations in the delivery of school-based health promotion interventions? Health Educ J. 2009;68:44–62.CrossRef
37.
go back to reference Mowbray CT, Holter MC, Teague GB, Bybee D. Fidelity criteria: development, measurement, and validation. Am J Eval. 2003;24:315–40.CrossRef Mowbray CT, Holter MC, Teague GB, Bybee D. Fidelity criteria: development, measurement, and validation. Am J Eval. 2003;24:315–40.CrossRef
38.
go back to reference McHugh RK, Murray HW, Barlow DH. Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: the promise of Transdiagnostic interventions. Behav Res Ther. 2009;47:946–53.CrossRef McHugh RK, Murray HW, Barlow DH. Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: the promise of Transdiagnostic interventions. Behav Res Ther. 2009;47:946–53.CrossRef
39.
go back to reference Sanetti LMH, DiGennaro Reed FD. Barriers to implementing treatment integrity procedures in school psychology research. Assess Eff Interv. 2012;37:195–202. Sanetti LMH, DiGennaro Reed FD. Barriers to implementing treatment integrity procedures in school psychology research. Assess Eff Interv. 2012;37:195–202.
40.
go back to reference Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, Baird J. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:1–7.CrossRef Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, Baird J. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:1–7.CrossRef
41.
go back to reference Glasgow R, Vogt T, Boles S. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7.CrossRef Glasgow R, Vogt T, Boles S. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7.CrossRef
42.
go back to reference Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:65–6. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:65–6.
43.
go back to reference Green L, Kreuter M. Health promotion planning: an educational and environmental approach. Mountain View: Mayfield Publishing; 1991. Green L, Kreuter M. Health promotion planning: an educational and environmental approach. Mountain View: Mayfield Publishing; 1991.
44.
go back to reference Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;9:e108585. Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;9:e108585.
45.
go back to reference Saunders R, Evans M, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6:134–47.CrossRef Saunders R, Evans M, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6:134–47.CrossRef
46.
go back to reference Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:2-11. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:2-11.
47.
go back to reference Karas S, Plankis L. Consideration of treatment fidelity to improve manual therapy research. J Man Manip Ther. 2016;24:233–7.CrossRef Karas S, Plankis L. Consideration of treatment fidelity to improve manual therapy research. J Man Manip Ther. 2016;24:233–7.CrossRef
48.
go back to reference Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41:327–50.CrossRef Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41:327–50.CrossRef
49.
go back to reference Song M-K, Happ MB, Sandelowski M. Development of a tool to assess fidelity to a psycho-educational intervention. J Adv Nurs. 2010;66:673–82.CrossRef Song M-K, Happ MB, Sandelowski M. Development of a tool to assess fidelity to a psycho-educational intervention. J Adv Nurs. 2010;66:673–82.CrossRef
Metadata
Title
Surveying knowledge, practice and attitudes towards intervention fidelity within trials of complex healthcare interventions
Authors
Daragh McGee
Fabiana Lorencatto
Karen Matvienko-Sikar
Elaine Toomey
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Trials / Issue 1/2018
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/s13063-018-2838-6

Other articles of this Issue 1/2018

Trials 1/2018 Go to the issue