Skip to main content
Top
Published in: Systematic Reviews 1/2014

Open Access 01-12-2014 | Protocol

A protocol for a systematic review of the use of process evaluations in knowledge translation research

Authors: Shannon D Scott, Thomas Rotter, Lisa Hartling, Thane Chambers, Katherine H Bannar-Martin

Published in: Systematic Reviews | Issue 1/2014

Login to get access

Abstract

Background

Experimental designs for evaluating knowledge translation (KT) interventions for professional behavior change can provide strong estimates of intervention effectiveness but offer limited insight how the intervention worked or not. Furthermore, trials provide little insight into the ways through which interventions lead to behavior change and how they are moderated by different facilitators and barriers. As a result, the ability to generalize the findings from one study to a different context, organization, or clinical problem is severely compromised. Consequently, researchers have started to explore the causal mechanisms in complementary studies (process evaluations) alongside experimental designs for evaluating KT interventions. This study focuses on improving process evaluations by synthesizing current evidence on process evaluations conducted alongside experimental designs for evaluating KT interventions.

Methods/Design

A medical research librarian will develop and implement search strategies designed to identify evidence that is relevant to process evaluations in health research. Studies will not be excluded based on design. Included studies must contain a process evaluation component aimed at understanding or evaluating a KT intervention targeting professional behavior change. Two reviewers will perform study selection, quality assessment, and data extraction using standard forms. Disagreements will be resolved through discussion or third party adjudication. Data to be collected include study design, details about data collection approaches and types, theoretical influences, approaches to evaluate intervention dose delivered, intervention dose received, intervention fidelity, intervention reach, data analysis, and study outcomes. This study is not registered with PROSPERO.

Discussion

There is widespread acceptance that the generalizability of quantitative trials of KT interventions would be significantly enhanced to other contexts, health professional groups, and clinical conditions through complementary process evaluations alongside trials. This systematic review will serve as a ‘state of the science’ on methodological approaches to process evaluations and will allow us to: 1) take stock of current research approaches and 2) develop concrete recommendations for knowledge users (e.g., quality consultants and health services researchers) designing future KT process evaluations.
Appendix
Available only for authorised users
Literature
1.
go back to reference Bhattacharyya OK, Estey EA, Zwarenstein M: Methodologies to evaluate the effectiveness of knowledge translation interventions: a primer for researchers and health care managers. J Clin Epidemiol. 2011, 64 (1): 32-40. 10.1016/j.jclinepi.2010.02.022.CrossRefPubMed Bhattacharyya OK, Estey EA, Zwarenstein M: Methodologies to evaluate the effectiveness of knowledge translation interventions: a primer for researchers and health care managers. J Clin Epidemiol. 2011, 64 (1): 32-40. 10.1016/j.jclinepi.2010.02.022.CrossRefPubMed
2.
go back to reference Walter I, Nutley S, Davies H: Developing a taxonomy of interventions used to increase the impact of research. 2003, ᅟ: University of St. Andrews Walter I, Nutley S, Davies H: Developing a taxonomy of interventions used to increase the impact of research. 2003, ᅟ: University of St. Andrews
3.
go back to reference Kotaska A: Inappropriate use of randomised trials to evaluate complex phenomena: case study of vaginal breech delivery. BMJ. 2004, 329: 1039-1042. 10.1136/bmj.329.7473.1039.CrossRefPubMedPubMedCentral Kotaska A: Inappropriate use of randomised trials to evaluate complex phenomena: case study of vaginal breech delivery. BMJ. 2004, 329: 1039-1042. 10.1136/bmj.329.7473.1039.CrossRefPubMedPubMedCentral
4.
go back to reference Seers K: Evaluating complex interventions. Worldviews Evid-Based Nurs. 2007, 4 (2): 67-68. 10.1111/j.1741-6787.2007.00083.x.CrossRefPubMed Seers K: Evaluating complex interventions. Worldviews Evid-Based Nurs. 2007, 4 (2): 67-68. 10.1111/j.1741-6787.2007.00083.x.CrossRefPubMed
5.
go back to reference Hawe P, Shiell A, Riley T: Complex interventions: how “out of control” can a randomised controlled trial be?. BMJ. 2004, 328: 1561-1563. 10.1136/bmj.328.7455.1561.CrossRefPubMedPubMedCentral Hawe P, Shiell A, Riley T: Complex interventions: how “out of control” can a randomised controlled trial be?. BMJ. 2004, 328: 1561-1563. 10.1136/bmj.328.7455.1561.CrossRefPubMedPubMedCentral
6.
go back to reference Wollf N: Using randomized controlled trials to evaluate socially complex services: problems, challenges and recommendations. J Ment Health Policy Econ. 2000, 3 (2): 97-109. 10.1002/1099-176X(200006)3:2<97::AID-MHP77>3.0.CO;2-S.CrossRef Wollf N: Using randomized controlled trials to evaluate socially complex services: problems, challenges and recommendations. J Ment Health Policy Econ. 2000, 3 (2): 97-109. 10.1002/1099-176X(200006)3:2<97::AID-MHP77>3.0.CO;2-S.CrossRef
8.
go back to reference Patton M: Developmental evaluation: applying complexity concepts to enhance innovation use. 2011, New York: Guilford Press Patton M: Developmental evaluation: applying complexity concepts to enhance innovation use. 2011, New York: Guilford Press
9.
go back to reference Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P: Framework for design and evaluation of complex interventions to improve health. BMJ. 2000, 321 (7262): 694-696. 10.1136/bmj.321.7262.694.CrossRefPubMedPubMedCentral Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P: Framework for design and evaluation of complex interventions to improve health. BMJ. 2000, 321 (7262): 694-696. 10.1136/bmj.321.7262.694.CrossRefPubMedPubMedCentral
10.
go back to reference Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008, 337 (7667): 979-983. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008, 337 (7667): 979-983.
11.
go back to reference Oakley A, Strange V, Bonell C, Allen E, Stephenson J: Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006, 332: 413-416. 10.1136/bmj.332.7538.413.CrossRefPubMedPubMedCentral Oakley A, Strange V, Bonell C, Allen E, Stephenson J: Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006, 332: 413-416. 10.1136/bmj.332.7538.413.CrossRefPubMedPubMedCentral
12.
go back to reference Anderson P, Benford M, Harris N, Karavali M, Piercy J: Real-world physician and patient behaviour across countries: disease-specific programmes - a means to understand. Curr Med Res Opin. 2008, 24 (11): 3063-3072. 10.1185/03007990802457040.CrossRefPubMed Anderson P, Benford M, Harris N, Karavali M, Piercy J: Real-world physician and patient behaviour across countries: disease-specific programmes - a means to understand. Curr Med Res Opin. 2008, 24 (11): 3063-3072. 10.1185/03007990802457040.CrossRefPubMed
13.
go back to reference Wallin L: Knowledge translation and implementation research in nursing. Int J Nurs Stud. 2009, 46 (4): 576-587. 10.1016/j.ijnurstu.2008.05.006.CrossRefPubMed Wallin L: Knowledge translation and implementation research in nursing. Int J Nurs Stud. 2009, 46 (4): 576-587. 10.1016/j.ijnurstu.2008.05.006.CrossRefPubMed
14.
go back to reference Hasson H: Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implement Sci. 2010, 5: 67-10.1186/1748-5908-5-67.CrossRefPubMedPubMedCentral Hasson H: Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implement Sci. 2010, 5: 67-10.1186/1748-5908-5-67.CrossRefPubMedPubMedCentral
15.
go back to reference Hasson H, Blomberg S, Duner A: Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implement Sci. 2012, 7: 23-10.1186/1748-5908-7-23.CrossRefPubMedPubMedCentral Hasson H, Blomberg S, Duner A: Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implement Sci. 2012, 7: 23-10.1186/1748-5908-7-23.CrossRefPubMedPubMedCentral
16.
go back to reference Grimshaw J, Zwarenstein M, Tetroe JM, Godin G, Graham ID, Lemrye L, Eccles MP, Johnston M, Francis JJ, O’Rourke K, Légare F, Presseau J: Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario. Canada Implement Sci. 2007, 2 (1): 38-10.1186/1748-5908-2-38.CrossRefPubMed Grimshaw J, Zwarenstein M, Tetroe JM, Godin G, Graham ID, Lemrye L, Eccles MP, Johnston M, Francis JJ, O’Rourke K, Légare F, Presseau J: Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario. Canada Implement Sci. 2007, 2 (1): 38-10.1186/1748-5908-2-38.CrossRefPubMed
17.
go back to reference Ramsay C, Thomas RE, Croal BL, Grimshaw JM, Eccles MP: Using the theory of planned behaviour as a process evaluation tool in randomised trials of knowledge translation strategies: a case study from UK primary care. Implement Sci. 2010, 5 (1): 71-10.1186/1748-5908-5-71.CrossRefPubMedPubMedCentral Ramsay C, Thomas RE, Croal BL, Grimshaw JM, Eccles MP: Using the theory of planned behaviour as a process evaluation tool in randomised trials of knowledge translation strategies: a case study from UK primary care. Implement Sci. 2010, 5 (1): 71-10.1186/1748-5908-5-71.CrossRefPubMedPubMedCentral
18.
go back to reference Herr K, Titler M, Fine PG, Sanders S, Cavanaugh JE, Swegle J, Tang X, Forcucci C: The Effect of a Translating Research into Practice (TRIP)-cancer intervention on cancer pain management in older adults in hospice. Pain Med. 2012, 13 (8): 1004-1017. 10.1111/j.1526-4637.2012.01405.x.CrossRefPubMedPubMedCentral Herr K, Titler M, Fine PG, Sanders S, Cavanaugh JE, Swegle J, Tang X, Forcucci C: The Effect of a Translating Research into Practice (TRIP)-cancer intervention on cancer pain management in older adults in hospice. Pain Med. 2012, 13 (8): 1004-1017. 10.1111/j.1526-4637.2012.01405.x.CrossRefPubMedPubMedCentral
19.
go back to reference Fretheim A, Aaserud M, Oxman AD: Rational prescribing in primary care (RaPP): economic evaluation of an intervention to improve professional practice. PLoS Med Publ Librar Sci. 2006, 3 (6): e216-10.1371/journal.pmed.0030216.CrossRef Fretheim A, Aaserud M, Oxman AD: Rational prescribing in primary care (RaPP): economic evaluation of an intervention to improve professional practice. PLoS Med Publ Librar Sci. 2006, 3 (6): e216-10.1371/journal.pmed.0030216.CrossRef
20.
go back to reference Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005, 58 (2): 107-112. 10.1016/j.jclinepi.2004.09.002.CrossRefPubMed Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005, 58 (2): 107-112. 10.1016/j.jclinepi.2004.09.002.CrossRefPubMed
21.
go back to reference Grant A, Shaun T, Tobias D, Robbie F, Bruce G: Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials [Electronic Resource]. 2013, 14: 15- Grant A, Shaun T, Tobias D, Robbie F, Bruce G: Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials [Electronic Resource]. 2013, 14: 15-
22.
go back to reference Lewin S, Glenton C, Oxman AD: Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ. 2009, 339 (7723): 732-734. Lewin S, Glenton C, Oxman AD: Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ. 2009, 339 (7723): 732-734.
23.
go back to reference Franx G, Oud M, de Lange J, Wensing M, Grol R: Implementing a stepped-care approach in primary care: results of a qualitative study. Implement Sci. 2012, 7: 8-10.1186/1748-5908-7-8.CrossRefPubMedPubMedCentral Franx G, Oud M, de Lange J, Wensing M, Grol R: Implementing a stepped-care approach in primary care: results of a qualitative study. Implement Sci. 2012, 7: 8-10.1186/1748-5908-7-8.CrossRefPubMedPubMedCentral
24.
go back to reference Francis JJ, Eccles MP, Johnston M, Whitty P, Grimshaw JM, Kaner EF, Smith L, Walker A: Explaining the effects of an intervention designed to promote evidence-based diabetes care: a theory-based process evaluation of a pragmatic cluster randomised controlled trial. Implement Sci. 2008, 3: 50-10.1186/1748-5908-3-50.CrossRefPubMedPubMedCentral Francis JJ, Eccles MP, Johnston M, Whitty P, Grimshaw JM, Kaner EF, Smith L, Walker A: Explaining the effects of an intervention designed to promote evidence-based diabetes care: a theory-based process evaluation of a pragmatic cluster randomised controlled trial. Implement Sci. 2008, 3: 50-10.1186/1748-5908-3-50.CrossRefPubMedPubMedCentral
25.
go back to reference Brady M, Stott DJ, Norrie J, Chalmers C, St George B, Sweeney PM, Langhorne P: Developing and evaluating the implementation of a complex intervention: using mixed methods to inform the design of a randomised controlled trial of an oral healthcare intervention after stroke. Trials [Electronic Resource]. 2011, 12 (1): 168- Brady M, Stott DJ, Norrie J, Chalmers C, St George B, Sweeney PM, Langhorne P: Developing and evaluating the implementation of a complex intervention: using mixed methods to inform the design of a randomised controlled trial of an oral healthcare intervention after stroke. Trials [Electronic Resource]. 2011, 12 (1): 168-
26.
go back to reference Scott S, Albrecht L, O’Leary K, Ball GDC, Dryden DM, Hartling L, Hofmeyer A, Jones CA, Kovacs Burns K, Newton AS, Thompson D, Klassen TP: A protocol of a systematic review of knowledge translation strategies in the allied health professions. Implementation Science. 2011, 6 (58): 1-23. Doi:10.1186/1748-5908-6-58 Scott S, Albrecht L, O’Leary K, Ball GDC, Dryden DM, Hartling L, Hofmeyer A, Jones CA, Kovacs Burns K, Newton AS, Thompson D, Klassen TP: A protocol of a systematic review of knowledge translation strategies in the allied health professions. Implementation Science. 2011, 6 (58): 1-23. Doi:10.1186/1748-5908-6-58
27.
go back to reference WHO: Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference. New York, 19-22. June, 1946; signed on 22 July 1946 by the representatives of 61 States (Official Records of the World Health Organization, no. 2, p. 100) and entered into force on 7 April 1948. WHO: Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference. New York, 19-22. June, 1946; signed on 22 July 1946 by the representatives of 61 States (Official Records of the World Health Organization, no. 2, p. 100) and entered into force on 7 April 1948.
29.
go back to reference O’Cathain A: Assessing the quality of mixed methods research: towards a comprehensive framework. Handbook of mixed methods in social and behavioral research. Edited by: Tashakkori ACT. 2010, Thousand Oaks: Sage, 531-555.CrossRef O’Cathain A: Assessing the quality of mixed methods research: towards a comprehensive framework. Handbook of mixed methods in social and behavioral research. Edited by: Tashakkori ACT. 2010, Thousand Oaks: Sage, 531-555.CrossRef
30.
go back to reference Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, Seller R: Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int J Nurs Stud. 2012, 49 (1): 47-53. 10.1016/j.ijnurstu.2011.07.002.CrossRefPubMed Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, Seller R: Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int J Nurs Stud. 2012, 49 (1): 47-53. 10.1016/j.ijnurstu.2011.07.002.CrossRefPubMed
31.
go back to reference Crowe M, Sheppard L: A review of critical appraisal tools show they lack rigor: alternative tool structure is proposed. J Clin Epidemiol. 2011, 64 (1): 79-89. 10.1016/j.jclinepi.2010.02.008.CrossRefPubMed Crowe M, Sheppard L: A review of critical appraisal tools show they lack rigor: alternative tool structure is proposed. J Clin Epidemiol. 2011, 64 (1): 79-89. 10.1016/j.jclinepi.2010.02.008.CrossRefPubMed
32.
go back to reference Mays N, Pope C, Popay J: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005, 10: 6-20. 10.1258/1355819054308576.CrossRefPubMed Mays N, Pope C, Popay J: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005, 10: 6-20. 10.1258/1355819054308576.CrossRefPubMed
33.
go back to reference Barbour RS: Mixing qualitative methods: quality assurance or qualitative quagmire?. Qual Health Res. 1998, 8 (3): 352-361. 10.1177/104973239800800306.CrossRefPubMed Barbour RS: Mixing qualitative methods: quality assurance or qualitative quagmire?. Qual Health Res. 1998, 8 (3): 352-361. 10.1177/104973239800800306.CrossRefPubMed
Metadata
Title
A protocol for a systematic review of the use of process evaluations in knowledge translation research
Authors
Shannon D Scott
Thomas Rotter
Lisa Hartling
Thane Chambers
Katherine H Bannar-Martin
Publication date
01-12-2014
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2014
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/2046-4053-3-149

Other articles of this Issue 1/2014

Systematic Reviews 1/2014 Go to the issue