Skip to main content
Top
Published in: Implementation Science 1/2007

Open Access 01-12-2007 | Study protocol

Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada

Authors: Jeremy M Grimshaw, Merrick Zwarenstein, Jacqueline M Tetroe, Gaston Godin, Ian D Graham, Louise Lemyre, Martin P Eccles, Marie Johnston, Jillian J Francis, Jan Hux, Keith O'Rourke, France Légaré, Justin Presseau

Published in: Implementation Science | Issue 1/2007

Login to get access

Abstract

Background

Randomised controlled trials of implementation strategies tell us whether (or not) an intervention results in changes in professional behaviour but little about the causal mechanisms that produce any change. Theory-based process evaluations collect data on theoretical constructs alongside randomised trials to explore possible causal mechanisms and effect modifiers. This is similar to measuring intermediate endpoints in clinical trials to further understand the biological basis of any observed effects (for example, measuring lipid profiles alongside trials of lipid lowering drugs where the primary endpoint could be reduction in vascular related deaths).
This study protocol describes a theory-based process evaluation alongside the Ontario Printed Educational Message (OPEM) trial. We hypothesize that the OPEM interventions are most likely to operate through changes in physicians' behavioural intentions due to improved attitudes or subjective norms with little or no change in perceived behavioural control. We will test this hypothesis using a well-validated social cognition model, the theory of planned behaviour (TPB) that incorporates these constructs.

Methods/design

We will develop theory-based surveys using standard methods based upon the TPB for the second and third replications, and survey a subsample of Ontario family physicians from each arm of the trial two months before and six months after the dissemination of the index edition of informed, the evidence based newsletter used for the interventions. In the third replication, our study will converge with the "TRY-ME" protocol (a second study conducted alongside the OPEM trial), in which the content of educational messages was constructed using both standard methods and methods informed by psychological theory. We will modify Dillman's total design method to maximise response rates. Preliminary analyses will initially assess the internal reliability of the measures and use regression to explore the relationships between predictor and dependent variable (intention to advise diabetic patients to have annual retinopathy screening and to prescribe thiazide diuretics for first line treatment of uncomplicated hypertension). We will then compare groups using methods appropriate for comparing independent samples to determine whether there have been changes in the predicted constructs (attitudes, subjective norms, or intentions) across the study groups as hypothesised, and will assess the convergence between the process evaluation results and the main trial results.

Trial registration number

Current controlled trial ISRCTN72772651
Literature
1.
go back to reference Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA: Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ. 1998, 317: 465-468.CrossRefPubMedPubMedCentral Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA: Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ. 1998, 317: 465-468.CrossRefPubMedPubMedCentral
2.
go back to reference NHS Centre for Reviews and Dissemination: Getting evidence into practice. Effective Health Care Bulletin. 1999, 5: 1-16. NHS Centre for Reviews and Dissemination: Getting evidence into practice. Effective Health Care Bulletin. 1999, 5: 1-16.
3.
go back to reference Zwarenstein M, Hux JE, Kelsall DL, Paterson JM, Grimshaw JM, Davis DA: The Ontario Printed Educational Message Trial (OPEM). Implementation Science. 2005, Zwarenstein M, Hux JE, Kelsall DL, Paterson JM, Grimshaw JM, Davis DA: The Ontario Printed Educational Message Trial (OPEM). Implementation Science. 2005,
4.
go back to reference Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay C, Vale L: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8: Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay C, Vale L: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8:
5.
go back to reference Eccles M, Grimshaw J, Steen N, Parkin D, Purves I, McColl E: The design and analysis of a randomized controlled trial to evaluate computerized decision support in primary care: the COGENT study. Fam Pract. 2000, 17: 180-186. 10.1093/fampra/17.2.180.CrossRefPubMed Eccles M, Grimshaw J, Steen N, Parkin D, Purves I, McColl E: The design and analysis of a randomized controlled trial to evaluate computerized decision support in primary care: the COGENT study. Fam Pract. 2000, 17: 180-186. 10.1093/fampra/17.2.180.CrossRefPubMed
6.
go back to reference Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M: Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003, 326: 314-10.1136/bmj.326.7384.314.CrossRefPubMedPubMedCentral Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M: Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003, 326: 314-10.1136/bmj.326.7384.314.CrossRefPubMedPubMedCentral
7.
go back to reference Ferlie EB, Shortell SM: Improving the quality of health care in the United Kingdom and the United States: a framework for change. The Milbank Quarterly. 2001, 79: 281-315. 10.1111/1468-0009.00206.CrossRefPubMedPubMedCentral Ferlie EB, Shortell SM: Improving the quality of health care in the United Kingdom and the United States: a framework for change. The Milbank Quarterly. 2001, 79: 281-315. 10.1111/1468-0009.00206.CrossRefPubMedPubMedCentral
8.
go back to reference Francis J, Grimshaw JM, Zwarenstein M, Eccles MP, Shiller S, Johston M, O'Rourke K, Presseau J, Tetroe J: Testing a TheoRY-informed MEssage ("TRY-ME"): A Sub-trial Within a Randomized Controlled Trial of Printed Educational Materials (The Ontario Printed Educational Message (OPEM) Trial). Implementation Science. 2006, 2 (39): Francis J, Grimshaw JM, Zwarenstein M, Eccles MP, Shiller S, Johston M, O'Rourke K, Presseau J, Tetroe J: Testing a TheoRY-informed MEssage ("TRY-ME"): A Sub-trial Within a Randomized Controlled Trial of Printed Educational Materials (The Ontario Printed Educational Message (OPEM) Trial). Implementation Science. 2006, 2 (39):
9.
go back to reference Ajzen I: The theory of planned behaviour. Organizational Behaviour and Human Decision Process. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.CrossRef Ajzen I: The theory of planned behaviour. Organizational Behaviour and Human Decision Process. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.CrossRef
10.
go back to reference Francis JJ, Eccles MP, Johnston M, Walker AE, Grimshaw JM, Foy R: Constructing questionnaires based on the theory of planned behaviour. A manual for health services researchers. 2004, Centre for Health Services Research, University of Newcastle upon Tyne, UK Francis JJ, Eccles MP, Johnston M, Walker AE, Grimshaw JM, Foy R: Constructing questionnaires based on the theory of planned behaviour. A manual for health services researchers. 2004, Centre for Health Services Research, University of Newcastle upon Tyne, UK
11.
go back to reference Godin G, Kok G: The theory of planned behavior: a review of its applications to health-related behaviors. Am J Health Promot. 1996, 11: 87-98.CrossRefPubMed Godin G, Kok G: The theory of planned behavior: a review of its applications to health-related behaviors. Am J Health Promot. 1996, 11: 87-98.CrossRefPubMed
12.
go back to reference Dillman D: Mail and internet surveys. The tailored design method. 2000, New York: John Wiley, 2 Dillman D: Mail and internet surveys. The tailored design method. 2000, New York: John Wiley, 2
13.
go back to reference Cummings SM, Savitz LA, Konrad TR: Reported response rates to mailed physician questionnaires. Health Services Research. 2001, 35: 1347-1355.PubMedPubMedCentral Cummings SM, Savitz LA, Konrad TR: Reported response rates to mailed physician questionnaires. Health Services Research. 2001, 35: 1347-1355.PubMedPubMedCentral
14.
go back to reference Field JL, Cadoret CA, Brown ML, Ford M, Greene SM, Hill J: Surveying Physicians. Do components of the"Total Design Approach" to optimizing survey response rates apply to physicians?. Med Care. 2002, 40: 596-606. 10.1097/00005650-200207000-00006.CrossRefPubMed Field JL, Cadoret CA, Brown ML, Ford M, Greene SM, Hill J: Surveying Physicians. Do components of the"Total Design Approach" to optimizing survey response rates apply to physicians?. Med Care. 2002, 40: 596-606. 10.1097/00005650-200207000-00006.CrossRefPubMed
15.
go back to reference Rabaneck L, Viscole CM, Horwitz RJ: Problems in the conduct and analysis of randomized clinical trials. Archives of Internal Medicine. 1992, 152: 507-512. 10.1001/archinte.152.3.507.CrossRef Rabaneck L, Viscole CM, Horwitz RJ: Problems in the conduct and analysis of randomized clinical trials. Archives of Internal Medicine. 1992, 152: 507-512. 10.1001/archinte.152.3.507.CrossRef
16.
go back to reference Gilliss CL, Kulkin IL: Monitoring nursing interventions and data collection in a randomized clinical trial. Western Journal of Nursing Research. 1991, 13: 416-422. 10.1177/019394599101300312.CrossRefPubMed Gilliss CL, Kulkin IL: Monitoring nursing interventions and data collection in a randomized clinical trial. Western Journal of Nursing Research. 1991, 13: 416-422. 10.1177/019394599101300312.CrossRefPubMed
17.
go back to reference McCullagh P, Nelder JA: Generalized linear models. 1999, London: Chapman & Hall Ltd McCullagh P, Nelder JA: Generalized linear models. 1999, London: Chapman & Hall Ltd
18.
go back to reference Cox DR, Wermuth N: Multivariate dependencies: models, analysis, and interpretation. 1998, Chapman and Hall Ltd Cox DR, Wermuth N: Multivariate dependencies: models, analysis, and interpretation. 1998, Chapman and Hall Ltd
19.
go back to reference Greenland S, O"Rourke K: Meta-Analysis. Modern Epidemiology. Edited by: Rothman KJ, Greenland S, Lash T. 2007, Lippincott Williams and Wilkins, 3, Greenland S, O"Rourke K: Meta-Analysis. Modern Epidemiology. Edited by: Rothman KJ, Greenland S, Lash T. 2007, Lippincott Williams and Wilkins, 3,
20.
go back to reference Thompson SG, Higgins JP: How should meta-regression analyses be undertaken and interpreted?. Stat Med. 2002, 15: 1559-1573. 10.1002/sim.1187.CrossRef Thompson SG, Higgins JP: How should meta-regression analyses be undertaken and interpreted?. Stat Med. 2002, 15: 1559-1573. 10.1002/sim.1187.CrossRef
21.
go back to reference Shadish WR, Cook TD, Campbell DT: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. 2002, Boston: Houghton-Mifflin Shadish WR, Cook TD, Campbell DT: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. 2002, Boston: Houghton-Mifflin
Metadata
Title
Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada
Authors
Jeremy M Grimshaw
Merrick Zwarenstein
Jacqueline M Tetroe
Gaston Godin
Ian D Graham
Louise Lemyre
Martin P Eccles
Marie Johnston
Jillian J Francis
Jan Hux
Keith O'Rourke
France Légaré
Justin Presseau
Publication date
01-12-2007
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2007
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/1748-5908-2-38

Other articles of this Issue 1/2007

Implementation Science 1/2007 Go to the issue