Skip to main content
Top
Published in: The Journal of Behavioral Health Services & Research 4/2012

01-10-2012

Measuring Enactment of Innovations and the Factors that Affect Implementation and Sustainability: Moving Toward Common Language and Shared Conceptual Understanding

Authors: Jeanne Century, EdD, Amy Cassata, PhD, Mollie Rudnick, MS Ed, Cassie Freeman, MA

Published in: The Journal of Behavioral Health Services & Research | Issue 4/2012

Login to get access

Abstract

This article describes research that focuses on the concern that researchers are unable to fully realize the potential value of their collective efforts because they do not have shared conceptual or operational tools for communicating assumptions, ideas, research strategies, or findings with others outside, or even within their disciplines. This research, through the lens of measuring implementation of educational programs, has taken steps toward bringing researchers’ varied pictures of understanding into a coherent landscape. This article describes a conceptual framework for describing aspects of implementation, a conceptual framework for describing the factors that affect implementation, and tools for measuring each. It describes the challenges addressed in the development of these approaches, and the application of these approaches to current studies in education and other fields in the social sciences. In doing so, it demonstrates that meaningful communication between researchers and accumulation of knowledge across fields is possible, and necessary.
Footnotes
1
See Researchers Without Borders at http://​www.​researcherswitho​utborders.​org
 
2
For more information on the search and review process of this project, see technical reports at http://​cemse.​uchicago.​edu/​research-and-evaluation/​research/​innovation/​
 
3
For more information on STEM school components go to: http://​researcherswitho​utborders.​org/​group/​stem
 
Literature
1.
go back to reference Aarons, G., Hurlburt, M., Horwitz, SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration Policy Mental Health. 2011; 38:4-23.PubMedCrossRef Aarons, G., Hurlburt, M., Horwitz, SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration Policy Mental Health. 2011; 38:4-23.PubMedCrossRef
2.
go back to reference Durlak, J., Dupre, E. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal Community Psychology. 2008; 41:327-350.PubMedCrossRef Durlak, J., Dupre, E. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal Community Psychology. 2008; 41:327-350.PubMedCrossRef
3.
go back to reference Proctor, E., Silmere, H., Raghavan, R., et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration Policy Mental Health. 2011; 38:65-76.PubMedCrossRef Proctor, E., Silmere, H., Raghavan, R., et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration Policy Mental Health. 2011; 38:65-76.PubMedCrossRef
4.
go back to reference Hulleman, C.S. & Cordray, D.S. (2009). Moving from the lab to the field: The role of fidelity and achieved intervention strength. Journal of Research on Educational Effectiveness, 2, 88-110.CrossRef Hulleman, C.S. & Cordray, D.S. (2009). Moving from the lab to the field: The role of fidelity and achieved intervention strength. Journal of Research on Educational Effectiveness, 2, 88-110.CrossRef
5.
go back to reference Mowbray, C. T., Holter, M. C., Teague, G. B., et al. Fidelity criteria: Development, measurement, and validation. The American Journal of Evaluation. 2003;24(3):315-340. Mowbray, C. T., Holter, M. C., Teague, G. B., et al. Fidelity criteria: Development, measurement, and validation. The American Journal of Evaluation. 2003;24(3):315-340.
6.
go back to reference O’Donnell, C.L. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research. 2008; 78(1): 33-84.CrossRef O’Donnell, C.L. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research. 2008; 78(1): 33-84.CrossRef
7.
go back to reference Domitrovich, C., Bradshaw, C., Poduska, J., et al. Advances in school mental health promotion. 2008; 1(3): 6- 28.CrossRef Domitrovich, C., Bradshaw, C., Poduska, J., et al. Advances in school mental health promotion. 2008; 1(3): 6- 28.CrossRef
8.
go back to reference Fixsen D., Blase K., Naoom S., et al. Core implementation components. Research on Social Work Practice. 2009;19(5):531.CrossRef Fixsen D., Blase K., Naoom S., et al. Core implementation components. Research on Social Work Practice. 2009;19(5):531.CrossRef
9.
go back to reference Sanetti, L.M.H., Kratochwill, T.R. Toward developing a science of treatment integrity: Introduction to the special series. School Psychology Review. 2009;38(4):445–459. Sanetti, L.M.H., Kratochwill, T.R. Toward developing a science of treatment integrity: Introduction to the special series. School Psychology Review. 2009;38(4):445–459.
10.
go back to reference Damschroder, L., Aron, D., Keith, R., et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. 2009; 4:50. Damschroder, L., Aron, D., Keith, R., et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. 2009; 4:50.
11.
go back to reference The road ahead: research partnerships to transform services. A report by the National Advisory Mental Health Council’s Workgroup on Services and Clinical Epidemiology Research. Bethesda, MD: Department of Health and Human Services, National Institute of Mental Health; 2001. The road ahead: research partnerships to transform services. A report by the National Advisory Mental Health Council’s Workgroup on Services and Clinical Epidemiology Research. Bethesda, MD: Department of Health and Human Services, National Institute of Mental Health; 2001.
12.
go back to reference Backer, T., David, S.L. Synthesis of behavioral science learning about technology transfer. National Institute on Drug Abuse. NIH Publication. 1995; 95-4035: 262-289. Backer, T., David, S.L. Synthesis of behavioral science learning about technology transfer. National Institute on Drug Abuse. NIH Publication. 1995; 95-4035: 262-289.
13.
go back to reference Greenhalgh, T., Robert, G., MacFarlane, F., et al. Diffusion of innovations in service organizations: systematic review and recommendations. 2004; 82(4): 581-629. Greenhalgh, T., Robert, G., MacFarlane, F., et al. Diffusion of innovations in service organizations: systematic review and recommendations. 2004; 82(4): 581-629.
14.
go back to reference Wandersman, A., Duffy, J., Flaspohler, P., et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. 2008; 41: 171-181. 8 Wandersman, A., Duffy, J., Flaspohler, P., et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. 2008; 41: 171-181. 8
15.
go back to reference Century, J., Rudnick, M. & Freeman, C. Measuring Fidelity of Implementation: A Foundation for Shared Language and Accumulation of Knowledge. American Journal of Evaluation. 2010;31(2):199-218.CrossRef Century, J., Rudnick, M. & Freeman, C. Measuring Fidelity of Implementation: A Foundation for Shared Language and Accumulation of Knowledge. American Journal of Evaluation. 2010;31(2):199-218.CrossRef
16.
go back to reference Hall, G. E., Hord, S. M. Change in Schools: Facilitating the Process. New York, State University of New York Press. 1987:117 Hall, G. E., Hord, S. M. Change in Schools: Facilitating the Process. New York, State University of New York Press. 1987:117
17.
go back to reference Bond, G., Williams, J., Evans, L., et al. Psychiatric Rehabilitation Fidelity Toolkit. Cambridge, MA: Human Services Research Institute. 2000. Bond, G., Williams, J., Evans, L., et al. Psychiatric Rehabilitation Fidelity Toolkit. Cambridge, MA: Human Services Research Institute. 2000.
18.
go back to reference Huntley, M. A. Operationalizing the concept of "fidelity of implementation" for NSF-funded mathematics curricula. Unpublished manuscript. 2005. Huntley, M. A. Operationalizing the concept of "fidelity of implementation" for NSF-funded mathematics curricula. Unpublished manuscript. 2005.
19.
go back to reference Wang, M. C., Nojan, M., Strom, C. D., et al. The utility of degree of implementation measures in program implementation and evaluation research. Curriculum Inquiry. 1984;14:249–286.CrossRef Wang, M. C., Nojan, M., Strom, C. D., et al. The utility of degree of implementation measures in program implementation and evaluation research. Curriculum Inquiry. 1984;14:249–286.CrossRef
20.
go back to reference Gresham F.M, Gansle K, Noell G.H. Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis. 1993; 26:257–263PubMedCrossRef Gresham F.M, Gansle K, Noell G.H. Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis. 1993; 26:257–263PubMedCrossRef
21.
go back to reference Rezmovic, E. L. Program implementation and evaluation results. Evaluation and Program Planning, 1982;5:111-118.CrossRef Rezmovic, E. L. Program implementation and evaluation results. Evaluation and Program Planning, 1982;5:111-118.CrossRef
22.
go back to reference Hall, G. E., & Loucks, S. F. A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 1977;14(3):263-276. Hall, G. E., & Loucks, S. F. A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 1977;14(3):263-276.
23.
go back to reference Fullan, M., & Pomfret, A. Research on curriculum and instruction implementation. Review of Educational Research. 1977;47(2):335-397. Fullan, M., & Pomfret, A. Research on curriculum and instruction implementation. Review of Educational Research. 1977;47(2):335-397.
24.
go back to reference Fullan, M. Evaluating program implementation: What can be learned from follow through. Curriculum Inquiry. 1983;13(2):215-227.CrossRef Fullan, M. Evaluating program implementation: What can be learned from follow through. Curriculum Inquiry. 1983;13(2):215-227.CrossRef
25.
go back to reference Gersten, R. M. & Carnine, D. Measuring implementation of the direct instruction model in an urban school district: An observational approach. Paper presented at the Annual Meeting of the American Educational Research Association, Boston, MA. 1980. Gersten, R. M. & Carnine, D. Measuring implementation of the direct instruction model in an urban school district: An observational approach. Paper presented at the Annual Meeting of the American Educational Research Association, Boston, MA. 1980.
26.
go back to reference Lastica, J & O’Donnell, C. Considering the role of fidelity of implementation in science education research: Fidelity as teacher and student adherence to structure. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. 2007. Lastica, J & O’Donnell, C. Considering the role of fidelity of implementation in science education research: Fidelity as teacher and student adherence to structure. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. 2007.
27.
go back to reference Dane, A. V., Schneider, B. H. Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review. 1998; 18:23-45.PubMedCrossRef Dane, A. V., Schneider, B. H. Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review. 1998; 18:23-45.PubMedCrossRef
28.
go back to reference Herie, M. & Martin, G. Knowledge diffusion in social work: A new approach to bridging the gap. Social Work. 2002;47(1):85—95PubMedCrossRef Herie, M. & Martin, G. Knowledge diffusion in social work: A new approach to bridging the gap. Social Work. 2002;47(1):85—95PubMedCrossRef
29.
go back to reference Cranney, M., Warren, E., Barton, S., et al. Why do GPs not implement evidence–based guidelines? A descriptive study. Family Practice. 2001;18(4):353–355.CrossRef Cranney, M., Warren, E., Barton, S., et al. Why do GPs not implement evidence–based guidelines? A descriptive study. Family Practice. 2001;18(4):353–355.CrossRef
30.
go back to reference Freeman, A. C., & Sweeney, K. Why general practitioners do not implement evidence: qualitative study. British Medical Journal. 2001:323(7321):1100–1102.PubMedCrossRef Freeman, A. C., & Sweeney, K. Why general practitioners do not implement evidence: qualitative study. British Medical Journal. 2001:323(7321):1100–1102.PubMedCrossRef
31.
go back to reference Louis, K. S. & Jones, L. M. Dissemination with impact: What research suggests for practice in career and technical education. St. Paul, MN: University of Minnesota, National Research Center for Career and Technical Education. 2001 Louis, K. S. & Jones, L. M. Dissemination with impact: What research suggests for practice in career and technical education. St. Paul, MN: University of Minnesota, National Research Center for Career and Technical Education. 2001
32.
go back to reference Robertson, T. (The process of innovation and the diffusion of innovation. Journal of Marketing. 1967;31(1):14-19.CrossRef Robertson, T. (The process of innovation and the diffusion of innovation. Journal of Marketing. 1967;31(1):14-19.CrossRef
Metadata
Title
Measuring Enactment of Innovations and the Factors that Affect Implementation and Sustainability: Moving Toward Common Language and Shared Conceptual Understanding
Authors
Jeanne Century, EdD
Amy Cassata, PhD
Mollie Rudnick, MS Ed
Cassie Freeman, MA
Publication date
01-10-2012
Publisher
Springer US
Published in
The Journal of Behavioral Health Services & Research / Issue 4/2012
Print ISSN: 1094-3412
Electronic ISSN: 2168-6793
DOI
https://doi.org/10.1007/s11414-012-9287-x

Other articles of this Issue 4/2012

The Journal of Behavioral Health Services & Research 4/2012 Go to the issue