Top

Published in:

01-03-2011

Putting the Pieces Together: An Integrated Model of Program Implementation

Authors: Cady Berkel, Anne M. Mauricio, Erin Schoenfelder, Irwin N. Sandler

Published in: Prevention Science | Issue 1/2011

Abstract

Considerable evidence indicates that variability in implementation of prevention programs is related to the outcomes achieved by these programs. However, while implementation has been conceptualized as a multidimensional construct, few studies examine more than a single dimension, and no theoretical framework exists to guide research on the effects of implementation. We seek to address this need by proposing a theoretical model of the relations between the dimensions of implementation and outcomes of prevention programs that can serve to guide future implementation research. In this article, we focus on four dimensions of implementation, which we conceptualize as behaviors of program facilitators (fidelity, quality of delivery, and adaptation) and behaviors of participants (responsiveness) and present the evidence supporting these as predictors of program outcomes. We then propose a theoretical model by which facilitator and participant dimensions of implementation influence participant outcomes. Finally, we provide recommendations and directions for future implementation research.

We distinguish between the concepts of attendance and dosage. Although attendance is sometimes described as “dosage” in the implementation literature (e.g., Charlebois et al. 2004a; Spoth and Redmond 2002), this confounds a participant behavior with an element of program design. We suggest that dosage should refer to the number of sessions offered, and is thus under the purview of program designers and administrators, while attendance should be used to describe number of sessions participants attend.

Backer, T. E. (2002). Finding the balance: Program fidelity and adaptation in substance abuse prevention. Rockville, MD: Center for Substance Abuse Prevention.

Baydar, N., Reid, M. J., & Webster-Stratton, C. (2003). The role of mental health factors and program engagement in the effectiveness of a preventive parenting program for Head Start mothers. Child Development, 74, 1433–1453.CrossRefPubMed

Berkel, C., Hagan, M., Jones, S., Ayers, T. S., Wolchik, S. A., & Sandler, I. N. (2009, November). Longitudinal examination of facilitator implementation: A case study across multiple cohorts of delivery. Paper presented at the annual meeting of the American Evaluation Association, Orlando, FL.

Blake, S. M., Simkin, L., Ledsky, R., Perkins, C., & Calabrese, J. M. (2001). Effects of a parent-child communications intervention on young adolescents’ risk for early onset of sexual intercourse. Family Planning Perspectives, 33, 52–61.CrossRefPubMed

Blakely, C. H., Mayer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W. S., Roitman, D. B., et al. (1987). The fidelity-adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253–268.CrossRef

Boruch, R. F., & Gomez, H. (1977). Sensitivity, bias and theory in impact evaluations. Professional Psychology, 8, 411–434.CrossRef

Botvin, G. J. (2004). Advancing prevention science and practice: Challenges, critical issues, and future directions. Prevention Science, 5, 69–72.CrossRefPubMed

Botvin, G. J., Dusenbury, L., Baker, E., James-Ortiz, S., & Kerner, J. F. (1989). A skills training approach to smoking prevention among Hispanic youth. Journal of Behavioral Medicine, 12, 279–296.CrossRefPubMed

Brach, C., & Fraser, I. (2000). Can cultural competency reduce racial and ethnic health disparities? A review and conceptual model. Medical Care Research & Review, 57, 181–217.

Bush, P. J., Zuckerman, A. E., Taggart, V. S., Theiss, P. K., Peleg, E. O., & Smith, S. A. (1989). Cardiovascular risk factor prevention in Black school children: The “Know Your Body” evaluation project. Health Education Quarterly, 16, 215–227.PubMed

Castro, F. G., Barrera, M., & Martinez, C. R. (2004). The cultural adaptation of prevention interventions: Resolving tensions between fidelity and fit. Prevention Science, 5, 41–45.CrossRefPubMed

Charlebois, P., Brendgen, M., Vitaro, F., Normandeau, S., & Boudreau, J.-F. (2004). Examining dosage effects on prevention outcomes: Results from a multi-modal longitudinal preventive intervention for young disruptive boys. Journal of School Psychology, 42, 201–220.CrossRef

Charlebois, P., Vitaro, F., Normandeau, S., Brendgen, M., & Rondeau, N. (2004). Trainers’ behavior and participants’ persistence in a longitudinal preventive intervention for disruptive boys. Journal of Primary Prevention, 25, 375–388.CrossRef

Coatsworth, J. D., Duncan, L. G., Pantin, H., & Szapocznik, J. (2006a). Patterns of retention in a preventive intervention with ethnic minority families. Journal of Primary Prevention, 27, 171–193.CrossRefPubMed

Coatsworth, J. D., Duncan, L. G., Pantin, H., & Szapocznik, J. (2006b). Retaining ethnic minority parents in a preventive intervention: The quality of group process. Journal of Primary Prevention, 27, 367–389.CrossRefPubMed

Conduct Problems Prevention Research Group. (1999). Initial impact of the Fast Track prevention trial for conduct problems: II. Classroom effects. Journal of Consulting & Clinical Psychology, 67, 648–657.CrossRef

Cross, T. L., Bazron, B. J., Dennis, K. W., & Isaacs, M. R. (1989). Towards a culturally competent system of care: A monograph on effective services for minority children who are severely emotionally disturbed: Volume I. Washington, DC: National Technical Assistance Center for Children’s Mental Health, Georgetown University Child Development Center.

Cross, W., West, J., Wyman, P. A., & Schmeelk-Cone, K. H. (2009). Disaggregating and measuring implementor adherence and competence. Paper presented at the 2nd Annual NIH Conference on the Science of Dissemination and Implementation, Bethesda, MD.

Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45.CrossRefPubMed

Diamond, G. M., Liddle, H. A., Hogue, A., & Dakof, G. A. (1999). Alliance-building interventions with adolescents in family therapy: A process study. Psychotherapy: Theory, Research, Practice, Training, 36, 355–368.CrossRef

Dillman Carpentier, F., Mauricio, A. M., Gonzales, N. A., Millsap, R. E., Meza, C. M., Dumka, L. E., et al. (2007). Engaging Mexican origin families in a school-based preventive intervention. Journal of Primary Prevention, 28, 521–546.CrossRefPubMed

Durlak, J. (1998). Why program implementation is important. Journal of Prevention & Intervention in the Community, 17, 5–18.CrossRef

Durlak, J., & DuPre, E. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.CrossRefPubMed

Dusenbury, L. A., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.CrossRefPubMed

Dusenbury, L. A., Brannigan, R., Hansen, W. B., Walsh, J., & Falco, M. (2005). Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research, 20, 308–313.CrossRefPubMed

Eames, C., Daley, D., Hutchings, J., Whitaker, C. J., Jones, K., Hughes, J. C., et al. (2009). Treatment fidelity as a predictor of behaviour change in parents attending group-based parent training. Child Care, Health & Development, 35, 603–612.CrossRef

Eddy, J. M., Dishion, T. J., & Stoolmiller, M. (1998). The analysis of intervention change in children and families: Methodological and conceptual issues embedded in intervention studies. Journal of Abnormal Child Psychology, 26, 53–69.CrossRefPubMed

Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5, 47–52.CrossRefPubMed

Emshoff, J., Blakely, C., Gray, D., Jakes, S., Brounstein, P., Coulter, J., et al. (2003). An ESID case study at the federal level. American Journal of Community Psychology, 32, 345–357.CrossRefPubMed

Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development study. American Journal of Community Psychology, 41, 235–249.CrossRefPubMed

Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behavior Therapy, 36, 3–13.CrossRefPubMed

Garvey, C., Julion, W., Fogg, L., Kratovil, A., & Gross, D. (2006). Measuring participation in a prevention trial with parents of young children. Research in Nursing & Health, 29, 212–222.CrossRef

Giles, S., Jackson-Newsom, J., Pankratz, M., Hansen, W., Ringwalt, C., & Dusenbury, L. (2008). Measuring quality of delivery in a substance use prevention program. Journal of Primary Prevention, 29, 489–501.CrossRefPubMed

Gingiss, P., Roberts-Gray, C., & Boerm, M. (2006). Bridge-It: A system for predicting implementation fidelity for school-based tobacco prevention programs. Prevention Science, 7, 197–207.CrossRefPubMed

Gonzales, N. A., Wolchik, S. A., Sandler, I. N., Winslow, E. B., Martinez, C. R., & Cooley, M. (2006, June). Quality management methods to adapt interventions for cultural diversity. Paper presented at the annual meeting of the Society for Prevention Research, San Antonio, TX.

Haggerty, K. P., Fleming, C. B., Lonczak, H. S., Oxford, M. L., Harachi, T. W., & Catalano, R. F. (2002). Predictors of participation in parenting workshops. Journal of Primary Prevention, 22, 375–387.CrossRef

Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991). Program integrity as a moderator of prevention program effectiveness: Results for fifth-grade students in the Adolescent Alcohol Prevention Trial. Journal of Studies on Alcohol, 52, 568–579.PubMed

Hubble, M. A., Duncan, B. L., & Miller, S. D. (1999). The heart and soul of change: What works in therapy. Washington, DC: American Psychological Association.CrossRef

Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27, 599–641.CrossRefPubMed

Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63.CrossRefPubMed

Kerr, D. M., Kent, L., & Lam, T. C. (1985). Measuring program implementation with a classroom observation instrument: The Interactive Teaching Map. Evaluation Review, 9, 461–482.CrossRef

Kosterman, R., Hawkins, J. D., Haggerty, K. P., Spoth, R., & Redmond, C. (2001). Preparing for the Drug Free Years: Session-specific effects of a universal parent-training intervention with rural families. Journal of Drug Education, 31, 47–68.CrossRefPubMed

Kumpfer, K. L., Alvarado, R., Smith, P., & Bellamy, N. (2002). Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science, 3, 241–246.CrossRefPubMed

Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education & Behavior, 31, 242–257.CrossRef

Lochman, J. E., Boxmeyer, C., Powell, N., Qu, L., Wells, K., & Windle, M. (2009). Dissemination of the Coping Power program: Importance of intensity of counselor training. Journal of Consulting & Clinical Psychology, 77, 397–409.CrossRef

McGraw, S. A., Sellers, D. E., Stone, E. J., Bebchuk, J., Edmundson, E. W., Johnson, C. C., et al. (1996). Using process data to explain outcomes: An illustration from the Child and Adolescent Trial for Cardiovascular Health (CATCH). Evaluation Review, 20, 291–312.CrossRefPubMed

NRC/IOM. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Research Council and Institute of Medicine.

Nye, C. L., Zucker, R. A., & Fitzgerald, H. E. (1995). Early intervention in the path to alcohol problems through conduct problems: Treatment involvement and child behavior change. Journal of Consulting & Clinical Psychology, 63, 831–840.CrossRef

Palinkas, L. A., Aarons, G. A., Chorpita, B. F., Hoagwood, K., Landsverk, J., & Weisz, J. R. (2009). Cultural exchange and the implementation of evidence-based practices: Two case studies. Research on Social Work Practice, 19, 602–612.CrossRef

Parcel, G. S., Ross, J. G., Lavin, A. T., Portnoy, B., Nelson, G. D., & Winters, F. (1991). Enhancing implementation of the Teenage Health Teaching Modules. Journal of School Health, 61, 35–38.CrossRefPubMed

Patterson, G. R., & Forgatch, M. S. (1985). Therapist behavior as a determinant for client noncompliance: A paradox for the behavior modifier. Journal of Consulting & Clinical Psychology, 53, 846–851.CrossRef

Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. P., Dwyer, J. H., Johnson, C. A., et al. (1990). Effects of program implementation on adolescent drug use behavior: The Midwestern Prevention Project (MPP). Evaluation Review, 14, 264–289.CrossRef

Prado, G., Pantin, H., Schwartz, S. J., Lupei, N. S., & Szapocznik, J. (2006). Predictors of engagement and retention into a parent-centered, ecodevelopmental HIV preventive intervention for Hispanic adolescents and their families. Journal of Pediatric Psychology, 31, 874–890.CrossRefPubMed

Resnicow, K., Davis, M., Smith, M., Lazarus-Yaroch, A., Baranowski, T., Baranowski, J., et al. (1998). How best to measure implementation of school health curricula: A comparison of three measures. Health Education Research, 13, 239–250.CrossRefPubMed

Ringwalt, C. L., Vincus, A., Ennett, S., Johnson, R., & Rohrbach, L. A. (2004). Reasons for teachers’ adaptation of substance use prevention curricula in schools with non-White student populations. Prevention Science, 5, 61–67.CrossRefPubMed

Ringwalt, C. L., Pankratz, M., Jackson-Newsom, J., Gottfredson, N., Hansen, W. B., Giles, S. et al. (2010). Three-year trajectory of teachers’ fidelity to a drug prevention curriculum. Prevention Science, 11, 67–76.

Rogers, E. M. (1995). Diffusion of drug abuse prevention programs: Spontaneous diffusion, agenda setting, and reinvention. Bethesda, MD: NIDA Research Monograph.

Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: The Free Press.

Rohrbach, L. A., Gunning, M., Sun, P., & Sussman, S. (2010). The Project Towards No Drug Abuse (TND) dissemination trial: Implementation fidelity and immediate outcomes. Prevention Science, 11, 77–88.CrossRefPubMed

Shelef, K., Diamond, G. M., Diamond, G. S., & Liddle, H. A. (2005). Adolescent and parent alliance and treatment outcome in Multidimensional Family Therapy. Journal of Consulting & Clinical Psychology, 73, 689–698.CrossRef

Slavin, R. E. (1990). Comprehensive cooperative learning models: Embedding cooperative learning in the curriculum and school. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 261–283). New York: Praeger.

Spoth, R. L., & Redmond, C. (2002). Project Family prevention trials based in community-university partnerships: Toward scaled-up preventive interventions. Prevention Science, 3, 203–221.CrossRefPubMed

Spoth, R., Guyll, M., Trudeau, L., & Goldberg-Lillehoj, C. (2002). Two studies of proximal outcomes and implementation quality of universal preventive interventions in a community-university collaboration context. Journal of Community Psychology, 30, 499–518.CrossRef

Stanton, B., Harris, C., Cottrell, L., Li, X., Gibson, C., Guo, J., et al. (2006). Trial of an urban adolescent sexual risk-reduction intervention for rural youth: A promising but imperfect fit. Journal of Adolescent Health, 38, 25–36.CrossRef

Tobler, N. S., & Stratton, H. H. (1997). Effectiveness of school-based drug prevention programs: A meta-analysis of the research. Journal of Primary Prevention, 18, 71–128.CrossRef

Tolan, P. H., Hanish, L. D., McKay, M. M., & Dickey, M. H. (2002). Evaluating process in child and family interventions: Aggression prevention as an example. Journal of Family Psychology, 16, 220–236.CrossRefPubMed

Widaman, K. F., & Kagan, S. (1987). Cooperativeness and achievement: Interaction of student cooperativeness with cooperative versus competitive classroom organization. Journal of School Psychology, 25, 355–365.CrossRef

Wilson, B. D. M., & Miller, R. L. (2003). Examining strategies for culturally grounded HIV prevention: A review. AIDS Education & Prevention, 15, 184–202.CrossRef

Wilson, S. J., & Lipsey, M. W. (2007). School-based interventions for aggressive and disruptive behavior: Update of a meta-analysis. American Journal of Preventive Medicine, 33, S130–S143.CrossRefPubMed

Wolchik, S. A., Sandler, I. N., Jones, S., Gonzales, N., Doyle, K., Winslow, E., et al. (2009). The New Beginnings Program for divorcing and separating families: Moving from efficacy to effectiveness. Family Court Review, 47, 416–435.CrossRefPubMed

Title: Putting the Pieces Together: An Integrated Model of Program Implementation
Authors: Cady Berkel
Anne M. Mauricio
Erin Schoenfelder
Irwin N. Sandler
Publication date: 01-03-2011
Publisher: Springer US
Published in: Prevention Science / Issue 1/2011
Print ISSN: 1389-4986
Electronic ISSN: 1573-6695
DOI: https://doi.org/10.1007/s11121-010-0186-1

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Putting the Pieces Together: An Integrated Model of Program Implementation

Abstract

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2011

Social Networking Technology, Social Network Composition, and Reductions in Substance Use Among Homeless Adolescents

Improving Attention and Preventing Reading Difficulties among Low-Income First-Graders: A Randomized Study

When to Intervene: Elementary School, Middle School or Both? Effects of keepin’ It REAL on Substance Use Trajectories of Mexican Heritage Youth

Using Community Based Participatory Research to Create a Culturally Grounded Intervention for Parents and Youth to Prevent Risky Behaviors

The Prevalence of Evidence-based Drug Use Prevention Curricula in U.S. Middle Schools in 2008

Short-Term Efficacy of Click City®: Tobacco: Changing Etiological Mechanisms Related to the Onset of Tobacco Use