Skip to main content
Top
Published in: Implementation Science 1/2015

Open Access 01-12-2015 | Research

Factors influencing implementation dose and fidelity thereof and related student outcomes of an evidence-based national HIV prevention program

Authors: Bo Wang, Bonita Stanton, Lynette Deveaux, Maxwell Poitier, Sonja Lunn, Veronica Koci, Richard Adderley, Linda Kaljee, Sharon Marshall, Xiaoming Li, Glenda Rolle

Published in: Implementation Science | Issue 1/2015

Login to get access

Abstract

Background

Teachers’ implementation of evidence-based prevention programs in schools is inconsistent. Using data gathered from the national implementation among grade six students in The Bahamas of an evidence-based HIV intervention [Focus on Youth in the Caribbean (FOYC)], this study examines differences in the degree of implementation (“dose”) and adherence to the core activities (“fidelity of implementation”) by teachers according to theoretically and historically relevant teachers’ characteristics, attitudes, and experiences pre-intervention and post-intervention. The relationship of implementation dose and implementation fidelity is assessed according to student outcomes.

Methods

Beginning in 2008, the Bahamian Ministry of Education (MOE) included FOYC in the grade six curriculum nationwide. Consistent with standard practice, teachers were offered MOE training workshops in FOYC prior to delivery. The MOE conducted an anonymous curricular assessment among the grade six students at the beginning and end of the school year. Teachers agreeing to participate in the research component were asked to complete a pre-implementation and post-implementation assessment of attitudes and prior experiences.

Results

Teachers taught 15.6 out of 30 core activities, 24 out of the 46 total activities, and 4.6 out of 8 sessions on average. Three teachers’ implementation groups were identified: 1) High Implementation Group (31.7% of the teachers), characterized by high levels of implementation dose and fidelity of implementation; 2) Moderate Implementation Group (52.8%), showing moderate levels of implementation dose but high levels of fidelity of implementation; and 3) Low Implementation Group (15.6%), with low levels of implementation dose and fidelity of implementation. Low Implementation Group teachers compared to teachers in the two higher performing groups had less training in interactive teaching, limited prior exposure to the FOYC curriculum, incomplete attendance at FOYC training workshops, and low levels of comfort in teaching FOYC lessons. Students taught by teachers in the Low Implementation Group demonstrated poorer outcomes relevant to the four student outcomes (HIV/AIDS knowledge, preventive reproductive health skills, self-efficacy, and intention to use protection if they were to have sex).

Conclusions

Both implementation dose and implementation fidelity are related to student outcomes. Teachers at risk for limited implementation can be identified pre-intervention, thus opening the possibility for focused pre-intervention training.
Literature
1.
go back to reference Spoth R, Rohrbach LA, Greenberg M, Leaf P, Brown CH, Fagan A, et al. Addressing core challenges for the next generation of type 2 translation research and systems: the translation science to population impact (TSci Impact) framework. Prev Sci. 2013;14(4):319–51.PubMedPubMedCentralCrossRef Spoth R, Rohrbach LA, Greenberg M, Leaf P, Brown CH, Fagan A, et al. Addressing core challenges for the next generation of type 2 translation research and systems: the translation science to population impact (TSci Impact) framework. Prev Sci. 2013;14(4):319–51.PubMedPubMedCentralCrossRef
3.
go back to reference Gottfredson DC, Gottfredson GD. Quality of school-based prevention programs: results from a national survey. J Res Crime Delinquency. 2002;39:3–35.CrossRef Gottfredson DC, Gottfredson GD. Quality of school-based prevention programs: results from a national survey. J Res Crime Delinquency. 2002;39:3–35.CrossRef
4.
go back to reference Lyles CM, Kay LS, Crepaz N, Herbst JH, Passin WF, Kim AS, et al. Best-evidence interventions: findings from a systematic review of HIV behavioral interventions for US populations at high risk, 2000–2004. Am J Public Health. 2007;97(1):133–43.PubMedPubMedCentralCrossRef Lyles CM, Kay LS, Crepaz N, Herbst JH, Passin WF, Kim AS, et al. Best-evidence interventions: findings from a systematic review of HIV behavioral interventions for US populations at high risk, 2000–2004. Am J Public Health. 2007;97(1):133–43.PubMedPubMedCentralCrossRef
5.
go back to reference Ringwalt CL, Ennett S, Johnson R, Rohrbach LA, Simons-Rudolph A, Vincus A, et al. Factors associated with fidelity to substance use prevention curriculum guides in the nation’s middle schools. Health Educ Behav. 2003;30(3):375–91.PubMedCrossRef Ringwalt CL, Ennett S, Johnson R, Rohrbach LA, Simons-Rudolph A, Vincus A, et al. Factors associated with fidelity to substance use prevention curriculum guides in the nation’s middle schools. Health Educ Behav. 2003;30(3):375–91.PubMedCrossRef
6.
go back to reference Rohrbach LA, Graham JW, Hansen WB. Diffusion of a school-based substance abuse prevention program: predictors of program implementation. Prev Med. 1993;22(2):237–60.PubMedCrossRef Rohrbach LA, Graham JW, Hansen WB. Diffusion of a school-based substance abuse prevention program: predictors of program implementation. Prev Med. 1993;22(2):237–60.PubMedCrossRef
7.
go back to reference Adelman HS, Taylor L. On sustainability of project innovations as systemic change. J Educ Psychol Consultation. 2003;14(1):1–25.CrossRef Adelman HS, Taylor L. On sustainability of project innovations as systemic change. J Educ Psychol Consultation. 2003;14(1):1–25.CrossRef
8.
go back to reference Gersten R, Chard D, Baker S. Factors enhancing sustained use of research-based instructional practices. J Learn Disabil. 2000;33(5):445–57.PubMedCrossRef Gersten R, Chard D, Baker S. Factors enhancing sustained use of research-based instructional practices. J Learn Disabil. 2000;33(5):445–57.PubMedCrossRef
9.
go back to reference Boardman AG, Argüelles ME, Vaughn S, Hughes MT, Klingner J. Special education teachers’ views of research-based practices. J Special Educ. 2005;39(3):168–80.CrossRef Boardman AG, Argüelles ME, Vaughn S, Hughes MT, Klingner J. Special education teachers’ views of research-based practices. J Special Educ. 2005;39(3):168–80.CrossRef
10.
go back to reference Han SS, Weiss B. Sustainability of teacher implementation of school-based mental health programs. J Abnorm Child Psychol. 2005;33(6):665–79.PubMedCrossRef Han SS, Weiss B. Sustainability of teacher implementation of school-based mental health programs. J Abnorm Child Psychol. 2005;33(6):665–79.PubMedCrossRef
11.
go back to reference Kincaid D, Childs K, Blase KA, Wallace F. Identifying barriers and facilitators in implementing schoolwide positive behavior support. J Positive Behav Interv. 2007;9(3):174–84.CrossRef Kincaid D, Childs K, Blase KA, Wallace F. Identifying barriers and facilitators in implementing schoolwide positive behavior support. J Positive Behav Interv. 2007;9(3):174–84.CrossRef
12.
go back to reference McIntosh K, Horner RH, Sugai G. Sustainability of systems-level evidence-based practices in schools: current knowledge and future directions. In: Sailor W, Dunlap G, Sugai G, Horner RH, editors. Handbook of positive behavior support. New York: Springer; 2009. p. 327–52.CrossRef McIntosh K, Horner RH, Sugai G. Sustainability of systems-level evidence-based practices in schools: current knowledge and future directions. In: Sailor W, Dunlap G, Sugai G, Horner RH, editors. Handbook of positive behavior support. New York: Springer; 2009. p. 327–52.CrossRef
13.
go back to reference McIntosh K, Predy LK, Upreti G, Hume AE, Turri MG, Mathews S. Perceptions of contextual features related to implementation and sustainability of school-wide positive behavior support. J Positive Behav Interv. 2013. doi:10.1177/1098300712470723 McIntosh K, Predy LK, Upreti G, Hume AE, Turri MG, Mathews S. Perceptions of contextual features related to implementation and sustainability of school-wide positive behavior support. J Positive Behav Interv. 2013. doi:10.1177/1098300712470723
14.
go back to reference Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.PubMedCrossRef Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.PubMedCrossRef
15.
go back to reference McIntosh K, Mercer SH, Humes AE, Frank JL, Turri MG, Matthews S. Factors related to sustained implementation of school-wide positive behavior support. Except Child. 2013;79(3):293–311. McIntosh K, Mercer SH, Humes AE, Frank JL, Turri MG, Matthews S. Factors related to sustained implementation of school-wide positive behavior support. Except Child. 2013;79(3):293–311.
16.
go back to reference Basen-Engquist K, O’Hara-Tompkins N, Lovato CY, Lewis MJ, Parcel GS, Gingiss P. The effect of two types of teacher training on implementation of Smart Choices: a tobacco prevention curriculum. J School Health. 1994;64(8):334–9.PubMedCrossRef Basen-Engquist K, O’Hara-Tompkins N, Lovato CY, Lewis MJ, Parcel GS, Gingiss P. The effect of two types of teacher training on implementation of Smart Choices: a tobacco prevention curriculum. J School Health. 1994;64(8):334–9.PubMedCrossRef
17.
go back to reference Dusenbury L, Falco M. Eleven components of effective drug abuse prevention curricula. J School Health. 1995;65(10):420–5.PubMedCrossRef Dusenbury L, Falco M. Eleven components of effective drug abuse prevention curricula. J School Health. 1995;65(10):420–5.PubMedCrossRef
18.
go back to reference Perry CL, Murray DM, Griffin G. Evaluating the statewide dissemination of smoking prevention curricula: factors in teacher compliance. J School Health. 1990;60:501–4.PubMedCrossRef Perry CL, Murray DM, Griffin G. Evaluating the statewide dissemination of smoking prevention curricula: factors in teacher compliance. J School Health. 1990;60:501–4.PubMedCrossRef
19.
go back to reference Beets MW, Flay BR, Vuchinich S, Acock AC, Li K-K, Allred C. School climate and teachers’ beliefs and attitudes associated with implementation of the positive action program: a diffusion of innovations model. Prev Sci. 2008;9(4):264–75.PubMedCrossRef Beets MW, Flay BR, Vuchinich S, Acock AC, Li K-K, Allred C. School climate and teachers’ beliefs and attitudes associated with implementation of the positive action program: a diffusion of innovations model. Prev Sci. 2008;9(4):264–75.PubMedCrossRef
20.
go back to reference Parcel GS, O’Hara-Tompkins NM, Harrist RB, Basen-Engquist KM, McCormick LK, Gottlieb NH, et al. Diffusion of an effective tobacco prevention program. Part II: evaluation of the adoption phase Health Educ Res. 1995;10(3):297–307.PubMed Parcel GS, O’Hara-Tompkins NM, Harrist RB, Basen-Engquist KM, McCormick LK, Gottlieb NH, et al. Diffusion of an effective tobacco prevention program. Part II: evaluation of the adoption phase Health Educ Res. 1995;10(3):297–307.PubMed
21.
go back to reference Draper CE, Nemutandani SM, Grimsrud AT, Rudolph M, Kolbe-Alexander TL, de Kock L, et al. Qualitative evaluation of a physical activity-based chronic disease prevention program in a low-income, rural South African setting. Rural Remote Health. 2010;10(3):1467.PubMed Draper CE, Nemutandani SM, Grimsrud AT, Rudolph M, Kolbe-Alexander TL, de Kock L, et al. Qualitative evaluation of a physical activity-based chronic disease prevention program in a low-income, rural South African setting. Rural Remote Health. 2010;10(3):1467.PubMed
22.
go back to reference Mihalic SF, Fagan AA, Argamaso S. Implementing the LifeSkills Training drug prevention program: factors related to implementation fidelity. Implementation Science. 2008;3(5):1–16. Mihalic SF, Fagan AA, Argamaso S. Implementing the LifeSkills Training drug prevention program: factors related to implementation fidelity. Implementation Science. 2008;3(5):1–16.
23.
go back to reference Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50.PubMedCrossRef Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50.PubMedCrossRef
24.
go back to reference Blakely CH, Mayer JP, Gottschalk RG, Schmitt N, Davidson WS, Roitman DB, et al. The fidelity-adaptation debate: implications for the implementation of public sector social programs. Am J Community Psychol. 1987;15(3):253–68.CrossRef Blakely CH, Mayer JP, Gottschalk RG, Schmitt N, Davidson WS, Roitman DB, et al. The fidelity-adaptation debate: implications for the implementation of public sector social programs. Am J Community Psychol. 1987;15(3):253–68.CrossRef
25.
go back to reference Smith DJ, Schneider BH, Smith PK, Ananiadou K. The effectiveness of whole-school antibullying programs: a synthesis of evaluation research. School Psychol Rev. 2004;33(4):547–60. Smith DJ, Schneider BH, Smith PK, Ananiadou K. The effectiveness of whole-school antibullying programs: a synthesis of evaluation research. School Psychol Rev. 2004;33(4):547–60.
26.
go back to reference Derzon JH, Sale E, Springer JF, Brounstein P. Estimating intervention effectiveness: synthetic projection of field evaluation results. J Prim Prev. 2005;26(4):321–43.PubMedCrossRef Derzon JH, Sale E, Springer JF, Brounstein P. Estimating intervention effectiveness: synthetic projection of field evaluation results. J Prim Prev. 2005;26(4):321–43.PubMedCrossRef
27.
go back to reference Klingner JK, Ahwee S, Pilonieta P, Menendez R. Barriers and facilitators in scaling up research-based practices. Except Child. 2003;69(4):411–29. Klingner JK, Ahwee S, Pilonieta P, Menendez R. Barriers and facilitators in scaling up research-based practices. Except Child. 2003;69(4):411–29.
28.
go back to reference Shin Y, Miller-Day M, Pettigrew J, Hecht ML, Krieger JL. Typology of delivery quality: latent profile analysis of teacher engagement and delivery techniques in a school-based prevention intervention, keepin’ it REAL curriculum. Health Educ Res. 2014;29(6):897–905.PubMedPubMedCentralCrossRef Shin Y, Miller-Day M, Pettigrew J, Hecht ML, Krieger JL. Typology of delivery quality: latent profile analysis of teacher engagement and delivery techniques in a school-based prevention intervention, keepin’ it REAL curriculum. Health Educ Res. 2014;29(6):897–905.PubMedPubMedCentralCrossRef
29.
go back to reference Rodgers R. Cognitive and physiological processes in fear appeals and attitude change: a revised theory of protection motivation. In: Caaioppi T, Petty R, editors. Social psychology. New York, NY: Guilford Press; 1983. p. 153–76. Rodgers R. Cognitive and physiological processes in fear appeals and attitude change: a revised theory of protection motivation. In: Caaioppi T, Petty R, editors. Social psychology. New York, NY: Guilford Press; 1983. p. 153–76.
30.
go back to reference Deveaux L, Stanton B, Lunn S, Cottrell L, Yu S, Brathwaite N, et al. Reduction in human immunodeficiency virus risk among youth in developing countries. Arch Pediatr Adolesc Med. 2007;161(12):1130–9.PubMedCrossRef Deveaux L, Stanton B, Lunn S, Cottrell L, Yu S, Brathwaite N, et al. Reduction in human immunodeficiency virus risk among youth in developing countries. Arch Pediatr Adolesc Med. 2007;161(12):1130–9.PubMedCrossRef
31.
go back to reference Gong J, Stanton B, Lunn S, Deveaux L, Li X, Marshall S, et al. Effects through 24 months of an HIV/AIDS prevention intervention program based on protection motivation theory among preadolescents in the Bahamas. Pediatrics. 2009;123:e917–28.PubMedPubMedCentralCrossRef Gong J, Stanton B, Lunn S, Deveaux L, Li X, Marshall S, et al. Effects through 24 months of an HIV/AIDS prevention intervention program based on protection motivation theory among preadolescents in the Bahamas. Pediatrics. 2009;123:e917–28.PubMedPubMedCentralCrossRef
32.
go back to reference Dinaj-Koci V, Brathwaite N, Deveaux L, Lunn S, Cottrell L, Harris C, et al. When things are not as they appear: assessing the adequacy of cluster randomization when outcome events are rare at baseline. AIDS Res Treat. 2012. doi:10.1155/2012/806384 Dinaj-Koci V, Brathwaite N, Deveaux L, Lunn S, Cottrell L, Harris C, et al. When things are not as they appear: assessing the adequacy of cluster randomization when outcome events are rare at baseline. AIDS Res Treat. 2012. doi:10.1155/2012/806384
33.
go back to reference Chen X, Lunn S, Deveaux L, Li X, Brathwaite N, Cottrell L, et al. A cluster randomized controlled trial of an adolescent HIV prevention program among Bahamian youth: effect at 12 months post-intervention. AIDS Behav. 2009;13:499–508.PubMedCrossRef Chen X, Lunn S, Deveaux L, Li X, Brathwaite N, Cottrell L, et al. A cluster randomized controlled trial of an adolescent HIV prevention program among Bahamian youth: effect at 12 months post-intervention. AIDS Behav. 2009;13:499–508.PubMedCrossRef
34.
go back to reference Wang B, Deveaux L, Knowles V, Koci V, Rolle G, Lunn S, et al. Fidelity of implementation of an evidence-based HIV prevention program among Bahamian sixth grade students. Prev Sci. 2015;16:110–21.PubMedCrossRef Wang B, Deveaux L, Knowles V, Koci V, Rolle G, Lunn S, et al. Fidelity of implementation of an evidence-based HIV prevention program among Bahamian sixth grade students. Prev Sci. 2015;16:110–21.PubMedCrossRef
35.
go back to reference Stanton B, Deveaux L, Lunn S, Yu S, Brathwaite N, Li X, et al. Condom-use skills checklist: a proxy for assessing condom-use knowledge and skills when direct observation is not possible. J Health Popul Nutr. 2009;27:406–13.PubMedPubMedCentralCrossRef Stanton B, Deveaux L, Lunn S, Yu S, Brathwaite N, Li X, et al. Condom-use skills checklist: a proxy for assessing condom-use knowledge and skills when direct observation is not possible. J Health Popul Nutr. 2009;27:406–13.PubMedPubMedCentralCrossRef
36.
37.
go back to reference Sinyor M, Schaffer A, Streiner DL. Characterizing suicide in Toronto: an observational study and cluster analysis. Can J Psychiatry. 2014;59(1):26–33.PubMedPubMedCentral Sinyor M, Schaffer A, Streiner DL. Characterizing suicide in Toronto: an observational study and cluster analysis. Can J Psychiatry. 2014;59(1):26–33.PubMedPubMedCentral
38.
go back to reference Swaminathan S, Thomas T, Yusuf S, Vaz M. Clustering of diet, physical activity and overweight in parents and offspring in South India. Eur J Clin Nutr. 2013;67:128–34.PubMedCrossRef Swaminathan S, Thomas T, Yusuf S, Vaz M. Clustering of diet, physical activity and overweight in parents and offspring in South India. Eur J Clin Nutr. 2013;67:128–34.PubMedCrossRef
39.
go back to reference Ennett ST, Ringwalt CL, Thorne J, Rohrbach LA, Vincus A, Simons-Rudolph A, et al. A comparison of current practice in school-based substance use prevention programs with meta-analysis findings. Prev Sci. 2003;4(1):1–14.PubMedCrossRef Ennett ST, Ringwalt CL, Thorne J, Rohrbach LA, Vincus A, Simons-Rudolph A, et al. A comparison of current practice in school-based substance use prevention programs with meta-analysis findings. Prev Sci. 2003;4(1):1–14.PubMedCrossRef
40.
go back to reference Domitrovich CE, Bradshaw CP, Poduska JM, Hoagwood K, Buckley JA, Olin S, et al. Maximizing the implementation quality of evidence-based preventive interventions in schools: a conceptual framework. Adv School Mental Health Promot. 2008;1(3):6–28.CrossRef Domitrovich CE, Bradshaw CP, Poduska JM, Hoagwood K, Buckley JA, Olin S, et al. Maximizing the implementation quality of evidence-based preventive interventions in schools: a conceptual framework. Adv School Mental Health Promot. 2008;1(3):6–28.CrossRef
Metadata
Title
Factors influencing implementation dose and fidelity thereof and related student outcomes of an evidence-based national HIV prevention program
Authors
Bo Wang
Bonita Stanton
Lynette Deveaux
Maxwell Poitier
Sonja Lunn
Veronica Koci
Richard Adderley
Linda Kaljee
Sharon Marshall
Xiaoming Li
Glenda Rolle
Publication date
01-12-2015
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2015
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-015-0236-y

Other articles of this Issue 1/2015

Implementation Science 1/2015 Go to the issue