Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
The Journal of Behavioral Health Services & Research
Published in: The Journal of Behavioral Health Services & Research 4/2012

01-10-2012

Examining the Association Between Implementation and Outcomes

State-wide Scale-up of School-wide Positive Behavior Intervention and Supports

Authors: Elise T. Pas, PhD, Catherine P. Bradshaw, PhD

Published in: The Journal of Behavioral Health Services & Research | Issue 4/2012

Login to get access

Abstract

Although there is an established literature supporting the efficacy of a variety of prevention programs, there has been less empirical work on the translation of such research to everyday practice or when scaled-up state-wide. There is a considerable need for more research on factors that enhance implementation of programs and optimize outcomes, particularly in school settings. The current paper examines how the implementation fidelity of an increasingly popular and widely disseminated prevention model called, School-wide Positive Behavioral Interventions and Supports (SW-PBIS), relates to student outcomes within the context of a state-wide scale-up effort. Data come from a scale-up effort of SW-PBIS in Maryland; the sample included 421 elementary and middle schools trained in SW-PBIS. SW-PBIS fidelity, as measured by one of three fidelity measures, was found to be associated with higher math achievement, higher reading achievement, and lower truancy. School contextual factors were related to implementation levels and outcomes. Implications for scale-up efforts of behavioral and mental health interventions and measurement considerations are discussed.
Footnotes
1
This scale measures pre-implementation readiness and therefore the variability and the internal consistency on this scale are low.
 
Literature
1.
Flay BR, Biglan A, Boruch RF, et al. Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science. 2005;6(3):151–175.PubMedCrossRef
2.
Dusenbury L, Brannigan R, Falco M, et al. A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research. 2003;18(2):237–256.PubMedCrossRef
3.
Ringwalt CL, Ennett S, Johnson R, et al. Factors associated with fidelity to substance use prevention curriculum guides in the nation's middle schools. Health Education & Behavior. 2003;30(3):375–391.CrossRef
4.
5.
Woolf SH. The meaning of translational research and why it matters. Journal of the American Medical Association. 2008;299:211–213.PubMedCrossRef
6.
Woolf SH. Potential health and economic consequences of misplaced priorities. Journal of the American Medical Association. 2007;297:523–526.PubMedCrossRef
7.
Spoth R. Translating family-focused prevention science into effective practice: Toward a translational impact paradigm. Current Directions in Psychological Science. 2008;17(6):415–421.PubMedCrossRef
8.
Ringwalt C, Vincus AA, Hanley S, et al. The prevalence of evidence-based drug use prevention curricula in U.S. middle schools in 2008. Prevention Science. 2011;12(1):63–69.PubMedCrossRef
9.
Sloboda Z, Pyakuryal A, Stephens PC, et al. Reports of substance abuse prevention programming available in schools. Prevention Science. 2008;9(4):276–287.PubMedCrossRef
10.
Gottfredson GD, Gottfredson DC, Czeh ER, et al. National study of delinquency prevention in schools. Ellicott City, MD: Gottfredson Associates; 2000.CrossRef
11.
Gottfredson GD, Jones EM, Gore TW. Implementation and evaluation of a cognitive–behavioral intervention to prevent problem behavior in a disorganized school. Prevention Science. 2002;3(1):43–56.PubMedCrossRef
12.
Domitrovich CE, Greenberg MT. The study of implementation: Current findings from effective programs that prevent mental disorders in school-aged children. Journal of Educational & Psychological Consultation. 2000;11(2):193–221.CrossRef
13.
Durlak JA. Successful prevention programs for children and adolescents. New York: Plenum; 1997.
14.
Sugai G, Horner RH, Gresham FM. Behaviorally effective school environments. In: Shinn MR, Walker HM, Stoner G, eds. Interventions for academic and behavior problems: II. Preventive and remedial approaches. Bethesda, MD US: National Association of School Psychologists; 2002:315–350.
15.
Domitrovich CE, Bradshaw CP, Poduska JM, et al. Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion. 2008;1(3):6–28.CrossRef
16.
Schoenwald SK, Hoagwood K. Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services. 2001;52(9):1190–1197.PubMedCrossRef
17.
Stuart E, Cole S, Bradshaw CP, et al. The use of propensity scores to assess the generalizability of results from randomized trials. The Journal of the Royal Statistical Society, Series A. 2011; 174(2):369–386.CrossRef
18.
O’Connell ME, Boat T, Warner KE. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: The National Academies Press. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults: Research Advances and Promising Interventions; 2009.
19.
Bradshaw CP, Koth CW, Bevans KB, et al. The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly. 2008;23(4):462–473.CrossRef
20.
Bradshaw CP, Koth CW, Thornton LA, et al. Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science. 2009;10(2):100–115.PubMedCrossRef
21.
Bradshaw CP, Mitchell MM, Leaf PJ. Examining the effects of schoolwide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions. 2010;12(3):133–148.CrossRef
22.
Horner RH, Sugai G, Smolkowski K, et al. A randomized, wait-list controlled effectiveness trial assessing School-Wide Positive Behavior Support in elementary schools. Journal of Positive Behavior Interventions. 2009;11(3):133–144.CrossRef
23.
Bradshaw CP, Waasdorp TE, Leaf PJ. Effects of School-Wide Positive Behavioral Interventions and Supports on child behavior problems and adjustment. Manuscript submitted for publication. 2012.
24.
Waasdorp TE, Bradshaw CP, Leaf PJ. The impact of School-wide Positive Behavioral Interventions and Supports (SWPBIS) on bullying and peer rejection: A randomized controlled effectiveness trial. Archives of Child and Adolescent Medicine. 2012;116(2):149–156.CrossRef
25.
Barrett SB, Bradshaw CP, Lewis-Palmer T. Maryland statewide PBIS initiative: Systems, evaluation, and next steps. Journal of Positive Behavior Interventions. 2008;10(2):105–114.CrossRef
26.
Muscott HS, Mann EL, LeBrun MR. Positive behavioral interventions and supports in New Hampshire. Effects of large-scale implementation of schoolwide positive behavior support on student discipline and academic achievement. Journal of Positive Behavior Interventions. 2008;10(3):190–205.CrossRef
27.
Bradshaw CP, Pas ET. A state-wide scale-up of Positive Behavioral Interventions and Supports (PBIS): A description of the development of systems of support and analysis of adoption and implementation. School Psychology Review. 2011;40(4):530–548.
28.
Adelman HS, Taylor L. Toward a scale-up model for replicating new approaches to schooling. Journal of Educational & Psychological Consultation. 1997;8(2):197.CrossRef
29.
Kreuter MW, Bernhardt JM. Reframing the dissemination challenge: a marketing and distribution perspective. American Journal of Public Health. 2009;99(12):2123–2127.PubMedCrossRef
30.
Rohrbach LA, Grana R, Sussman S, et al. Type II Translation: Transporting prevention interventions from research to real-world settings. Evaluation & the Health Professions. 2006;29(3):302–333.CrossRef
31.
Bloomquist ML, August GJ, Horowitz JL, et al. Moving from science to service: Transposing and sustaining the early risers prevention program in a community service system. The Journal of Primary Prevention. 2008;29(4):307–321.PubMedCrossRef
32.
Fixsen DL, Naoom SF, Blase KA, et al. Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network; 2005.
33.
Spoth RL, Greenberg, MT. Toward a comprehensive strategy for effective practitioner–scientist partnerships and larger-scale community health and well-being. American Journal of Community Psychology. 2005;35:107–126.PubMedCrossRef
34.
35.
Botvin GJ, Baker E, Dusenbury L, et al. Long-term follow-up results of a randomized drug abuse prevention trial in a White middle-class population. Journal of the American Medical Association.1995;273(14):1106–1112.PubMedCrossRef
36.
Derzon JH, Sale E, Springer JF, et al. Estimating intervention effectiveness: Synthetic projection of field evaluation results. The Journal of Primary Prevention. 2005;26(4):321–343.PubMedCrossRef
37.
Durlak JA, DuPre EP. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008;41(3–4):327–350.PubMedCrossRef
38.
Hulleman CS, Cordray, DS. Moving from the lab to the field: The role of fidelity and achieved relative intervention strength. Journal of Research on Educational Effectiveness. 2009;2(1):88–110.CrossRef
39.
Ialongo NS, Werthamer L, Kellam SG, et al. Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology. 1999;27(5):599–641.PubMedCrossRef
40.
Sanders MR. Triple P-Positive Parenting Program: Towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clinical Child & Family Psychology Review. 1999;2(2):71–90.CrossRef
41.
Nowak C, Heinrichs N. A comprehensive meta-analysis of Triple P-Positive Parenting Program using hierarchical linear modeling: Effectiveness and moderating variables. Clinical Child and Family Psychology Review. 2008;11(3):114–144.PubMedCrossRef
42.
Kam C-M, Greenberg MT, Walls CT. Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science. 2003;4(1):55–63.PubMedCrossRef
43.
Jowers KL, Bradshaw CP, Gately S. Taking school-based substance abuse prevention to scale: District-wide implementation of Keep A Clear Mind. Journal of Alcohol and Drug Education. 2007;51(3):73–91.
44.
Gottfredson GD, Gottfredson DC, Payne AA, et al. School climate predictors of school disorder: Results from a national study of delinquency prevention in schools. Journal of Research in Crime and Delinquency. 2005;42(4):412–444.CrossRef
45.
Glisson C, Green P. The effects of organizational culture and climate on the access to mental health care in Child welfare and Juvenile Justice systems. Administration and Policy in Mental Health and Mental Health Services Research. 2006;33(4):433–448.PubMedCrossRef
46.
Bradshaw CP, Debnam K, Koth CW, et al. Preliminary validation of the Implementation Phases Inventory for assessing fidelity of schoolwide positive behavior supports. Journal of Positive Behavior Interventions. 2009;11(3):145–160.CrossRef
47.
Sugai G, Lewis-Palmer T, Todd A, et al. School-wide Evaluation Tool (SET). Eugene, OR: Center for Positive Behavioral Supports, University of Oregon; 2001.
48.
Kincaid D, Childs K, George H. School-wide Benchmarks of Quality (Revised). Tampa, Florida: University of South Florida; 2010.
49.
Cohen R, Kincaid D, Childs KE. Measuring school-wide positive behavior support implementation: Development and validation of the benchmarks of quality. Journal of Positive Behavior Interventions. 2007;9(4):203–213.CrossRef
50.
Bond GR, Drake RE, McHugo GJ, et al. Strategies for improving fidelity in the National Evidence-Based Practices Project. Research on Social Work Practice. 2009;19(5):569–581.CrossRef
51.
Horner RH, Todd AW, Lewis-Palmer T, et al. The School-Wide Evaluation Tool (SET): A research instrument for assessing School-Wide Positive Behavior Support. Journal of Positive Behavior Interventions. 2004;6(1):3–12.CrossRef
52.
Bradshaw CP, Reinke WM, Brown LD, et al. Implementation of School-Wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children (West Virginia University Press). 2008:1–26.
53.
Vincent C, Spaulding S, Tobin TJ. A reexamination of the psychometric properties of the School-Wide Evaluation Tool (SET). Journal of Positive Behavior Interventions. 2010;12(3):161–179.CrossRef
54.
Muthén LK, & Muthén BO. Mplus user’s guide. Los Angeles, CA: Muthén and Muthén; 1997–2009.
55.
Bollen KA. Structural equations with latent variables. Oxford, England: John Wiley & Sons; 1989.
56.
Kline RB. Principles and practice of structural equation modeling. 2nd ed. New York, NY: Guilford; 2005.
57.
Yu CY, Muthén BO. Evaluation of model fit indices for latent variable models with categorical and continuous outcomes. Los Angeles, CA; 2001.
58.
McDonald RP, Ho MR. Principles and practice in reporting structural equation analyses. Psychological Methods. 2002;7(1):64–82.PubMedCrossRef
59.
Wilson SJ, Lipsey MW. School-Based Interventions for Aggressive and Disruptive Behavior: Update of a Meta-Analysis. American Journal of Preventive Medicine. 2007;33(2):S130–S143.PubMedCrossRef
60.
Thomas R, Zimmer-Gembeck MJ. Behavioral outcomes of Parent–Child Interaction Therapy and Triple P-Positive Parenting Program: A review and meta-analysis. Journal of Abnormal Child Psychology. 2007;35(3):475–495.PubMedCrossRef
61.
Birnbaum AS, Lytle LA, Hannan PJ, et al. School functioning and violent behavior among young adolescents: A contextual analysis. Health Education Research. 2003;18(3):389–403.PubMedCrossRef
62.
Hoagwood K, Burns BJ, Kiser L, et al. Evidence-based practice in child and adolescent mental health services. Psychiatric Services. 2001;52(9):1179–1189.PubMedCrossRef
63.
Sandler I, Ostrom A, Bitner MJ, et al. Developing effective prevention services for the real world: A prevention science development model. American Journal of Community Psychology. 2005;35:127–142.PubMedCrossRef
Metadata
Title
Examining the Association Between Implementation and Outcomes
State-wide Scale-up of School-wide Positive Behavior Intervention and Supports
Authors
Elise T. Pas, PhD
Catherine P. Bradshaw, PhD
Publication date
01-10-2012
Publisher
Springer US
Published in
The Journal of Behavioral Health Services & Research / Issue 4/2012
Print ISSN: 1094-3412
Electronic ISSN: 2168-6793
DOI
https://doi.org/10.1007/s11414-012-9290-2

Other articles of this Issue 4/2012

The Journal of Behavioral Health Services & Research 4/2012 Go to the issue

OriginalPaper

The Assimilation of Evidence-Based Healthcare Innovations: A Management-Based Perspective

OriginalPaper

A Concept Mapping Approach to Guide and Understand Dissemination and Implementation

OriginalPaper

Tobacco Use and Substance Use Disorders as Predictors of Postoperative Weight Loss 2 Years After Bariatric Surgery

OriginalPaper

Quality of Communication Between Primary Health Care and Mental Health Care: An Examination of Referral and Discharge Letters

BriefCommunication

Addressing Challenges to Providing Peer-Based Recovery Support

OriginalPaper

Evidence-Based Practices in Community Mental Health: Outcome Evaluation