Top

Published in:

01-06-2013 | Original Paper

Intervening with Practitioners to Improve the Quality of Prevention: One-Year Findings from a Randomized Trial of Assets-Getting To Outcomes

Authors: Matthew Chinman, Joie Acosta, Patricia Ebener, Q. Burkhart, Patrick S. Malone, Susan M. Paddock, Michael Clifford, Maryann Corsello, Tim Duffey, Sarah Hunter, Margaret Jones, Michel Lahti, Andrea Phillips, Susan Savell, Peter C. Scales, Nancy Tellett-Royce

Published in: Journal of Prevention | Issue 3/2013

Abstract

There continues to be a gap in prevention outcomes achieved in research trials versus those achieved in “real-world” practice. This article reports interim findings from a randomized controlled trial evaluating Assets-Getting To Outcomes (AGTO), a two-year intervention designed to build prevention practitioners’ capacity to implement positive youth development–oriented practices in 12 community coalitions in Maine. A survey of coalition members was used to assess change on individual practitioners’ prevention capacity between baseline and one year later. Structured interviews with 32 program directors (16 in the intervention group and 16 in the control group) were used to assess changes in programs’ prevention practices during the same time period. Change in prevention capacity over time did not differ significantly between the intervention and control groups. However, in secondary analyses of only those assigned to the AGTO intervention, users showed greater improvement in their self-efficacy to conduct Assets–based programming and increases in the frequency with which they engaged in AGTO behaviors, whereas among non-users, self-efficacy to conduct Assets–based programming declined. Interview ratings showed improvement in several key areas of performance among intervention programs. Improvement was associated with the number of technical assistance hours received. These results suggest that, after one year, AGTO is beginning to improve the capacity of community practitioners who make use of it.

“Getting To Outcomes” and “GTO” are trademarks registered by the University of South Carolina and the RAND Corporation.

“Developmental Assets” is a trademark registered by the Search Institute.

Of the 60 programs initially randomized, nine folded between the time the study was planned and begun, leaving only 51 programs.

Acosta, J., & Chinman, M. (2011). Building capacity for outcomes measurement in human service organizations: The “Getting To Outcomes” (GTO) method. In J. L. Magnabosco & R. W. Manderscheid (Eds.), Outcomes measurement in the human services: Cross-cutting issues and methods (2nd ed.). Washington, DC: NASW Press.

Ajzen, I., & Fishbein, M. (1977). Attitude-behavior relations: A theoretical analysis and review of empirical research. Psychological Bulletin, 84, 888–918.CrossRef

Backer, T. E. (2001). Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state-of-the-art review. Rockville, MD: Substance Abuse and Mental Health Administration.

Bandura, A. (2004). Health promotion by social cognitive means. Health Education & Behavior, 31, 143–164.CrossRef

Battelle. (2008). Annual findings report: Drug-free communities support program national evaluation. Washington, DC: Executive Office of the President Office of Administration Office of National Drug Control Policy.

Benson, P. L. (2002). Toward asset-building communities: How does change occur? In R. M. Lerner & P. L. Benson (Eds.), Developmental assets and asset-building communities: Implications for research policy and practice. Norwell, MA: Kluwer Academic.

Benson, P. L., & Scales, P. C. (2011). Developmental assets. In R. J. R. Levesque (Ed.), Encyclopedia of adolescence. New York, NY: Springer.

Benson, P. L., Scales, P. C., Leffert, N., & Roehlkepartain, E. R. (1999). A fragile foundation: The state of developmental assets among American youth. Minneapolis, MN: Search Institute.

Benson, P. L., Scales, P. C., & Mannes, M. (2003). Developmental strengths and their sources: Implications for the study and practice of community building. In R. M. Lerner, F. Jacobs, & D. Wertlieb (Eds.), Handbook of applied developmental science. Applying developmental science for youth and families (Vol. 1, pp. 369–406). Thousand Oaks, CA: Sage.

Benson, P. L., Scales, P. C., Hamilton, S. F., & Sesma, A. (2006). Positive youth development: Theory, research, and applications. In W. Damon, & R. M. Lerner (Eds.), Handbook of child psychology (6th ed., pp. 894–941). New York, NY: Wiley.

Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273, 1106–1112.PubMedCrossRef

Bryt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. Journal of Clinical Epidemiology, 46, 423–429.CrossRef

Butterfoss, F. D., Goodman, R. M., & Wandersman, A. (1993). Community coalitions for prevention and health promotion. Health Education Research, 8, 315–330.PubMedCrossRef

Cantor, D., Crosse, S., Hagen, C. A., Mason, M. J., Siler, A. J., & von Glatz, A. (2001). A closer look at drug and violence prevention efforts in American schools: Report on the study on school violence and prevention. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.

Chinman, M., Imm, P., & Wandersman, A. (2004). Getting To Outcomes 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation (No. TR-TR101). Retrieved June 1, 2011, from http://www.rand.org/publications/TR/TR101/.

Chinman, M., Hannah, G., Wandersman, A., Ebener, P., Hunter, S., Imm, P., et al. (2005). Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology, 3, 143–157.CrossRef

Chinman, M., Hunter, S., Ebener, P., Paddock, S., Stillman, L., Imm, P., et al. (2008). The Getting To Outcomes demonstration and evaluation: An illustration of the Prevention Support System. American Journal of Community Psychology, 41, 206–224.PubMedCrossRef

Chinman, M., Tremain, B., Imm, P., & Wandersman, A. (2009). Strengthening prevention performance using technology: A formative evaluation of interactive Getting To Outcomes. The American Journal of Orthopsychiatry, 79, 469–481.PubMedCrossRef

Chinman, M., Acosta, J., Ebener, P., Burkhart, Q., Clifford, M., Corsello, M., et al. (2012). Establishing and evaluating the key functions of an interactive systems framework using an assets-getting to outcomes intervention. American Journal of Community Psychology, 50, 295–310.PubMedCrossRef

Cividin, T. M., & Ottoson, J. M. (1997). Linking reasons for continuing professional education participation with post-program application. Journal of Continuing Education in the Health Professions, 17, 46–55.CrossRef

Collins, C., Harshbarger, C., Sawyer, R., & Hamdallah, M. (2006). The Diffusion of Effective Behavioral Interventions project: Development, implementation, and lessons learned. AIDS Education and Prevention, 18(4 Suppl. A), 5–20.

Crosse, S., Burr, M., Cantor, D., Hagen, C. A., & Hantman, I. (2001). Wide scope, questionable quality: Drug and violence prevention efforts in American schools. Report on the study on school violence and prevention. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.

Crosse, S., Williams, B., Hagen, C. A., Harmon, M., Ristow, L., DiGaetano, R., et al. (2011). Prevalence and implementation fidelity of research-based prevention programs in public schools: Final report. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.

Donner, A., & Klar, N. (2004). Pitfalls of and controversies in cluster randomization trials. American Journal of Public Health, 94, 416–422.PubMedCrossRef

Ellickson, P. L., & Bell, R. M. (1990). Drug prevention in junior high: A multi-site longitudinal test. Science, 247, 1299–1305.PubMedCrossRef

Ennett, S. T., Ringwalt, C. L., Thorne, J., Rohrbach, L. A., Vincus, A., Simons-Rudolph, A., et al. (2003). A comparison of current practice in school-based substance abuse prevention programs with meta-analysis findings. Prevention Science, 4, 1–14.PubMedCrossRef

Feinberg, M. E., Ridenour, T. A., & Greenberg, M. T. (2008). The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania. The Journal of Primary Prevention, 29(2), 145–165.PubMedCrossRef

Fetterman, D. M., & Wandersman, A. (2005). Empowerment evaluation principles in practice. New York: The Guilford Press.

Fishbein, M., & Ajzen, I. (1974). Attitudes toward objects as predictive of single and multiple behavioral criteria. Psychological Review, 81, 59–74.CrossRef

Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

Fisher, D., Imm, P. S., Chinman, M., & Wandersman, A. (2006). Getting To Outcomes with developmental assets: Ten steps to measuring success in youth programs and communities. Minneapolis, MN: Search Institute.

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Flaspohler, P., Duffy, J., Wandersman, A., Stillman, L., & Maras, M. A. (2008). Unpacking prevention capacity: An intersection of research-to-practice models and community-centered models. American Journal of Community Psychology, 41, 182–196.PubMedCrossRef

Green, L. (2001). From research to “Best Practices” in other settings and populations. American Journal of Health Behavior, 25, 165–178.PubMedCrossRef

Hall, G. E., & Hord, S. M. (2001). Implementing change. Boston, MA: Allyn and Bacon.

Hall, G. E., & Loucks, S. F. (1977). A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 14, 263–276.CrossRef

Hall, G. E., Loucks, S. F., Rutherford, W. L., & Newlove, B. W. (1975). Levels of use of the innovation: A framework for analyzing innovation adoption. Journal of Teacher Education, 26, 52–56.CrossRef

Hallfors, D., & Godette, D. (2002). Will the ‘principles of effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Education Research, 17, 461–470.PubMedCrossRef

Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbott, R. D., Fagan, A. A., et al. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities that Care. Archives of Pediatric and Adolescent Medicine, 163, 789–798.CrossRef

Hunter, S., Paddock, S., Ebener, P., Burkhart, Q., & Chinman, M. (2009). Promoting evidence based practices: The adoption of a prevention support system in community settings. The Journal of Community Psychology, 37, 579–593.CrossRef

Jemmott, J. B., Jemmott, L. S., Fong, G. T., & McCaffree, K. (1999). Reducing HIV risk-associated sexual behavior among African American adolescents: Testing the generality of intervention effects. American Journal of Community Psychology, 27, 161–187.PubMedCrossRef

Jemmott, J. B., Jemmott, L. S., Spears, H., Hewitt, N., & Cruz-Collins, M. (1992a). Self-efficacy, hedonistic expectancies, and condom-use intentions among inner-city black adolescent women: A social cognitive approach to AIDS risk behavior. Journal of Adolescent Health, 13, 512–519.PubMedCrossRef

Jemmott, J. B., Jemmott, L. S., Spears, H., Hewitt, N., & Cruz-Collins, M. (1992b). Increasing condom-use intentions among sexually active inner-city black adolescent women: Effects of an AIDS prevention program. Nursing Research, 41, 273–279.PubMed

Keener, D. C. (2007). Towards a science of capacity building: An examination of technical assistance following a training of prevention professionals. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (Accession Order No. AAT 3272459).

Kitson, A., Harvey, G., & McCormack, B. (1998). Enabling the implementation of evidence based practice: A conceptual framework. Quality in Health Care, 7, 149–158.PubMedCrossRef

Leffert, N., Benson, P. L., Scales, P. C., Sharma, A. R., Drake, D. R., & Blyth, D. A. (1998). Developmental assets: Measurement and prediction of risk behaviors among adolescents. Applied Developmental Science, 2, 209–230.CrossRef

Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197–209.PubMedCrossRef

Livet, M., & Wandersman, A. (2005). Organizational functioning: Facilitating effective interventions and increasing the odds of programming success. In D. M. Fetterman & A. Wandersman (Eds.), Empowerment evaluation in practice (pp. 123–154). New York, NY: Guilford.

Maine Office of Substance Abuse. (2006). Maine youth drug and alcohol survey local summary reports. Retrieved March 25, 2009 from http://www.maine.gov/maineosa/survey/home.php.

Mihalic, S. F., & Irwin, K. (2003). Blueprints for violence prevention: From research to real-world settings—Factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice, 1, 1–23.CrossRef

Miller, T., & Hendrie, D. (2009). Substance abuse prevention dollars and cents: A cost-benefit analysis (DHHS Pub. No. (SMA) 07-4298). Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration.

Mitchell, R. E., Stone-Wiggins, B., Stevenson, J. F., & Florin, P. (2004). Cultivating capacity: Outcomes of a statewide support system for prevention coalitions. Journal of Prevention & Intervention in the Community, 27, 67–87.CrossRef

Murray, D. M. (1998). Design and analysis of group-randomized trials. Oxford, UK: Oxford University Press.

O’Donnell, L. O., Scattergood, P., Adler, M., San Doval, A., Barker, M., et al. (2000). The role of technical assistance in the replication of effective HIV interventions. AIDS Education and Prevention, 12(Supl A), 99–111.

Ottoson, J. M. (1997). After the applause: Exploring multiple influences on application following an adult education program. Adult Education Quarterly, 47, 92–107.CrossRef

Philliber, S., & Nolte, K. (2008). Implementation science: Promoting science-based approaches to prevent teen pregnancy. Prevention Science, 9, 166–177.PubMedCrossRef

Reuter, P., & Timpane, J. M. (2001). Assessing options for the safe and drug free schools and communities act (No. MR-1328). Santa Monica, CA: RAND Corporation.

Ringwalt, C. L., Ennett, S., Vincus, A., Thorne, J., Rohrbach, L. A., & Simons-Rudolph, A. (2002). The prevalence of effective substance use prevention curricula in US middle schools. Prevention Science, 3, 257–265.PubMedCrossRef

Robertson, E. B., David, S. L., & Rao, S. A. (2003). Preventing drug use among children and adolescents: A research-based guide for parents, educators, and community leaders (2nd ed.) (NIH Publication No. 04-4212[A]). Bethesda, MD: National Institute on Drug Abuse.

Rycroft-Malone, J., Harvey, G., Seers, K., Kitson, A., McCormack, B., & Titchen, A. (2002). An exploration of the factors that influence the implementation of evidence into practice. Journal of Clinical Nursing, 13, 913–924.CrossRef

Scales, P. C. (1999). Reducing risks and building developmental assets: Essential actions for promoting adolescent health. Journal of School Health, 69, 113–119.PubMedCrossRef

Scales, P. C., Benson, P. L., Leffert, N., & Blyth, D. A. (2000). Contribution of Developmental Assets to the prediction of thriving among adolescents. Applied Developmental Science, 4, 27–46.CrossRef

Silvia, E. S., & Thorne, J. (1997). School-based drug prevention programs: A longitudinal study in selected school districts. Research Triangle Park, NC: Research Triangle Institute.

Spoth, R., Redmond, C., Shin, C., Greenberg, M., Clair, S., & Feinberg, M. (2007). Substance-use outcomes at 18 months past baseline: The PROSPER Community-University partnership trial. American Journal of Preventive Medicine, 32, 395–402.PubMedCrossRef

Stetler, C. B., Legro, M. W., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., et al. (2006). Role of “external facilitation” in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science, 1, 1–15.CrossRef

Stevenson, J. F., Florin, P., Mills, D. S., & Andrade, M. (2002). Building evaluation capacity in human service organizations: A case study. Evaluation and Program Planning, 25, 233–243.CrossRef

U.S. Census Bureau. (2000). State and county Quickfacts: Maine. Retrieved March 25, 2009, from http://quickfacts.census.gov.

Wandersman, A. (2009). Four keys to success (theory, implementation, evaluation, and resource/system support): High hopes and challenges in participation. American Journal of Community Psychology, 43, 3–21.PubMedCrossRef

Wandersman, A., & Florin, P. (2003). Community interventions and effective prevention: Bringing researchers/evaluators, funders and practitioners together for accountability. American Psychologist, 58, 441–448.PubMedCrossRef

Wandersman, A., Chien, V. H., & Katz, J. (2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology, 50, 445–449.PubMedCrossRef

Weiss, E., Miller-Anderson, R., & Lasker, R. (2002). Making the most of collaboration: Exploring the relationship between partnership synergy and partnership functioning. Health Education and Behavior, 29, 683–698.PubMedCrossRef

Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of practice. Boston, MA: Harvard Business School Press.

Yin, R. K., Kaftarian, S. J., Yu, P., & Jansen, M. A. (1997). Outcomes from CSAP’s Community Partnership Program: Findings from the National Cross Site Evaluation. Evaluation and Program Planning, 20, 345–355.CrossRef

Title: Intervening with Practitioners to Improve the Quality of Prevention: One-Year Findings from a Randomized Trial of Assets-Getting To Outcomes
Authors: Matthew Chinman
Joie Acosta
Patricia Ebener
Q. Burkhart
Patrick S. Malone
Susan M. Paddock
Michael Clifford
Maryann Corsello
Tim Duffey
Sarah Hunter
Margaret Jones
Michel Lahti
Andrea Phillips
Susan Savell
Peter C. Scales
Nancy Tellett-Royce
Publication date: 01-06-2013
Publisher: Springer US
Published in: Journal of Prevention / Issue 3/2013
Print ISSN: 2731-5533
Electronic ISSN: 2731-5541
DOI: https://doi.org/10.1007/s10935-013-0302-7

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Intervening with Practitioners to Improve the Quality of Prevention: One-Year Findings from a Randomized Trial of Assets-Getting To Outcomes

Abstract

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 3/2013

Genesee County REACH Windshield Tours: Enhancing Health Professionals Understanding of Community Conditions that Influence Infant Mortality

Examining Adaptations of Evidence-Based Programs in Natural Contexts

The Tug-of-War: Fidelity Versus Adaptation Throughout the Health Promotion Program Life Cycle

Entre-Parents: Initial Outcome Evaluation of a Preventive-Parenting Program for French-Speaking Parents

Psychoecological Model of Alcohol Use in Mexican American Adolescents

Letter to Prospective Authors from the Editor