Top

Published in:

Open Access 01-12-2018 | Study protocol

Developing an evidence-based program sustainability training curriculum: a group randomized, multi-phase approach

Published in: Implementation Science | Issue 1/2018

Abstract

Background

The emergence of dissemination and implementation (D&I) science has driven a rapid increase in studies of how new scientific discoveries are translated and developed into evidence-based programs and policies. However, D&I science has paid much less attention to what happens to programs once they have been implemented. Public health programs can only deliver benefits if they reach maturity and sustain activities over time. In order to achieve the full benefits of significant investment in public health research and program development, there must be an understanding of the factors that relate to sustainability to inform development of tools and trainings to support strategic long-term program sustainability. Tobacco control programs, specifically, vary in their abilities to support and sustain themselves over time. As of 2018, most states still do not meet the CDC-recommended level for funding their TC program, allowing tobacco use to remain the leading cause of preventable disease and death in the USA. The purpose of this study is to empirically develop, test, and disseminate training programs to improve the sustainability of evidence-based state tobacco control programs and thus, tobacco-related health outcomes.

Methods

This paper describes the methods of a group randomized, multi-phase study that evaluates the empirically developed “Program Sustainability Action Planning Training” and technical assistance in US state-level tobacco control programs. Phase 1 includes developing the sustainability action planning training curriculum and technical assistance protocol and developing measures to assess long-term program sustainability. Phase 2 includes a group randomized trial to test the effectiveness of the training and technical assistance in improving sustainability outcomes in 24 state tobacco control programs (12 intervention, 12 comparison). Phase 3 includes the active dissemination of final training curricula materials to a broader public health audience.

Discussion

Empirical evidence has established that program sustainability can improve through training and technical assistance; however, to our knowledge, no evidence-based sustainability training curriculum program exists. Therefore, systematic methods are needed to develop, test, and disseminate a training that improves the sustainability of evidence-based programs.

Trial registration

NCT03598114. Registered 25 July 2018—retrospectively registered.

Schell S, Luke D, Schooley M, Elliott M, Mueller N, Bunger A. Public health program capacity for sustainability: a new framework. Implement Sci. 2013;8(15). https://doi.org/10.1186/1748-5908-8-15.

Gruen R, Elliott J, Nolan M, Lawton P, Parkhill A, McLaren C, et al. Sustainability science: an integrated approach for health-programme planning. Lancet. 2008;372:1579–89. https://doi.org/10.1016/S0140-6736(08)61659-1.CrossRefPubMed

Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26:320–47. https://doi.org/10.1016/j.amepre.2012.06.010.CrossRef

Hanson H, Salmoni A, Volpe R. Defining program sustainability: differing view of stakeholders. Can J Public Heal. 2009;100(3):304–9.

Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7(17). https://doi.org/10.1186/1748-5908-7-17.

Wright C, Catty J, Watt H, Burns T. A systematic review of home treatment services: classification and sustainability. Soc Psychiatry Psychiatr Epidemiol. 2004;39(10):789–96.CrossRef

Masse L, McKay H, Valente M, Brant R, Naylor P. Physical activity implementation in schools: a 4-year follow-up. Am J Prev Meed. 2012;43(4):369–77. https://doi.org/10.1016/j.amepre.2012.06.010.CrossRef

Scheirer M, Dearing J. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67. https://doi.org/10.2105/AJPH.2011.300193.CrossRefPubMedPubMedCentral

Savaya R, Spiro S, Elran-Barak R. Sustainability of social programs: a comparative case study analysis. Am J Eval. 2008;29(4):478–93. https://doi.org/10.1177/1098214008325126.CrossRef

10.

Akerlund KM. Prevention program sustainability: the state’s perspective. J Community Psychol. 2000;28(3):353–62.CrossRef

11.

Department of Health and Human Services (US). The health consequences of smoking – 50 years of progress: a report of the Surgeon General. Rockville; 2014.

12.

Farrelly M, Pechacek T, Thomas K, Nelson D. The impact of tobacco control programs on adult smoking. Am J Public Health. 2008;98(2):304–9. https://doi.org/10.2105/AJPH.2006.106377.CrossRefPubMedPubMedCentral

13.

Johnson K, Collins D, Wandersman A. Sustaining innovations in community prevention systems: a data-informed sustainability strategy. J Community Psychol. 2013;41(3):322–40.CrossRef

14.

Calhoun A, Mainor A, Moreland-Russell S, Maier R, Brossart L, Luke D. Using the Program Sustainability Assessment Tool to assess and plan for sustainability. Prev Chronic Dis. 2014;11:131–85. https://doi.org/10.5888/pcd11.130185.CrossRef

15.

Feinberg M, Ridenour T, Greenberg M. The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania. J Prim Prev. 2008;29(2):145–65.CrossRef

16.

Perkins D, Feinberg M, Greenberg M, Johnson L, Chilenski S, Micemoyer C, et al. Team factors that predict to sustainability indicators for community-based prevention teams. Eval Program Plann. 2011;34(3). https://doi.org/10.1016/j.evalprogplan.2010.10.003.CrossRef

17.

Feinberg M, Bontempo D, Greenberg M. Predictors and level of sustainability of community prevention coalitions. Am J Prev Med. 2008;34(6):495–501. https://doi.org/10.1016/j.ampere.2008.01.030.CrossRefPubMed

18.

Sheirer M. Linking sustainability research to intervention types. Am J Public Health. 2013;103:e1–8. https://doi.org/10.2105/AJPH.2012.300976.CrossRef

19.

Weiss C. Nothing as practice as good theory: exploring theory-based evaluation for comprehensive community initiatives for children and families. In: Connell J, Kubisch A, Schorr L, Weiss C, editors. New approaches to evaluating community initiatives: volume 1, concepts, methods, and contexts. Washington, D.C.: The Aspen Institute; 1995.

20.

Huey-Tysh C. Theory driven evaluation. Newbury Park: Sage Publications Ltd.; 1990.

21.

Johnson K, Hays C, Center H, Daley C. Building capacity and sustainable prevention innovations: a sustainability planning model. Eval Program Plann. 2004;27(2):135–49.CrossRef

22.

Kolb A, Kolb D. The learning way: meta-cognitive aspects of experiential learning. Simul Gaming. 2009;40(3):297–327.CrossRef

23.

de Vries H, Eggers S, Bolman C. The role of action planning and plan enactment for smoking cessation. BMC Public Health. 2013;13(393). https://doi.org/10.1186/1471-2458-13-393.

24.

van Osch L, Lechner L, Reubsaet A, de Vries H. From theory to practice: an explorative study into instrumentality and specificity of implementation intentions. Psychol Health. 2010;25(3):351–64.CrossRef

25.

de Vet E, Oenema A, Brug J. More or better: do the number and specificity of implementation intentions matter in increasing physical activity? Psychol Sport Exerc. 2011;12(4):471–7.CrossRef

26.

Verbiest M, Presseau J, Chavannes N, et al. Use of action planning to increase provision of smoking cessation care by general practitioners: role of plan specificity and enactment. Implent Sci. 2014;9(180). https://doi.org/10.1186/s13012-014-0180-2.

27.

Sheeran P, Milne S, Webb T, Gollwitzer P. Implementation intentions and health behavior. In: Connor M, Norman P, editors. Predicting health behavior research and practice with social cognition models. 2nd ed. Berkshire: Open University Press, McGraw-Hill Education; 2005. p. 276–323.

28.

Kolb D. Experiential learning: experience as the source of learning and development. 1st ed. Englewood Cliffs: Prentice Hall, Inc.; 1984. https://doi.org/10.1002/job.4030080408.CrossRef

29.

Abdulwahed M, Nagy Z. Applying Kolb’s experiential learning cycle for laboratory education. J Eng Educ. 2009;98:283–94 Available from: https://dspace.lboro.ac.uk/dspace-jspui/handle/2134/5412.CrossRef

30.

Rushovich B, Bartley L, Steward R, Bright C. Technical assistance: a comparison between providers and recipients. Hum Serv Organ Manag Leadersh Gov. 2015;39(4):362–79. https://doi.org/10.1080/23303131.2015.1052124.CrossRef

31.

American Lung Association. State of tobacco control 2017; 2017. Available from: http://www.lung.org/assets/documents/tobacco/state-of-tobacco-control.pdf. Accessed 21 Feb 2018.

32.

Segal L, Matin A. A funding crisis for public health and safety: state-by-state public health funding and key health facts. Trust for America’s Health. 2017:2017 Available from: https://www.tfah.org/report-details/a-funding-crisis-for-public-health-and-safety-state-by-state-public-health-funding-and-key-health-facts-2017/. Accessed 21 Feb 2018.

33.

Savaya R, Spiro S. Predictors of sustainability of social programs. Am J Eval. 2012;33(1):26–43. https://doi.org/10.1177/1098214011408066.CrossRef

34.

Centers for Disease Control and Prevention. National state-based tobacco control programs. CDC-RFA-DP15–1509. Available from: https://apply07.grants.gov/apply/opportunities/instructions/oppCDC-RFA-DP15-1509-cfda93.305-cidNCCDPHP-NR-instructions.pdf. Accessed 21 Feb 2018.

35.

Miller T, Levy D. Cost-outcome analysis in injury prevention and control: eighty-four recent estimates for the United States. Med Care. 2000;38(6):562–82.CrossRef

36.

Luke D, Calhoun A, Robichaux C, Elliott M, Moreland-Russell S. The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:130184. https://doi.org/10.5888/pcd11.130184.CrossRefPubMed

37.

Aarons G, Miller E, Green A, Perrott J, Bradway R. Adaptation happens: a qualitative case study of implementation of The Incredible Years evidence-based parent training programme in a residential substance abuse treatment programme. J Child Serv. 2012;7(4):233–45. https://doi.org/10.1108/17466661211286463.CrossRef

38.

Coffey J, Horner R. The sustainability of school wide positive behavior interventions and supports. Career Dev Transit Except Individ. 2012;78(4):407–22.

39.

Cooper B, Bumbarger B, Moore J. Sustaining evidence-based prevention programs: correlates in a large-scale dissemination initiative. Prev Sci. 2015;16:145–57. https://doi.org/10.1007/s11121-013-0427-1.CrossRefPubMed

40.

O’Loughlin J, Renaud L, Richard L, Gomez L, Paradis G. Correlates of the sustainability of community-based heart health promotion intervention. Prev Med. 1998;27:702–12.CrossRef

41.

Patrizi P, Gross E, Freedman S. Strength in flexibility: lessons from a cluster of capacity building grants in the juvenile justice field. Eval Program Plann. 2006:162–70.CrossRef

42.

Pluye P, Potvin L, Denis JL. Making public health programs last: conceptualizing sustainability. Eval Program Plann. 2004:121–33.

43.

Savaya R, Elsworth G, Rogers P. Projected sustainability of innovative social programs. Eval Rev. 2009;33(2):189–205. https://doi.org/10.1177/0193841X08322860.CrossRefPubMed

44.

Savaya R, Spiro S, Elran-Barak R. Sustainability of social programs: variables that differentiate between programs that survive and those that do not. Am J Eval. 2008;29:478–93.CrossRef

45.

Spoth R, Guyll M, Redmond C, Greenberg M, Feinberg M. Six-year sustainability of evidence-based intervention implementation quality by community-university partnerships: the PROSPER study. Am J Community Psychol. 2011;48:412–25. https://doi.org/10.1007/s10464-011-9430-5.CrossRefPubMedPubMedCentral

46.

Stevens B, Peikes D. When the funding stops: do grantees of the Local Initiative Fund Partners Program sustain themselves? Eval Program Plann. 2006;29:153–61.CrossRef

47.

Centers for Disease Control and Prevention. Best practices for comprehensive tobacco control programs – 2014. Atlanta; 2014.

48.

Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New Jersey: Lawrence Erlbalm Associates; 1988.

49.

Del Re A. Compute.es: compute effect sizes. 2014. Available from: http://cran.r-project.org/web/packages/compute.es/. Accessed 9 Apr 2018.

50.

Champely S. Pwr: basic functions for power analysis. 2015. Available from: http://cran.r-project.org/package=pwr. Accessed 9 Apr 2018.

51.

Team R. R: a language and environment for statistical computing. 2014. Available from: http://www.r-project.org. Accessed 9 Apr 2018.

Title: Developing an evidence-based program sustainability training curriculum: a group randomized, multi-phase approach
Publication date: 01-12-2018
Published in: Implementation Science / Issue 1/2018
Electronic ISSN: 1748-5908
DOI: https://doi.org/10.1186/s13012-018-0819-5

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Developing an evidence-based program sustainability training curriculum: a group randomized, multi-phase approach

Abstract

Background

Methods

Discussion

Trial registration

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Discussion

Trial registration

Please log in to get access to this content

Other articles of this Issue 1/2018

Implementation of a knowledge mobilization model to prevent peripheral venous catheter-related adverse events: PREBACP study—a multicenter cluster-randomized trial protocol

Mental models of audit and feedback in primary care settings

Standardizing an approach to the evaluation of implementation science proposals

Barriers and facilitators to implementing evidence-based interventions among third sector organisations: a systematic review

Designing and implementing two facilitation interventions within the ‘Facilitating Implementation of Research Evidence (FIRE)’ study: a qualitative analysis from an external facilitators’ perspective

A new scale for the evaluation of clinical practice guidelines applicability: development and appraisal