Top

Implementation Science

Published in:

Open Access 01-12-2015 | Study protocol

The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services

Authors: Anna S. Lau, Lauren Brookman-Frazee

Published in: Implementation Science | Issue 1/2015

Abstract

Background

Research to date has largely focused on predictors of adoption and initial implementation of evidence-based practices (EBPs), yet sustained implementation is crucial to deliver a return on investments in dissemination. Furthermore, most studies focus on single EBPs, limiting opportunities to study the fit between practice characteristics EBPs and implementation contexts.

Methods/design

This observational study will characterize implementation sustainment and identify organizational and therapist characteristics that predict sustainment of multiple practices being implemented within a fiscal mandate in the largest public mental health system in the USA. Specific aims are to (1) characterize sustainment outcomes (volume/penetration, EBP concordant care); (2) use mixed methods to characterize inner context (agency- and therapist-level) factors and early implementation conditions; and (3) identify inner context factors and early implementation conditions that predict sustainment outcomes. This study will undertake original data collection and analysis of existing data sources to achieve its aims. Archived reports and documents will be used to characterize early implementation conditions in 102 agencies. Administrative claims data will be used to characterize volume and penetration outcomes over 8 years. Therapist and program manager surveys will be administered to characterize sustained EBP concordant care and inner context determinants of sustainment. An in-depth study in a subset of agencies will yield interview data and recordings of treatment sessions for validation of the EBP concordant care scale.

Discussion

This project will yield new understanding of whether and how multiple EBPs can be sustained in public mental health systems undergoing a policy-driven community implementation effort. We will produce generalizable models for characterizing sustainment, including feasible and flexible measurement of practice across multiple EBPs. The findings will inform the development of implementation interventions to promote sustained delivery of EBPs to maximize their public health impact.

Available only for authorised users

Garland AF, Bickman L, Chorpita BF. Change what? Identifying quality improvement targets by investigating usual mental health care. Adm Policy Ment Health. 2010;37(1):15–26. doi:10.1007/s10488-010-0279-y.CrossRefPubMedPubMedCentral

Weisz JR, Jensen-Doss A, Hawley KM. Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. Am Psychol. 2006;61(7):671–89. doi:10.1037/0003-066X.61.7.671.CrossRefPubMed

Hoagwood KE, Olin SS, Horwitz S, McKay M, Cleek A, Gleacher A, et al. Scaling up evidence-based practices for children and families in New York State: toward evidence-based policies on implementation for state mental health systems. J Clin Child Adolesc Psychol. 2014;43(2):145–57. doi:10.1080/15374416.2013.869749.CrossRefPubMedPubMedCentral

McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. Am Psychol. 2010;65(2):73–84. doi:10.1037/a0018121.CrossRefPubMed

Nakamura BJ, Higa-McMillan CK, Okamura KH, Shimabukuro S. Knowledge of and attitudes towards evidence-based practices in community child mental health practitioners. Adm Policy Ment Health. 2011;38(4):287–300. doi:10.1007/s10488-011-0351-2.CrossRefPubMed

Starin AC, Atkins MS, Wehrmann KC, Mehta T, Hesson-McInnis MS, Marinez-Lora A, et al. Moving science into state child and adolescent mental health systems: Illinois’ evidence-informed practice initiative. J Clin Child Adolesc Psychol. 2014;43(2):169–78. doi:10.1080/15374416.2013.848772.CrossRefPubMed

Trupin E, Kerns S. Introduction to the special issue: legislation related to children’s evidence-based practice. Adm Policy Ment Health. 2015:1–5. doi:10.1007/s10488-015-0666-5.

Chorpita BF, Yim LM, Donkervoet JC, Arensdorf A, Amundsen MJ, McGee C, et al. Toward large-scale implementation of empirically supported treatments for children: a review and observations by the Hawaii Empirical Basis to Services Task Force. Clin Psychol Sci Pr. 2002;9(2):165–90. doi:10.1093/clipsy/9.2.165.CrossRef

Cooper JL, Aratani Y, Knitzer J, Douglas-Hall A, Masi R, Banghart PL et al. Unclaimed children revisited: the status of children’s mental health policy in the United States. Columbia University, National Center for Children in Poverty;2008. http://hdl.handle.net/10022/AC:P:8917. Accessed 29 Jan 2016.

10.

Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):2059–67. doi:10.2105/AJPH.2011.300193.CrossRefPubMedPubMedCentral

11.

Chorpita BF, Daleiden EL. Mapping evidence-based treatments for children and adolescents: application of the distillation and matching model to 615 treatments from 322 randomized trials. J Consult Clin Psychol. 2009;77(3):566–79. doi:10.1037/a0014565.CrossRefPubMed

12.

Chorpita BF, Bernstein A, Daleiden EL. Empirically guided coordination of multiple evidence-based treatments: an illustration of relevance mapping in children’s mental health services. J Consult Clin Psychol. 2011;79(4):470–80. doi:10.1037/a0023982.CrossRefPubMed

13.

Los Angeles County Department of Mental Health. About DMH. http://dmh.lacounty.gov/wps/portal/dmh/aboutdmh. Accessed 29 Jan 2016.

14.

Los Angeles County Department of Mental Health. Status Report on the MHSA PEI Plan. 2011. http://file.lacounty.gov/dmh/cms1_165911.pdf. Accessed 29 Jan 2016.

15.

Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23. doi:10.1007/s10488-010-0327-7.CrossRefPubMedPubMedCentral

16.

Glisson C, Schoenwald SK. The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatments. Ment Health Serv Res. 2005;7(4):243–59. doi:10.1007/s11020-005-7456-1.CrossRefPubMed

17.

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. doi:10.1007/s10488-010-0319-7.CrossRefPubMedPubMedCentral

18.

Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17. doi:10.1186/1748-5908-7-17.CrossRef

19.

Chambers DA, Rupp A. Sharing state mental health data for research: building toward ongoing learning in mental health care systems. Adm Policy Ment Health. 2015;42(5):586–7. doi:10.1007/s10488-015-0624-2.CrossRefPubMed

20.

Hoagwood KE, Essock S, Morrissey J, Libby A, Donahue S, Druss B, et al. Use of pooled state administrative data for mental health services research. Adm Policy Ment Health. 2015;43(1):67–78. doi:10.1007/s10488-014-0620-y.CrossRefPubMedCentral

21.

Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011;38(1):32–43. doi:10.1007/s10488-010-0321-0.CrossRefPubMedPubMedCentral

22.

Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the evidence-based practice attitude scale (EBPAS). Ment Health Serv Res. 2004;6(2):61–74. doi:10.1023/B:MHSR.0000024351.12294.65.CrossRefPubMedPubMedCentral

23.

Aarons GA, Sommerfeld DH, Walrath-Greene CM. Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implement Sci. 2009;4:83. doi:10.1186/1748-5908-4-83.CrossRefPubMedPubMedCentral

24.

Aarons GA, Sawitzky AC. Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychol Serv. 2006;3(1):61–72. doi:10.1037/1541-1559.3.1.61.CrossRefPubMedPubMedCentral

25.

Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: an organizational analysis. J Appl Psychol. 2001;86(5):811–24. doi:10.1037/0021-9010.86.5.811.CrossRefPubMed

26.

Glisson C. The organizational context of children’s mental health services. Clin Child Fam Psychol Rev. 2002;5(4):233–53. doi:10.1023/A:1020972906177.CrossRefPubMed

27.

Palinkas LA, Aarons GA. A view from the top: executive and management challenges in a statewide implementation of an evidence-based practice to reduce child neglect. Int J Child Health Hum Dev. 2009;2(1):47–55.

28.

Aarons GA, Palinkas LA. Implementation of evidence-based practice in child welfare: service provider perspectives. Adm Policy Ment Health. 2007;34(4):411–9. doi:10.1007/s10488-007-0121-3.CrossRefPubMed

29.

Aarons GA, Sommerfeld DH. Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Adolesc Psychiatry. 2012;51(4):423–31. doi:10.1016/j.jaac.2012.01.018.CrossRefPubMed

30.

Aarons GA, Sommerfeld DH, Hecht DB, Silovsky JF, Chaffin MJ. The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: evidence for a protective effect. J Consult Clin Psychol. 2009;77(2):270–80. doi:10.1037/a0013223.CrossRefPubMedPubMedCentral

31.

Casper ES. The theory of planned behavior applied to continuing education for mental health professionals. Psychiatr Serv. 2007;58(10):1324–9. doi:10.1176/ps.2007.58.10.1324.CrossRefPubMed

32.

Reding ME, Chorpita BF, Lau AS, Innes-Gomberg D. Providers’ attitudes toward evidence-based practices: is it just about providers, or do practices matter, too? Adm Policy Ment Health. 2014;41(6):767–76. doi:10.1007/s10488-013-0525-1.CrossRefPubMedPubMedCentral

33.

Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, et al. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implement Sci. 2012;7:32. doi:10.1186/1748-5908-7-32.CrossRefPubMedPubMedCentral

34.

Stirman SW, Gutner CA, Crits-Christoph P, Edmunds J, Evans AC, Beidas RS. Relationships between clinician-level attributes and fidelity-consistent and fidelity-inconsistent modifications to an evidence-based psychotherapy. Implement Sci. 2015;10:115. doi:10.1186/s13012-015-0308-z.CrossRef

35.

Southam-Gerow MA, Ringeisen HL, Sherrill JT. Integrating interventions and services research: progress and prospects. Clin Psychol Sci Pr. 2006;13(1):18. doi:10.1111/j.1468-2850.2006.00001.x.CrossRef

36.

Aarons GA, Cafri G, Lugo L, Sawitzky A. Expanding the domains of attitudes towards evidence-based practice: the evidence based practice attitude scale-50. Adm Policy Ment Health. 2010;39(5):331–40. doi:10.1007/s10488-010-0302-3.CrossRefPubMedCentral

37.

Aarons GA, Glisson C, Hoagwood K, Kelleher K, Landsverk J, Cafri G. Psychometric properties and U.S. national norms of the evidence-based practice attitude scale (EBPAS). Psychol Assess. 2010;22(2):356–65. doi:10.1037/a0019188.CrossRefPubMed

38.

Aisenberg E. Evidence-based practice in mental health care to ethnic minority communities: has its practice fallen short of its evidence? Soc Work. 2008;53(4):2970306. doi:10.1093/sw/53.4.297.CrossRef

39.

Bernal G, Scharró-del-Río MR. Are empirically supported treatments valid for ethnic minorities? Toward an alternative approach for treatment research. Cultur Divers Ethnic Minor Psychol. 2001;7(4):328–42. doi:10.1037/1099-9809.7.4.328.CrossRefPubMed

40.

Lau AS. Making the case for selective and directed cultural adaptations of evidence-based treatments: examples from parent training. Clin Psychol Sci Pr. 2006;13(4):295–310. doi:10.1111/j.1468-2850.2006.00042.x.CrossRef

41.

Forehand R, Dorsey S, Jones DJ, Long N, McMahon RJ. Adherence and flexibility: they can (and do) coexist! Psychological Services. 2010;17(3):258–64. doi:10.1111/j.1468-2850.2010.01217.x.

42.

Mazzucchelli TG, Sanders MR. Facilitating practitioner flexibility within an empirically supported intervention: lessons from a system of parenting support. Clin Psychol Sci Pr. 2010;17(3):238–52. doi:10.1111/j.1468-2850.2010.01215.x.CrossRef

43.

Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56. doi:10.1093/her/18.2.237.CrossRefPubMed

44.

Chambers DA, Wang PS, Insel TR. Maximizing efficiency and impact in effectiveness and services research. Gen Hosp Psychiatry. 2010;32(5):453–5. doi:10.1016/j.genhosppsych.2010.07.011.CrossRefPubMed

45.

Schoenwald SK, Garland AF, Southam-Gerow MA, Chorpita BF, Chapman JE. Adherence measurement in treatments for disruptive behavior disorders: pursuing clear vision through varied lenses. Clin Psychol Sci Pr. 2011;18(4):331–41. doi:10.1111/j.1468-2850.2011.01264.x.CrossRef

46.

Palinkas LA, Horwitz SM, Chamberlain P, Hurlburt MS, Landsverk J. Mixed-methods designs in mental health services research: a review. Psychiatr Serv. 2011;62(3):255–63. doi:10.1176/appi.ps.62.3.255.CrossRefPubMed

47.

Beretvas SN. Cross-classified and multiple-membership models. In: Hox JJ, Roberts JK, editors. Handbook for advanced multilevel analysis. New York: Routledge; 2011. p. 313–34.

48.

McLeod BD, Weisz JR. The therapy process observational coding system for child psychotherapy-strategies scale. J Clin Child Adolesc Psychol. 2010;39(3):436–43. doi:10.1080/15374411003691750.CrossRefPubMed

49.

Garland AF, Brookman-Frazee L, Hurlburt MS, Accurso EC, Zoffness RJ, Haine-Schlagel R, et al. Mental health care for children with disruptive behavior problems: a view inside therapists’ offices. Psychiatr Serv. 2010;61(8):788–95. doi:10.1176/appi.ps.61.8.788.CrossRefPubMedPubMedCentral

50.

Child & Adolescent Mental Health Division. Child and adolescent mental health performance standards. Hawa’ii Department of Health, Child & Adolescent Mental Health Division, Honolulu, HI. 2012. http://health.hawaii.gov/camhd/files/2013/07/Orange-Book.pdf. Accessed 2 Feb 2016.

51.

Orimoto TE, Higa-McMillan CK, Mueller CW, Daleiden EL. Assessment of therapy practices in community treatment for children and adolescents. Psychiatr Serv. 2012;63(4):343–50. doi:10.1176/appi.ps.201100129.CrossRefPubMed

52.

Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. 2013;8:65. doi:10.1186/1748-5908-8-65.CrossRefPubMedPubMedCentral

53.

Stirman SW, Calloway A, Toder K, Miller CJ, Devito AK, Meisel SN, et al. Community mental health provider modifications to cognitive therapy: implications for sustainability. Psychiatr Serv. 2013;64(10):1056–9. doi:10.1176/appi.ps.201200456.CrossRef

54.

Palinkas LA, Prussing E, Reznik VM, Landsverk JA. The San Diego East County school shootings: a qualitative study of community-level post-traumatic stress. Prehosp Disaster Med. 2004;19(1):113–21. doi:10.1017/S1049023X00001564.PubMed

55.

Hogue A. When technology fails: getting back to nature. Clin Psychol Sci Pr. 2010;17(1):77–81. doi:10.1111/j.1468-2850.2009.01196.x.CrossRef

Title: The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services
Authors: Anna S. Lau
Lauren Brookman-Frazee
Publication date: 01-12-2015
Publisher: BioMed Central
Published in: Implementation Science / Issue 1/2015
Electronic ISSN: 1748-5908
DOI: https://doi.org/10.1186/s13012-016-0388-4

Keynote webinar | Spotlight on medication adherence

Springer Medicine

The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services

Abstract

Background

Methods/design

Discussion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods/design

Discussion

Please log in to get access to this content

Other articles of this Issue 1/2015

Assessing fidelity of cognitive behavioral therapy in rural VA clinics: design of a randomized implementation effectiveness (hybrid type III) trial

Developing the ethics of implementation research in health

Factors affecting implementation of perinatal mental health screening in women of refugee background

Juvenile Justice—Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS): a cluster randomized trial targeting system-wide improvement in substance use services

High impact of implementation on school-based smoking prevention: the X:IT study—a cluster-randomized smoking prevention trial

Erratum to: ‘Toward the sustainability of health interventions implemented in sub-Saharan Africa: a systematic review and conceptual framework’