Skip to main content
Top
Published in: Implementation Science 1/2013

Open Access 01-12-2013 | Study protocol

Policy to implementation: evidence-based practice in community mental health – study protocol

Authors: Rinad S Beidas, Gregory Aarons, Frances Barg, Arthur Evans, Trevor Hadley, Kimberly Hoagwood, Steven Marcus, Sonja Schoenwald, Lucia Walsh, David S Mandell

Published in: Implementation Science | Issue 1/2013

Login to get access

Abstract

Background

Evidence-based treatments (EBTs) are not widely available in community mental health settings. In response to the call for implementation of evidence-based treatments in the United States, states and counties have mandated behavioral health reform through policies and other initiatives. Evaluations of the impact of these policies on implementation are rare. A systems transformation about to occur in Philadelphia, Pennsylvania, offers an important opportunity to prospectively study implementation in response to a policy mandate.

Methods/design

Using a prospective sequential mixed-methods design, with observations at multiple points in time, we will investigate the responses of staff from 30 community mental health clinics to a policy from the Department of Behavioral Health encouraging and incentivizing providers to implement evidence-based treatments to treat youth with mental health problems. Study participants will be 30 executive directors, 30 clinical directors, and 240 therapists. Data will be collected prior to the policy implementation, and then at two and four years following policy implementation. Quantitative data will include measures of intervention implementation and potential moderators of implementation (i.e., organizational- and leader-level variables) and will be collected from executive directors, clinical directors, and therapists. Measures include self-reported therapist fidelity to evidence-based treatment techniques as measured by the Therapist Procedures Checklist-Revised, organizational variables as measured by the Organizational Social Context Measurement System and the Implementation Climate Assessment, leader variables as measured by the Multifactor Leadership Questionnaire, attitudes towards EBTs as measured by the Evidence-Based Practice Attitude Scale, and knowledge of EBTs as measured by the Knowledge of Evidence- Based Services Questionnaire. Qualitative data will include semi-structured interviews with a subset of the sample to assess the implementation experience of high-, average-, and low-performing agencies. Mixed methods will be integrated through comparing and contrasting results from the two methods for each of the primary hypotheses in this study.

Discussion

Findings from the proposed research will inform both future policy mandates around implementation and the support required for the success of these policies, with the ultimate goal of improving the quality of treatment provided to youth in the public sector.
Appendix
Available only for authorised users
Literature
1.
go back to reference Chambless DL, Ollendick TH: Empirically supported psychological interventions: controversies and evidence. Annu Rev Psychol. 2001, 52: 685-716. 10.1146/annurev.psych.52.1.685.CrossRefPubMed Chambless DL, Ollendick TH: Empirically supported psychological interventions: controversies and evidence. Annu Rev Psychol. 2001, 52: 685-716. 10.1146/annurev.psych.52.1.685.CrossRefPubMed
2.
go back to reference Balas EA, Boren SA: Managing clinical knowledge for health care improvement. Yearbook of Medical Informatics. Edited by: Bemmel J, McCray AT. 2000, Stuttgart: Schattauer Verlagsgesellschaft, 65-70. Balas EA, Boren SA: Managing clinical knowledge for health care improvement. Yearbook of Medical Informatics. Edited by: Bemmel J, McCray AT. 2000, Stuttgart: Schattauer Verlagsgesellschaft, 65-70.
3.
go back to reference New Freedom Commission on Mental Health: Achieving the Promise: Transforming Mental Health Care in America. Executive Summary. 2003, Rockville New Freedom Commission on Mental Health: Achieving the Promise: Transforming Mental Health Care in America. Executive Summary. 2003, Rockville
4.
go back to reference Aarons GA, Sommerfeld DH, Willging CE: The soft underbelly of system change: The role of leadership and organizational climate in turnover during statewide behavioral health reform. Psychol Serv. 2011, 8: 269-281.CrossRefPubMedPubMedCentral Aarons GA, Sommerfeld DH, Willging CE: The soft underbelly of system change: The role of leadership and organizational climate in turnover during statewide behavioral health reform. Psychol Serv. 2011, 8: 269-281.CrossRefPubMedPubMedCentral
5.
go back to reference Brownson RC, Chriqui JF, Stamatakis KA: Understanding evidence-based public health policy. Am J Public Health. 2009, 99: 1576-83. 10.2105/AJPH.2008.156224.CrossRefPubMedPubMedCentral Brownson RC, Chriqui JF, Stamatakis KA: Understanding evidence-based public health policy. Am J Public Health. 2009, 99: 1576-83. 10.2105/AJPH.2008.156224.CrossRefPubMedPubMedCentral
6.
go back to reference Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood K, Mayberg S, Green P, Research Network on Youth Mental health: Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health. 2008, 35: 98-113. 10.1007/s10488-007-0148-5.CrossRefPubMed Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood K, Mayberg S, Green P, Research Network on Youth Mental health: Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health. 2008, 35: 98-113. 10.1007/s10488-007-0148-5.CrossRefPubMed
7.
go back to reference Glisson C, James LR: The cross-level effects of culture and climate in human service teams. J Organ Behavior. 2002, 23: 767-794. 10.1002/job.162.CrossRef Glisson C, James LR: The cross-level effects of culture and climate in human service teams. J Organ Behavior. 2002, 23: 767-794. 10.1002/job.162.CrossRef
8.
go back to reference Glisson C, Green P: Organizational climate, services, and outcomes in child welfare systems. Child Abuse Negl. 2011, 35: 582-91. 10.1016/j.chiabu.2011.04.009.CrossRefPubMed Glisson C, Green P: Organizational climate, services, and outcomes in child welfare systems. Child Abuse Negl. 2011, 35: 582-91. 10.1016/j.chiabu.2011.04.009.CrossRefPubMed
9.
go back to reference Glisson C, Schoenwald SK, Hemmelgam A, Green P, Dukes D, Armstrong KS, Chapman JE: A randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol. 2010, 78: 537-50.CrossRefPubMedPubMedCentral Glisson C, Schoenwald SK, Hemmelgam A, Green P, Dukes D, Armstrong KS, Chapman JE: A randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol. 2010, 78: 537-50.CrossRefPubMedPubMedCentral
10.
go back to reference Klein K, Knight A: Innovation implementation: overcoming the challenge. Cur Dir Psychol Sci. 2005, 14: 243-246. 10.1111/j.0963-7214.2005.00373.x.CrossRef Klein K, Knight A: Innovation implementation: overcoming the challenge. Cur Dir Psychol Sci. 2005, 14: 243-246. 10.1111/j.0963-7214.2005.00373.x.CrossRef
11.
go back to reference Weiner BJ, Beiden CM, Bergmire DM, Johnston M: The meaning and measurement of implementation climate. Implement Sci. 2011, 6: 78-10.1186/1748-5908-6-78.CrossRefPubMedPubMedCentral Weiner BJ, Beiden CM, Bergmire DM, Johnston M: The meaning and measurement of implementation climate. Implement Sci. 2011, 6: 78-10.1186/1748-5908-6-78.CrossRefPubMedPubMedCentral
12.
go back to reference Klein KJ, Conn AB, Sorra JS: Implementing computerized technology: an organizational analysis. J Appl Psych. 2001, 86: 811-824.CrossRef Klein KJ, Conn AB, Sorra JS: Implementing computerized technology: an organizational analysis. J Appl Psych. 2001, 86: 811-824.CrossRef
13.
go back to reference Bass B, Avolio B: Transformational Leadership Development: Manual for the Multifactor Leadership Questionnaire. 1990, Palo Alto: Consulting Psychologist Press Bass B, Avolio B: Transformational Leadership Development: Manual for the Multifactor Leadership Questionnaire. 1990, Palo Alto: Consulting Psychologist Press
14.
go back to reference Aarons GA: Transformational and transactional leadership: association with attitudes toward evidence-based practice. Psychiatr Serv. 2006, 57: 1162-9. 10.1176/appi.ps.57.8.1162.CrossRefPubMedPubMedCentral Aarons GA: Transformational and transactional leadership: association with attitudes toward evidence-based practice. Psychiatr Serv. 2006, 57: 1162-9. 10.1176/appi.ps.57.8.1162.CrossRefPubMedPubMedCentral
15.
go back to reference Aarons GA, Hurlburt M, Horwitz SM: Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011, 38: 4-23. 10.1007/s10488-010-0327-7.CrossRefPubMed Aarons GA, Hurlburt M, Horwitz SM: Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011, 38: 4-23. 10.1007/s10488-010-0327-7.CrossRefPubMed
16.
go back to reference Brownson RC, Royer C, Ewing R, McBride TD: Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006, 30: 164-72. 10.1016/j.amepre.2005.10.004.CrossRefPubMed Brownson RC, Royer C, Ewing R, McBride TD: Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006, 30: 164-72. 10.1016/j.amepre.2005.10.004.CrossRefPubMed
17.
go back to reference Dinh-Zarr TB, Sleet D, Shults RA, Zaza S, Elder RW, Nichols JL, Thompson RS, Sosin DM, Task Force on Community Preventative Services:Reviews of evidence regarding interventions to increase the use of safety belts. Am J Prev Med. 2001, 21: 48-65.CrossRefPubMed Dinh-Zarr TB, Sleet D, Shults RA, Zaza S, Elder RW, Nichols JL, Thompson RS, Sosin DM, Task Force on Community Preventative Services:Reviews of evidence regarding interventions to increase the use of safety belts. Am J Prev Med. 2001, 21: 48-65.CrossRefPubMed
18.
go back to reference Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M: Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011, 38: 65-76. 10.1007/s10488-010-0319-7.CrossRefPubMed Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M: Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011, 38: 65-76. 10.1007/s10488-010-0319-7.CrossRefPubMed
19.
go back to reference Schoenwald SK, Henngeller SW, Brondino MJ, Rowland MD: Multisystemic therapy: monitoring treatment fidelity. Fam Process. 2000, 39: 83-103. 10.1111/j.1545-5300.2000.39109.x.CrossRefPubMed Schoenwald SK, Henngeller SW, Brondino MJ, Rowland MD: Multisystemic therapy: monitoring treatment fidelity. Fam Process. 2000, 39: 83-103. 10.1111/j.1545-5300.2000.39109.x.CrossRefPubMed
20.
go back to reference Weersing VR, Weisz JR, Donenberg G: Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. J Clin Child Psychol. 2002, 31: 168-180.CrossRef Weersing VR, Weisz JR, Donenberg G: Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. J Clin Child Psychol. 2002, 31: 168-180.CrossRef
21.
go back to reference McLeod BD, Weisz JR: The Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale. J Clin Child Adolesc Psychol. 2010, 39: 436-43. 10.1080/15374411003691750.CrossRefPubMed McLeod BD, Weisz JR: The Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale. J Clin Child Adolesc Psychol. 2010, 39: 436-43. 10.1080/15374411003691750.CrossRefPubMed
22.
go back to reference Beidas RS, Kendall PC: Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol-Sci Pr. 2010, 17: 1-30. 10.1111/j.1468-2850.2009.01187.x.CrossRef Beidas RS, Kendall PC: Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol-Sci Pr. 2010, 17: 1-30. 10.1111/j.1468-2850.2009.01187.x.CrossRef
23.
go back to reference Lehman WJ, Greener J, Simpson D: Assessing organizational readiness for change. J Subst Abuse Treat. 2002, 22: 197-209. 10.1016/S0740-5472(02)00233-7.CrossRefPubMed Lehman WJ, Greener J, Simpson D: Assessing organizational readiness for change. J Subst Abuse Treat. 2002, 22: 197-209. 10.1016/S0740-5472(02)00233-7.CrossRefPubMed
24.
go back to reference Helfrich CD, Li YF, Sharp ND, Sales AE: Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implement Sci. 2009, 4: 38-10.1186/1748-5908-4-38.CrossRefPubMedPubMedCentral Helfrich CD, Li YF, Sharp ND, Sales AE: Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implement Sci. 2009, 4: 38-10.1186/1748-5908-4-38.CrossRefPubMedPubMedCentral
25.
go back to reference Glisson C, Green P, Williams NJ: Assessing the Organizational Social Context (OSC) of child welfare systems: implications for research and practice. Child Abuse Negl. 2012, 36: 621-32. 10.1016/j.chiabu.2012.06.002.CrossRefPubMed Glisson C, Green P, Williams NJ: Assessing the Organizational Social Context (OSC) of child welfare systems: implications for research and practice. Child Abuse Negl. 2012, 36: 621-32. 10.1016/j.chiabu.2012.06.002.CrossRefPubMed
26.
go back to reference Aarons G: Organizational Climate for Evidence-Based Practice Implementation: Development of a New Scale. Proceedings of the Annual Association for Behavioral and Cognitive Therapies: November 2011; Toronto. 2011 Aarons G: Organizational Climate for Evidence-Based Practice Implementation: Development of a New Scale. Proceedings of the Annual Association for Behavioral and Cognitive Therapies: November 2011; Toronto. 2011
27.
go back to reference Antonakis J, Avolio B, Sivasubramaniam N: Context and leadership: an examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. Leadership Quart. 2003, 14: 261-295. 10.1016/S1048-9843(03)00030-4.CrossRef Antonakis J, Avolio B, Sivasubramaniam N: Context and leadership: an examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. Leadership Quart. 2003, 14: 261-295. 10.1016/S1048-9843(03)00030-4.CrossRef
28.
go back to reference Aarons GA: Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004, 6: 61-74.CrossRefPubMedPubMedCentral Aarons GA: Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004, 6: 61-74.CrossRefPubMedPubMedCentral
29.
go back to reference Aarons GA, Sawitzky AC: Organizational culture and climate and mental health provider attitudes towards evidence-based practice. Psychol Serv. 2006, 3: 61-72.CrossRefPubMedPubMedCentral Aarons GA, Sawitzky AC: Organizational culture and climate and mental health provider attitudes towards evidence-based practice. Psychol Serv. 2006, 3: 61-72.CrossRefPubMedPubMedCentral
30.
go back to reference Aarons GA: Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child Adolesc Psychiatr Clin N Am. 2005, 14: 255-271. 10.1016/j.chc.2004.04.008.CrossRefPubMedPubMedCentral Aarons GA: Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child Adolesc Psychiatr Clin N Am. 2005, 14: 255-271. 10.1016/j.chc.2004.04.008.CrossRefPubMedPubMedCentral
31.
go back to reference Stumpf RE, Higa-McMillan CK, Chorpita BF: Implementation of evidence-based services for youth: assessing provider knowledge. Behav Modif. 2009, 33: 48-65.CrossRefPubMed Stumpf RE, Higa-McMillan CK, Chorpita BF: Implementation of evidence-based services for youth: assessing provider knowledge. Behav Modif. 2009, 33: 48-65.CrossRefPubMed
32.
go back to reference Bradley EH, Curry LA, Ramanadhan S, Rowe L, Nembhard I, Krumholz H: Research in action: using positive deviance to improve quality of health care. Implement Sci. 2009, 4: 25-10.1186/1748-5908-4-25.CrossRefPubMedPubMedCentral Bradley EH, Curry LA, Ramanadhan S, Rowe L, Nembhard I, Krumholz H: Research in action: using positive deviance to improve quality of health care. Implement Sci. 2009, 4: 25-10.1186/1748-5908-4-25.CrossRefPubMedPubMedCentral
33.
go back to reference Palinkas LA, Horwitz SM, Chamberlain P, Hurlburt MS, Landsverk J: Mixed-methods designs in mental health services research: a review. Psychiatr Serv. 2011, 62: 255-63. 10.1176/appi.ps.62.3.255.CrossRefPubMed Palinkas LA, Horwitz SM, Chamberlain P, Hurlburt MS, Landsverk J: Mixed-methods designs in mental health services research: a review. Psychiatr Serv. 2011, 62: 255-63. 10.1176/appi.ps.62.3.255.CrossRefPubMed
34.
go back to reference Guba Y, Lincoln E: Naturalistic Inquiry. 1985, Beverly Hills: Sage Guba Y, Lincoln E: Naturalistic Inquiry. 1985, Beverly Hills: Sage
35.
go back to reference Palinkas LA, Aarons GA, Horwitz SM, Chamberlain P, Hurlburt M, Landsverk J: Mixed method designs in implementation research. Adm Policy Ment Health. 2011, 38: 44-53. 10.1007/s10488-010-0314-z.CrossRefPubMed Palinkas LA, Aarons GA, Horwitz SM, Chamberlain P, Hurlburt M, Landsverk J: Mixed method designs in implementation research. Adm Policy Ment Health. 2011, 38: 44-53. 10.1007/s10488-010-0314-z.CrossRefPubMed
36.
go back to reference Creswell JW, Klassen AC, Plano Clark VL: Smith. 2011, Best Practices for Mixed Methods Research in the Health Sciences, Rockville: KC Creswell JW, Klassen AC, Plano Clark VL: Smith. 2011, Best Practices for Mixed Methods Research in the Health Sciences, Rockville: KC
37.
go back to reference Williams ME, Rogers KC, Carson MC, Sherer S, Hudson BO: Opportunities arising from transformation from treatment as usual to evidence-based practice. Prof Psychol- Res Pr. 2011, 43: 9-16.CrossRef Williams ME, Rogers KC, Carson MC, Sherer S, Hudson BO: Opportunities arising from transformation from treatment as usual to evidence-based practice. Prof Psychol- Res Pr. 2011, 43: 9-16.CrossRef
38.
go back to reference Schoenwald SK, Garland A, Chapman JE, Frazier S, Sheidow A, Southam-Gerow M: Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011, 38: 32-43. 10.1007/s10488-010-0321-0.CrossRefPubMedPubMedCentral Schoenwald SK, Garland A, Chapman JE, Frazier S, Sheidow A, Southam-Gerow M: Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011, 38: 32-43. 10.1007/s10488-010-0321-0.CrossRefPubMedPubMedCentral
Metadata
Title
Policy to implementation: evidence-based practice in community mental health – study protocol
Authors
Rinad S Beidas
Gregory Aarons
Frances Barg
Arthur Evans
Trevor Hadley
Kimberly Hoagwood
Steven Marcus
Sonja Schoenwald
Lucia Walsh
David S Mandell
Publication date
01-12-2013
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2013
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/1748-5908-8-38

Other articles of this Issue 1/2013

Implementation Science 1/2013 Go to the issue