Top

Administration and Policy in Mental Health and Mental Health Services Research

Published in:

01-09-2016 | Original Article

Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice

Author: Nathaniel J. Williams

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 5/2016

Abstract

A step toward the development of optimally effective, efficient, and feasible implementation strategies that increase evidence-based treatment integration in mental health services involves identification of the multilevel mechanisms through which these strategies influence implementation outcomes. This article (a) provides an orientation to, and rationale for, consideration of multilevel mediating mechanisms in implementation trials, and (b) systematically reviews randomized controlled trials that examined mediators of implementation strategies in mental health. Nine trials were located. Mediation-related methodological deficiencies were prevalent and no trials supported a hypothesized mediator. The most common reason was failure to engage the mediation target. Discussion focuses on directions to accelerate implementation strategy development in mental health.

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274. doi:10.1146/annurev-publhealth-032013-182447.CrossRefPubMedPubMedCentral

Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public sectors. Administration and Policy in Mental Health, 38, 4–23. doi:10.1007/s10488-010-0327-7.CrossRefPubMed

Atkins, M. S., Frazier, S. L., Leathers, S. J., Graczyk, P. A., Talbott, E., Jakobsons, L., et al. (2008). Teacher key opinion leaders and mental health consultation in low-income urban schools. Journal of Consulting and Clinical Psychology, 76, 905–908. doi:10.1037/a0013036.CrossRefPubMed

Baer, J. S., Wells, E. A., Rosengren, D. B., Hartzler, B., Beadnell, B., & Dunn, C. (2009). Agency context and tailored training in technology transfer: A pilot evaluation of motivational interviewing training for community counselors. Journal of Substance Abuse Treatment, 37, 191–202. doi:10.1016/j.jsat.2009.01.003.CrossRefPubMedPubMedCentral

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182. doi:10.1037/0022-3514.51.6.1173.CrossRefPubMed

Barwick, M. A., Schachter, H. M., Bennett, L. M., McGowan, J., Ly, M., Wilson, A., et al. (2012). Knowledge translation efforts in child and youth mental health: A systematic review. Journal of Evidence Based Social Work, 9, 369–395. doi:10.1080/15433714.2012.663667.CrossRefPubMedPubMedCentral

Bauer, D. J., Preacher, K. J., & Gil, K. M. (2006). Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods, 11, 142–163. doi:10.1037/1082-989x.11.2.142.supp.CrossRefPubMed

Berwick, D. M. (2008). The science of improvement. Journal of the American Medical Association, 299, 1182–1184. doi:10.1001/jama.299.10.1182.CrossRefPubMed

Biesanz, J. C., Falk, C. F., & Savalei, V. (2010). Assessing mediational models: Testing and interval estimation for indirect effects. Multivariate Behavioral Research, 45, 661–701. doi:10.1080/00273171.2010.498292.CrossRefPubMed

Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246. doi:10.1037/0021-9010.83.2.234.CrossRef

Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013). Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science, 8, 22. doi:10.1186/1748-5908-8-22.CrossRefPubMedPubMedCentral

Chen, H. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.

Collins, P. Y., Patel, V., Joestl, S. S., March, D., Insel, T. R., & Daar, A. S. (2011). Grand challenges in global mental health. Nature, 475, 27–30. doi:10.1038/475027a.CrossRefPubMedPubMedCentral

Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. doi:10.1186/1748-5908-4-50.CrossRefPubMedPubMedCentral

Davies, P., Walker, A. E., & Grimshaw, J. M. (2010). A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implementation Science, 5(14), 5908-5. doi:10.1186/1748-5908-5-14.

Doss, B. D. (2004). Changing the way we study change in psychotherapy. Clinical Psychology: Science and Practice, 11, 368–386. doi:10.1186/1748-5908-5-14.

Edwards, J. R., & Lambert, L. S. (2007). Methods for integrating moderation and mediation: A general analytical framework using moderated path analysis. Psychological Methods, 12, 1–22. doi:10.1037/1082-989x.12.1.1.supp.CrossRefPubMed

Enders, C. (2013). Centering predictors and contextual effects. In M. A. Scott, J. S. Simonoff, & B. D. Marx (Eds.), The SAGE handbook of multilevel modeling (pp. 89–108). Thousand Oaks, CA: Sage Publications Inc.CrossRef

Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health, 40, 6–22. doi:10.1371/journal.pmed.1001434.CrossRefPubMedPubMedCentral

Garner, B. R., Godley, S. H., & Bair, C. M. L. (2011). The impact of pay-for-performance on therapists’ intentions to deliver high quality treatment. Journal of Substance Abuse Treatment, 41, 97–103. doi:10.1016/j.jsat.2011.01.012.CrossRefPubMedPubMedCentral

Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78, 537–550. doi:10.1037/a0019160.CrossRefPubMedPubMedCentral

Glisson, C., & Williams, N. J. (2015). Assessing and changing organizational social contexts for effective mental health services. Annual Review of Public Health, 36, 507–523. doi:10.1146/annurev-publhealth-031914-122435.CrossRefPubMed

Godin, G., Belanger-Gravel, A., Eccles, M., & Grimshaw, J. (2008). Healthcare professionals’ intentions and behaviors: A systematic review of studies based on social cognitive theories. Implementation Science, 3(36), 1–12.

Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: A systematic review and recommendations. Milbank Quarterly, 82, 581–629. doi:10.1111/j.0887-378x.2004.00325.x.CrossRefPubMedPubMedCentral

Grol, R. P., Bosch, M. C., Hulscher, M. E., Eccles, M. P., & Wensing, M. (2007). Planning and studying improvement in patient care: The use of theoretical perspectives. Milbank Quarterly, 85, 93–138. doi:10.1111/j.1468-0009.2007.00478.x.CrossRefPubMedPubMedCentral

Hayes, A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science, 24, 1918–1927. doi:10.1177/0956797613480187.CrossRefPubMed

Holth, P., Torsheim, T., Sheidow, A. J., Ogden, T., & Henggeler, S. W. (2011). Intensive quality assurance of therapist adherence to behavioral interventions for adolescent substance use problems. Journal of Child and Adolescent Substance Abuse, 20, 289–313. doi:10.1080/1067828x.2011.581974.CrossRefPubMedPubMedCentral

Insel, T. R. (2009). Translating scientific opportunity into public health impact: A strategic plan for research on mental illness. Archives of General Psychiatry, 66, 128–133. doi:10.1001/archgenpsychiatry.2008.540.CrossRefPubMed

Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: Author.

Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619. doi:10.1080/1067828x.2011.581974.CrossRef

Kauth, M. R., Sullivan, G., Blevins, D., Cully, J. A., Landes, R. D., Said, Q., & Teasdale, T. A. (2010). Employing external facilitation to implement cognitive behavioral therapy in VA clinics: A pilot study. Implementation Science, 5, 75. doi:10.1186/1748-5908-5-75.CrossRefPubMedPubMedCentral

Kazdin, A. E. (2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 1–27. doi:10.1146/annurev.clinpsy.3.022806.091432.CrossRefPubMed

Kenny, D. A., Korchmaros, J. D., & Bolger, N. (2003). Lower level mediation in multilevel models. Psychological Methods, 8, 115–128. doi:10.1037/1082-989x.8.2.115.CrossRefPubMed

Kessler, R. C., Aguilar-Gaxiola, S., Alonso, J., Chatterji, S., Lee, S., Ormel, J., et al. (2009). The global burden of mental disorders: An update from the WHO World Mental Health (WMH) Surveys. Epidemiologia e Psichiatria Sociale, 18(1), 23–33. doi:10.1017/s1121189x00001421.CrossRefPubMedPubMedCentral

Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195–229. doi:10.5465/AMR.1994.9410210745.

Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent properties. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco: Jossey-Bass.

Kraemer, H. C., Kazdin, A. E., Offord, D. R., Kessler, R. C., Jensen, P. S., & Kupfer, D. J. (1997). Coming to terms with the terms of risk. Archives of General Psychiatry, 54, 337–343. doi:10.1001/archpsyc.1997.01830160065009.CrossRefPubMed

Kreft, I. G. G., de Leeuw, J., & Aiken, L. S. (1995). The effects of different forms of centering in hierarchical linear models. Multivariate Behavioral Research, 30, 1–21. doi:10.1207/s15327906mbr3001_1.CrossRefPubMed

Krull, J. L., & MacKinnon, D. P. (2001). Multilevel modeling of individual and group level mediated effects. Multivariate Behavioral Research, 36, 249–277. doi:10.1207/S15327906MBR3602_06.CrossRefPubMed

Landsverk, J., Brown, C. H., Reutz, J. R., Palinkas, L., & Horwitz, S. M. (2011). Design elements in implementation research: A structured review of child welfare and mental health studies. Administration and Policy in Mental Health, 38, 54–63. doi:10.1007/s10488-010-0315-y.CrossRefPubMedPubMedCentral

LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852. doi:10.1177/1094428106296642.CrossRef

Lochman, J. E., Boxmeyer, C., Powell, N., Qu, L., Wells, K., & Windle, M. (2009). Dissemination of the coping power program: Importance of intensity of counselor training. Journal of Consulting and Clinical Psychology, 77, 397–409. doi:10.1037/a0014514.CrossRefPubMed

MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Lawrence Erlbaum Associates.

MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 58, 593–614. doi:10.1146/annurev.psych.58.110405.085542.CrossRefPubMedPubMedCentral

MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104. doi:10.1037/1082-989X.7.1.83.CrossRefPubMedPubMedCentral

MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128. doi:10.1207/s15327906mbr3901_4.CrossRefPubMedPubMedCentral

Mathieu, J. E., DeShon, R. P., & Bergh, D. D. (2008). Mediational inferences in organizational research: Then, now, and beyond. Organizational Research Methods, 11, 203–223. doi:10.1177/1094428107310089.CrossRef

Mathieu, J. E., & Taylor, S. R. (2007). A framework for testing meso-mediational relationships in organizational behavior. Journal of Organizational Behavior, 28, 141–172. doi:10.1002/job.436.CrossRef

McHugh, R. K., & Barlow, D. H. (2012). The reach of evidence-based psychological interventions. In R. K. McHugh & D. H. Barlow (Eds.), Dissemination and implementation of evidence-based psychological interventions (pp. 3–15). New York: Oxford University Press.

Michie, S., van Stralen, M. M., & West, R. (2011). The behavior change wheel: A new method for characterizing and designing behavior change interventions. Implementation Science, 6, 42. doi:10.1186/1748-5908-6-42.CrossRefPubMedPubMedCentral

Neuhaus, J. M., & Kalbfleisch, J. D. (1998). Between- and within-cluster covariate effects in the analysis of clustered data. Biometrics, 54, 638–645. doi:10.2307/3109770.CrossRefPubMed

Novins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 1009–1025. doi:10.1016/j.jaac.2013.07.012.CrossRefPubMedPubMedCentral

Pituch, K. A., Murphy, D. L., & Tate, R. L. (2010). Three-level models for indirect effects in school- and class-randomized experiments in education. Journal of Experimental Education, 78, 60–95. doi:10.1080/00220970903224685.CrossRef

Powell, B. J., Proctor, E. K., & Glass, J. E. (2014). A systematic review of strategies for implementing empirically supported mental health interventions. Research on Social Work Practice, 24, 192–212. doi:10.1177/1049731513505778.CrossRefPubMed

Preacher, K. J., & Hayes, A. F. (2004). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879–891. doi:10.3758/brm.40.3.879.CrossRef

Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42, 185–227. doi:10.1080/00273170701341316.CrossRefPubMed

Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6, 77–98. doi:10.1080/19312458.2012.679848.CrossRef

Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209–233. doi:10.1037/a0020141.CrossRefPubMed

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38, 65–76. doi:10.1007/s10488-010-0319-7.CrossRefPubMed

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis (2nd ed.). Thousand Oaks, CA: Sage publications.

Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260. doi:10.1006/pmed.1993.1020.CrossRefPubMed

Rousseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In L. L. Cummings & B. M. Straw (Eds.), Research in organizational behavior (Vol. 7, pp. 1–37). Greenwich, CT: JAI Press.

Saloner, B., Carson, N., & Lê Cook, B. (2014). Episodes of mental health treatment among a nationally representative sample of children and adolescents. Medical Care Research and Review, 71, 261–279. doi:10.1177/1077558713518347.CrossRefPubMed

Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology, 13, 290–312. doi:10.2307/270723.CrossRef

Steel, Z., Marnane, C., Iranpour, C., Chey, T., Jackson, J. W., Patel, V., & Silove, D. (2014). The global prevalence of common mental disorders: A systematic review and meta-analysis 1980–2013. International Journal of Epidemiology, 43, 476–493. doi:10.1093/ije/dyu038.CrossRefPubMedPubMedCentral

Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice: Models for dissemination and implementation research. American Journal of Preventive Medicine, 43, 337–350. doi:10.1016/j.amepre.2012.05.024.CrossRefPubMedPubMedCentral

Taylor, A. B., MacKinnon, D. P., & Tein, J. Y. (2008). Tests of the three-path mediated effect. Organizational Research Methods, 11, 241–269. doi:10.1177/1094428107300344.CrossRef

Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70, 750–761. doi:10.1001/jamapsychiatry.2013.1176.CrossRefPubMed

Williams, N. J. (2015). Mechanisms of change in an organizational culture and climate intervention for increasing clinicians’ evidence-based practice adoption in mental health (Unpublished doctoral dissertation). University of Tennessee, Knoxville. http://trace.tennessee.edu/utk_graddiss/3482.

Williams, N. J., & Glisson, C. (2014). The role of organizational culture and climate in the dissemination and implementation of empirically-supported treatments for youth. In R. S. Beidas & P. C. Kendall (Eds.), Dissemination and implementation of evidence based practices in child and adolescent mental health (pp. 61–81). New York: Oxford University Press.

Williams, J. R., Williams, W. O., Dusablon, T., Blais, M. P., Tregear, S. J., Banks, D., & Hennessy, K. D. (2014). Evaluation of a randomized intervention to increase adoption of comparative effectiveness research by community health organizations. Journal of Behavioral Health Services and Research, 41, 308–323. doi:10.1007/s11414-013-9369-4.CrossRefPubMed

Zhang, Z., Zyphur, M. J., & Preacher, K. J. (2009). Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods, 12, 695–719. doi:10.1177/1094428108327450.CrossRef

Title: Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice
Author: Nathaniel J. Williams
Publication date: 01-09-2016
Publisher: Springer US
Published in: Administration and Policy in Mental Health and Mental Health Services Research / Issue 5/2016
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-015-0693-2

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 5/2016

Sociocultural Factors and Parent–Therapist Agreement on Explanatory Etiologies for Youth Mental Health Problems

Constraints and Benefits of Child Welfare Contracts with Behavioral Health Providers: Conditions that Shape Service Access

Availability of Youth Services in U.S. Mental Health Treatment Facilities

Contracting and Procurement for Evidence-Based Interventions in Public-Sector Human Services: A Case Study

Patient Preferences of a Low-Income Hispanic Population for Mental Health Services in Primary Care

Leadership, Organizational Climate, and Perceived Burden of Evidence-Based Practice in Mental Health Services