Top

Administration and Policy in Mental Health and Mental Health Services Research

Published in:

01-01-2017 | Original Article

A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy

Authors: Shannon Wiltsey Stirman, Kristin Pontoski, Torrey Creed, Regina Xhezo, Arthur C. Evans, Aaron T. Beck, Paul Crits-Christoph

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 1/2017

Abstract

Despite the central role of training and consultation in the implementation of evidence-based psychological interventions (EBPIs), comprehensive reviews of research on training have highlighted serious gaps in knowledge regarding best practices. Consultation after initial didactic training appears to be of critical importance, but there has been very little research to determine optimal consultation format or interventions. This observational study compared two consultation formats that included review of session audio and feedback in the context of a program to train clinicians (n = 85) in community mental health clinics to deliver cognitive therapy (CT). A “gold standard” condition in which clinicians received individual feedback after expert consultants reviewed full sessions was compared to a group consultation format in which short segments of session audio were reviewed by a group of clinicians and an expert consultant. After adjusting for potential baseline differences between individuals in the two consultation conditions, few differences were found in terms of successful completion of the consultation phase or in terms of competence in CT at the end of consultation or after a 2 year follow-up. However, analyses did not support hypotheses regarding non-inferiority of the group consultation condition. While both groups largely maintained competence, clinicians in the group consultation condition demonstrated increases in competence over the follow-up period, while a sub-group of those in the individual condition experienced decreases. These findings, if replicated, have important implications for EBP implementation programs, as they suggest that observation and feedback is feasible in community mental health setting, and that employing this method in a group format is an effective and efficient consultation strategy that may enhance the implementation and sustainability of evidence-based psychotherapies.

Baer, J. S., Rosengren, D. B., Dunn, C. W., Wells, E. A., Ogle, R. L., & Hartzler, B. (2004). An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug and Alcohol Dependence, 73(1), 99–106. doi:10.1016/j.drugalcdep.2003.10.001.CrossRefPubMed

Beck, A. T. (2005). The current state of cognitive therapy: A 40-year retrospective. Archives of General Psychiatry, 62(9), 953–959. doi:10.1001/archpsyc.62.9.953.CrossRefPubMed

Beidas, R., Edmunds, J., Cannuscio, C., Gallagher, M., Downey, M. M., & Kendall, P. (2013). Therapists perspectives on the effective elements of consultation following training. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 507–517. doi:10.1007/s10488-013-0475-7.CrossRefPubMedPubMedCentral

Beidas, R., Edmunds, J., Marcus, S., & Kendall, P. (2012). Training and consultation to promote implementation of an empirically supported treatment: A randomized trial. Psychiatric Services, 63(7), 660–665. doi:10.1176/appi.ps.201100401.CrossRefPubMedPubMedCentral

Beidas, R., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1. doi:10.1111/j.1468-2850.2009.01187.x.

Blackwelder, W. (2004). Current issues in clinical equivalence trials. Journal of Dental Research, 83(suppl 1), C113–C115.CrossRefPubMed

Branson, A., Shafran, R., & Myles, P. (2015). Investigating the relationship between competence and patient outcome with CBT. Behaviour Research and Therapy, 68, 19–26. doi:10.1016/j.brat.2015.03.002.CrossRefPubMed

Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self evaluation of cognitive therapy performance: Do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36(5), 581. doi:10.1017/S1352465808004438.CrossRef

Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications Inc.

Carroll, K. M., Martino, S., & Rounsaville, B. J. (2010). No train, no gain? Clinical Psychology: Science and Practice, 17(1), 36–40. doi:10.1111/j.1468-2850.2009.01190.x.

Chorpita, B. F., & Regan, J. (2009). Dissemination of effective mental health treatment procedures: Maximizing the return on a significant investment. Behaviour Research and Therapy, 47(11), 990–993. doi:10.1016/j.brat.2009.07.002.CrossRefPubMed

Cichetti, D. (1994). Guidelines, criteria, and rules of thumb for normed and standardized instruments in psychology. Psychological Assessment, 6(4), 284–290.CrossRef

Creed, T. A., Stirman, S. W., Evans, A. C., & Beck, A. T. (2014). A model for implementation of cognitive therapy in community mental health: The Beck Initiative. The Behavior Therapist, 37(3).

D’Agostino, R. B., Massaro, J. M., & Sullivan, L. M. (2003). Non-inferiority trials: Design concepts and issues-the encounters of academic consultants in statistics. Statistics in Medicine, 22(2), 169–186.CrossRefPubMed

Eckardt, P. (2012). Propensity score estimates in multilevel models for causal inference. Nursing Research, 61(3), 213–223.CrossRefPubMed

Edmunds, J. M., Beidas, R. S., & Kendall, P. C. (2013). Dissemination and implementation of evidence-based practices: Training and consultation as implementation strategies. Clinical Psychology: Science and Practice, 20(2), 152–165. doi:10.1111/cpsp.12031.

Feldstein, A. C., Glasgow, R. E., & Smith, D. H. (2008). A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Joint Commission Journal on Quality and Patient Safety, 34(4), 228–243. doi:10.2165/00148365-200806040-00001.CrossRefPubMed

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friendman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: The National Implementation Research Network, Louis de la Parte Florida Mental Health Institute, University of South Florida.

Fleming, L. M., Glass, J. A., Fujisaki, S., & Toner, S. L. (2010). Group process and learning: A grounded theory model of group supervision. Training and Education in Professional Psychology, 4(3), 194.CrossRef

Franklin, M. E., Abramowitz, J. S., Kozak, M. J., Levitt, J. T., & Foa, E. B. (2000). Effectiveness of exposure and ritual prevention for obsessive-compulsive disorder: Randomized compared with nonrandomized samples. Journal of Consulting and Clinical Psychology, 68(4), 594. doi:10.1037/0022-006X.68.4.594.CrossRefPubMed

Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15(3), 234–249. doi:10.1037/a0019623.CrossRefPubMedPubMedCentral

Henggeler, S. W., Chapman, J. E., Rowland, M. D., Sheidow, A. J., & Cunningham, P. B. (2013). Evaluating training methods for transporting contingency management to therapists. Journal of Substance Abuse Treatment, 45(5), 466–474.CrossRefPubMedPubMedCentral

Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. doi:10.1016/j.cpr.2010.02.005.CrossRefPubMedPubMedCentral

Juster, H. R., Heimberg, R. G., & Engelberg, B. (1995). Self selection and sample selection in a treatment study of social phobia. Behavioral Research Therapy, 33(3), 321–324.CrossRef

Karlin, B. E., Ruzek, J. I., Chard, K. M., Eftekhari, A., Monson, C. M., Hembree, E. A., et al. (2010). Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the veterans health administration. Journal of Traumatic Stress, 23(6), 663–673. doi:10.1002/jts.20588.CrossRefPubMed

Kolko, D. J., Baumann, B. L., Herschell, A. D., Hart, J. A., Holden, E. A., & Wisniewski, S. R. (2012). Implementation of AF-CBT by community practitioners serving child welfare and mental health: A randomized trial. Child maltreatment. doi:10.1177/1077559511427346.PubMedCentral

Lewandowsky, S., & Thomas, J. L. (2009). Expertise: Acquisition, limitations, and control. Reviews of human factors and ergonomics, 5(1), 140–165.CrossRef

McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist, 65, 73–84. doi:10.1037/a0018121.CrossRefPubMed

Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72(6), 1050–1062. doi:10.1037/0022-006X.72.6.1050.CrossRefPubMed

Milne, D. (2014). Beyond the “Acid Test”: A conceptual review and reformulation of outcome evaluation in clinical supervision. American Journal of Psychotherapy, 68(2), 213–230.PubMed

Nacasch, N., Huppert, J. D., Su, Y.-J., Kivity, Y., Dinshtein, Y., Yeh, R., & Foa, E. B. (2014). Are 60-minute prolonged exposure sessions with 20-minute imaginal exposure to traumatic memories sufficient to successfully treat PTSD? A randomized noninferiority clinical trial. Behavior Therapy, 46, 328–341.CrossRefPubMed

Ögren, M.-L. E. C. (2009). Group supervision in psychotherapy. Main findings from a Swedish research project on psychotherapy supervision in a group format. British Journal of Guidance & Counselling, 37(2), 129–139. doi:10.1080/03069880902728614.CrossRef

Rakovshik, S. G., & McManus, F. (2010). Establishing evidence-based training in cognitive behavioral therapy: A review of current empirical findings and theoretical guidance. Clinical Psychology Review, 30(5), 496–516. doi:10.1016/j.cpr.2010.03.004.CrossRefPubMed

Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173.CrossRef

Ruzek, J. I., & Rosen, R. C. (2009). Disseminating evidence-based treatments for PTSD in organizational settings: A high priority focus area. Behaviour Research and Therapy, 47(11), 980–989. doi:10.1016/j.brat.2009.07.008.CrossRefPubMed

Schafer, J. L., & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13(4), 279–313. doi:10.1037/a0014268.CrossRefPubMed

Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-010-0321-0.PubMedPubMedCentral

Shaw, B. F., Elkin, I., Yamaguchi, J., Olmsted, M., Vallis, T. M., Dobson, K. S., et al. (1999). Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology, 67(6), 837.CrossRefPubMed

Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S. A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73(1), 106–115. doi:10.1037/0022-006X.73.1.106.CrossRefPubMedPubMedCentral

Simons, A. D., Padesky, C. A., Montemarano, J., Lewis, C. C., Murakami, J., Lamb, K., et al. (2010). Training and dissemination of cognitive behavior therapy for depression in adults: A preliminary examination of therapist competence and client outcomes. Journal of Consulting and Clinical Psychology, 78(5), 751–756. doi:10.1037/a0020569.CrossRefPubMed

Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. London: Oxford University Press.CrossRef

Stirman, S. W., Bhar, S. S., Spokas, M., Brown, G. K., Creed, T. A., Perivoliotis, D., et al. (2010). Training and consultation in evidence-based psychosocial treatments in public mental health settings: The ACCESS model. Professional Psychology: Research and Practice, 41(1), 48–56. doi:10.1037/a0018099.CrossRef

Stirman, S. W., Buchhofer, R., McLaulin, J. B., Evans, A. C., & Beck, A. T. (2009). The Beck initiative: A partnership to implement cognitive therapy in a community behavioral health system. Psychiatric Services, 60, 1302–1310. doi:10.1176/appi.ps.60.10.1302.CrossRefPubMedPubMedCentral

Stirman, S. W., Crits-Christoph, P., & DeRubeis, R. J. (2004). Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clinical Psychology: Science and Practice, 11(4), 343–359. doi:10.1093/clipsy/bph091.

Stirman, S. W., Gutierrez-Colina, A., Toder, K., Castro, F., Esposito, G., Barg, F., et al. (2013). Clinicians’ perspectives on cognitive therapy in community mental health settings: Implications for training and implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40(4), 274–285. doi:10.1007/s10488-012-0418-8.CrossRefPubMedPubMedCentral

Stirman, S. W., Miller, C. J., Toder, K., Calloway, A., Crits-Christoph, P., & Beck, A. T. (2012). Perspectives on cognitive therapy training within community mental health settings: Implications for clinician satisfaction and skill development. Depression Research and Treatment, 2012, 1–11. doi:10.1155/2012/391084.CrossRef

Swales, M. A., Taylor, B., & Hibbs, R. A. (2012). Implementing dialectical behaviour therapy: Programme survival in routine healthcare settings. Journal of Mental Health, 21(6), 548–555.CrossRefPubMed

Tracey, T. J., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69(3), 218.CrossRefPubMed

Weissman, M. M., Verdeli, H., Gameroff, M. J., Bledsoe, S. E., Betts, K., Mufson, L., et al. (2006). National survey of psychotherapy training in psychiatry, psychology, and social work. Archives of General Psychiatry, 63(8), 925–934.CrossRefPubMed

Young, J., & Beck, A. T. (1980). Cognitive therapy scale: Rating manual. Unpublished manuscript.

Title: A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy
Authors: Shannon Wiltsey Stirman
Kristin Pontoski
Torrey Creed
Regina Xhezo
Arthur C. Evans
Aaron T. Beck
Paul Crits-Christoph
Publication date: 01-01-2017
Publisher: Springer US
Published in: Administration and Policy in Mental Health and Mental Health Services Research / Issue 1/2017
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-015-0700-7

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy

Abstract

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2017

Introduction to the Special Issue: Legislation Related to Children’s Evidence-Based Practice

Critical Time Intervention for People Leaving Shelters in the Netherlands: Assessing Fidelity and Exploring Facilitators and Barriers

The Consistencies and Vagaries of the Washington State Inventory of Evidence-Based Practice: The Definition of “Evidence-Based” in a Policy Context

Developing a Quality Assurance System for Multiple Evidence Based Practices in a Statewide Service Improvement Initiative

What Factors are Associated with Consumer Initiation of Shared Decision Making in Mental Health Visits?

Community Engagement Strategies for Implementation of a Policy Supporting Evidence-Based Practices: A Case Study of Washington State