Skip to main content
Top
Published in: Administration and Policy in Mental Health and Mental Health Services Research 5/2017

01-09-2017 | Original Paper

Benchmarking Family Therapy for Adolescent Behavior Problems in Usual Care: Fidelity, Outcomes, and Therapist Performance Differences

Authors: Aaron Hogue, Sarah Dauber, Craig E. Henderson

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 5/2017

Login to get access

Abstract

This study evaluated whether community therapists delivering family therapy for adolescent behavior problems in usual care achieved performance benchmarks established in controlled trials for treatment fidelity and outcomes, with particular focus on individual differences in therapist performance. The study contained N = 38 adolescents (50 % male; mean age 15 years) whose self-reported race/ethnicity was Hispanic (74 %), African American (11 %), multiracial (11 %), and other (4 %). Clients were treated by 13 therapists in one community mental health clinic that delivered family therapy as the routine standard of care. Therapists provided self-report data on adherence to core family therapy techniques; these scores were inflation-adjusted based on concordance with observer reports. Results showed that community therapists surpassed the fidelity benchmark for core family therapy techniques established by research therapists during a controlled trial. Regarding change in client functioning at 6-month follow-up, community therapists were equivalent to the benchmark for internalizing symptoms and superior for externalizing symptoms and delinquent acts. Community therapists also demonstrated a high degree of performance uniformity: Each one approximated the fidelity benchmark, and only two produced relatively weak outcomes on any of the client change indicators. Caveats for interpreting therapist performance data, given the small sample size, are described. Recommendations are made for developing therapist-report fidelity measures and utilizing statistical process control methods to diagnose therapist differences and enhance quality assurance procedures.
Literature
go back to reference Achenbach, T. M., & Rescorla, L. A. (2001). ASEBA School Age Forms and Profiles. Burlington, Vt.: ASEBA. Achenbach, T. M., & Rescorla, L. A. (2001). ASEBA School Age Forms and Profiles. Burlington, Vt.: ASEBA.
go back to reference Adelson, J. L., & Owen, J. (2012). Bringing the psychotherapist back: Basic concepts for reading articles examining therapist effects using multilevel modeling. Psychotherapy, 49, 152.CrossRefPubMed Adelson, J. L., & Owen, J. (2012). Bringing the psychotherapist back: Basic concepts for reading articles examining therapist effects using multilevel modeling. Psychotherapy, 49, 152.CrossRefPubMed
go back to reference American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th edn., text revision). Washington, DC: APA. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th edn., text revision). Washington, DC: APA.
go back to reference Baldwin, S. A., Christian, S., Berkeljon, A., Shadish, W., & Bean, R. (2012). The effects of family therapies for adolescent delinquency and substance abuse: A meta-analysis. Journal of Marital and Family Therapy, 38, 281–304.CrossRefPubMed Baldwin, S. A., Christian, S., Berkeljon, A., Shadish, W., & Bean, R. (2012). The effects of family therapies for adolescent delinquency and substance abuse: A meta-analysis. Journal of Marital and Family Therapy, 38, 281–304.CrossRefPubMed
go back to reference Barber, J. P., Sharpless, B., Klostermann, S., & McCarthy, K. S. (2007). Assessing intervention competence and its relation to therapy outcome: A selected review derived from the outcome literature. Professional Psychology: Research and Practice, 38, 493–500.CrossRef Barber, J. P., Sharpless, B., Klostermann, S., & McCarthy, K. S. (2007). Assessing intervention competence and its relation to therapy outcome: A selected review derived from the outcome literature. Professional Psychology: Research and Practice, 38, 493–500.CrossRef
go back to reference Barth, R. P., Kolivoski, K. M., Lindsey, M. A., Lee, B. R., & Collins, K. S. (2014). Translating the common elements approach: Social work’s experiences in education, practice, and research. Journal of Clinical Child & Adolescent Psychology, 43(2), 301–311.CrossRef Barth, R. P., Kolivoski, K. M., Lindsey, M. A., Lee, B. R., & Collins, K. S. (2014). Translating the common elements approach: Social work’s experiences in education, practice, and research. Journal of Clinical Child & Adolescent Psychology, 43(2), 301–311.CrossRef
go back to reference Bearman, S. K., Weisz, J. R., Chorpita, B. F., Hoagwood, K., Ward, A., & Research Network on Youth Mental Health. (2013). More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40, 518–529.CrossRefPubMed Bearman, S. K., Weisz, J. R., Chorpita, B. F., Hoagwood, K., Ward, A., & Research Network on Youth Mental Health. (2013). More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40, 518–529.CrossRefPubMed
go back to reference Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30. Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30.
go back to reference Blatt, S. J., Sanislow, C. A. III, Zuroff, D. C., & Pilkonis, P. A. (1996). Characteristics of effective therapists: Further analyses of data from the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical psychology, 64, 1276.CrossRefPubMed Blatt, S. J., Sanislow, C. A. III, Zuroff, D. C., & Pilkonis, P. A. (1996). Characteristics of effective therapists: Further analyses of data from the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical psychology, 64, 1276.CrossRefPubMed
go back to reference Bond, G. R., Becker, D. R., & Drake, R. E. (2011). Measurement of fidelity of implementation of evidence-based practices: Case example of the IPS Fidelity Scale. Clinical Psychology: Science and Practice, 18, 126–141. Bond, G. R., Becker, D. R., & Drake, R. E. (2011). Measurement of fidelity of implementation of evidence-based practices: Case example of the IPS Fidelity Scale. Clinical Psychology: Science and Practice, 18, 126–141.
go back to reference Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self-evaluation of cognitive therapy performance: Do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36, 581–587.CrossRef Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self-evaluation of cognitive therapy performance: Do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36, 581–587.CrossRef
go back to reference Callahan, C. D., & Barisa, M. T. (2005). Statistical process control and rehabilitation outcome: The single-subject design reconsidered. Rehabilitation Psychology, 50, 24–33.CrossRef Callahan, C. D., & Barisa, M. T. (2005). Statistical process control and rehabilitation outcome: The single-subject design reconsidered. Rehabilitation Psychology, 50, 24–33.CrossRef
go back to reference Carroll, K. M., Nich, C., & Rounsaville, B. J. (1998). Utility of therapist session checklists to monitor delivery of coping skills treatment for cocaine abusers. Psychotherapy Research, 8, 307–320.CrossRef Carroll, K. M., Nich, C., & Rounsaville, B. J. (1998). Utility of therapist session checklists to monitor delivery of coping skills treatment for cocaine abusers. Psychotherapy Research, 8, 307–320.CrossRef
go back to reference Carroll, K. M., Nich, C., Sifry, R., Nuro, K. F., Frankforter, T. L., et al. (2000). A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug & Alcohol Dependence, 57, 225–238.CrossRef Carroll, K. M., Nich, C., Sifry, R., Nuro, K. F., Frankforter, T. L., et al. (2000). A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug & Alcohol Dependence, 57, 225–238.CrossRef
go back to reference Chorpita, B., Daleiden, E., Ebesutani, C., Young, J., Becker, K., Nakamura, B., et al (2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 154–172. Chorpita, B., Daleiden, E., Ebesutani, C., Young, J., Becker, K., Nakamura, B., et al (2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 154–172.
go back to reference Chorpita, B. F., Becker, K. D., & Daleiden, E. L. (2007). Understanding the common elements of evidence-based practice: Misconceptions and clinical examples. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 647–652.CrossRef Chorpita, B. F., Becker, K. D., & Daleiden, E. L. (2007). Understanding the common elements of evidence-based practice: Misconceptions and clinical examples. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 647–652.CrossRef
go back to reference Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35, 114–123.CrossRefPubMed Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35, 114–123.CrossRefPubMed
go back to reference Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: Erlbaum. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: Erlbaum.
go back to reference Crits-Christoph, P., Gibbons, M. B. C., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011). The dependability of alliance assessments: The alliance-outcome correlation is larger than you might think. Journal of Consulting and Clinical Psychology, 79, 267–278.CrossRefPubMedPubMedCentral Crits-Christoph, P., Gibbons, M. B. C., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011). The dependability of alliance assessments: The alliance-outcome correlation is larger than you might think. Journal of Consulting and Clinical Psychology, 79, 267–278.CrossRefPubMedPubMedCentral
go back to reference Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20.CrossRefPubMed Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20.CrossRefPubMed
go back to reference Curran, P. J., & Hussong, A. M. (2009). Integrative data analysis: The simultaneous analysis of multiple data sets. Psychological Methods, 14, 81–100.CrossRefPubMedPubMedCentral Curran, P. J., & Hussong, A. M. (2009). Integrative data analysis: The simultaneous analysis of multiple data sets. Psychological Methods, 14, 81–100.CrossRefPubMedPubMedCentral
go back to reference Curtis, N. M., Ronan, K. R., Heiblum, N., & Crellin, K. (2009). Dissemination and effectiveness of multisystemic treatment in New Zealand: a benchmarking study. Journal of Family Psychology, 23, 119.CrossRefPubMed Curtis, N. M., Ronan, K. R., Heiblum, N., & Crellin, K. (2009). Dissemination and effectiveness of multisystemic treatment in New Zealand: a benchmarking study. Journal of Family Psychology, 23, 119.CrossRefPubMed
go back to reference Delgadillo, J., McMillan, D., Leach, C., Lucock, M., Gilbody, S., & Wood, N. (2014). Benchmarking routine psychological services: a discussion of challenges and methods. Behavioural and Cognitive Psychotherapy, 42, 16–30.CrossRefPubMed Delgadillo, J., McMillan, D., Leach, C., Lucock, M., Gilbody, S., & Wood, N. (2014). Benchmarking routine psychological services: a discussion of challenges and methods. Behavioural and Cognitive Psychotherapy, 42, 16–30.CrossRefPubMed
go back to reference Deming, W. E. (1986). Out of the Crisis. Cambridge, MA: MIT Press. Deming, W. E. (1986). Out of the Crisis. Cambridge, MA: MIT Press.
go back to reference Dey, M. L., Sluyter, G. V., & Keating, J. E. (1994). Statistical process control and direct care staff performance. Journal of Mental Health Administration, 21, 201–209.CrossRefPubMed Dey, M. L., Sluyter, G. V., & Keating, J. E. (1994). Statistical process control and direct care staff performance. Journal of Mental Health Administration, 21, 201–209.CrossRefPubMed
go back to reference Elkin, I. (1999). A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clinical Psychology: Science and Practice, 6, 10–32. Elkin, I. (1999). A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clinical Psychology: Science and Practice, 6, 10–32.
go back to reference Elliott, D. S., Huizinga, D., & Ageton, S. S. (1985). Explaining delinquency and drug use. Beverly Hills, CA: Sage Publications. Elliott, D. S., Huizinga, D., & Ageton, S. S. (1985). Explaining delinquency and drug use. Beverly Hills, CA: Sage Publications.
go back to reference Erickson, S. J., Tonigan, J. S., & Winhusen, T. (2012). Therapist effects in a NIDA CTN intervention trial with pregnant substance abusing women: Findings from a RCT with MET and TAU conditions. Alcoholism Treatment Quarterly, 30, 224–237.CrossRef Erickson, S. J., Tonigan, J. S., & Winhusen, T. (2012). Therapist effects in a NIDA CTN intervention trial with pregnant substance abusing women: Findings from a RCT with MET and TAU conditions. Alcoholism Treatment Quarterly, 30, 224–237.CrossRef
go back to reference Fals-Stewart, W., & Birchler, G. R. (2002). Behavioral couples therapy with alcoholic men and their intimate partners: The comparative effectiveness of bachelor’s and master’s level counselors. Behavior Therapy, 33, 123–147.CrossRef Fals-Stewart, W., & Birchler, G. R. (2002). Behavioral couples therapy with alcoholic men and their intimate partners: The comparative effectiveness of bachelor’s and master’s level counselors. Behavior Therapy, 33, 123–147.CrossRef
go back to reference Flicker, S. M., Waldron, H. B., Turner, C. W., Brody, J. L., & Hops, H. (2008). Ethnic matching and treatment outcome with Hispanic and Anglo substance-abusing adolescents in family therapy. Journal of Family Psychology, 22, 439.CrossRefPubMed Flicker, S. M., Waldron, H. B., Turner, C. W., Brody, J. L., & Hops, H. (2008). Ethnic matching and treatment outcome with Hispanic and Anglo substance-abusing adolescents in family therapy. Journal of Family Psychology, 22, 439.CrossRefPubMed
go back to reference Garland, A. F., Bickman, L., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research, 37, 15–26.CrossRefPubMed Garland, A. F., Bickman, L., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research, 37, 15–26.CrossRefPubMed
go back to reference Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health and Mental Health Services Research, 40, 6–22.CrossRefPubMed Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health and Mental Health Services Research, 40, 6–22.CrossRefPubMed
go back to reference Garland, A. F., Hawley, K. M., Brookman-Frazee, L. I., & Hurlburt, M. (2008). Identifying common, core elements of evidence-based practice for children with disruptive behavior disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 47, 505–514.CrossRef Garland, A. F., Hawley, K. M., Brookman-Frazee, L. I., & Hurlburt, M. (2008). Identifying common, core elements of evidence-based practice for children with disruptive behavior disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 47, 505–514.CrossRef
go back to reference Green, R. S. (1999). The application of statistical process control to manage global client outcomes in behavioral healthcare. Evaluation and Program Planning, 22, 199–210.CrossRefPubMed Green, R. S. (1999). The application of statistical process control to manage global client outcomes in behavioral healthcare. Evaluation and Program Planning, 22, 199–210.CrossRefPubMed
go back to reference Haley, J. (1987). Problem-solving therapy. San Francisco: Jossey-Bass. Haley, J. (1987). Problem-solving therapy. San Francisco: Jossey-Bass.
go back to reference Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic.
go back to reference Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999). Multisystemic treatment of substance-abusing and-dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research, 1, 171–184.CrossRefPubMed Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999). Multisystemic treatment of substance-abusing and-dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research, 1, 171–184.CrossRefPubMed
go back to reference Henggeler, S. W., & Sheidow, A. J. (2012). Empirically supported family-based treatments for conduct disorder and delinquency in adolescents. Journal of Marital and Family Therapy, 38, 30–58.CrossRefPubMed Henggeler, S. W., & Sheidow, A. J. (2012). Empirically supported family-based treatments for conduct disorder and delinquency in adolescents. Journal of Marital and Family Therapy, 38, 30–58.CrossRefPubMed
go back to reference Hill, C. E. (1991). Almost everything you ever wanted to know about how to do process research on counseling and psychotherapy but didn’t know who to ask. In C. E. Hill & L. J. Schneider (Eds.), Research in Counseling (pp. 85–118). Hillsdale, NJ: Erlbaum. Hill, C. E. (1991). Almost everything you ever wanted to know about how to do process research on counseling and psychotherapy but didn’t know who to ask. In C. E. Hill & L. J. Schneider (Eds.), Research in Counseling (pp. 85–118). Hillsdale, NJ: Erlbaum.
go back to reference Hill, C. E., O’Grady, K. E., & Elkin, I. (1992). Applying the collaborative study psychotherapy rating scale to rate therapist adherence in cognitive-behavior therapy, interpersonal therapy, and clinical management. Journal of Consulting and Clinical Psychology, 60, 73–79.CrossRefPubMed Hill, C. E., O’Grady, K. E., & Elkin, I. (1992). Applying the collaborative study psychotherapy rating scale to rate therapist adherence in cognitive-behavior therapy, interpersonal therapy, and clinical management. Journal of Consulting and Clinical Psychology, 60, 73–79.CrossRefPubMed
go back to reference Hoagwood, K. E. (2005). Family-based services in children’s mental health: A research review and synthesis. Journal of Child Psychology and Psychiatry, 46, 690–713.CrossRefPubMed Hoagwood, K. E. (2005). Family-based services in children’s mental health: A research review and synthesis. Journal of Child Psychology and Psychiatry, 46, 690–713.CrossRefPubMed
go back to reference Hoagwood, K. E. (2013). Don’t mourn: Organize. Reviving mental health services research for healthcare quality improvement. Clinical Psychology: Science and Practice, 20, 120–126. Hoagwood, K. E. (2013). Don’t mourn: Organize. Reviving mental health services research for healthcare quality improvement. Clinical Psychology: Science and Practice, 20, 120–126.
go back to reference Hogue, A., & Dauber, S. (2013). Assessing fidelity to evidence-based practices in usual care: The example of family therapy for adolescent behavior problems. Evaluation and Program Planning, 37, 21–30.CrossRefPubMed Hogue, A., & Dauber, S. (2013). Assessing fidelity to evidence-based practices in usual care: The example of family therapy for adolescent behavior problems. Evaluation and Program Planning, 37, 21–30.CrossRefPubMed
go back to reference Hogue, A., Dauber, S., Chinchilla, P., Fried, A., Henderson, C. E., et al. (2008). Assessing fidelity in individual and family therapy for adolescent substance abuse. Journal of Substance Abuse Treatment, 35, 137–147.CrossRefPubMed Hogue, A., Dauber, S., Chinchilla, P., Fried, A., Henderson, C. E., et al. (2008). Assessing fidelity in individual and family therapy for adolescent substance abuse. Journal of Substance Abuse Treatment, 35, 137–147.CrossRefPubMed
go back to reference Hogue, A., Dauber, S., & Henderson, C. E. (2014). Therapist self-report of evidence-based practices in usual care for adolescent behavior problems: Factor and construct validity. Administration and Policy in Mental Health and Mental Health Services Research, 41, 126–139.CrossRefPubMed Hogue, A., Dauber, S., & Henderson, C. E. (2014). Therapist self-report of evidence-based practices in usual care for adolescent behavior problems: Factor and construct validity. Administration and Policy in Mental Health and Mental Health Services Research, 41, 126–139.CrossRefPubMed
go back to reference Hogue, A., Dauber, S., Henderson, C. E., Bobek, M., Johnson, C., Lichvar, E., & Morgenstern, J. (2015). Randomized trial of family therapy versus non-family treatment for adolescent behavior problems in usual care. Journal of Clinical Child and Adolescent Psychology, 44, 954–969.CrossRefPubMed Hogue, A., Dauber, S., Henderson, C. E., Bobek, M., Johnson, C., Lichvar, E., & Morgenstern, J. (2015). Randomized trial of family therapy versus non-family treatment for adolescent behavior problems in usual care. Journal of Clinical Child and Adolescent Psychology, 44, 954–969.CrossRefPubMed
go back to reference Hogue, A., Dauber, S., Henderson, C. E., & Liddle, H. A. (2014). Reliability of therapist self-report on treatment targets and focus in family-based intervention. Administration and Policy in Mental Health and Mental Health Services Research, 41, 697–705.CrossRefPubMed Hogue, A., Dauber, S., Henderson, C. E., & Liddle, H. A. (2014). Reliability of therapist self-report on treatment targets and focus in family-based intervention. Administration and Policy in Mental Health and Mental Health Services Research, 41, 697–705.CrossRefPubMed
go back to reference Hogue, A., Dauber, S., Lichvar, E., Bobek, M., & Henderson, C. E. (2015). Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: Correspondence between therapists and observers. Administration and Policy in Mental Health and Mental Health Services Research, 42, 229–243.CrossRefPubMed Hogue, A., Dauber, S., Lichvar, E., Bobek, M., & Henderson, C. E. (2015). Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: Correspondence between therapists and observers. Administration and Policy in Mental Health and Mental Health Services Research, 42, 229–243.CrossRefPubMed
go back to reference Hogue, A., Dauber, S., Samuolis, J., & Liddle, H. A. (2006). Treatment techniques and outcomes in multidimensional family therapy for adolescent behavior problems. Journal of Family Psychology, 20, 535–543.CrossRefPubMedPubMedCentral Hogue, A., Dauber, S., Samuolis, J., & Liddle, H. A. (2006). Treatment techniques and outcomes in multidimensional family therapy for adolescent behavior problems. Journal of Family Psychology, 20, 535–543.CrossRefPubMedPubMedCentral
go back to reference Hogue, A., Henderson, C. E., Ozechowski, T. J., & Robbins, M. S. (2014). Evidence base on outpatient behavioral treatments for adolescent substance use: Updates and recommendations 2007–2013. Journal of Clinical Child and Adolescent Psychology, 43, 697–720.CrossRef Hogue, A., Henderson, C. E., Ozechowski, T. J., & Robbins, M. S. (2014). Evidence base on outpatient behavioral treatments for adolescent substance use: Updates and recommendations 2007–2013. Journal of Clinical Child and Adolescent Psychology, 43, 697–720.CrossRef
go back to reference Hogue, A., Henderson, C. E., & Schmidt, A. T. (2016). Multidimensional predictors of treatment outcome in usual care for adolescent conduct problems and substance use. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-016-0724-7.CrossRef Hogue, A., Henderson, C. E., & Schmidt, A. T. (2016). Multidimensional predictors of treatment outcome in usual care for adolescent conduct problems and substance use. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.​1007/​s10488-016-0724-7.CrossRef
go back to reference Hogue, A., & Liddle, H. A. (2009). Family-based treatment for adolescent substance abuse: Controlled trials and new horizons in services research. Journal of Family Therapy, 31, 126–154.CrossRefPubMedPubMedCentral Hogue, A., & Liddle, H. A. (2009). Family-based treatment for adolescent substance abuse: Controlled trials and new horizons in services research. Journal of Family Therapy, 31, 126–154.CrossRefPubMedPubMedCentral
go back to reference Hogue, A., Ozechowski, T. J., Robbins, M. R., & Waldron, H. B. (2013). Making fidelity an intramural game: Localizing quality assurance procedures to promote sustainability of evidence-based practices in usual care. Clinical Psychology: Science and Practice, 20, 60–77. Hogue, A., Ozechowski, T. J., Robbins, M. R., & Waldron, H. B. (2013). Making fidelity an intramural game: Localizing quality assurance procedures to promote sustainability of evidence-based practices in usual care. Clinical Psychology: Science and Practice, 20, 60–77.
go back to reference Hoyer, R. W., & Ellis, W. C. (1996). A graphical exploration of SPC: Part 1-SPC’s definitions and procedures. Quality Progress, 29, 65–72. Hoyer, R. W., & Ellis, W. C. (1996). A graphical exploration of SPC: Part 1-SPC’s definitions and procedures. Quality Progress, 29, 65–72.
go back to reference Hunsley, J., & Lee, C. M. (2007). Research-informed benchmarks for psychological treatments: Efficacy studies, effectiveness studies, and beyond. Professional Psychology: Research and Practice, 38, 21.CrossRef Hunsley, J., & Lee, C. M. (2007). Research-informed benchmarks for psychological treatments: Efficacy studies, effectiveness studies, and beyond. Professional Psychology: Research and Practice, 38, 21.CrossRef
go back to reference Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37, 230–244.CrossRefPubMed Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37, 230–244.CrossRefPubMed
go back to reference Institute of Medicine. (2006). Improving the quality of healthcare for mental and substance-use conditions. Washington, D.C.: National Academy Press. Institute of Medicine. (2006). Improving the quality of healthcare for mental and substance-use conditions. Washington, D.C.: National Academy Press.
go back to reference Laska, K. M., Smith, T. L., Wislocki, A. P., Minami, T., & Wampold, B. E. (2013). Uniformity of evidence-based treatments in practice? Therapist effects in the delivery of cognitive processing therapy for PTSD. Journal of Counseling Psychology, 60, 31.CrossRefPubMed Laska, K. M., Smith, T. L., Wislocki, A. P., Minami, T., & Wampold, B. E. (2013). Uniformity of evidence-based treatments in practice? Therapist effects in the delivery of cognitive processing therapy for PTSD. Journal of Counseling Psychology, 60, 31.CrossRefPubMed
go back to reference Löfholm, C. A., Eichas, K., & Sundell, K. (2014). The Swedish implementation of multisystemic therapy for adolescents: Does treatment experience predict treatment adherence? Journal of Clinical Child & Adolescent Psychology, 43, 643–655.CrossRef Löfholm, C. A., Eichas, K., & Sundell, K. (2014). The Swedish implementation of multisystemic therapy for adolescents: Does treatment experience predict treatment adherence? Journal of Clinical Child & Adolescent Psychology, 43, 643–655.CrossRef
go back to reference Martino, S., Ball, S., Nich, C., Frankforter, T. L., & Carroll, K. M. (2009). Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychotherapy Research, 19, 181–193.CrossRefPubMedPubMedCentral Martino, S., Ball, S., Nich, C., Frankforter, T. L., & Carroll, K. M. (2009). Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychotherapy Research, 19, 181–193.CrossRefPubMedPubMedCentral
go back to reference McHugh, R., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatment: A review of current efforts. American Psychologist, 65, 73–84.CrossRefPubMed McHugh, R., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatment: A review of current efforts. American Psychologist, 65, 73–84.CrossRefPubMed
go back to reference McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodríguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20, 14–32. McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodríguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20, 14–32.
go back to reference Minami, T., Serlin, R. C., Wampold, B. E., Kircher, J. C., & Brown, G. S. (2008). Using clinical trials to benchmark effects produced in clinical practice. Quality and Quantity, 42, 513–525.CrossRef Minami, T., Serlin, R. C., Wampold, B. E., Kircher, J. C., & Brown, G. S. (2008). Using clinical trials to benchmark effects produced in clinical practice. Quality and Quantity, 42, 513–525.CrossRef
go back to reference Minami, T., Wampold, B. E., Serlin, R. C., Hamilton, E. G., Brown, G. S., & Kircher, J. C. (2008). Benchmarking the effectiveness of psychotherapy treatment for adult depression in a managed care environment: A preliminary study. Journal of Consulting and Clinical Psychology, 76, 116–124.CrossRefPubMed Minami, T., Wampold, B. E., Serlin, R. C., Hamilton, E. G., Brown, G. S., & Kircher, J. C. (2008). Benchmarking the effectiveness of psychotherapy treatment for adult depression in a managed care environment: A preliminary study. Journal of Consulting and Clinical Psychology, 76, 116–124.CrossRefPubMed
go back to reference Minuchin, S., & Fishman, H. C. (1981). Family therapy techniques. Cambridge, MA: Harvard University Press. Minuchin, S., & Fishman, H. C. (1981). Family therapy techniques. Cambridge, MA: Harvard University Press.
go back to reference Muthen, B., & Muthen, L. (2012). Mplus User’s Guide. Los Angeles, CA: Muthen & Muthen. Muthen, B., & Muthen, L. (2012). Mplus User’s Guide. Los Angeles, CA: Muthen & Muthen.
go back to reference Noyez, L. (2009). Control charts, cusum techniques, and funnel plots: A review of methods for monitoring performance in healthcare. Interactive Cardiovascular and Thoracic Surgery, 9, 494–499.CrossRefPubMed Noyez, L. (2009). Control charts, cusum techniques, and funnel plots: A review of methods for monitoring performance in healthcare. Interactive Cardiovascular and Thoracic Surgery, 9, 494–499.CrossRefPubMed
go back to reference Ozechowski, T. J., & Waldron, H. B. (2010). Assertive outreach strategies for narrowing the adolescent substance abuse treatment gap: Implications for research, practice, and policy. Journal of Behavioral Health Services & Research, 37, 40–63.CrossRef Ozechowski, T. J., & Waldron, H. B. (2010). Assertive outreach strategies for narrowing the adolescent substance abuse treatment gap: Implications for research, practice, and policy. Journal of Behavioral Health Services & Research, 37, 40–63.CrossRef
go back to reference Project MATCH Research Group (1998). Therapist effects in three treatments for alcohol problems. Psychotherapy Research, 8, 455–474.CrossRef Project MATCH Research Group (1998). Therapist effects in three treatments for alcohol problems. Psychotherapy Research, 8, 455–474.CrossRef
go back to reference Schoenwald, S. K. (2011). It’s a bird, It’s a plane, It’s… fidelity measurement in the real world. Clinical Psychology: Science and Practice, 18, 142–147. Schoenwald, S. K. (2011). It’s a bird, It’s a plane, It’s… fidelity measurement in the real world. Clinical Psychology: Science and Practice, 18, 142–147.
go back to reference Schoenwald, S. K., Carter, R. E., Chapman, J. E., & Sheidow, A. J. (2008). Therapist adherence and organizational effects on change in youth behavior problems one year after multisystemic therapy. Administration and Policy in Mental Health and Mental Health Services Research, 35, 379–394.CrossRefPubMed Schoenwald, S. K., Carter, R. E., Chapman, J. E., & Sheidow, A. J. (2008). Therapist adherence and organizational effects on change in youth behavior problems one year after multisystemic therapy. Administration and Policy in Mental Health and Mental Health Services Research, 35, 379–394.CrossRefPubMed
go back to reference Schoenwald, S. K., Chapman, J. E., Sheidow, A. J., & Carter, R. E. (2009). Long-term youth criminal outcomes in MST transport: The impact of therapist adherence and organizational climate and structure. Journal of Clinical Child & Adolescent Psychology, 38, 91–105.CrossRef Schoenwald, S. K., Chapman, J. E., Sheidow, A. J., & Carter, R. E. (2009). Long-term youth criminal outcomes in MST transport: The impact of therapist adherence and organizational climate and structure. Journal of Clinical Child & Adolescent Psychology, 38, 91–105.CrossRef
go back to reference Schoenwald, S. K., Henggeler, S. W., Brondino, M. J., & Rowland, M. D. (2000). Multisystemic therapy: Monitoring treatment fidelity. Family Process, 39, 83–103.CrossRefPubMed Schoenwald, S. K., Henggeler, S. W., Brondino, M. J., & Rowland, M. D. (2000). Multisystemic therapy: Monitoring treatment fidelity. Family Process, 39, 83–103.CrossRefPubMed
go back to reference Schoenwald, S. K., Letourneau, E. J., & Halliday-Boykins, C. (2005). Predicting therapist adherence to a transported family-based treatment for youth. Journal of Clinical Child and Adolescent Psychology, 34(4), 658–670.CrossRefPubMed Schoenwald, S. K., Letourneau, E. J., & Halliday-Boykins, C. (2005). Predicting therapist adherence to a transported family-based treatment for youth. Journal of Clinical Child and Adolescent Psychology, 34(4), 658–670.CrossRefPubMed
go back to reference Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., et al. (1998). The Mini International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59, 22–33.PubMed Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., et al. (1998). The Mini International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59, 22–33.PubMed
go back to reference Southam-Gerow, M. A., McLeod, B. D., Arnold, C. C., Rodríguez, A., Cox, J. R., Reise, S. P., … & Kendall, P. C., et al. (2016). Initial development of a treatment adherence measure for cognitive–behavioral therapy for child anxiety. Psychological Assessment, 28, 70–80.CrossRefPubMed Southam-Gerow, M. A., McLeod, B. D., Arnold, C. C., Rodríguez, A., Cox, J. R., Reise, S. P., … & Kendall, P. C., et al. (2016). Initial development of a treatment adherence measure for cognitive–behavioral therapy for child anxiety. Psychological Assessment, 28, 70–80.CrossRefPubMed
go back to reference Spilka, M. J., & Dobson, K. S. (2015). Promoting the internationalization of evidence-based practice: Benchmarking as a strategy to evaluate culturally transported psychological treatments. Clinical Psychology: Science and Practice, 22, 58–75. Spilka, M. J., & Dobson, K. S. (2015). Promoting the internationalization of evidence-based practice: Benchmarking as a strategy to evaluate culturally transported psychological treatments. Clinical Psychology: Science and Practice, 22, 58–75.
go back to reference Tanner-Smith, E. E., Wilson, S. J., & Lipsey, M. W. (2012). The comparative effectiveness of outpatient treatment for adolescent substance abuse: A meta-analysis. Journal of Substance Abuse Treatment, 44, 145–158.CrossRefPubMedPubMedCentral Tanner-Smith, E. E., Wilson, S. J., & Lipsey, M. W. (2012). The comparative effectiveness of outpatient treatment for adolescent substance abuse: A meta-analysis. Journal of Substance Abuse Treatment, 44, 145–158.CrossRefPubMedPubMedCentral
go back to reference Wampold, B. E., & Serlin, R. C. (2000). The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods, 5, 425.CrossRefPubMed Wampold, B. E., & Serlin, R. C. (2000). The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods, 5, 425.CrossRefPubMed
go back to reference Weersing, R. V., & Weisz, J. R. (2002). Community clinic treatment of depressed youth: Benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology, 70, 299–310.CrossRefPubMed Weersing, R. V., & Weisz, J. R. (2002). Community clinic treatment of depressed youth: Benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology, 70, 299–310.CrossRefPubMed
go back to reference Weersing, R. V., Weisz, J. R., & Donenberg, G. R. (2002). Development of the Therapy Procedures Checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child Psychology, 31, 168–180.CrossRef Weersing, R. V., Weisz, J. R., & Donenberg, G. R. (2002). Development of the Therapy Procedures Checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child Psychology, 31, 168–180.CrossRef
go back to reference Weersing, V. R. (2005). Benchmarking the effectiveness of psychotherapy: Program evaluation as a component of evidence-based practice. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 1058–1062.CrossRef Weersing, V. R. (2005). Benchmarking the effectiveness of psychotherapy: Program evaluation as a component of evidence-based practice. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 1058–1062.CrossRef
go back to reference Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., et al. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69, 274–282.CrossRefPubMed Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., et al. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69, 274–282.CrossRefPubMed
go back to reference Wiborg, J. F., Knoop, H., Wensing, M., & Bleijenberg, G. (2012). Therapist effects and the dissemination of cognitive behavior therapy for chronic fatigue syndrome in community-based mental health care. Behaviour Research and Therapy, 50, 393–396.CrossRefPubMed Wiborg, J. F., Knoop, H., Wensing, M., & Bleijenberg, G. (2012). Therapist effects and the dissemination of cognitive behavior therapy for chronic fatigue syndrome in community-based mental health care. Behaviour Research and Therapy, 50, 393–396.CrossRefPubMed
go back to reference Zucker, D. M. (1990). An analysis of variance pitfall: The fixed effects analysis in a nested design. Educational and Psychological Measurement, 50, 731–738.CrossRef Zucker, D. M. (1990). An analysis of variance pitfall: The fixed effects analysis in a nested design. Educational and Psychological Measurement, 50, 731–738.CrossRef
Metadata
Title
Benchmarking Family Therapy for Adolescent Behavior Problems in Usual Care: Fidelity, Outcomes, and Therapist Performance Differences
Authors
Aaron Hogue
Sarah Dauber
Craig E. Henderson
Publication date
01-09-2017
Publisher
Springer US
Published in
Administration and Policy in Mental Health and Mental Health Services Research / Issue 5/2017
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI
https://doi.org/10.1007/s10488-016-0769-7

Other articles of this Issue 5/2017

Administration and Policy in Mental Health and Mental Health Services Research 5/2017 Go to the issue