Skip to main content
Top
Published in: The Journal of Behavioral Health Services & Research 3/2018

01-07-2018

Measuring Treatment Integrity: Use of and Experience with Measurements in Child and Youth Care Organizations

Authors: Pauline Brigitta Goense, PhD, LLM, Leonieke Boendermaker, PhD, Tom van Yperen, PhD

Published in: The Journal of Behavioral Health Services & Research | Issue 3/2018

Login to get access

Abstract

Performance feedback and supervision are essential to the adoption of evidence-based interventions with high treatment integrity in child and youth care organizations. Little is known about the use of treatment integrity measurements in these organizations. For this study, 12 interventions for children and young people in the Netherlands with externalizing behavioral problems were selected. For each intervention, an expert, two supervisors, and two therapists were approached for an interview. In total, 54 semi-structured interviews were conducted. The results show that almost all interventions used treatment integrity instruments (N = 11, 91.7%). Only two used measurements for both QA procedures (certification and recertification) and supervision purposes. Therapists regard treatment integrity measurements as valuable when they are used for multiple purposes and feedback is provided. The results of this study suggest the feasibility of the use of measurements for multiple purposes. Collaborative action is required to develop instruments that effectively contribute to continuous improvement.
Literature
1.
go back to reference Fixsen DL, Ogden T. Facing the Challenges of Implementation. Zeitschrift für Psychologie. 2014;222(1):1–3.CrossRef Fixsen DL, Ogden T. Facing the Challenges of Implementation. Zeitschrift für Psychologie. 2014;222(1):1–3.CrossRef
2.
3.
go back to reference Schoenwald SK, Chapman JE, Sheidow AJ, et al. Long-Term Youth Criminal Outcomes in MST Transport: The Impact of Therapist Adherence and Organizational Climate and Structure. Journal of Clinical Child and Adolescent Psychology. 2009;38(1):91–105.CrossRefPubMedPubMedCentral Schoenwald SK, Chapman JE, Sheidow AJ, et al. Long-Term Youth Criminal Outcomes in MST Transport: The Impact of Therapist Adherence and Organizational Climate and Structure. Journal of Clinical Child and Adolescent Psychology. 2009;38(1):91–105.CrossRefPubMedPubMedCentral
4.
go back to reference Sexton T, Turner CW. The Effectiveness of Functional Family Therapy for Youth with Behavioral Problems in a Community Practice Setting. Journal of Family Psychology. 2010;24:339–348.CrossRefPubMedPubMedCentral Sexton T, Turner CW. The Effectiveness of Functional Family Therapy for Youth with Behavioral Problems in a Community Practice Setting. Journal of Family Psychology. 2010;24:339–348.CrossRefPubMedPubMedCentral
6.
go back to reference Schoenwald SK, Garland AF. A Review of Treatment Adherence Measurement Methods. Psychological Assessment. 2013;25(1):146–156.CrossRefPubMed Schoenwald SK, Garland AF. A Review of Treatment Adherence Measurement Methods. Psychological Assessment. 2013;25(1):146–156.CrossRefPubMed
7.
go back to reference Fixsen DL, Naoom SF, Blase KA, et al. Implementation Research: A Synthesis of the Literature. Tampa, FL: Louis de la Parte Florida Mental Health Institute, National Implementation Research Network, 2005. Fixsen DL, Naoom SF, Blase KA, et al. Implementation Research: A Synthesis of the Literature. Tampa, FL: Louis de la Parte Florida Mental Health Institute, National Implementation Research Network, 2005.
8.
go back to reference Garland AF, Schoenwald SK. Use of Effective and Efficient Quality Control Methods to Implement Psychosocial Interventions. Clinical Psychology: Science and Practice. 2013;20:33–43. Garland AF, Schoenwald SK. Use of Effective and Efficient Quality Control Methods to Implement Psychosocial Interventions. Clinical Psychology: Science and Practice. 2013;20:33–43.
9.
go back to reference Goense PB, Boendermaker L, van Yperen T. Support Systems for Treatment Integrity. Research on Social Work Practice. 2015;1–5. Goense PB, Boendermaker L, van Yperen T. Support Systems for Treatment Integrity. Research on Social Work Practice. 2015;1–5.
10.
go back to reference Hogue A, Ozechowski TJ, Robbins MS, et al. Making Fidelity an Intramural Game: Localizing Quality Assurance Procedures to Promote Sustainability of Evidence-Based Practices in Usual Care. Clinical Psychology: Science and Practice. 2013;20:60–77. Hogue A, Ozechowski TJ, Robbins MS, et al. Making Fidelity an Intramural Game: Localizing Quality Assurance Procedures to Promote Sustainability of Evidence-Based Practices in Usual Care. Clinical Psychology: Science and Practice. 2013;20:60–77.
11.
go back to reference Kerby NT (Ed). Helping Others Help Children; Clinical Supervision of Child Psychotherapy. Washington, DC: American Psychological Association, 2006. Kerby NT (Ed). Helping Others Help Children; Clinical Supervision of Child Psychotherapy. Washington, DC: American Psychological Association, 2006.
12.
go back to reference Miller WR, Sorensen JL, Selzer JA, et al. Disseminating Evidence Based Practices in Substance Abuse Treatment: A Review with Suggestions. Journal of Substance Abuse Treatment. 2006;31:25–39.CrossRefPubMed Miller WR, Sorensen JL, Selzer JA, et al. Disseminating Evidence Based Practices in Substance Abuse Treatment: A Review with Suggestions. Journal of Substance Abuse Treatment. 2006;31:25–39.CrossRefPubMed
13.
go back to reference Schoenwald SK, Sheidow AJ, Chapman JE. Clinical Supervision in Treatment Transport: Effects on Adherence and Outcomes. Journal of Consulting and Clinical Psychology. 2009;77(3):410–421.CrossRefPubMedPubMedCentral Schoenwald SK, Sheidow AJ, Chapman JE. Clinical Supervision in Treatment Transport: Effects on Adherence and Outcomes. Journal of Consulting and Clinical Psychology. 2009;77(3):410–421.CrossRefPubMedPubMedCentral
14.
go back to reference Perepletchikova F, Treat TA, Kazdin AE. Treatment Integrity in Psychotherapy Research: Analysis of the Studies and Examination of the Associated Factors. Journal of Consulting and Clinical Psychology. 2007;75(6):829–841.CrossRefPubMed Perepletchikova F, Treat TA, Kazdin AE. Treatment Integrity in Psychotherapy Research: Analysis of the Studies and Examination of the Associated Factors. Journal of Consulting and Clinical Psychology. 2007;75(6):829–841.CrossRefPubMed
15.
go back to reference Barber JP, Gallop R, Crits-Christoph P, et al. The role of Therapist Adherence, Therapist Competence, and Alliance in Predicting Outcome of Individual Drug Counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16(2):229–240.CrossRef Barber JP, Gallop R, Crits-Christoph P, et al. The role of Therapist Adherence, Therapist Competence, and Alliance in Predicting Outcome of Individual Drug Counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16(2):229–240.CrossRef
16.
go back to reference Barber JP, Sharpless BA, Klostermann S, et al. Assessing Intervention Competence and Its Relation to Therapy Outcome. Professional Psychology: Research and Practice. 2007;38(5):493–500.CrossRef Barber JP, Sharpless BA, Klostermann S, et al. Assessing Intervention Competence and Its Relation to Therapy Outcome. Professional Psychology: Research and Practice. 2007;38(5):493–500.CrossRef
17.
go back to reference Barber J, Triffleman E, Marmar C. Considerations in Treatment Integrity: Implications and Recommendations for PTSD Research. Journal of Traumatic Stress. 2007;20(5):793–805.CrossRefPubMed Barber J, Triffleman E, Marmar C. Considerations in Treatment Integrity: Implications and Recommendations for PTSD Research. Journal of Traumatic Stress. 2007;20(5):793–805.CrossRefPubMed
18.
go back to reference Goense P, Boendermaker L, Van Yperen T, et al. Implementation of Treatment Integrity Procedures: An Analysis of Outcome Studies of Youth Interventions Targeting Externalizing Behavioral Problems. Zeitschrift für Psychologie. 2014;222(1):12–21.CrossRef Goense P, Boendermaker L, Van Yperen T, et al. Implementation of Treatment Integrity Procedures: An Analysis of Outcome Studies of Youth Interventions Targeting Externalizing Behavioral Problems. Zeitschrift für Psychologie. 2014;222(1):12–21.CrossRef
19.
go back to reference McLeod BD, Southam-Gerow MA, Tully CB, et al. Making a Case for Treatment Integrity as a Psychosocial Treatment Quality Indicator for Youth Mental Health Care. Clinical Psychology. 2013;20(1):1–23. McLeod BD, Southam-Gerow MA, Tully CB, et al. Making a Case for Treatment Integrity as a Psychosocial Treatment Quality Indicator for Youth Mental Health Care. Clinical Psychology. 2013;20(1):1–23.
20.
go back to reference Schoenwald SK, Garland AF, Chapman JE, et al. Toward the Effective and Efficient Measurement of Implementation Fidelity. Administration and Policy in Mental Health. 2011;38:32–43.CrossRefPubMedPubMedCentral Schoenwald SK, Garland AF, Chapman JE, et al. Toward the Effective and Efficient Measurement of Implementation Fidelity. Administration and Policy in Mental Health. 2011;38:32–43.CrossRefPubMedPubMedCentral
21.
go back to reference Nápoles AM, Santoyo-Olsson J, Stewart AL. Methods for Translating Evidence-Based Behavioral Interventions for Health-Disparity Communities. Preventing Chronic Disease. 2013;10. Nápoles AM, Santoyo-Olsson J, Stewart AL. Methods for Translating Evidence-Based Behavioral Interventions for Health-Disparity Communities. Preventing Chronic Disease. 2013;10.
22.
go back to reference Knox LM, Aspy CB. Quality Improvement as a Tool for Translating Evidence-Based Interventions into Practice: What the Youth Violence Prevention Community Can Learn from Healthcare. American Journal of Community Psychology. 2011;48(1–2):56–64.CrossRefPubMed Knox LM, Aspy CB. Quality Improvement as a Tool for Translating Evidence-Based Interventions into Practice: What the Youth Violence Prevention Community Can Learn from Healthcare. American Journal of Community Psychology. 2011;48(1–2):56–64.CrossRefPubMed
23.
go back to reference Sutherland KS, McLeod BD, Conroy MA, et al. Measuring Implementation of Evidence-Based Programs Targeting Young Children at Risk for Emotional/Behavioral Disorders: Conceptual Issues and Recommendations. Journal of Early Intervention. 2013;35(2):129–149.CrossRef Sutherland KS, McLeod BD, Conroy MA, et al. Measuring Implementation of Evidence-Based Programs Targeting Young Children at Risk for Emotional/Behavioral Disorders: Conceptual Issues and Recommendations. Journal of Early Intervention. 2013;35(2):129–149.CrossRef
24.
go back to reference Kazdin A. Methodology, design, and evaluation in psychotherapy research. In: AE Bergin, SL Garfield (Eds). Handbook of Psychotherapy and Behavior Change, Fourth Edition. Oxford: John Wiley & Sons, 1994, pp. 19–71. Kazdin A. Methodology, design, and evaluation in psychotherapy research. In: AE Bergin, SL Garfield (Eds). Handbook of Psychotherapy and Behavior Change, Fourth Edition. Oxford: John Wiley & Sons, 1994, pp. 19–71.
25.
go back to reference McLeod BD, Islam NY, Wheat E. Designing, conducting, and evaluating therapy process research. In: J Comer, P Kendall (Eds). The Oxford Handbook of Research Strategies for Clinical Psychology. New York: Oxford University Press, 2013, pp. 142–164. McLeod BD, Islam NY, Wheat E. Designing, conducting, and evaluating therapy process research. In: J Comer, P Kendall (Eds). The Oxford Handbook of Research Strategies for Clinical Psychology. New York: Oxford University Press, 2013, pp. 142–164.
26.
go back to reference Schoenwald SK. It’s a Bird, It’s a Plane, It’s…Fidelity Measurement in the Real World. Clinical Psychology: Science and Practice. 2011;18(2):142–147. Schoenwald SK. It’s a Bird, It’s a Plane, It’s…Fidelity Measurement in the Real World. Clinical Psychology: Science and Practice. 2011;18(2):142–147.
27.
go back to reference Goense PB, Pronk S, Boendermaker L, et al. Leren op de Werkvloer. Organisatie en Inhoud van Leren op de Werkvloer. Delft, NL: Uitgeverij Eburon, 2015. Goense PB, Pronk S, Boendermaker L, et al. Leren op de Werkvloer. Organisatie en Inhoud van Leren op de Werkvloer. Delft, NL: Uitgeverij Eburon, 2015.
28.
go back to reference Weisz JR, Jensen-Doss A, Hawley KM. Evidence-Based Youth Psychotherapies Versus Usual Clinical Care: A Meta-Analysis of Direct Comparisons. American Psychologist. 2006;61:671–689.CrossRefPubMed Weisz JR, Jensen-Doss A, Hawley KM. Evidence-Based Youth Psychotherapies Versus Usual Clinical Care: A Meta-Analysis of Direct Comparisons. American Psychologist. 2006;61:671–689.CrossRefPubMed
29.
go back to reference Goldstein AP, Glick B, Gibbs JC. Aggression Replacement Training; A Comprehensive Intervention for Aggressive Youth, Revised Edition. Champaign, IL: Research Press, 1998. Goldstein AP, Glick B, Gibbs JC. Aggression Replacement Training; A Comprehensive Intervention for Aggressive Youth, Revised Edition. Champaign, IL: Research Press, 1998.
30.
go back to reference MAXQDA. MAXQDA 11: The Art of Data Analysis, Reference Manual for Windows. Berlin, DE: VERBI Software, Consult, Sozialforschung, 2012. MAXQDA. MAXQDA 11: The Art of Data Analysis, Reference Manual for Windows. Berlin, DE: VERBI Software, Consult, Sozialforschung, 2012.
31.
go back to reference Hsieh H, Shannon SE. Three Approaches to Qualitative Content Analysis. Qualitative Health Research. 2005;15:1277–1288.CrossRefPubMed Hsieh H, Shannon SE. Three Approaches to Qualitative Content Analysis. Qualitative Health Research. 2005;15:1277–1288.CrossRefPubMed
32.
go back to reference Morrow SL. Quality and Trustworthiness in Qualitative Research in Counseling Psychology. Journal of Counseling Psychology. 2005;52:250–260.CrossRef Morrow SL. Quality and Trustworthiness in Qualitative Research in Counseling Psychology. Journal of Counseling Psychology. 2005;52:250–260.CrossRef
Metadata
Title
Measuring Treatment Integrity: Use of and Experience with Measurements in Child and Youth Care Organizations
Authors
Pauline Brigitta Goense, PhD, LLM
Leonieke Boendermaker, PhD
Tom van Yperen, PhD
Publication date
01-07-2018
Publisher
Springer US
Published in
The Journal of Behavioral Health Services & Research / Issue 3/2018
Print ISSN: 1094-3412
Electronic ISSN: 2168-6793
DOI
https://doi.org/10.1007/s11414-018-9600-4

Other articles of this Issue 3/2018

The Journal of Behavioral Health Services & Research 3/2018 Go to the issue