Top

Administration and Policy in Mental Health and Mental Health Services Research

Published in:

01-03-2015 | Original Article

Integrity of Evidence-Based Practice: Are Providers Modifying Practice Content or Practice Sequencing?

Authors: Alayna L. Park, Bruce F. Chorpita, Jennifer Regan, John R. Weisz, The Research Network on Youth Mental Health

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 2/2015

Abstract

This study examined patterns of evidence-based treatment (EBT) implementation within community settings by evaluating integrity along separate dimensions of practice content (PC; a session included the prescribed procedure) and practice sequencing (a session occurred in the prescribed sequence) within a recent randomized effectiveness trial. We measured whether sessions showed integrity to PC and to flexible or linear practice sequences. Findings revealed that providers tended to incorporate content from the EBT protocol in most treatment sessions, but that the sequencing of the sessions was often modified, suggesting that providers are amenable to evidence-based procedures, but not necessarily their prescribed arrangement.

American Psychiatric Association. (2000). American Psychiatric Association Diagnostic and statistical manual of mental disorders: DSM-IV-TR. Washington, DC: American Psychiatric Association.

Barkley, R. A. (1997). Defiant children: A provider’s manual for assessment and parent training (2nd ed.). New York: Guilford Press, New York, NY.

Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114–1119. doi:10.1097/CHI.0b013e3181825af8.CrossRefPubMedCentralPubMed

Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to providers on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62(12), 1423–1429. doi:10.1176/appi.ps.002052011.CrossRefPubMed

Calhoun, K. S., Moras, K., Pilkonis, P. A., & Rehm, L. P. (1998). Empirically supported treatments: Implications for training. Journal of Consulting and Clinical Psychology, 66(1), 151–162. doi:10.1037/0022-006X.66.1.151.CrossRefPubMed

Carroll, K. M., & Rounsaville, B. J. (2007). A vision of the next generation of behavioral therapies research in the addictions. Addiction, 102(6), 850–862.CrossRefPubMedCentralPubMed

Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18. doi:10.1037/0022-006X.66.1.7.CrossRefPubMed

Chapman, J. E., McCart, M. R., Letourneau, E. J., & Sheidow, A. J. (2013). Comparison of youth, caregiver, therapist, trained, and treatment expert raters of therapist adherence to a substance abuse treatment protocol. Journal of Consulting and Clinical Psychology, 81(4), 674–680. doi:10.1037/a0033021.CrossRefPubMedCentralPubMed

Chorpita, B. F., & Daleiden, E. L. (2014). Structuring the collaboration of science and service in pursuit of a shared vision. Journal of Clinical Child and Adolescent Psychology, 43(2), 323–338. doi:10.1080/15374416.2013.828297.

Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005). Modularity in the design and application of therapeutic interventions. Applied and Preventive Psychology, 11(3), 141–156. doi:10.1016/j.appsy.2005.05.002.CrossRef

Chorpita, B. F., & Southam-Gerow, M. (2006). Fears and anxieties. New York: Guilford Press.

Chorpita, B. F., & Weisz, J. R. (2005). Modular approach to therapy for children with anxiety, depression, or conduct problems. Honolulu, HI: University of Hawaii at Manoa; Boston: Judge Baker Children’s Center; Harvard Medical School.

Chorpita, B. F., Weisz, J. R., Daleiden, E. L., Schoenwald, S. K., Palinkas, L. A., Miranda, J., et al. (2013). Long-term outcomes for the child STEPs randomized effectiveness trial: A comparison of modular and standard treatment designs with usual care. Journal of Consulting and Clinical Psychology, 81(6), 999–1009. doi:10.1037/a0034200.CrossRefPubMed

Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating integrity: Predictive validity for a measure of competent adherence to the Oregon Model of Parent Management Training. Behavior Therapy, 36(1), 3–13. doi:10.1037/0022-006X.67.5.711, 10.1037//0022-006X.65.5.821.

Hoagwood, K. E., Vogel, J. M., Levitt, J. M., D’Amico, P. J., Paisner, W. I., & Kaplan, S. J. (2007). Implementing an evidence-based trauma treatment in a state system after September 11: The CATS Project. Journal of the American Academy of Child and Adolescent Psychiatry, 46(6), 773–779. doi:10.1097/chi.0b013e3180413def.CrossRefPubMed

Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 230–244. doi:10.1007/s10488-009-0251-x.CrossRefPubMedCentralPubMed

Kendall, P. C., & Beidas, R. S. (2007). Smoothing the trail for dissemination of evidence-based practices for youth: Flexibility within fidelity. Professional Psychology: Research and Practice, 38(1), 13–20. doi:10.1037/0735-7028.38.1.13.CrossRef

Kendall, P. C., Gosch, E., Furr, J. M., & Sood, E. (2008). Flexibility within fidelity. Journal of the American Academy of Child and Adolescent Psychiatry, 47(9), 987–993. doi:10.1097/CHI.0b013e31817eed2f.CrossRefPubMed

Kendall, P. C., Kane, M., Howard, B., & Siqueland, L. (1990). Cognitive-behavioral treatment of anxious children. Ardmore, PA: Workbook Publishing.

Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions. Journal of Clinical Psychology, 61(2), 165–174.CrossRefPubMed

Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001). The effects of providing providers with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11(1), 49–68. doi:10.1093/ptr/11.1.49.CrossRef

Lonigan, C. J., Elbert, J. C., & Johnson, S. B. (1998). Empirically supported psychosocial interventions for children: An overview. Journal of Clinical Child Psychology, 27(2), 138–145.CrossRefPubMed

McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodríguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20(1), 14–32. doi:10.1111/cpsp.12020.

National Advisory Mental Health Council Workgroup on Child and Adolescent Mental Health Intervention and Deployment. (2001). Blueprint for change: Research on child and adolescent mental health. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health.

Office of the Surgeon General. (1999). Mental health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services.

Office of the Surgeon General. (2004). Report of the Surgeon General’s Conference on children’s mental health: A national action agenda. Rockville, MD: U.S. Department of Health and Human Services.

Palinkas, L. A., Aarons, G. A., Chorpita, B. F., Hoagwood, K., Landsverk, J., & Weisz, J. R. (2009). Cultural exchange and the implementation of evidence-based practices: Two case studies. Research on Social Work Practice, 19(5), 602–612. doi:10.1177/1049731509335529.CrossRef

Palinkas, L. A., Weisz, J. R., Chorpita, B. F., Garland, A., Hoagwood, K. E., Landsverk, J., et al. (2013). Use of evidence-based treatments for youth mental health subsequent to a randomized controlled effectiveness trial: A qualitative study. Psychiatric Services, 64(11), 1110–1118. doi:10.1176/appi.ps.004682012.CrossRefPubMed

Perepletchikova, F., Hilt, L. M., Chereji, E., & Kazdin, A. E. (2009). Barriers to implementing treatment integrity procedures: Survey of treatment outcome researchers. Journal of Consulting and Clinical Psychology, 77(2), 212–218. doi:10.1037/a0015232.CrossRefPubMed

Perepletchikova, F., & Kazdin, A. E. (2009). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12(4), 365–383. doi:10.1093/clipsy/bpi045.

President’s New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America: Final report. DHHS Publication number SMA-03-3832. Rockville, MD.

Proctor, E. K. (2004). Leverage points for the implementation of evidence-based practice. Brief Treatment and Crisis Intervention, 4(3), 227. doi:10.1093/brief-treatment/mhh020.CrossRef

Regan, J., Daleiden, E. L., & Chorpita, B. F. (2013). Integrity in mental health systems: An expanded framework for managing uncertainty in clinical care. Clinical Psychology: Science and Practice, 20(1), 78–98. doi:10.1111/cpsp.12024.

Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. doi:10.1007/s10488-010-0321-0.CrossRefPubMedCentralPubMed

Schoenwald, S. K., Kelleher, K., & Weisz, J. R. (2008). Building bridges to evidence-based practice: The MacArthur Foundation Child System and Treatment Enhancement Projects (Child STEPs). Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 66–72. doi:10.1007/s10488-006-0047-1.2006-08150-01010.1007/s10488-006-0047-1.CrossRefPubMed

Southam-Gerow, M. A., Chorpita, B. F., Miller, L. M., & Gleacher, A. A. (2008). Are children with anxiety disorders self-referred to a university clinic like those from the public mental health system? Administration and Policy in Mental Health and Mental Health Services Research, 35, 168–180. doi:10.1007/s10488-007-0154-7.CrossRefPubMed

Southam-Gerow, M. A., & McLeod, B. D. (2013). Advances in applying treatment integrity research for dissemination and implementation science: Introduction to special issue. Clinical Psychology: Science and Practice, 20, 1–13.

Southam-Gerow, M. A., Weisz, J. R., Chu, B. C., McLeod, B. D., Gordis, E. B., & Connor-Smith, J. K. (2010). Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child and Adolescent Psychiatry, 49(10), 1043–1052.CrossRefPubMedCentralPubMed

Stirman, S. W., Calloway, A., Toder, K., Miller, C. J., DeVito, A. K., Meisel, S. N., et al. (2013a). Community mental health provider modifications to cognitive therapy: Implications for sustainability. Psychiatric Services, 64(10), 1056–1059. doi:10.1176/appi.ps.201200456.CrossRef

Stirman, S. W., Miller, C. J., Toder, K., & Calloway, A. (2013b). Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science, 8(1), 65. doi:10.1186/1748-5908-8-65.CrossRefPubMedCentralPubMed

Travis, J. K., & Brestan-Knight, E. (2013). A pilot study examining trainee treatment session fidelity when parent–child interaction therapy (PCIT) is implemented in community settings. The Journal of Behavioral Health Services and Research, 40(3), 342–354. doi:10.1007/s11414-013-9326-2.CrossRefPubMed

Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., et al. (2013). Tracking evidence based practice with youth: Validity of the match and standard manual consultation records. Journal of Clinical Child and Adolescent Psychology, 42(1), 44–55. doi:10.1080/15374416.2012.700505.CrossRefPubMed

Weersing, V. R., Weisz, J. R., & Donenberg, G. R. (2002). Development of the therapy procedures checklist: A provider-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology, 31(2), 168–180. doi:10.1037//0003-066X.51.7.689.CrossRefPubMed

Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S., et al. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. doi:10.1007/s10488-007-0151-x.2008-00915-01110.1007/s10488-007-0151-x.CrossRefPubMed

Weisz, J. R., & Gray, J. S. (2008). Evidence-based psychotherapy for children and adolescents: Data from the present and a model for the future. Child and Adolescent Mental Health, 13(2), 54–65. doi:10.1111/j.1475-3588.2007.00475.x.CrossRef

Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61(7), 671–689. doi:10.1207/s15374424jccp3401_11.2005-01144-01110.1207/s15374424jccp3401_11.CrossRefPubMed

Weisz, J. R., & Kazdin, A. E. (2010). The present and future of evidence-based psychotherapies for children and adolescents (pp. 557–572). New York: Guilford Press.

Weisz, J. R., Moore, P. S., Southam-Gerow, M. A., Weersing, V. R., Valeri, S. M., & McCarty, C. A. (2005). Provider’s manual PASCET: Primary and secondary control enhancement training program (3rd ed.). Los Angeles: University of California.

Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects of psychotherapy with children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin, 117(3), 450–468.CrossRefPubMed

Weller, E. B., Weller, R. A., Rooney, M. T., & Fristad, M. A. (1999a). Children’s interview for psychiatric syndromes—Parent version (P-ChIPS). Arlington, VA: American Psychiatric Association, Arlington, VA.

Weller, E. B., Weller, R. A., Rooney, M. T., & Fristad, M. A. (1999b). Children’s interview for psychiatric syndromes: ChIPS. Arlington, VA: American Psychiatric Association, Arlington, VA.

Title: Integrity of Evidence-Based Practice: Are Providers Modifying Practice Content or Practice Sequencing?
Authors: Alayna L. Park
Bruce F. Chorpita
Jennifer Regan
John R. Weisz
The Research Network on Youth Mental Health
Publication date: 01-03-2015
Publisher: Springer US
Published in: Administration and Policy in Mental Health and Mental Health Services Research / Issue 2/2015
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-014-0559-z

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Integrity of Evidence-Based Practice: Are Providers Modifying Practice Content or Practice Sequencing?

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 2/2015

Individual and Organizational Predictors of Pediatric Psychiatric Inpatient Admission in Connecticut Hospitals: A 6 Month Secondary Analysis

Strategies for mHealth Research: Lessons from 3 Mobile Intervention Studies

Validity of Therapist Self-Report Ratings of Fidelity to Evidence-Based Practices for Adolescent Behavior Problems: Correspondence between Therapists and Observers

Quality of Care for Medicaid-Enrolled Youth with Bipolar Disorders

Maslow and Mental Health Recovery: A Comparative Study of Homeless Programs for Adults with Serious Mental Illness

The Dynamics of Psychiatric Bed Use in General Hospitals