Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Administration and Policy in Mental Health and Mental Health Services Research
Published in: Administration and Policy in Mental Health and Mental Health Services Research 3/2010

Open Access 01-05-2010 | original paper

Methodological Challenges of Characterizing Usual Care Psychotherapeutic Practice

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 3/2010

Login to get access

Abstract

There is minimal existing research providing detailed, reliable data characterizing usual community-based psychotherapy practice, and, thus, limited established methods for such research. This article identifies methodological challenges of usual care descriptive research, including, (a) general design considerations, (b) measurement, (c) data analytic, and (d) ethical challenges. Case examples drawn from studies reported in this special issue are used to illustrate the implications, strengths, and weaknesses of different methodological decisions. Central themes include achieving an acceptable balance of scientific rigor, feasibility, and generalizable practice relevance, as well as working collaboratively with practice partners to select and implement study methods.
Appendix
Available only for authorised users
Literature
Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6, 61–74.CrossRefPubMed
Aarons, G. A., & Sawitzky, A. (2006). Organizational climate mediates the effect of culture on work attitudes and turnover in mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 33, 289–301.CrossRefPubMed
Addis, M. E., & Krasnow, A. D. (2000). A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology, 68, 331–339.CrossRefPubMed
American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060–1073.CrossRef
Baumann, B. L., Kolko, D. J., Collins, K., & Herschell, A. D. (2006). Understanding practitioners’ characteristics and perspectives prior to the dissemination of an evidence-based intervention. Child Abuse and Neglect, 30, 771–787.CrossRefPubMed
Bearsley-Smith, C., Sellick, K., Chesters, J., Francis, K., & Gippsland Adolescent Depression Research Group. (2008). Treatment content in child and adolescent mental health services: Development of the treatment recording sheet. Administration and Policy Mental Health, 35, 423–435.CrossRef
Beutler, L. E. (2002). The dodo bird is extinct. Clinical Psychology: Science and Practice, 9, 30–34.CrossRef
Beutler, L. E., & Baker, M. (1998). The movement toward empirical validation: At what level should we analyze, and who are the consumers? (pp. 43–65). Thousand Oaks, CA: Sage Press.
Bickman, L. (2000). The most dangerous and difficult question in mental health services research. Mental Health Services Research, 2, 71–72.CrossRef
Brookman-Frazee, L., Garland, A. F., Taylor, R., & Zoffness, R. (2009a). Therapists’ attitudes towards psychotherapeutic strategies in community-based psychotherapy with children with disruptive behavior problems. Administration and Policy in Mental Health and Mental Health Services, 36, 1–12.CrossRef
Brookman-Frazee, L., Haine, R. A., Baker-Ericzén, M., Zoffness, R., & Garland, A. F. (2009b). Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy (this issue).
Carroll, K. M., & Rounsaville, B. J. (2007). A vision of the next generation of behavioral therapies research in the addictions. Addiction, 102, 850–862.CrossRefPubMed
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7–18.CrossRefPubMed
Chambless, D. L., Sanderson, W. C., Shoham, V., Bennett Johnson, S., Pope, K. S., Crits-Christoph, P., et al. (1996). An update on empirically validated therapies. Clinical Psychologist, 49, 5–18.
Chorpita, B. F., Becker, K. D., & Daleiden, E. L. (2007). Understanding the common elements of evidence-based practice: misconceptions and clinical examples. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 647–652.CrossRefPubMed
Cichetti, D. V. (1994). Guidelines, criteria, and rule of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.CrossRef
Eyberg, S. M., Nelson, M. M., & Boggs, S. R. (2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology, 37(1), 215–237.CrossRefPubMed
Garland, A. F., Brookman-Frazee, L., Hurlburt, M., Accurso, E. C., Zoffness, R., Haine, R. A., & Ganger, W. A. (under review). Characterizing community-based psychotherapy for children with disruptive behavior problems.
Garland, A. F., Brookman-Frazee, L., McLeod, B. (2008a). Scoring manual for the PRAC study therapy process observational coding system for child psychotherapy: strategies scale.
Garland, A. F., Hawley, K. M., Brookman-Frazee, L., & Hurlburt, M. S. (2008b). Identifying common elements of evidence-based psychosocial treatments for children’s disruptive behavior problems. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 505–514.CrossRefPubMed
Garland, A. F., Hurlburt, M. S., & Hawley, K. M. (2006a). Examining psychotherapy processes in a services research context. Clinical Psychology: Science and Practice, 13, 30–46.CrossRef
Garland, A. F., McCabe, K. M., & Yeh, M. (2008c). Ethical challenges in practice-based mental health services research: Examples from research with children and families. Clinical Psychology: Science and Practice, 15, 118–124.CrossRef
Garland, A. F., Plemmons, D., & Koontz, L. (2006b). Research-practice partnerships in mental health: Lessons from participants. Administration and Policy in Mental Health and Mental Health Services, 33, 517–528.CrossRef
Glisson, C., Dukes, D., & Green, P. (2006). The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse and Neglect, 30, 855–880.CrossRefPubMed
Glisson, C., Landsverk, J., Schoenwald, S., et al. (2008a). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research, 35, 98–113.CrossRefPubMed
Glisson, C., Schoenwald, S. K., Kelleher, K., Landsverk, J., Hoagwood, K. E., Mayberg, S., & Green, P. (2008b). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health and Mental Health Services Research. Special Issue: Improving Mental Health Services, 35, 124–133.
Goldfried, M. R. (1980). Toward the delineation of therapeutic change principles. American Psychologist, 35, 991–999.CrossRefPubMed
Heaton, K. J., Hill, C. E., & Edwards, L. A. (1995). Comparing molecular and molar methods of judging therapist techniques. Psychotherapy Research, 5, 141–153.
Hogue, A., Dauber, S., Liddle, H. A., & Samuolis, J. (2004). Linking session focus to treatment outcome in evidence-based treatments for adolescent substance abuse. Psychotherapy: Theory, Research, Practice, Training, 41, 83–96.CrossRef
Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy, 33, 332–345.
Hogue, A., Liddle, H. A., Rowe, C., Turner, R. M., Dakof, G. A., & LaPann, K. (1998). Treatment adherence and differentiation in individual versus family therapy for adolescent substance abuse. Journal of Counseling Psychology, 45, 104–114.CrossRef
Hox, J. (2002). Multilevel analysis techniques and applications. Mahwah, NJ: Erlbaum.
Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2009). Child and family therapy process: Concordance of therapist and observational (this issue).
Jensen, J. P., Bergin, A. E., & Greaves, D. W. (1990). The meaning of eclecticism: New survey and analysis of components. Professional Psychology Research and Practice, 21, 124–130.CrossRef
Jensen-Doss, A., Cusack, K. J., & de Arellano, M. A. (2008). Workshop-based training in trauma-focused CBT: An in-depth analysis of impact on provider practices. Community Mental Health Journal, 44, 227–244.CrossRefPubMed
Kataoka, S., Fuentes, S., O’Donoghue, V. P., Castillo-Campos, P., Bonilla, A., Halsey, K., et al. (2006). A community participatory research partnership: The development of a faith-based intervention for children exposed to violence. Ethnicity and Disease, 16, 89–97.
Kazdin, A. E., & Nock, M. K. (2003). Delineating mechanisms of change in child and adolescent therapy: Methodological issues and research recommendations. Journal of Child Psychology and Psychiatry and Allied Disciplines, 44, 1116–1129.CrossRef
Kazdin, A. E., Siegel, T. C., & Bass, D. (1990). Drawing on clinical practice to inform research on child and adolescent psychotherapy: Survey of practitioners. Professional Psychology: Research and Practice, 21, 189–198.CrossRef
Kazdin, A. E., & Weisz, J. R. (1998). Identifying and developing empirically supported child and adolescent treatments. Journal of Consulting and Clinical Psychology, 66, 19–36.CrossRefPubMed
Kazdin, A. E., & Weisz, J. R. (2003). Evidence-based psychotherapies for children and adolescents. New York, NY: Guilford Press.
Knudsen, H. K., Johnson, J. A., & Roman, P. M. (2003). Retaining counseling staff at substance abuse treatment centers: Effects of management practices. Journal of Substance Abuse Treatment, 24, 129–135.CrossRefPubMed
Lambert, M., & Hill, C. (1994). Assessing psychotherapy outcomes and processes. Handbook of psychotherapy and behavior change (4th ed., pp. 72–113). New York, NY: Wiley.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.CrossRefPubMed
Mayo, E. (1933). The human problems of industrial civilization. New York, NY: MacMillan.
McLeod, B. D. (2001). The therapy process observational coding system for child psychotherapy. Los Angeles: University of California. Unpublished instrument.
McLeod, B. D., & Weisz, J. R. (2008). The Therapy Process Observational Coding System for child psychotherapy strategies: Measure characteristics and prediction of outcome in usual clinical practice. (under review).
Mufson, L. H., Pollack-Dorta, K., Olfson, M., Weissman, M. M., & Hoagwood, K. (2004). Effectiveness research: Transporting Interpersonal Psychotherapy for Depressed Adolescents (IPT-A) from the lab to school-based health clinics. Clinical Child and Family Psychology Review, 7, 251–261.CrossRefPubMed
National Advisory Mental Health Council. (1998). Bridging science and service: A report by the National Advisory Mental Health Council’s Clinical treatment and Services Workgroup. Bethesda, MD: National Institutes of Health/National Institute of Mental Health.
Nelson, T. D., Steele, R. G., & Mize, J. A. (2006). Practitioner attitudes toward evidence-based practice: Themes and challenges. Administration and Policy In Mental Health, 33, 398–409.CrossRefPubMed
Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting Clinical Psychology, 75, 829–841.CrossRef
Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100, 322–331.CrossRef
Schley, C., Ryall, C., Crothers, L., Radovini, S., Fletcher, K., Marriage, K., et al. (2008). Early intervention in the real world: Early intervention with difficult to engage, ‘high-risk’ youth: Evaluating an intensive outreach approach in youth mental health. Early Intervention in Psychiatry, 2, 195–200.CrossRef
Schoenwald, S. K., Heiblum, N., Saldana, L., & Henggeler, S. W. (2008). The international implementation of multisystemic therapy. Evaluation & The Health Professions, 31, 211–225.CrossRef
Sheehan, A. K., Walrath-Greene, C., Fisher, S., Crossbear, S., & Walker, J. (2007). Evidence-based practice knowledge, use, and factors that influence decisions: Results from an evidence-based practice survey of providers in American Indian/Alaska Native communities. American Indian and Alaska Native Mental Health Research, 14, 29–48.PubMed
Vinnars, B., Barber, J. B., Norén, K., Gallop, R., & Weinryb, R. M. (2005). Manualized Supportive-Expressive Psychotherapy versus nonmanualized Community-Delivered Psychodynamic Therapy for patients with personality disorders: Bridging efficacy and effectiveness. American Journal of Psychiatry, 162, 1933–1940.CrossRefPubMed
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620–630.CrossRefPubMed
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H. (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: Empiricially, “all must have prizes”. Psychological Bulletin, 122, 203–215.CrossRef
Weersing, V. R., Weisz, J. R., & Donenberg, G. R. (2002). Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology, 31, 168–180.PubMed
Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61, 671–689.CrossRefPubMed
Wells, K. B., Sturm, R., Sherbourne, C. D., & Meredith, L. S. (1996). Caring for depression. Cambridge, MA: Harvard University Press.
West, S. G., Duan, H., Pequegnat, W., Gaist, P., Des Jarlais, D. C., Holtgrave, D., et al. (2008). Alternatives to the randomized control trial. American Journal of Public Health, 98, 1359–1366.CrossRefPubMed
Westfall, J. M., Mold, J., & Fagnan, L. (2007). Practice-based research—“Blue Highways” on the NIH roadmap. The Journal of the American Medical Association, 297, 403–406.CrossRef
Zima, B. T., Hurlburt, M. S., Knapp, P., Ladd, H., Tang, L., Duan, N., et al. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 130–144.CrossRefPubMed
Zoffness, R., Garland, A., Brookman-Frazee, L., & Roesch, S. (2009). Case management as a significant component of usual care psychotherapy for youth with disruptive behavior problems. Child & Youth Care Forum, 38, 185–200.CrossRef
Metadata
Title
Methodological Challenges of Characterizing Usual Care Psychotherapeutic Practice
Publication date
01-05-2010
Published in
Administration and Policy in Mental Health and Mental Health Services Research / Issue 3/2010
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI
https://doi.org/10.1007/s10488-009-0237-8

Other articles of this Issue 3/2010

Administration and Policy in Mental Health and Mental Health Services Research 3/2010 Go to the issue

Original Paper

Factors Associated with Use of Evidence-Based Practice Strategies in Usual Care Youth Psychotherapy

Original Paper

Usual Care Psychotherapy for Depression in a Large Managed Behavioral Health Organization

Original Paper

The Content of Substance Abuse and Mental Health Counseling Reported by Patients in a National Survey

Original Paper

Development of a Clinician Report Measure to Assess Psychotherapy for Depression in Usual Care Settings

Original Paper

Assessment of Evidence-Based Psychotherapy Practices in Usual Care: Challenges, Promising Approaches, and Future Directions

Original Paper

Exploring the Black Box: Measuring Youth Treatment Process and Progress in Usual Care