Top

Administration and Policy in Mental Health and Mental Health Services Research

Published in:

01-11-2011 | Original Paper

Improvement in Symptoms Versus Functioning: How Do Our Best Treatments Measure Up?

Authors: Kimberly D. Becker, Bruce F. Chorpita, Eric L. Daleiden

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 6/2011

Abstract

We examined the effects of redefining standards of evidence for treatments targeting childhood mental health problems by expanding outcomes beyond symptom reduction to include functioning. Over 750 treatment protocols from 435 randomized controlled trials were rated based on empirical evidence. Nearly two-thirds (63.9%) demonstrated at least a minimum level of evidence for reducing symptoms; however, only 18.8% of treatments demonstrated evidence for reducing functional impairment. Of those treatments with empirical support for symptom reduction, the majority did not demonstrate empirical support for improvement in functioning because measures of functioning were not included in the studies in which these treatments were tested. However, even when measures of functioning were included, it was much more difficult for treatments to achieve improvement. Among treatments that achieved improvement in functioning, the most notable were Collaborative Problem Solving for disruptive behavior and Cognitive Behavioral Therapy plus Medication for traumatic stress because they demonstrated no support for symptom reduction but good support for improvement in functioning. Results are discussed within the context of evaluating the standards of evidence for treatments and the opportunity to move towards a multidimensional framework whose utility has the potential to exceed the sum of its parts.

Achenbach, T. (1991). Manual for the child behavior checklist/4–18 and 1991 profile. Burlington: Department of Psychiatry, University of Vermont.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders, (4th ed.), text revision. Washington, DC: American Psychiatric Association

Bacon, S., Collins, M., & Plake, E. (2002). Does the Global Assessment of Functioning assess functioning? Journal of Mental Health Counseling, 24, 202–212.

Bates, M. (2001). The Child and Adolescent Functional Assessment Scale (CAFAS): Review and current status. Clinical Child and Family Psychology Review, 4, 63–84.PubMedCrossRef

Becker, K. D., Nakamura, B., Young, J., & Chorpita, B. (2009). What better place than here? What better time than now? ABCT’s burgeoning role in the dissemination and implementation of evidence-based practices. The Behavior Therapist, 32, 89–96.

Bickman, L., Lambert, E., Karver, M., & Andrade, A. (1998). Two low-cost measures of child and adolescent functioning for services research. Evaluation and Program Planning, 21, 263–275.CrossRef

Bickman, L., Lambert, E., & Schilling, S. (1999). Time patterns of mental health outcomes. Mental Health Services Research, 1, 67–82.CrossRef

Bickman, L., Rosof-Williams, J., Salzer, M., Summerfelt, W., Noser, K., Wilson, S., et al. (2000). What information do clinicians value for monitoring adolescent client progress and outcomes? Professional Psychology: Research and Practice, 31, 70–74.CrossRef

Bird, H. (1999). The assessment of functional impairment. In D. Shaffer, C. Lucas, & J. Richters (Eds.), Diagnostic assessment in child and adolescent psychopathology (pp. 209–229). NY: Guilford.

Bird, H., Yager, T., Staghezza, B., & Gould, M. (1990). Impairment in the epidemiological measurement of childhood psychopathology in the community. Journal of the American Academy of Child & Adolescent Psychiatry, 29, 796–803.CrossRef

Burlingame, G., Lambert, M., Reisinger, C., Neff, W., & Mosier, J. (1995). Pragmatics of tracking mental health outcomes in a managed care setting. Mental Health Services Research, 22, 226–236.

Canino, G., Costello, E., & Angold, A. (1999). Assessing functional impairment for child mental health services research: A review of measures. Journal of Mental Health Services Research, 1, 93–108.CrossRef

Chambless, D., Baker, M., Baucom, D., Beurtler, L., Calhoun, K., Crits-Cristoph, P., et al. (1998). Update on empirically validated therapies, II. The Clinical Psychologist, 51, 3–16.

Chambless, D., Sanderson, W., Shoham, V., Bennett Johnson, S., Pope, K., Crits-Christoph, P., et al. (1996). An update on empirically validated therapies. The Clinical Psychologist, 49, 5–18.

Chorpita, B. (2001). Reflections on clinical significance: What do our best treatments accomplish and how can we best find out? Clinical Psychology: Science and Practice, 8, 451–454.CrossRef

Chorpita, B., & Daleiden, E. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77, 566–579.PubMedCrossRef

Chorpita, B., Daleiden, E., Ebesutani, C., Young, J., Becker, K. D., Nakamura, B. et al. (in press). Evidence based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice.

Chorpita, B., & Regan, J. (2009). Dissemination of effective mental health treatment procedures: Maximizing the return on a significant investment. Behaviour Research and Therapy, 47, 990–993.PubMedCrossRef

Chorpita, B., Yim, L., Donkervoet, J., Arensdorf, A., Amundsen, M., McGee, C., et al. (2002). Toward large-scale implementation of empirically supported treatments for children: A review and observations by the Hawaii Empirical Basis to Services Task Force. Clinical Psychology: Science and Practice, 9, 165–190.CrossRef

Cohen, J., Mannarino, A., Perel, J., & Staron, V. (2007). A pilot randomized controlled trial of combined trauma-focused CBT and sertraline for childhood PTSD symptoms. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 811–819.PubMedCrossRef

De Los Reyes, A., & Kazdin, A. (2008). When the evidence says, ‘yes, no, and maybe so’: Attending to and interpreting inconsistent findings among evidence-based interventions. Current Directions in Psychological Science, 17, 47–51.PubMedCrossRef

Drake, R., Latimer, E., Leff, H., McHugo, G., & Burns, B. (2004). What is evidence? Child and Adolescent Psychiatric Clinics of North America, 13, 717–728.PubMedCrossRef

Drake, R., Rosenberg, S., Teague, G., Bartels, S., & Torrey, W. (2003). Fundamental principles of evidence-based medicine applied to mental health care. Psychiatric Clinics of North America, 26, 811–820.PubMedCrossRef

Dulcan, M. (2005). Practitioner perspectives on evidence-based practice. Child and Adolescent Psychiatric Clinics of North America, 14, 225–240.PubMedCrossRef

Goldman, H. (2005). Do you walk to school, or do you carry your lunch? Psychiatric Services, 56, 419.CrossRef

Goldman, H., Ganju, V., Drake, R., Gorman, P., Hogan, M., Hyde, P., et al. (2001). Policy implications for implementing evidence-based practices. Psychiatric Services, 52, 1591–1597.PubMedCrossRef

Goldman, H., Skodol, A., & Lave, T. (1992). Revising axis V for DSM-IV: A review of measures of social functioning. American Journal of Psychiatry, 149, 1148–1156.PubMed

Greene, R., Ablon, J., Monuteaux, M., Goring, J., Henin, A., Raezer, L., et al. (2004). Effectiveness of collaborative problem solving in affectively dysregulated youth with oppositional defiant disorder: Initial findings. Journal of Consulting and Clinical Psychology, 72, 1157–1164.PubMedCrossRef

Hayes, S., Barlow, D., & Nelson-Gray, R. (1999). The scientist practitioner: Research and accountability in the age of managed care. Needham heights (2nd ed.). MA US: Allyn & Bacon.

Hernandez, M., Hodges, S., & Cascardi, M. (1998). The ecology of outcomes: System accountability in children’s mental health. The Journal of Behavioral Health Services & Research, 25, 136–150.CrossRef

Hoagwood, K., Jensen, P., Petti, T., & Burns, B. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 1055–1063.CrossRef

Hodges, K., Doucette-Gates, A., & Kim, C. (2000). Predicting service utilization with the child and adolescent functional assessment scale in a sample of youths with serious emotional disturbance served by center for mental health Services-funded demonstrations. The Journal of Behavioral Health Services & Research, 27, 47–59.CrossRef

Individuals with Disabilities Education Improvement Act (2004) H.R. 1350, 108th Congress.

Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.

Jacobson, N., Roberts, L., Berns, S., & McGlinchey, J. (1999). Methods for defining and determining the clinical significance of treatment effects: Description, application, and alternatives. Journal of Consulting and Clinical Psychology, 67, 300–307.PubMedCrossRef

Kagan, J. (1964). Matching familiar figures test. Cambridge, MA: Harvard University.

Karver, M., & Bickman, L. (2002). Positive functioning: Does it add validity to maladaptive functioning items? Evaluation and Program Planning, 25, 85–93.CrossRef

Kazdin, A., & Bass, D. (1989). Power to detect differences between alternative treatments in comparative psychotherapy outcome research. Journal of Consulting and Clinical Psychology, 57, 138–147.PubMedCrossRef

Kazdin, A., & Kendall, P. (1998). Current progress and future plans for developing effective treatments: Comments and perspectives. Journal of Clinical Child Psychology, 27, 217–226.PubMedCrossRef

McGlinchey, J., Zimmerman, M., & Atkins, D. (2008). Clinical significance and remission in treating major depressive disorder: Parallels between related outcome constructs. Harvard Review of Psychiatry, 16, 25–34.PubMedCrossRef

McLeod, B., & Weisz, J. (2004). Using dissertations to examine potential bias in child and adolescent clinical trials. Journal of Consulting and Clinical Psychology, 72, 235–251.PubMedCrossRef

Nakamura, B., Chorpita, B., Hisch, M., Daleiden, E., Slavin, L., Amundson, M. et al. (2009). Large-scale implementation of evidence-based treatments for children ten years later: Hawaii’s evidence based services initiative in children’s mental health (submitted).

Pickles, A., Rowe, R., Simonoff, E., Foley, D., Rutter, M., & Silberg, J. (2001). Child psychiatric symptoms and psychosocial impairment: Relationship and prognostic significance. British Journal of Psychiatry, 179, 230–235.PubMedCrossRef

Plante, T., Couchman, C., & Diaz, A. (1995). Measuring treatment outcome and client satisfaction among children and families. Journal of Mental Health Administration, 22, 61–269.CrossRef

PracticeWise. (2005). Psychosocial and combined treatments coding manual. Satellite Beach, FL: Author.

Rapp, C., Bond, G., Becker, D., Carpinello, S., Nikkel, R., & Gintoli, G. (2005). The role of state mental health authorities in promoting improved client outcomes through evidence-based practice. Community Mental Health Journal, 41, 347–363.PubMedCrossRef

Reynolds, C., & Richmond, B. (1985). Revised children’s manifest anxiety scale. Los Angeles: Western Psychological Services.

Rogers, S., & Vismara, L. (2008). Evidence-based comprehensive treatments for early autism. Journal of Clinical Child & Adolescent Psychology, 37, 8–38.CrossRef

Schaffer, D., et al. (1983). A children’s global assessment scale (CGAS). Archives of General Psychiatry, 40, 1228–1231.

Schoenwald, S., Hoagwood, K., Atkins, M., Evans, S., & Ringeisen, H. (2010). Workforce development and the organization of work: The science we need. Administration and Policy In Mental Health, 37, 71–80.PubMedCrossRef

Sechrest, L., McKnight, P., & McKnight, K. (1996). Calibration of measures for psychotherapy outcome studies. American Psychologist, 51, 1065–1071.PubMedCrossRef

Silverman, W., & Hinshaw, S. (2008). The second special issue on evidence-based psychosocial treatments for children and adolescents: A 10-year update. Journal of Clinical Child and Adolescent Psychology, 37, 1–7.CrossRef

Simonoff, E., et al. (1997). The Virginia twin study of adolescent behavioral development: Influences of age, gender and impairment on rates of disorder. Archives of General Psychiatry, 54, 801–808.PubMed

Sparrow, S., Balla, D., & Cicchetti, D. (1984). Vineland adaptive behavior scales: Interview edition survey form manual. Circle Pines, MN: American Guidance Service.

Striley, C., Stiffman, A., & Spitznagel, E. (2003). Functioning mediates between symptoms and provider assessment. Mental Health Services Research, 5, 155–171.PubMedCrossRef

Swets, J. (1992). The science of choosing the right decision threshold in high-stakes diagnostics. American Psychologist, 47, 522–532.PubMedCrossRef

Task Force on Promotion, Dissemination of Psychological Procedures D. o. C. P. (1995). Training in and dissemination of empirically validated psychological treatments: Report and recommendations. The Clinical Psychologist, 48, 3–23.

Wampold, B., Imel, Z., & Miller, S. (2009). Barriers to the dissemination of empirically supported treatments: Matching messages to the evidence. The Behavior Therapist, 32, 144–155.

Weisz, J., Hawley, K., & Jensen-Doss, A. (2004). Empirically tested psychotherapies for youth internalizing and externalizing problems and disorders. Child and Adolescent Psychiatric Clinics of North America, 13, 729–815.PubMedCrossRef

Weisz, J., Weersing, V., & Henngeler, S. (2005). Jousting with straw men: Comment on Westen, Novotny, and Thompson-Brenner. Psychological Bulletin, 131, 418–426.PubMedCrossRef

Westen, D., & Bradley, R. (2005). Empirically supported complexity: Rethinking evidence-based practice in psychotherapy. Current Directions in Psychological Science, 14, 266–271.CrossRef

Westen, D., Novotny, C., & Thompson-Brenner, H. (2004). The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin, 130, 631–663.PubMedCrossRef

Winters, N., Collett, B., & Myers, K. (2005). Ten-year review of rating scales, VII: Scales assessing functional impairment. Journal of the Academy of Child and Adolescent Psychiatry, 44, 309–338.CrossRef

Zimmerman, M., McGlinchey, J. B., Posternak, M. A., Friedman, M., Attiullah, M., & Boerescu, D. (2006). How should remission from depression be defined? The depressed patient’s perspective. The American Journal of Psychiatry, 163, 148–150.PubMedCrossRef

Title: Improvement in Symptoms Versus Functioning: How Do Our Best Treatments Measure Up?
Authors: Kimberly D. Becker
Bruce F. Chorpita
Eric L. Daleiden
Publication date: 01-11-2011
Publisher: Springer US
Published in: Administration and Policy in Mental Health and Mental Health Services Research / Issue 6/2011
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-010-0332-x

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Improvement in Symptoms Versus Functioning: How Do Our Best Treatments Measure Up?

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 6/2011

Changes in Psychotherapy Utilization Among Consumers of Services for Major Depressive Disorder in the Community Mental Health System

Mental Health Services Expenditures among Children Placed in Out-of-Home Care

Announcement

Trajectories of Symptom Reduction During Treatment for Behavior Problems in Pediatric Primary-Care Settings

Easier Said than Done: Intervention Sustainability in an Urban After-School Program

A Comparison of Treatment Outcomes Among Chronically Homelessness Adults Receiving Comprehensive Housing and Health Care Services Versus Usual Local Care