Skip to main content
Top
Published in: Administration and Policy in Mental Health and Mental Health Services Research 3/2012

01-05-2012 | Original Paper

Are Children Improving? Results from Outcome Measurement in a Large Mental Health System

Authors: Emily V. Trask, Ann F. Garland

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 3/2012

Login to get access

Abstract

Standardized outcome assessment was implemented in a large county mental health system. This study examines changes in children’s emotional and behavioral problems after 6 months of treatment and identifies factors associated with treatment improvement. Results from 3,215 youth indicated that externalizing and internalizing problems significantly decreased from intake to 6 months and that ethnicity, gender, and age were associated with caregiver-reported changes in symptomatology. While there was improvement associated with entry into treatment, the magnitude of symptom reduction was small to medium and not associated with amount of care, providing only limited support for the impact of routine out-patient care.
Literature
go back to reference Achenbach, T. (1991). Manual for the child behavior checklist/4–18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry. Achenbach, T. (1991). Manual for the child behavior checklist/4–18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
go back to reference Achenbach, T. M., & Edelbrock, C. S. (1979). The child behavior profile: II. Boys aged 12–16 and girls aged 6–11 and 12–16. Journal of Consulting and Clinical Psychology, 47, 223–233.PubMedCrossRef Achenbach, T. M., & Edelbrock, C. S. (1979). The child behavior profile: II. Boys aged 12–16 and girls aged 6–11 and 12–16. Journal of Consulting and Clinical Psychology, 47, 223–233.PubMedCrossRef
go back to reference Alegria, M., Atkins, M., Farmer, E., Slaton, E., & Stelk, W. (2010). One size does not fit all: Taking diversity, culture, and context seriously. Administration and Policy in Mental Health and Mental Health Services Research, 37, 48–60.PubMedCrossRef Alegria, M., Atkins, M., Farmer, E., Slaton, E., & Stelk, W. (2010). One size does not fit all: Taking diversity, culture, and context seriously. Administration and Policy in Mental Health and Mental Health Services Research, 37, 48–60.PubMedCrossRef
go back to reference Andrade, A. R., Lambert, W., & Bickman, L. (2000). Dose effect in child psychotherapy: Outcomes associated with negligible treatment. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 161–168.PubMedCrossRef Andrade, A. R., Lambert, W., & Bickman, L. (2000). Dose effect in child psychotherapy: Outcomes associated with negligible treatment. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 161–168.PubMedCrossRef
go back to reference Angold, A., Costello, J., Burns, B. J., Erkanli, A., & Farmer, E. M. Z. (2000). Effectiveness of nonresidential specialty mental health services for children and adolescents in the “real world”. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 154–160.PubMedCrossRef Angold, A., Costello, J., Burns, B. J., Erkanli, A., & Farmer, E. M. Z. (2000). Effectiveness of nonresidential specialty mental health services for children and adolescents in the “real world”. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 154–160.PubMedCrossRef
go back to reference Bickman, L., Andrade, A. R., & Lambert, W. (2004). Dose response in child and adolescent mental health services. Mental Health Services Research, 4, 57–70.CrossRef Bickman, L., Andrade, A. R., & Lambert, W. (2004). Dose response in child and adolescent mental health services. Mental Health Services Research, 4, 57–70.CrossRef
go back to reference Blais, M. A., Frank, R. G., Nierenberg, A. A., & Rauch, S. L. (2009). Measuring outcomes makes good sense. Psychiatric Services, 60, 112.PubMedCrossRef Blais, M. A., Frank, R. G., Nierenberg, A. A., & Rauch, S. L. (2009). Measuring outcomes makes good sense. Psychiatric Services, 60, 112.PubMedCrossRef
go back to reference Brookman-Frazee, L., Haine, R. A., & Garland, A. F. (2006). Measuring outcomes of real-world youth psychotherapy: Whom to ask and what to ask? Psychiatric Services, 57, 1373–1375.PubMedCrossRef Brookman-Frazee, L., Haine, R. A., & Garland, A. F. (2006). Measuring outcomes of real-world youth psychotherapy: Whom to ask and what to ask? Psychiatric Services, 57, 1373–1375.PubMedCrossRef
go back to reference Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
go back to reference Daleiden, E. L., Chorpita, B. F., Donkervoet, C. M., Arensdorf, A. A., & Brogan, M. (2006). Getting better at getting them better: Outcomes and evidence-based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 749–756.PubMedCrossRef Daleiden, E. L., Chorpita, B. F., Donkervoet, C. M., Arensdorf, A. A., & Brogan, M. (2006). Getting better at getting them better: Outcomes and evidence-based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 749–756.PubMedCrossRef
go back to reference Doucette, A. (2004). Child/adolescent measurement system (CAMS): User manual. Nashville, TN. Doucette, A. (2004). Child/adolescent measurement system (CAMS): User manual. Nashville, TN.
go back to reference Faraone, S. V., Biederman, J., & Mick, E. (2006). The age-dependent decline of attention deficit hyperactivity disorder: A meta-analysis of follow-up studies. Psychological Medicine, 36, 159–165.PubMedCrossRef Faraone, S. V., Biederman, J., & Mick, E. (2006). The age-dependent decline of attention deficit hyperactivity disorder: A meta-analysis of follow-up studies. Psychological Medicine, 36, 159–165.PubMedCrossRef
go back to reference Foster, M. E., Kelsch, C. C., Kamradt, B., Sosna, T., & Yang, Z. (2001). Expenditures and sustainability in systems of care. Journal of Emotional and Behavioral Disorders, 9, 53–62.CrossRef Foster, M. E., Kelsch, C. C., Kamradt, B., Sosna, T., & Yang, Z. (2001). Expenditures and sustainability in systems of care. Journal of Emotional and Behavioral Disorders, 9, 53–62.CrossRef
go back to reference Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R., Haine, R. A., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61, 788–795.PubMedCrossRef Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R., Haine, R. A., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61, 788–795.PubMedCrossRef
go back to reference Garland, B., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research. Advance online publication. doi:10.1007/s10488-010-0279-y. Garland, B., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research. Advance online publication. doi:10.​1007/​s10488-010-0279-y.
go back to reference Garland, A. F., Lau, A. S., Yeh, M., McCabe, K. M., Hough, R. L., & Landsverk, J. A. (2005). Racial and ethnic differences in utilization of mental health services among high-risk youths. American Journal of Psychiatry, 162, 1336–1343.PubMedCrossRef Garland, A. F., Lau, A. S., Yeh, M., McCabe, K. M., Hough, R. L., & Landsverk, J. A. (2005). Racial and ethnic differences in utilization of mental health services among high-risk youths. American Journal of Psychiatry, 162, 1336–1343.PubMedCrossRef
go back to reference Gold, J., Buonopane, R. J., Caggiano, R. A., Picciotto, M., Vogeli, C., Kanner, N. T., et al. (2009). Assessing outcomes in child psychiatry. American Journal of Managed Care, 15, 210–216.PubMed Gold, J., Buonopane, R. J., Caggiano, R. A., Picciotto, M., Vogeli, C., Kanner, N. T., et al. (2009). Assessing outcomes in child psychiatry. American Journal of Managed Care, 15, 210–216.PubMed
go back to reference Green, S. B., Salkind, N. J., & Akey, T. M. (2000). Using SPSS for windows: Analyzing and understanding data. New Jersey: Prentice Hall. Green, S. B., Salkind, N. J., & Akey, T. M. (2000). Using SPSS for windows: Analyzing and understanding data. New Jersey: Prentice Hall.
go back to reference Handwerk, M. L., Larzelere, R. E., Soper, S. H., & Friman, P. C. (1999). Parent and child discrepancies in reporting severity of problem behaviors in three out-of-home settings. Psychological Assessment, 11, 14–23.CrossRef Handwerk, M. L., Larzelere, R. E., Soper, S. H., & Friman, P. C. (1999). Parent and child discrepancies in reporting severity of problem behaviors in three out-of-home settings. Psychological Assessment, 11, 14–23.CrossRef
go back to reference Hankin, B. L., Abramson, L. Y., Silva, P. A., McGee, R., & Moffitt, T. E. (1998). Development of depression from preadolescence to young adulthood: Emerging gender differences in a 10-year longitudinal study. Journal of Abnormal Psychology, 107, 128–140.PubMedCrossRef Hankin, B. L., Abramson, L. Y., Silva, P. A., McGee, R., & Moffitt, T. E. (1998). Development of depression from preadolescence to young adulthood: Emerging gender differences in a 10-year longitudinal study. Journal of Abnormal Psychology, 107, 128–140.PubMedCrossRef
go back to reference Hatfield, D., & Ogles, B. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35, 485–491.CrossRef Hatfield, D., & Ogles, B. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35, 485–491.CrossRef
go back to reference Hill, J. C., & Schoener, E. P. (1996). Age-dependent decline of attention deficit hyperactivity disorder. American Journal of Psychiatry, 153, 1143–1146.PubMed Hill, J. C., & Schoener, E. P. (1996). Age-dependent decline of attention deficit hyperactivity disorder. American Journal of Psychiatry, 153, 1143–1146.PubMed
go back to reference Hoagwood, K., & Kolko, D. (2009). Introduction to the Special Section on Practice Contexts: A Glimpse into the Nether World of Public Mental Health Services for Children and Families. Administration and Policy in Mental Health and Mental Health Services Research, 36, 35–36.PubMedCrossRef Hoagwood, K., & Kolko, D. (2009). Introduction to the Special Section on Practice Contexts: A Glimpse into the Nether World of Public Mental Health Services for Children and Families. Administration and Policy in Mental Health and Mental Health Services Research, 36, 35–36.PubMedCrossRef
go back to reference Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.PubMedCrossRef Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.PubMedCrossRef
go back to reference Kataoka, S. H., Zhang, L., & Wells, K. B. (2002). Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance status. The American Journal of Psychiatry, 159, 1548–1555.PubMedCrossRef Kataoka, S. H., Zhang, L., & Wells, K. B. (2002). Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance status. The American Journal of Psychiatry, 159, 1548–1555.PubMedCrossRef
go back to reference Lambert, W., & Bickman, L. (2004). The “clock-setting” cure: How children’s symptoms might improve after ineffective treatment. Psychiatric Services, 55, 381–382.PubMedCrossRef Lambert, W., & Bickman, L. (2004). The “clock-setting” cure: How children’s symptoms might improve after ineffective treatment. Psychiatric Services, 55, 381–382.PubMedCrossRef
go back to reference Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181–1209.PubMedCrossRef Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181–1209.PubMedCrossRef
go back to reference Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis (Vol. 49). Thousand Oaks, CA: Sage Publications. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis (Vol. 49). Thousand Oaks, CA: Sage Publications.
go back to reference Liptzin, B. (2009). Quality improvement, pay for performance, and “outcomes measurement”: What makes sense? Psychiatric Services, 60, 108–111.PubMedCrossRef Liptzin, B. (2009). Quality improvement, pay for performance, and “outcomes measurement”: What makes sense? Psychiatric Services, 60, 108–111.PubMedCrossRef
go back to reference Manteuffel, B., Stephens, R. L., & Santiago, R. (2002). Overview of the national evaluation of the comprehensive community mental health services for children and their families program and summary of current findings. Children’s Services: Social Policy, Research, and Practice, 5, 3–20.CrossRef Manteuffel, B., Stephens, R. L., & Santiago, R. (2002). Overview of the national evaluation of the comprehensive community mental health services for children and their families program and summary of current findings. Children’s Services: Social Policy, Research, and Practice, 5, 3–20.CrossRef
go back to reference Manteuffel, B., Stephens, R. L., Sondheimer, D. L., & Fisher, S. K. (2008). Characteristics, service experiences, and outcomes of transition-aged youth in systems of care: Programmatic and policy implications. The Journal of Behavioral Health Services Research, 35, 469–487.PubMedCrossRef Manteuffel, B., Stephens, R. L., Sondheimer, D. L., & Fisher, S. K. (2008). Characteristics, service experiences, and outcomes of transition-aged youth in systems of care: Programmatic and policy implications. The Journal of Behavioral Health Services Research, 35, 469–487.PubMedCrossRef
go back to reference Maramba, G. G., Nagayama, H., & Gordon, C. (2002). Meta-analysis of ethnic match as a predictor of dropout, utilization, and level of functioning. Cultural Diversity and Ethnic Minority Psychology, 8, 290–297.PubMedCrossRef Maramba, G. G., Nagayama, H., & Gordon, C. (2002). Meta-analysis of ethnic match as a predictor of dropout, utilization, and level of functioning. Cultural Diversity and Ethnic Minority Psychology, 8, 290–297.PubMedCrossRef
go back to reference Maxwell, S. E., & Delaney, H. D. (2000). Designs with Concomitant Variables: ANCOVA and Blocking. In S. Maxwell & H. Delaney (Eds.), Designing Experiments and Analyzing Data. A Model Comparison Perspective (p. 256). Mahwah, New Jersey: Lawrence Erlbaum Associates. Maxwell, S. E., & Delaney, H. D. (2000). Designs with Concomitant Variables: ANCOVA and Blocking. In S. Maxwell & H. Delaney (Eds.), Designing Experiments and Analyzing Data. A Model Comparison Perspective (p. 256). Mahwah, New Jersey: Lawrence Erlbaum Associates.
go back to reference Ogles, B. M., Carlson, B., Hatfield, D., & Karpenko, V. (2008). Models of case mix adjustment for Ohio mental health consumer outcomes among children and adolescents. Administration and Policy in Mental Health and Mental Health Services Research, 35, 295–304.PubMedCrossRef Ogles, B. M., Carlson, B., Hatfield, D., & Karpenko, V. (2008). Models of case mix adjustment for Ohio mental health consumer outcomes among children and adolescents. Administration and Policy in Mental Health and Mental Health Services Research, 35, 295–304.PubMedCrossRef
go back to reference Robinson, E. A., Eyberg, S. M., & Ross, W. A. (1980). The standardization of an inventory of child conduct problem behaviors. Journal of Clinical Child and Adolescent Psychology, 9, 22–28.CrossRef Robinson, E. A., Eyberg, S. M., & Ross, W. A. (1980). The standardization of an inventory of child conduct problem behaviors. Journal of Clinical Child and Adolescent Psychology, 9, 22–28.CrossRef
go back to reference Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage Publications. Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage Publications.
go back to reference Sederer, L. I., Hermann, R., & Dickey, B. (1995). The imperative of outcome assessment in Psychiatry. American Journal of Medical Quality, 10, 127–132.PubMedCrossRef Sederer, L. I., Hermann, R., & Dickey, B. (1995). The imperative of outcome assessment in Psychiatry. American Journal of Medical Quality, 10, 127–132.PubMedCrossRef
go back to reference Snowden, L. R. (2003). Bias in mental health assessment and intervention: Theory and evidence. American Journal of Public Health, 93, 239–243.PubMedCrossRef Snowden, L. R. (2003). Bias in mental health assessment and intervention: Theory and evidence. American Journal of Public Health, 93, 239–243.PubMedCrossRef
go back to reference Sourander, A., Helstela, L., & Helenius, H. (1999). Parent-adolescent agreement on emotional and behavioral problems. Social Psychiatry and Psychiatric Epidemiology, 34, 657–663.PubMedCrossRef Sourander, A., Helstela, L., & Helenius, H. (1999). Parent-adolescent agreement on emotional and behavioral problems. Social Psychiatry and Psychiatric Epidemiology, 34, 657–663.PubMedCrossRef
go back to reference Turchik, J. A., Karpenko, V., & Ogles, B. M. (2007). Further evidence of the utility and validity of a measure of outcome for children and adolescents. Journal of Emotional and Behavioral Disorders, 15, 119–128.CrossRef Turchik, J. A., Karpenko, V., & Ogles, B. M. (2007). Further evidence of the utility and validity of a measure of outcome for children and adolescents. Journal of Emotional and Behavioral Disorders, 15, 119–128.CrossRef
go back to reference U.S. Department of Health and Human Services. (1999). Mental Health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National institute of Mental Health. U.S. Department of Health and Human Services. (1999). Mental Health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National institute of Mental Health.
go back to reference Walrath, C., Garraza, L. G., Stephens, R., Azur, M., Miech, R., & Leaf, P. (2009). Trends in characteristics of children served by the Children’s Mental Health Initiative: 1994–2007. Administration and Policy in Mental Health and Mental Health Services Research, 36, 361–373.PubMedCrossRef Walrath, C., Garraza, L. G., Stephens, R., Azur, M., Miech, R., & Leaf, P. (2009). Trends in characteristics of children served by the Children’s Mental Health Initiative: 1994–2007. Administration and Policy in Mental Health and Mental Health Services Research, 36, 361–373.PubMedCrossRef
go back to reference Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010). Youth psychotherapy change trajectories and outcomes in usual care: Community mental health versus managed care settings. Journal of Consulting and Clinical Psychology, 78, 144–155.PubMedCrossRef Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010). Youth psychotherapy change trajectories and outcomes in usual care: Community mental health versus managed care settings. Journal of Consulting and Clinical Psychology, 78, 144–155.PubMedCrossRef
go back to reference Weisz, J. R. (2004). Psychotherapy for children and adolescents: Evidence-based treatments and case examples. Cambridge, UK: Cambridge University Press.CrossRef Weisz, J. R. (2004). Psychotherapy for children and adolescents: Evidence-based treatments and case examples. Cambridge, UK: Cambridge University Press.CrossRef
go back to reference Weisz, J. R., & Jensen, A. L. (2001). Child and adolescent psychotherapy in research and practice contexts: Review of the evidence and suggestions for improving the field. European Journal of Child and Adolescent Psychiatry, 10, 12–18.CrossRef Weisz, J. R., & Jensen, A. L. (2001). Child and adolescent psychotherapy in research and practice contexts: Review of the evidence and suggestions for improving the field. European Journal of Child and Adolescent Psychiatry, 10, 12–18.CrossRef
go back to reference Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional psychology, research and practice, 24, 190–195.CrossRef Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional psychology, research and practice, 24, 190–195.CrossRef
go back to reference Zima, B. T., Hurlburt, M. S., Knapp, P., Ladd, H., Tang, L., et al. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 130–144.PubMedCrossRef Zima, B. T., Hurlburt, M. S., Knapp, P., Ladd, H., Tang, L., et al. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 130–144.PubMedCrossRef
Metadata
Title
Are Children Improving? Results from Outcome Measurement in a Large Mental Health System
Authors
Emily V. Trask
Ann F. Garland
Publication date
01-05-2012
Publisher
Springer US
Published in
Administration and Policy in Mental Health and Mental Health Services Research / Issue 3/2012
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI
https://doi.org/10.1007/s10488-011-0353-0

Other articles of this Issue 3/2012

Administration and Policy in Mental Health and Mental Health Services Research 3/2012 Go to the issue