Top

Administration and Policy in Mental Health and Mental Health Services Research

Published in:

01-07-2014 | Original Article

Satisfaction in Child and Adolescent Mental Health Services: Translating Users’ Feedback into Measurement

Authors: Anna Brown, Tamsin Ford, Jessica Deighton, Miranda Wolpert

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 4/2014

Abstract

The present research addressed gaps in our current understanding of validity and quality of measurement provided by patient reported experience measures. We established the psychometric properties of a freely available experience of service questionnaire (ESQ), based on responses from 7,067 families of patients across 41 UK providers of child and adolescent mental health services, using the two-level latent trait modeling. Responses to the ESQ were subject to strong ‘halo’ effects, which were thought to represent the overall positive or negative affect towards one’s treatment. Two strongly related constructs measured by the ESQ were interpreted as specific aspects of global satisfaction, namely satisfaction with care, and with environment. The Care construct was sensitive to differences between less satisfied patients, facilitating individual and service-level problem evaluation. The effects of nesting within service providers were strong, with parental reports being the most reliable source of data for the between-provider comparisons. We provide a scoring protocol for converting the hand-scored ESQ to the model-based population-referenced scores with supplied standard errors, which can be used for benchmarking services as well as individual evaluations.

Available only for authorised users

Due to the small sample size at the provider level (N = 41), change in Chi-square based indices is not significant between models with different number of factors at the provider level. Residuals for correlations and SRMR are better indicators of relative model fit in this case.

Most EFA rotation methods would minimize the number of factors each item loads on—i.e. rotate to an independent clusters structure.

Anderson, J. A., Rivera, V. R., & Kutash, K. (1998). Measuring consumer satisfaction with children’s mental health. In M. H. Epstein, K. Kutash, & A. J. Duchnowski (Eds.), Outcomes for children with emotional and behavioral disorders and their families: Program and evaluation best practices (pp. 455–482). Austin: Pro-Ed.

Attkisson, C., & Greenfield, T. (1996). The client satisfaction questionnaire (CSQ) scales and the service satisfaction scale-30 (SSS-30). In L. Sederer & B. Dickey (Eds.), Outcomes assessment in clinical practice. Baltimore: Williams & Wilkins.

Attride-Stirling, J. (2002). Development of methods to capture users’ views of CAMHS in clinical governance reviews. http://www.corc.uk.net/resources/downloads/.

Ayton, A., Mooney, M., Sillifant, K., Powls, J., & Rasool, H. (2007). The development of the child and adolescent versions of the verona service satisfaction scale (CAMHSSS). Social Psychiatry and Psychiatric Epidemiology, 42, 892–901. doi:10.1007/s00127-007-0241-9.PubMedCrossRef

Barber, A. J., Tischler, V. A., & Healy, E. (2006). Consumer satisfaction and child behaviour problems in child and adolescent mental health services. Journal of Child Health Care, 10(1), 9–21. doi:10.1177/1367493506060200.PubMedCrossRef

Barnes, D., Devanney, C., Uglebjerg, A., Wistow, R. & Hartley, C. (2010). A profile of children’s health services, child and adolescent mental health services and maternity services in England 2008/9. Durham: Durham University. http://dro.dur.ac.uk/8220/1/8220.pdf.

Biering, P., & Jensen, V. (2011). The concept of patient satisfaction in adolescent psychiatric care: A qualitative study. Journal of Child and Adolescent Psychiatric Nursing, 24, 3–10. doi:10.1111/j.1744-6171.2010.00261.x.PubMedCrossRef

Brannan, A., Sonnichesen, E., & Heflinger, C. (1996). Measuring satisfaction with children’s mental health services: Validity and reliability of the satisfaction scales. Evaluation and Program Planning, 19, 131–141. doi:10.1016/0149-7189(96)00004-3.CrossRef

Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150. doi:10.1207/S15327906MBR3601_05.CrossRef

Day, C., Michelson, D., & Hassan, I. (2011). Child and adolescent service experience (ChASE): Measuring service quality and therapeutic process. British Journal of Clinical Psychology, 50, 452–464. doi:10.1111/j.2044-8260.2011.02008.x.PubMedCrossRef

Department of Health. (2009). Understanding what matters: A guide to using patient feedback to transform services. London: Department of Health. Retrieved from http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_099780.

Edwards, A., & Elwyn, G. (2001). Developing professional ability to involve patients in their care: Pull or push? Quality in Health Care, 10, 129–134. doi:10.1136/qhc.0100129.PubMedCentralPubMedCrossRef

Garland, A., Saltzman, M., & Aarons, G. (2000). Adolescent satisfaction with mental health services: Development of a multidimensional scale. Evaluation and Program Planning, 23, 165–175. doi:10.1016/S0149-7189(00)00009-4.CrossRef

Goodman, R. (2001). Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ). Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345. doi:10.1097/00004583-200111000-00015.

Hart, A., Saunders, A., & Thomas, H. (2005). Attuned practice: a service user study of specialist child and adolescent mental health, UK. Epidemiology and Psychiatric Sciences, 14, 22–31. doi:10.1017/S1121189X00001895.CrossRef

Holbrook, M. B. (1983). Using a structural model of halo effect to assess perceptual distortion due to affective overtones. Journal of Consumer Research, 10, 247–252. http://www.jstor.org/stable/2488928.

Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York: Routledge.

Kazdin, A., & Mazurick, J. (1994). Dropping out of child psychotherapy: Distinguishing early and late dropouts over the course of treatment. Journal of Consulting and Clinical Psychology, 62, 1069–1074. doi:10.1177/1359104596011012.PubMedCrossRef

Kennedy, I. (2010). Getting it right for children and young people: Overcoming cultural barriers in the NHS so as to meet their needs. London: Department of Health. http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_119445.

McNaughton, D. (1994). Measuring parent satisfaction with early childhood intervention programs. Topics in Early Childhood Special Education, 14, 26–48. doi:10.1177/027112149401400106.CrossRef

Murphy, K. R., Jako, R. A., & Anhalt, R. L. (1993). Nature and consequences of halo error: A critical analysis. Journal of Applied Psychology, 78, 218–225. doi:10.1037/0021-9010.78.2.218.CrossRef

Muthén, L. K. & Muthén, B. O. (1998-2010). Mplus user’s guide. 6th Ed. Los Angeles, CA: Muthén & Muthén. www.stamodel.com.

Muthén, B. & Satorra, A. (1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267–316. http://www.jstor.org/stable/271070.

Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized multilevel structural equation modeling. Psychometrika, 69, 167–190. doi:10.1007/BF02295939.CrossRef

Reise, S., & Haviland, M. (2005). Item response theory and the measurement of clinical change. Journal of Personality Assessment, 84, 228–238. doi:10.1207/s15327752jpa8403_02.PubMedCrossRef

Reise, S., & Waller, G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 25–46. doi:10.1146/annurev.clinpsy.032408.153553.CrossRef

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society.

SDQ: Normative SDQ data from Britain. (2001). http://www.sdqinfo.com/UKNorm.html.

Shapiro, J., Welker, C., & Jacobson, B. (1997). The youth client satisfaction questionnaire: Development, construct validation and factor structure. Journal of Child Clinical Psychology, 26, 87–98. doi:10.1207/s15374424jccp2601_9.CrossRef

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107–120. doi:10.1007/s11336-008-9101-0.PubMedCentralPubMedCrossRef

Stallard, P. (1995). Parental satisfaction with intervention: differences between respondents and non-respondents to postal questionnaire. British Journal of Clinical Psychology, 34, 397–405. doi:10.1111/j.2044-8260.1995.tb01474.x.PubMedCrossRef

Thissen, D., & Orlando, M. (2001). Item response theory for items scored in two categories. In D. Thissen & H. Wainer (Eds.), Test Scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum.

Williams, B., Coyle, J., & Healy, D. (1998). The meaning of patient satisfaction: An explanation of high reported levels. Social Science and Medicine, 47, 1351–1359. doi:10.1016/S0277-9536(98)00213-5.PubMedCrossRef

CORC Measures Website http://www.corc.uk.net/index.php?contentkey=81.

Young, S., Nicholson, J., & Davis, M. (1995). An overview of issues in research on consumer satisfaction with child and adolescent mental health services. Journal of Child and Family Studies, 4, 219–238. doi:10.1007/BF02234097.CrossRef

Title: Satisfaction in Child and Adolescent Mental Health Services: Translating Users’ Feedback into Measurement
Authors: Anna Brown
Tamsin Ford
Jessica Deighton
Miranda Wolpert
Publication date: 01-07-2014
Publisher: Springer US
Published in: Administration and Policy in Mental Health and Mental Health Services Research / Issue 4/2014
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-012-0433-9

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Satisfaction in Child and Adolescent Mental Health Services: Translating Users’ Feedback into Measurement

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 4/2014

National Trainers’ Perspectives on Challenges to Implementation of an Empirically-Supported Mental Health Treatment

Innovation Adoption: A Review of Theories and Constructs

Clinical Transformation: Manager’s Perspectives on Implementation of Evidence-Based Practice

Pilot Implementation of Computerized Cognitive Behavioral Therapy in a University Health Setting

Frequency, Level, and Rate of Improvement for Treatment Targets in a Children’s Mental Health Community-Based Intensive In-Home Therapeutic Setting

Correlates of Mental Health Service Use and Type Among Asian Americans