Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Psychiatry
Published in: BMC Psychiatry 1/2018

Open Access 01-12-2018 | Research article

Treatment results for severe psychiatric illness: which method is best suited to denote the outcome of mental health care?

Authors: Edwin de Beurs, Matthijs Blankers, Philippe Delespaul, Erik van Duijn, Niels Mulder, Annet Nugter, Wilma Swildens, Bea G. Tiemens, Jan Theunissen, Arno F. A. van Voorst, Jaap van Weeghel

Published in: BMC Psychiatry | Issue 1/2018

Login to get access

Abstract

Background

The present study investigates the suitability of various treatment outcome indicators to evaluate performance of mental health institutions that provide care to patients with severe mental illness. Several categorical approaches are compared to a reference indicator (continuous outcome) using pretest-posttest data of the Health of Nation Outcome Scales (HoNOS).

Methods

Data from 10 institutions and 3189 patients were used, comprising outcomes of the first year of treatment by teams providing long-term care.

Results

Findings revealed differences between continuous indicators (standardized pre-post difference score ES and ΔT) and categorical indicators (SEM, JTRCI, JTCS, JTRCI&CS, JTrevised) on their ranking of institutions, as well as substantial differences among categorical indicators; the outcome according to the traditional JT approach was most concordant with the continuous outcome indicators.

Conclusions

For research comparing group averages, a continuous outcome indicator such as ES or ΔT is preferred, as this best preserves information from the original variable. Categorical outcomes can be used to illustrate what is accomplished in clinical terms. For categorical outcome, the classical Jacobson-Truax approach is preferred over the more complex method of Parabiaghi et al. with eight outcome categories. The latter may be valuable in clinical practice as it allows for a more detailed characterization of individual patients.
Literature
1.
Lambert MJ. Presidential address: what we have learned from a decade of research aimed at improving psychotherapy outcome in routine care. Psychother Res. 2007;17(1):1–14.CrossRef
2.
de Beurs E, den Hollander-Gijsman ME, van rood YR, van der wee NJ, Giltay EJ, van Noorden MS, van der Lem R, E vF, Zitman FG. Routine outcome monitoring in the Netherlands: practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clin Psychol Psychother. 2011;18(1):1–12.
3.
Ellwood PM. Shattuck lecture--Outcomes management. A technology of patient experience N Engl J Med. 1988;318(23):1549–56.PubMed
4.
de Beurs E, Barendregt M, Warmerdam L, editors. Behandeluitkomsten: bron voor kwaliteitsbeleid in de GGZ [Treatment outcome: source of quality management in mental Health Care]. Amsterdam: Boom; 2017.
5.
Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A. Health of the nation outcome scales (HoNOS). Research and development Br J Psychiatry. 1998;172(1):11–8.CrossRefPubMed
6.
Pirkis JE, Burgess PM, Kirk PK, Dodson S, Coombs TJ, Williamson MK. A review of the psychometric properties of the health of the nation outcome scales (HoNOS) family of measures. Health Qual Life Outcomes. 2005;3(1):1–12.CrossRef
7.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
8.
Seidel JA, Miller SD, Chow DL. Effect size calculations for the clinician: methods and comparability. Psychother Res. 2014;24(4):470–84.CrossRefPubMed
9.
de Beurs E, Barendregt M, de Heer A, van Duijn E, Goeree B, Kloos M, Kooiman K, Lionarons H, Merks A. Comparing methods to denote treatment outcome in clinical research and benchmarking mental health care. Clin Psychol Psychother. 2016;23:308–18.CrossRefPubMed
10.
Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. J Consult Clin Psychol. 1999;67(3):300–7.CrossRefPubMed
11.
Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991;59(1):12–9.CrossRefPubMed
12.
Jacobson NS, Follette WC, Revenstorf D. Toward a standard definition of clinically significant change. Behav Ther. 1986;17:308–11.CrossRef
13.
Hageman WJJM, Arrindell WA. Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis. Behav Res Ther. 1999;37(12):1169–93.CrossRefPubMed
14.
15.
Tingey R, Lambert M, Burlingame G, Hansen N. Assessing clinical significance: proposed extensions to method. Psychother Res. 1996;6(2):109–23.CrossRefPubMed
16.
Wise EA. Methods for analyzing psychotherapy outcomes: a review of clinical significance, reliable change, and recommendations for future directions. J Pers Assess. 2004;82(1):50–9.CrossRefPubMed
17.
Atkins DC, Bedics JD, McGlinchey JB, Beauchaine TP. Assessing clinical significance: does it matter which method we use? J Consult Clin Psychol. 2005;73(5):982–9.CrossRefPubMed
18.
Ogles BM, Lunnen KM, Bonesteel K. Clinical significance: history, application, and current practice. Clin Psychol Rev. 2001;21(3):421–46.CrossRefPubMed
19.
Lutz W, Stulz N, Martinovich Z, Leon S, Saunders SM. Methodological background of decision rules and feedback tools for outcomes management in psychotherapy. Psychother Res. 2009;19(4–5):502–10.CrossRefPubMed
20.
de Beurs E, Barendregt M, Rogmans B, Robbers S, van Geffen M, Van Aggelen-Gerrits M, Houben H. Denoting treatment outcome in child and adolescent psychiatry: a comparison of continuous and categorical outcomes. Eur Child Adolesc Psychiatry. 2015;23:553–63.CrossRef
21.
Audin K, Margison FR, Clark JM, Barkham M. Value of HoNOS in assessing patient change in NHS psychotherapy and psychological treatment services. Br J Psychiatry. 2001;178(6):561–6.CrossRefPubMed
22.
Parabiaghi A, Kortrijk HE, Mulder CL. Defining multiple criteria for meaningful outcome in routine outcome measurement using the health of the nation outcome scales. Soc Psychiatry Psychiatr Epidemiol. 2014;49(2):291–305.CrossRefPubMed
23.
Parabiaghi A, Barbato A, D'Avanzo B, Erlicher A, Lora A. Assessing reliable and clinically significant change on health of the nation outcome scales: method for displaying longitudinal data. Aust N Z J Psychiatry. 2005;39(8):719–24.CrossRefPubMed
24.
Burgess P, Pirkis J, Coombs T. Modelling candidate effectiveness indicators for mental health services. Aust N Z J Psychiatry. 2009;43(6):531–8.CrossRefPubMed
25.
Mulder CL, van der Gaag M, Bruggeman R, Cahn W, Delespaul PA, Dries P, Faber G, de Haan L, van der Heijden FM, Kempen RW, et al. Routine Outcpme monitoring voor patiënten met ernstige psychiatrische aandoeningen: een consensus document [routine outcome monitoring for patients with severe mental illness: a consensus document]. Tijdschr Psychiatr. 2010;52(3):169–79.PubMed
26.
Trauer T, editor. Outcome measurement in mental health. Cambridge: Cambridge University Press; 2010.
27.
Brooks R. The reliability and validity of the health of the nation outcome scales: validation in relation to patient derived measures. Aust N Z J Psychiatry. 2000;34(3):504–11.CrossRefPubMed
28.
Slade M, Beck A, Bindman J, Thornicroft G, Wright S. Routine clinical outcome measures for patients with severe mental illness: CANSAS and HoNOS. Br J Psychiatry. 1999;174(5):404–8.CrossRefPubMed
29.
Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care. 1989;27(3, S):S178–89.CrossRefPubMed
30.
McCall WA. How to measure in education. New York: MacMillan; 1922.
31.
Stevens SS. On the theory of scales of measurement. Science. 1946;103(2684):677–80.CrossRef
32.
McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4(4):293–307.CrossRefPubMed
33.
34.
Markon KE, Chmielewski M, Miller CJ. The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. Psychol Bull. 2011;137(5):856–79.CrossRefPubMed
35.
Twomey C, Prina AM, Baldwin DS, Das-Munshi J, Kingdon D, Koeser L, Prince MJ, Stewart R, Tulloch AD, Cieza A. Utility of the health of the nation outcome scales (HoNOS) in predicting mental health service costs for patients with common mental health problems: historical cohort study. PLoS One. 2016;11(11):e0167103.CrossRefPubMedPubMedCentral
36.
Speak BL, Hay P, Muncer SJ. HoNOS–their utility for payment by results in mental health. Int J Health Care Qual Assur. 2015;28(2):115–28.CrossRefPubMed
37.
Muncer S, Speak B. Confirmatory factor analysis of a two scale model of the health of the nation outcome scales (HoNOS) across diagnostic categories. Psychiatry Res. 2017;247:12–4.CrossRefPubMed
Metadata
Title
Treatment results for severe psychiatric illness: which method is best suited to denote the outcome of mental health care?
Authors
Edwin de Beurs
Matthijs Blankers
Philippe Delespaul
Erik van Duijn
Niels Mulder
Annet Nugter
Wilma Swildens
Bea G. Tiemens
Jan Theunissen
Arno F. A. van Voorst
Jaap van Weeghel
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Psychiatry / Issue 1/2018
Electronic ISSN: 1471-244X
DOI
https://doi.org/10.1186/s12888-018-1798-4

Other articles of this Issue 1/2018

BMC Psychiatry 1/2018 Go to the issue

Research article

A systematic review of co-responder models of police mental health ‘street’ triage

Research article

Inclusion of people of color in psychedelic-assisted psychotherapy: a review of the literature

Research article

Anaemia and depression before and after birth: a cohort study based on linked population data

Study protocol

Evaluation of a multimodal school-based depression and suicide prevention program among Dutch adolescents: design of a cluster-randomized controlled trial

Study protocol

N-acetylcysteine as add-on to antidepressant medication in therapy refractory major depressive disorder patients with increased inflammatory activity: study protocol of a double-blind randomized placebo-controlled trial

Research article

Associations between health-related self-efficacy and suicidality