Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

ACC 2024 Congress

Catch up on all the top stories and latest evidence in cardiology research with our ACC 2024 Congress hub page.
ADVANCED SEARCH
Log in
Top
Social Psychiatry and Psychiatric Epidemiology
Published in: Social Psychiatry and Psychiatric Epidemiology 2/2014

01-02-2014 | Original Paper

Defining multiple criteria for meaningful outcome in routine outcome measurement using the Health of the Nation Outcome Scales

Authors: Alberto Parabiaghi, Hans E. Kortrijk, Cornelis L. Mulder

Published in: Social Psychiatry and Psychiatric Epidemiology | Issue 2/2014

Login to get access

Abstract

Purpose

Using the reliable and clinically significant change approach, we aimed to identify meaningful outcome indicators for the Health of the Nation Outcome Scales (HoNOS) and to combine them in a single model. We applied these indicators to the 1-year outcome of two large samples of people attending community mental health services in Italy (cohort 1) and the Netherlands (cohort 2).

Methods

Data were drawn from two studies on routine outcome assessment. The criteria for meaningful outcome were defined on both study cohorts and both language versions of the scale. The model combined (a) two criteria for adequate change (at least 4 or 8 points change), (b) two cut-offs for clinically significant change (a total score of 10 was the threshold between mild and moderate illness, 13 between moderate and severe illness), and (c) a method for classifying stable subjects in three degrees of severity (stable in mild, moderate or severe illness). Results were compared with those given by the effect size (ES) and analysis of variance and covariance (ANOVA and ANCOVA).

Results

For the proposed approach the outcome of cohort 1 was better than cohort 2, with 65–67 % of its subjects showing a positive outcome compared to only 45–46 %. The other reference methods (ES and ANOVA), however, showed a greater improvement for cohort 2. ANCOVA indicated that the differences were due to regression to the mean (RTM) which showed opposite effects across the two cohorts.

Conclusions

The proposed approach proved valuable and generalizable for interpreting outcome on HoNOS, scarcely influenced by the RTM effect. Its introduction could benefit outcome evaluation and management.
Appendix
Available only for authorised users
Footnotes
1
The population mean should be a known true mean in a population. In the present study, the mean of the aggregate sample (i.e. both sites) was used as an estimate of μ.
 
Literature
1.
Department of Health (2001) Mental health information strategy. Department of Health, London
2.
Commonwealth Department of Health and Ageing (2002) National outcome and casemix collection: overview of clinical measures and data items. Commonwealth Department of Health and Ageing, Canberra
3.
Ministry of Health (2005) National mental health information strategy 2005–2010. Ministry of Health, Wellington
4.
Trauer T (2010) Outcome measurement around the world. In: Trauer T (ed) Outcome measurement in mental health. Cambridge University Press, Cambridge, pp 13–101
5.
Norman GR, Sridhar FG, Guyatt GH (2001) Relation of distribution- and anchor-based approaches in interpretation of changes in health-related quality of life. Med Care 39:1039–1047PubMedCrossRef
6.
Burgess P, Pirkis J (2008) Key performance indicators for Australian Public Mental Health Services—potential contributions of MH-NOCC data: developing indicators of effectiveness Version 2.0, Brisbane, Queensland
7.
Eisen SV, Ranganathan G, Seal P, Spiro A 3rd (2007) Measuring clinically meaningful change following mental health treatment. J Behav Health Serv Res 34:272–289PubMedCrossRef
8.
Trauer T (2010) Assessment of change in outcome measurement. In: Trauer T (ed) Outcome measurement in mental health. Cambridge University Press, Cambridge, pp 206–218
9.
Speer DC (1999) What is the role of two-wave designs in clinical research? Comment on Hageman and Arrindell. Behav Res Ther 37:1203–1210PubMedCrossRef
10.
Cella D, Bullinger M, Scott C, Barofsky I et al (2002) Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clin Proc 77:384–392PubMedCrossRef
11.
12.
Crosby RD, Kolotkin RL, Williams GR (2003) Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol 56:395–407PubMedCrossRef
13.
Jacobson NS, Follette WC, Revenstorf D, Baucom DH et al (1984) Variability in outcome and clinical significance of behavioral marital therapy: a reanalysis of outcome data. J Consult Clin Psychol 52:497–504PubMedCrossRef
14.
Christensen L, Mendoza JL (1986) A method of assessing change in a single subject: an alteration of the RC index. Behav Therapy 17:305–308CrossRef
15.
Jacobson NS, Truax P (1991) Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 59:12–19PubMedCrossRef
16.
Hageman WJ, Arrindell WA (1993) A further refinement of the reliable change (RC) index by improving the pre-post difference score: introducing RCID. Behav Res Ther 31:693–700PubMedCrossRef
17.
Evans C, Margison F, Barkham M (1998) The contribution of reliable and clinically significant change methods to evidence-based mental health. Evid Based Ment Health 1:70–72CrossRef
18.
Hageman WJ, Arrindell WA (1999) Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis. Behav Res Ther 37:1169–1193PubMedCrossRef
19.
Hsu LM (1999) A comparison of three methods of identifying reliable and clinically significant client changes: commentary on Hageman and Arrindell. Behav Res Ther 37:1195–1202PubMedCrossRef
20.
Hageman WJ, Arrindell WA (1999) Clinically significant and practical! Enhancing precision does make a difference. Reply to McGlinchey and Jacobson, Hsu, and Speer. Behav Res Ther 37:1219–1233PubMedCrossRef
21.
McGlinchey JB, Atkins DC, Jacobson NS (2002) Clinical significance methods: which one to use and how useful are they? Behav Ther 33:529–550CrossRef
22.
Burgess P, Pirkis J, Coombs T (2009) Modelling candidate effectiveness indicators for mental health services. Aust N Z J Psychiatry 43:531–538PubMedCrossRef
23.
Cohen J (1969) Statistical power analysis for the behavioural sciences. Academic Press, London
24.
Kazis LE, Anderson JJ, Meenan RF (1989) Effect sizes for interpreting changes in health status. Med Care 27(Suppl 3):178–189CrossRef
25.
McHorney CA, Tarlov AR (1995) Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 4:293–307PubMedCrossRef
26.
Grundy CT, Lambert MJ, Grundy EM (1996) Assessing clinical significance: application to the Hamilton Rating Scale for depression. J Ment Health 5:25–33CrossRef
27.
Hafkenscheid A (2000) Psychometric measures of individual change: an empirical comparison with the Brief Psychiatric Rating Scale (BPRS). Acta Psychiatr Scand 101:235–242PubMedCrossRef
28.
Audin K, Margison FR, Clark JM, Barkham M (2001) Value of HoNOS in assessing patient change in NHS psychotherapy and psychological treatment services. Br J Psychiatry 178:561–566PubMedCrossRef
29.
Matthey S (2004) Calculating clinically significant change in postnatal depression studies using the Edinburgh Postnatal Depression Scale. J Affect Disord 78:269–272PubMedCrossRef
30.
Parabiaghi A, Barbato A, D’Avanzo B, Erlicher A et al (2005) Assessing reliable and clinically significant change on Health of the Nation Outcome Scales: method for displaying longitudinal data. Aust N Z J Psychiatry 39:719–725PubMedCrossRef
31.
Barbato A, Parabiaghi A, Panicali F, Battino N et al (2011) Do patients improve after short psychiatric admission? A cohort study in Italy. Nord J Psychiatry 65:251–258PubMedCrossRef
32.
Parabiaghi A, Rapisarda F, D’Avanzo B, Erlicher A et al (2011) Measuring clinical change in routine mental health care: differences between first time and longer term service users. Aust N Z J Psychiatry 45:558–568PubMedCrossRef
33.
Bauer S, Lambert MJ, Nielsen SL (2003) Clinical significance methods: a comparison of statistical techniques. J Pers Assess 82:60–70CrossRef
34.
Speer DC (1992) Clinically significant change: jacobson & Truax (1991) revisited. J Consult Clin Psych 60:402–408CrossRef
35.
Kortrijk HE, Staring AB, van Baars AW, Mulder CL (2010) Involuntary admission may support treatment outcome and motivation in patients receiving assertive community treatment. Soc Psychiatry Psychiatr Epidemiol 45:245–252PubMedCrossRef
36.
Kortrijk HE, Mulder CL, Drukker M, Wiersma D, Duivenvoorden HJ (2012) Duration of assertive community treatment and the interpretation of routine outcome data. Aust N Z J Psychiatry 46:240–248PubMedCrossRef
37.
Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A (1998) Health of the Nation Outcome Scales (HoNOS). Research and development. Br J Psychiatry 172:11–18PubMedCrossRef
38.
Pirkis JE, Burgess PM, Kirk PK, Dodson S, Coombs TJ, Williamson MK (2005) A review of the psychometric properties of the Health of the Nation Outcome Scales (HoNOS) family of measures. Health Qual Life Outcomes 3:76PubMedCentralPubMedCrossRef
39.
Rossi R, Blaco R, Castelli C, Civenti G et al (1999) Il costo dei pazienti psichiatrici per classi di gravita. Epidemiol Psichiatr Soc 7:198–208CrossRef
40.
Lora A, Bai G, Bianchi S, Bolongaro G et al (2001) La versione italiana della HoNOS (Health of the Nation Outcome Scales), una scala per la valutazione della gravita e dell’esito nei servizi di salute mentale. Epidemiol Psichiatr Soc 10:198–212CrossRef
41.
Mulder CL, Staring ABP, Loos J, Buwalda V et al (2004) De Health of the Nations Outcome Scales (HoNOS) als instrument voor routine outcome assessment. Tijdschr Psychiatr 46:273–285
42.
Lelliott P (1999) Definition of severe mental illness. In: Charlwood P, Mason A, Goldacre M, Cleary R, Wilkinson E (eds) Health outcome indicators: severe mental illness. Report of a working group to the Department of Health. National Centre for Health Outcomes Development, Oxford, pp 87–93
43.
Barnett AG, Van Der Pols JC, Dobson AJ (2005) Regression to the mean: what it is and how to deal with it. Int J Epidemiol 34:215–220PubMedCrossRef
44.
Gardner MJ, Heady JA (1973) Some effects of within-person variability. J Chronic Dis 26:781–795CrossRef
45.
Davis CE (1976) The effect of regression to the mean in epidemiologic and clinical studies. Am J Epidemiol 104:493–498PubMed
46.
47.
Harmon C, Hawkins EJ, Lambert MJ, Slade K, Whipple JS (2005) Improving outcomes for poorly responding clients: the use of clinical support tools and feedback to clients. J Clin Psychol 61:175–185PubMedCrossRef
48.
Drukker M, Bak M, à Campo J, Driessen G, van Os J, Delespaul P (2010) The cumulative needs for care monitor: a unique monitoring system in the south of the Netherlands. Soc Psychiatry Psychiatr Epidemiol 45:475–485PubMedCentralPubMedCrossRef
Metadata
Title
Defining multiple criteria for meaningful outcome in routine outcome measurement using the Health of the Nation Outcome Scales
Authors
Alberto Parabiaghi
Hans E. Kortrijk
Cornelis L. Mulder
Publication date
01-02-2014
Publisher
Springer Berlin Heidelberg
Published in
Social Psychiatry and Psychiatric Epidemiology / Issue 2/2014
Print ISSN: 0933-7954
Electronic ISSN: 1433-9285
DOI
https://doi.org/10.1007/s00127-013-0750-7

Other articles of this Issue 2/2014

Social Psychiatry and Psychiatric Epidemiology 2/2014 Go to the issue

Original Paper

Stressful psychosocial school environment and suicidal ideation in Chinese adolescents

Original Paper

Occupational functioning and work impairment in association with personality disorder trait-scores

Original Paper

Interpersonal functioning deficits in association with DSM-IV personality disorder dimensions

Original Paper

Gender differences in the relationship between school problems, school class context and psychological distress: results from the Young-HUNT 3 study

Original Paper

The impact of a diagnosis of personality disorder on service usage in an adult Community Mental Health Team

Original Paper

Parental socio-economic position during childhood as a determinant of self-harm in adolescence