Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
The Journal of Behavioral Health Services & Research
Published in: The Journal of Behavioral Health Services & Research 3/2007

01-07-2007 | Regular Article

Measuring Clinically Meaningful Change Following Mental Health Treatment

Authors: Susan V. Eisen, PhD, Gayatri Ranganathan, MS, Pradipta Seal, MS, Avron Spiro III, PhD

Published in: The Journal of Behavioral Health Services & Research | Issue 3/2007

Login to get access

Abstract

Assessment of clinically meaningful change is useful for treatment planning, monitoring progress, and evaluating treatment response. Outcome studies often assess statistically significant change, which may not be clinically meaningful. Study objectives are to: (1) evaluate responsiveness of the BASIS-24© using three methods for determining clinically meaningful change: reliable change index (RCI), effect size (ES), and standard error of measurement (SEM); and (2) determine which method provides an estimate of clinically meaningful change most concordant with other change measures. BASIS-24© assessments were obtained at two time points for 1,397 inpatients and 850 outpatients. The proportion showing clinically meaningful change using each method was compared to the proportion showing change in global mental health, retrospectively reported change, and clinician-assessed change. BASIS-24© demonstrated responsiveness at both aggregate and individual levels. Regarding clinically meaningful improvement and decline, SEM was most concordant with all three outcome measures; regarding no change, RCI was most concordant with all three measures.
Literature
1.
Newman FL, Ciarlo JA, Carpenter D. Guidelines for selecting psychological instruments for treatment planning and outcome assessment. In: Maruish M, ed. The Use of Psychological Testing for Treatment Planning and Outcomes Assessment 2nd Edition. Mahwah, NJ: Lawrence Erlbaum; 1999:153–170.
2.
Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. Journal of Chronic Disease. 1987;40:171–178.CrossRef
3.
Liang MH. Longitudinal construct validity: Establishment of clinical meaning in patient evaluation instruments. Medical Care. 2000;38(Suppl II):S84–S90.
4.
Hays RD, Anderson R, Revicki D. Psychometric considerations in evaluating health-related quality of life measures. Quality of Life Research. 1993;2:441–449.PubMedCrossRef
5.
Wyrwich KW, Wolinsky FD. Identifying meaningful intra-individual change standards for health-related quality of life measures. Journal of Evaluation in Clinical Practice. 2000;6:39–49.PubMedCrossRef
6.
McHorney CA, Tarlow AR. Individual patient monitoring in clinical practice: are available health status surveys adequate? Quality of Life Research. 1995;4(4):293–307.PubMedCrossRef
7.
Hays RD, Brodsky M, Johnston MF, et al. Evaluating the statistical significance of health-related quality-of-life change in individual patients. Evaluation & the Health Professions. 2005;28:160–171.CrossRef
8.
Wyrwich KW, Bullinger M, Aaronson N, et al. Estimating clinically significant differences in quality of life outcomes. Quality of Life Research. 2005;14:285–295.PubMedCrossRef
9.
Glossary. Health outcomes methodology. Medical Care. 2000;38(9 Suppl. II,II-7-II-13):2–10).
10.
Atkins DC, Bedics JD, McGlinchey JB, et al. Assessing clinical significance: Does it matter which method we use? Journal of Consulting & Clinical Psychology. 2005;73:982–989.CrossRef
11.
Eisen SV, Normand SLT, Belanger AJ, et al. The revised Behavior and Symptom Identification Scale (BASIS-24©): Reliability and validity. Medical Care. 2004;42:1230–1241.PubMedCrossRef
12.
Guyatt GH, Norman GR, Juniper EF, et al. A critical look at transition ratings. Journal of Clinical Epidemiology. 2002;55:900–908.PubMedCrossRef
13.
Spitzer RL, Gibbon M, Endicott J. Global Assessment Scale (GAS), Global Assessment of Functioning (GAF) Scale, Social and Occupational Functioning Assessment Scale (SOFAS). In: Handbook of Psychiatric Measures. Washington DC: American Psychiatric Association; 2000:96–100.
14.
Kazdin AE. Assessing the clinical or applied importance of behavior change through social validation. Behavior Modification. 1977;1:427–452.CrossRef
15.
Streiner DL, Norman GR.Health Measurement Scales. A Practical Guide to Their Development and Use (3rd ed.). New York: Oxford University Press 2003
16.
Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimally important difference. Controlled Clinical Trials. 1991;10:407–415.CrossRef
17.
Hurst H, Bolton J. Assessing the clinical significance of change scores recorded on subjective outcome measures. Journal of Manipulative Physiological Therapeutics. 2004;27:26–35.CrossRef
18.
Ferguson RJ, Robinson AB, Splaine M. Use of the reliable change index to evaluate clinical significance in SF-36 outcomes. Quality of Life Research. 2002;11:509–516.PubMedCrossRef
19.
Clancy C, Eisenberg J. Outcomes research care: measuring the end results of health care. Science. 1998;282:245–246.PubMedCrossRef
20.
Cohen J.Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
21.
Testa M. Interpreting quality of life clinical trial data for use in the clinical practice of antihypertensive therapy. Journal of Hypertension Supplement. 1987;5:S9–S13.
22.
Kazis Le, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Medical Care. 1989;27(Suppl 3):S178–S189.PubMedCrossRef
23.
Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life. The remarkable universality of half a standard deviation. Medical Care. 2003;5:582–592.CrossRef
24.
Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology. 1991;59:12–19.PubMedCrossRef
25.
Jacobson NS, Follette WC, Revenstorf. Psychotherapy outcome research. Methods for reporting variability and evaluating clinical significance. Behavior Therapy. 1984;15:336–352.CrossRef
26.
Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: description, application and alternatives. Journal of Consulting and Clinical Psychology. 1999;67:300–307.PubMedCrossRef
27.
Speer D. Methodological developments. Clinically significant change: Jacobson and Truax (1991) revisited. Journal of Consulting and Clinical Psychology. 1992;69(3):402–408.CrossRef
28.
Hageman WJM, Arrindell WA. A further refinement of the reliable change (RC) index by improving the pre–post difference score: introducing RCID. Behaviour Research and Therapy. 1993;31:693–700.PubMedCrossRef
29.
Anastasi A, Urbina S.Psychological Testing (7th ed.). Upper Saddle River, NJ: Prentice-Hall; 1997.
30.
Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. Journal of Clinical Epidemiology. 1999;52:861–873.PubMedCrossRef
31.
Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky FD. Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Medical Care. 1999;37:469–478.PubMedCrossRef
32.
Eisen SV, Dill DL, Grob MC. Reliability and validity of a brief patient-report instrument for psychiatric outcome evaluation. Hospital & Community Psychiatry. 1994;45:242–247.
33.
American Psychiatric Association.Diagnostic and Statistical Manual of Mental Disorders. 4th ed. (DSM-IV). Washington D.C.: American Psychiatric Association; 1994.
34.
Burlingame GM, Dunn TW, Chen S, et al. Selection of outcome assessment instruments for inpatients with severe and persistent mental illness. Psychiatric Services. 2005;56:444–451.PubMedCrossRef
35.
Eisen SV, Gerena M, Rangathan G, et al. Reliability and validity of the BASIS-24© Mental Health Survey for Whites, African-Americans and Latinos. Journal of Behavioral Health Services & Research. 2006;33:304–323.CrossRef
36.
Jerrell JM. Behavior and Symptom Identification Scale 32. Sensitivity to change over time. Journal of Behavioral Health Services & Research. 2005;32:341–346.CrossRef
37.
Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement. 1960;20:37–46.CrossRef
38.
Lambert MJ, Lambert JM. Use of psychological tests for assessing treatment outcome. In: Maruish ME, ed. The Use of Psychological Testing for Treatment Planning and Outcomes Assessment (2nd ed.). Mahwah, NJ: Lawrence Erlbaum; 1999:115–152.
39.
Jacobson NS, Roberts LJ, Berns SB, et al. Methods for defining and determining the clinical significance of treatment effects. Journal of Consulting and Clinical Psychology. 1999;67:300–307.PubMedCrossRef
40.
Ogles BM, Lambert MJ, Sawyer JD. Clinical significance of the National Institute of Mental Health treatment of depression collaborative research program data. Journal of Consulting and Clinical Psychology. 1995;63:321–326.PubMedCrossRef
41.
Guyatt H, Osoba D, Wu A, et al. Methods to explain the clinical significance of health status measures. Mayo Clinic Proceedings. 2002;77(4):373–383.
42.
Jones SH, Thornicroft G, Coffey M, et al. A brief mental health outcome scale-reliability and validity of the Global Assessment of Functioning (GAF). British Journal of Psychiatry. 1995;166:654–650.PubMedCrossRef
43.
Startup M, Jackson MC, Bendix S. The concurrent validity of the Global Assessment of Functioning (GAF). British Journal of Clinical Psychology. 2002;41:417–422.PubMedCrossRef
44.
Greenberg GA, Rosenheck RA. Using the GAS as a national mental health outcome measure in the Department of Veterans Affairs. Psychiatric Services. 2005;56:420–426.PubMedCrossRef
45.
Wilkinson G, Hesdon B, Wild D, et al. Self-report quality of life measure for people with schizophrenia: the SQLS. British Journal of Psychiatry. 2000;177:42–46.PubMedCrossRef
46.
Srebnick D, Hendryx M, Stevenson J, et al. Development of outcome indicators for monitoring the quality of public mental health care. Psychiatric Services. 1997;48:903–909.
47.
Norman GR, Sridhar FG, Guyatt GH. Relation of distribution- and anchor-based approaches in interpretation of changes in health-related quality of life. Medical Care. 2001;39:1039–1047.PubMedCrossRef
48.
Nunnally JC.Psychometric Theory. New York: McGraw-Hill; 1994.
49.
Gilbody SM, House AO, Sheldon T. Routine administration of health related quality of life (HRQoL) and needs assessment instruments to improve psychological outcome—a systematic review. Psychological Medicine. 2002;32:1345–1356.PubMedCrossRef
50.
Garland AF, Kruse M, Aarons GA. Clinicians and outcome measurement: What’s the use? Journal of Behavioral Health Services & Research. 2003;30:393–405.CrossRef
Metadata
Title
Measuring Clinically Meaningful Change Following Mental Health Treatment
Authors
Susan V. Eisen, PhD
Gayatri Ranganathan, MS
Pradipta Seal, MS
Avron Spiro III, PhD
Publication date
01-07-2007
Publisher
Springer US
Published in
The Journal of Behavioral Health Services & Research / Issue 3/2007
Print ISSN: 1094-3412
Electronic ISSN: 2168-6793
DOI
https://doi.org/10.1007/s11414-007-9066-2

Other articles of this Issue 3/2007

The Journal of Behavioral Health Services & Research 3/2007 Go to the issue

Letter to the Editor

Gain Is Not Always Good

Regular Article

Discontinuation of Antidepressant Medication Among Latinos in the USA

Regular Article

Case Management and Client Access to Health and Social Services in Outpatient Substance Abuse Treatment

Regular Article

Consistent Adherence To Guidelines Improves Opioid Dependent Patients’ First Year Outcomes

Regular Article

The Sustainability of Systems of Care for Children’s Mental Health: Lessons Learned

Regular Article

Does State Certification or Licensure Influence Outpatient Substance Abuse Treatment Program Practices?