Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Psychiatry
Published in: BMC Psychiatry 1/2012

Open Access 01-12-2012 | Research article

Reference values for generic instruments used in routine outcome monitoring: the leiden routine outcome monitoring study

Authors: Yvonne WM Schulte-van Maaren, Ingrid VE Carlier, Frans G Zitman, Albert M van Hemert, Margot WM de Waal, Martijn S van Noorden, Erik J Giltay

Published in: BMC Psychiatry | Issue 1/2012

Login to get access

Abstract

Introduction

The Brief Symptom Inventory (BSI), Mood & Anxiety Symptom Questionnaire −30 (MASQ-D30), Short Form Health Survey 36 (SF-36), and Dimensional Assessment of Personality Pathology-Short Form (DAPP-SF) are generic instruments that can be used in Routine Outcome Monitoring (ROM) of patients with common mental disorders. We aimed to generate reference values usually encountered in 'healthy' and ‘psychiatrically ill’ populations to facilitate correct interpretation of ROM results.

Methods

We included the following specific reference populations: 1294 subjects from the general population (ROM reference group) recruited through general practitioners, and 5269 psychiatric outpatients diagnosed with mood, anxiety, or somatoform (MAS) disorders (ROM patient group). The outermost 5% of observations were used to define limits for one-sided reference intervals (95th percentiles for BSI, MASQ-D30 and DAPP-SF, and 5th percentiles for SF-36 subscales). Internal consistency and Receiver Operating Characteristics (ROC) analyses were performed.

Results

Mean age for the ROM reference group was 40.3 years (SD=12.6) and 37.7 years (SD=12.0) for the ROM patient group. The proportion of females was 62.8% and 64.6%, respectively. The mean for cut-off values of healthy individuals was 0.82 for the BSI subscales, 23 for the three MASQ-D30 subscales, 45 for the SF-36 subscales, and 3.1 for the DAPP-SF subscales. Discriminative power of the BSI, MASQ-D30 and SF-36 was good, but it was poor for the DAPP-SF. For all instruments, the internal consistency of the subscales ranged from adequate to excellent.

Discussion and conclusion

Reference values for the clinical interpretation were provided for the BSI, MASQ-D30, SF-36, and DAPP-SF. Clinical information aided by ROM data may represent the best means to appraise the clinical state of psychiatric outpatients.
Appendix
Available only for authorised users
Literature
1.
De Beurs E, Den Hollander-Gijsman ME, Van Rood YR, Van der Wee NJ, Giltay EJ, Van Noorden MS, Van der Lem R, Van Fenema E, Zitman FG: Routine outcome monitoring in the Netherlands: practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clin Psychol Psychother. 2011, 18: 1-12. 10.1002/cpp.696.CrossRefPubMed
2.
3.
Solberg HE: Establishment and use of reference values. Fundamentals of clinical chemistry. 6 edition. Edited by: Burtis CA, Ashwood ER, Bruns DE. 2008, St. Louis, Missouri: Saunders Elsevier, 229-238.
4.
Geffre A, Friedrichs K, Harr K, Concordet D, Trumel C, Braun JP: Reference values: a review. Vet Clin Pathol. 2009, 38: 288-298. 10.1111/j.1939-165X.2009.00179.x.CrossRefPubMed
5.
Katayev A, Balciza C, Seccombe DW: Establishing reference intervals for clinical laboratory test results: is there a better way?. Am J Clin Pathol. 2010, 133: 180-186. 10.1309/AJCPN5BMTSF1CDYP.CrossRefPubMed
6.
Sasse EA, Doumas BT, Miller WG, D'Orazio P, Eckfeldt JH, Evans SA, Graham GA, Myers GL, Parsons PJ, Stanton NV: How to define and determine reference intervals in the clinical laboratory; approved guideline-Second edition. NCCLS document C28-A2. 2000, Wayne, PA: NCCLS
7.
Kendall PC, Marrs-Garcia A, Nath SR, Sheldrick RC: Normative comparisons for the evaluation of clinical significance. J Consult Clin Psychol. 1999, 67: 285-299.CrossRefPubMed
8.
Horn PS, Feng L, Li Y, Pesce AJ: Effect of outliers and nonhealthy individuals on reference interval estimation. Clin Chem. 2001, 47: 2137-2145.PubMed
9.
Reed AH, Henry RJ, Mason WB: Influence of statistical method used on the resulting estimate of normal range. Clin Chem. 1971, 17: 275-284.PubMed
10.
Fan J, Upadhye S, Worster A: Understanding receiver operating characteristic (ROC) curves. CJEM. 2006, 8: 19-20.PubMed
11.
De Beurs E, Zitman FG: De Brief Symptom Inventory (BSI): De betrouwbaarheid en validiteit van een handzaam alternatief voor de SCL-90. [The Brief Symptom Inventory (BSI): The reliability and validity of a brief alternative of the SCL-90]. Maandblad Geestelijke Volksgezondheid. 2006, 61: 120-141.
12.
DeRogatis LR: The Brief Symptom Inventory. 1975, Baltimore: MD., Clinical Psychometric Research
13.
Clark LA, Watson D: Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications. J Abnorm Psychol. 1991, 100: 316-336.CrossRefPubMed
14.
Wardenaar KJ, Van Veen T, Giltay EJ, De Beurs E, Penninx BW, Zitman FG: Development and validation of a 30-item short adaptation of the Mood and Anxiety Symptoms Questionnaire (MASQ). Psychiatry Res. 2010, 179: 101-106. 10.1016/j.psychres.2009.03.005.CrossRefPubMed
15.
Aaronson NK, Muller M, Cohen PDA, Essink-Bot ML, Fekkes M, Sanderman R, Sprangers MAG, Velde AT, Verrips E: Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease populations. J Clin Epidemiol. 1998, 51: 1055-1068. 10.1016/S0895-4356(98)00097-3.CrossRefPubMed
16.
Ware JE, Snow KK, Kosinski M, Gandek B: SF-36 Health Survey Manual and Interpretation Guide. 1993, Boston: New England Medical Center, The Health Institute
17.
Livesley WJ, Jackson DN: Manual for the dimensional assessment of personality pathology - basic questionnaire (DAPP-BQ). 2002, Port Huron: Sigma Press
18.
Van Kampen D, De Beurs E, Andrea H: A short form of the Dimensional Assessment of Personality Pathology-Basic Questionnaire (DAPP-BQ): the DAPP-SF. Psychiatry Res. 2008, 160: 115-128. 10.1016/j.psychres.2007.05.002.CrossRefPubMed
19.
DeRogatis LR, Melisaratos N: The Brief Symptom Inventory: an introductory report. Psychol Med. 1983, 13: 595-605. 10.1017/S0033291700048017.CrossRefPubMed
20.
Ware JE, Gandek B: Overview of the SF-36 Health Survey and the International Quality of Life Assessment (IQOLA) Project. J Clin Epidemiol. 1998, 51: 903-912. 10.1016/S0895-4356(98)00081-X.CrossRefPubMed
21.
Jenkinson C, Coulter A, Wright L: Short form 36 (SF36) health survey questionnaire: normative data for adults of working age. BMJ. 1993, 306: 1437-1440. 10.1136/bmj.306.6890.1437.CrossRefPubMedPubMedCentral
22.
Loge JH, Kaasa S: Short form 36 (SF-36) health survey: normative data from the general Norwegian population. Scand J Soc Med. 1998, 26: 250-258.PubMed
23.
De Beurs E, Rinne T, Van Kampen D, Verheul R, Andrea H: Reliability and validity of the Dutch Dimensional Assessment of Personality Pathology-Short Form (DAPP-SF), a shortened version of the DAPP-Basic Questionnaire. J Pers Disord. 2009, 23: 308-326. 10.1521/pedi.2009.23.3.308.CrossRefPubMed
24.
Tiemensma J, Biermasz NR, Middelkoop HA, van der Mast RC, Romijn JA, Pereira AM: Increased prevalence of psychopathology and maladaptive personality traits after long-term cure of Cushing's disease. J Clin Endocrinol Metab. 2010, 95: E129-E141. 10.1210/jc.2010-0512.CrossRefPubMed
25.
Bowling A, Bond M, Jenkinson C, Lamping DL: Short Form 36 (SF-36) Health Survey questionnaire: which normative data should be used? Comparisons between the norms provided by the Omnibus Survey in Britain, the Health Survey for England and the Oxford Healthy Life Survey. J Public Health Med. 1999, 21: 255-270. 10.1093/pubmed/21.3.255.CrossRefPubMed
26.
Schulte-van Maaren YWM, Carlier IV, Giltay EJ, Van Noorden MS, De Waal MW, Van der Wee NJ, Zitman FG: Reference values for mental health assessment instruments: objectives and methods of the Leiden Routine Outcome Monitoring Study. J Eval Clin Pract. 2012, 10.1111/j.1365-2753.2012.01830.x. [Epub ahead of print]
27.
Van Noorden MS, Giltay EJ, Den Hollander-Gijsman ME, Van der Wee NJ, Van Veen T, Zitman FG: Gender differences in clinical characteristics in a naturalistic sample of depressive outpatients: the Leiden Routine Outcome Monitoring Study. J Affect Disord. 2010, 125: 116-123. 10.1016/j.jad.2009.12.007.CrossRefPubMed
28.
29.
Carlier IV, Meuldijk D, Van Vliet IM, Van Fenema E, Van der Wee NJ, Zitman FG: Routine outcome monitoring and feedback on physical or mental health status: evidence and theory. J Eval Clin Pract. 2012, 18: 104-110. 10.1111/j.1365-2753.2010.01543.x.CrossRefPubMed
30.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, Dunbar GC: The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998, 59 (20): 22-33.PubMed
31.
Van Vliet IM, De Beurs E: The MINI-International Neuropsychiatric Interview. A brief structured diagnostic psychiatric interview for DSM-IV en ICD-10 psychiatric disorders. Tijdschr Psychiatr. 2007, 49: 393-397.PubMed
32.
Barnabei L, Marazia S, De Caterina R: Receiver operating characteristic (ROC) curves and the definition of threshold levels to diagnose coronary artery disease on electrocardiographic stress testing. Part I: The use of ROC curves in diagnostic medicine and electrocardiographic markers of ischaemia. J Cardiovasc Med (Hagerstown). 2007, 8: 873-881. 10.2459/JCM.0b013e3280126615.CrossRef
33.
Ware JE, Gandek B: Methods for testing data quality, scaling assumptions, and reliability: the IQOLA Project approach. International Quality of Life Assessment. J Clin Epidemiol. 1998, 51: 945-952. 10.1016/S0895-4356(98)00085-7.CrossRefPubMed
34.
Gandek B, Ware JE: Methods for validating and norming translations of health status questionnaires: the IQOLA Project approach. International Quality of Life Assessment. J Clin Epidemiol. 1998, 51: 953-959. 10.1016/S0895-4356(98)00086-9.CrossRefPubMed
35.
Apolone G, Mosconi P: The Italian SF-36 Health Survey: translation, validation and norming. J Clin Epidemiol. 1998, 51: 1025-1036. 10.1016/S0895-4356(98)00094-8.CrossRefPubMed
36.
Hammerlid E, Taft C: Health-related quality of life in long-term head and neck cancer survivors: a comparison with general population norms. Br J Cancer. 2001, 84: 149-156. 10.1054/bjoc.2000.1576.CrossRefPubMedPubMedCentral
37.
Razavi D, Gandek B: Testing Dutch and French translations of the SF-36 Health Survey among Belgian angina patients. J Clin Epidemiol. 1998, 51: 975-981. 10.1016/S0895-4356(98)00089-4.CrossRefPubMed
38.
Demiral Y, Ergor G, Unal B, Semin S, Akvardar Y, Kivircik B, Alptekin K: Normative data and discriminative properties of short form 36 (SF-36) in Turkish urban population. BMC Public Health. 2006, 6: 247-10.1186/1471-2458-6-247.CrossRefPubMedPubMedCentral
39.
Marazia S, Barnabei L, De Caterina R: Receiver operating characteristic (ROC) curves and the definition of threshold levels to diagnose coronary artery disease on electrocardiographic stress testing. Part II: the use of ROC curves in the choice of electrocardiographic stress test markers of ischaemia. J Cardiovasc Med (Hagerstown). 2008, 9: 22-31. 10.2459/JCM.0b013e32813ef418.CrossRef
40.
Zimmerman M, Chelminski I, Posternak M: A review of studies of the Montgomery-Asberg Depression Rating Scale in controls: implications for the definition of remission in treatment studies of depression. Int Clin Psychopharmacol. 2004, 19: 1-7. 10.1097/00004850-200401000-00001.CrossRefPubMed
41.
Tromp NB, Koot HM: Self- and parent report of adolescent personality pathology: informant agreement and relations to dysfunction. J Pers Disord. 2010, 24: 151-170. 10.1521/pedi.2010.24.2.151.CrossRefPubMed
42.
Hirschfeld RM, Klerman GL, Clayton PJ, Keller MB, McDonald-Scott P, Larkin BH: Assessing personality: effects of the depressive state on trait measurement. Am J Psychiatry. 1983, 140: 695-699.CrossRefPubMed
43.
Zimmerman M, Coryell WH: Diagnosing personality disorders in the community. A comparison of self-report and interview measures. Arch Gen Psychiatry. 1990, 47: 527-531. 10.1001/archpsyc.1990.01810180027005.CrossRefPubMed
Metadata
Title
Reference values for generic instruments used in routine outcome monitoring: the leiden routine outcome monitoring study
Authors
Yvonne WM Schulte-van Maaren
Ingrid VE Carlier
Frans G Zitman
Albert M van Hemert
Margot WM de Waal
Martijn S van Noorden
Erik J Giltay
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Psychiatry / Issue 1/2012
Electronic ISSN: 1471-244X
DOI
https://doi.org/10.1186/1471-244X-12-203

Other articles of this Issue 1/2012

BMC Psychiatry 1/2012 Go to the issue

Study protocol

The effectiveness of cognitive behavioral therapy on the quality of life of patients with inflammatory bowel disease: multi-center design and study protocol (KL!C- study)

Research article

Suicide rates and income in São Paulo and Brazil: a temporal and spatial epidemiologic analysis from 1996 to 2008

Research article

Depression in Dercum’s disease and in obesity: A case control study

Research article

Activities of daily living in dementia: revalidation of the E-ADL test and suggestions for further development

Study protocol

Effect of depression on mortality and cardiovascular morbidity in type 2 diabetes mellitus after 3 years follow up. The DIADEMA study protocol

Research article

Single-prolonged stress induces apoptosis in dorsal raphe nucleus in the rat model of posttraumatic stress disorder