Skip to main content
Top
Published in: BMC Psychiatry 1/2021

Open Access 01-12-2021 | Mood Disorders | Research

Validation of case definitions of depression derived from administrative data against the CIDI-SF as reference standard: results from the PROspective Québec (PROQ) study

Authors: Ana Paula Bruno Pena-Gralle, Denis Talbot, Xavier Trudel, Karine Aubé, Alain Lesage, Sophie Lauzier, Alain Milot, Chantal Brisson

Published in: BMC Psychiatry | Issue 1/2021

Login to get access

Abstract

Background

Administrative data have several advantages over questionnaire and interview data to identify cases of depression: they are usually inexpensive, available for a long period of time and are less subject to recall bias and differential classification errors. However, the validity of administrative data in the correct identification of depression has not yet been studied in general populations. The present study aimed to 1) evaluate the sensitivity and specificity of administrative cases of depression using the validated Composite International Diagnostic Interview – Short Form (CIDI-SF) as reference standard and 2) compare the known-groups validity between administrative and CIDI-SF cases of depression.

Methods

The 5487 participants seen at the last wave (2015–2018) of the PROQ cohort had CIDI-SF questionnaire data linked to hospitalization and medical reimbursement data provided by the provincial universal healthcare provider and coded using the International Classification of Disease. We analyzed the sensitivity and specificity of several case definitions of depression from this administrative data. Their association with known predictors of depression was estimated using robust Poisson regression models.

Results

Administrative cases of depression showed high specificity (≥ 96%), low sensitivity (19–32%), and rather low agreement (Cohen’s kappa of 0.21–0.25) compared with the CIDI-SF. These results were consistent over strata of sex, age and education level and with varying case definitions. In known-groups analysis, the administrative cases of depression were comparable to that of CIDI-SF cases (RR for sex: 1.80 vs 2.03 respectively, age: 1.53 vs 1.40, education: 1.52 vs 1.28, psychological distress: 2.21 vs 2.65).

Conclusion

The results obtained in this large sample of a general population suggest that the dimensions of depression captured by administrative data and by the CIDI-SF are partially distinct. However, their known-groups validity in relation to risk factors for depression was similar to that of CIDI-SF cases. We suggest that neither of these data sources is superior to the other in the context of large epidemiological studies aiming to identify and quantify risk factors for depression.
Appendix
Available only for authorised users
Literature
1.
go back to reference WHO. Depression and other common mental disorders. Geneva: WHO; 2017. WHO. Depression and other common mental disorders. Geneva: WHO; 2017.
2.
go back to reference Kruijshaar ME, Barendregt J, Vos T, de Graaf R, Spijker J, Andrews G. Lifetime prevalence estimates of major depression: an indirect estimation method and a quantification of recall bias. Eur J Epidemiol. 2005;20(1):103–11.CrossRef Kruijshaar ME, Barendregt J, Vos T, de Graaf R, Spijker J, Andrews G. Lifetime prevalence estimates of major depression: an indirect estimation method and a quantification of recall bias. Eur J Epidemiol. 2005;20(1):103–11.CrossRef
3.
go back to reference Kessler RC, Bromet EJ. The epidemiology of depression across cultures. Annu Rev Public Health. 2013;34:119–38.CrossRef Kessler RC, Bromet EJ. The epidemiology of depression across cultures. Annu Rev Public Health. 2013;34:119–38.CrossRef
4.
go back to reference Kessing LV. Epidemiology of subtypes of depression. Acta Psychiatr Scand. 2007;115:85–9.CrossRef Kessing LV. Epidemiology of subtypes of depression. Acta Psychiatr Scand. 2007;115:85–9.CrossRef
5.
go back to reference GBD. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet (London, England). 2018;392(10159):1789–858.CrossRef GBD. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet (London, England). 2018;392(10159):1789–858.CrossRef
6.
go back to reference Roy T, Lloyd CE. Epidemiology of depression and diabetes: a systematic review. J Affect Disord. 2012;142(Suppl):S8–21.CrossRef Roy T, Lloyd CE. Epidemiology of depression and diabetes: a systematic review. J Affect Disord. 2012;142(Suppl):S8–21.CrossRef
7.
go back to reference Penninx BW. Depression and cardiovascular disease: Epidemiological evidence on their linking mechanisms. Neurosci Biobehav Rev. 2017;74(Pt B):277–86.CrossRef Penninx BW. Depression and cardiovascular disease: Epidemiological evidence on their linking mechanisms. Neurosci Biobehav Rev. 2017;74(Pt B):277–86.CrossRef
8.
go back to reference Green RC, Cupples LA, Kurz A, Auerbach S, Go R, Sadovnick D, et al. Depression as a risk factor for Alzheimer disease: the MIRAGE study. Arch Neurol. 2003;60(5):753–9.CrossRef Green RC, Cupples LA, Kurz A, Auerbach S, Go R, Sadovnick D, et al. Depression as a risk factor for Alzheimer disease: the MIRAGE study. Arch Neurol. 2003;60(5):753–9.CrossRef
9.
go back to reference Marrie RA, Walker JR, Graff LA, Lix LM, Bolton JM, Nugent Z, et al. Performance of administrative case definitions for depression and anxiety in inflammatory bowel disease. J Psychosom Res. 2016;89:107–13.CrossRef Marrie RA, Walker JR, Graff LA, Lix LM, Bolton JM, Nugent Z, et al. Performance of administrative case definitions for depression and anxiety in inflammatory bowel disease. J Psychosom Res. 2016;89:107–13.CrossRef
10.
go back to reference Jette N, Quan H, Hemmelgarn B, Drosler S, Maass C, Moskal L, et al. The development, evolution, and modifications of ICD-10: challenges to the international comparability of morbidity data. Med Care. 2010;48(12):1105–10.CrossRef Jette N, Quan H, Hemmelgarn B, Drosler S, Maass C, Moskal L, et al. The development, evolution, and modifications of ICD-10: challenges to the international comparability of morbidity data. Med Care. 2010;48(12):1105–10.CrossRef
11.
go back to reference Craven MA, Bland R. Depression in primary care: current and future challenges. Can J Psychiatry. 2013;58(8):442–8.CrossRef Craven MA, Bland R. Depression in primary care: current and future challenges. Can J Psychiatry. 2013;58(8):442–8.CrossRef
12.
go back to reference Smolders M, Laurant M, Verhaak P, Prins M, van Marwijk H, Penninx B, et al. Adherence to evidence-based guidelines for depression and anxiety disorders is associated with recording of the diagnosis. Gen Hosp Psychiatry. 2009;31(5):460–9.CrossRef Smolders M, Laurant M, Verhaak P, Prins M, van Marwijk H, Penninx B, et al. Adherence to evidence-based guidelines for depression and anxiety disorders is associated with recording of the diagnosis. Gen Hosp Psychiatry. 2009;31(5):460–9.CrossRef
13.
go back to reference Joling KJ, van Marwijk HW, Piek E, van der Horst HE, Penninx BW, Verhaak P, et al. Do GPs' medical records demonstrate a good recognition of depression? A new perspective on case extraction. J Affect Disord. 2011;133(3):522–7.CrossRef Joling KJ, van Marwijk HW, Piek E, van der Horst HE, Penninx BW, Verhaak P, et al. Do GPs' medical records demonstrate a good recognition of depression? A new perspective on case extraction. J Affect Disord. 2011;133(3):522–7.CrossRef
14.
go back to reference John A, McGregor J, Fone D, Dunstan F, Cornish R, Lyons RA, et al. BMC Medical Informatics and Decision Making. 2016;16:35. John A, McGregor J, Fone D, Dunstan F, Cornish R, Lyons RA, et al. BMC Medical Informatics and Decision Making. 2016;16:35.
15.
go back to reference Solberg LI, Engebretson KI, Sperl-Hillen JM, Hroscikoski MC, O'Connor PJ. Are claims data accurate enough to identify patients for performance measures or quality improvement? The case of diabetes, heart disease, and depression. Am J Med Qual. 2006;21(4):238–45.CrossRef Solberg LI, Engebretson KI, Sperl-Hillen JM, Hroscikoski MC, O'Connor PJ. Are claims data accurate enough to identify patients for performance measures or quality improvement? The case of diabetes, heart disease, and depression. Am J Med Qual. 2006;21(4):238–45.CrossRef
16.
go back to reference Doktorchik C, Patten S, Eastwood C, Peng MK, Chen GM, Beck CA, et al. Validation of a case definition for depression in administrative data against primary chart data as a reference standard. BMC Psychiatry. 2019;19:9. Doktorchik C, Patten S, Eastwood C, Peng MK, Chen GM, Beck CA, et al. Validation of a case definition for depression in administrative data against primary chart data as a reference standard. BMC Psychiatry. 2019;19:9.
17.
go back to reference Fiest KM, Jette N, Quan HD, St Germaine-Smith C, Metcalfe A, Patten SB, et al. Systematic review and assessment of validated case definitions for depression in administrative data. BMC Psychiatry. 2014;14:289. Fiest KM, Jette N, Quan HD, St Germaine-Smith C, Metcalfe A, Patten SB, et al. Systematic review and assessment of validated case definitions for depression in administrative data. BMC Psychiatry. 2014;14:289.
18.
go back to reference Rawson NS, Malcolm E, D'Arcy C. Reliability of the recording of schizophrenia and depressive disorder in the Saskatchewan health care datafiles. Soc Psychiatry Psychiatr Epidemiol. 1997;32(4):191–9.CrossRef Rawson NS, Malcolm E, D'Arcy C. Reliability of the recording of schizophrenia and depressive disorder in the Saskatchewan health care datafiles. Soc Psychiatry Psychiatr Epidemiol. 1997;32(4):191–9.CrossRef
19.
go back to reference Trinh NH, Youn SJ, Sousa J, Regan S, Bedoya CA, Chang TE, et al. Using electronic medical records to determine the diagnosis of clinical depression. Int J Med Inform. 2011;80(7):533–40.CrossRef Trinh NH, Youn SJ, Sousa J, Regan S, Bedoya CA, Chang TE, et al. Using electronic medical records to determine the diagnosis of clinical depression. Int J Med Inform. 2011;80(7):533–40.CrossRef
20.
go back to reference Townsend L, Walkup JT, Crystal S, Olfson M. A systematic review of validated methods for identifying depression using administrative data. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):163–73.CrossRef Townsend L, Walkup JT, Crystal S, Olfson M. A systematic review of validated methods for identifying depression using administrative data. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):163–73.CrossRef
21.
go back to reference Howren A, Aviña-Zubieta JA, Puyat JH, Esdaile JM, Da Costa D, De Vera MA. Defining depression and anxiety in individuals with rheumatic diseases using administrative health databases: a systematic review. Arthritis Care Res (Hoboken). 2020;72(2):243–55.CrossRef Howren A, Aviña-Zubieta JA, Puyat JH, Esdaile JM, Da Costa D, De Vera MA. Defining depression and anxiety in individuals with rheumatic diseases using administrative health databases: a systematic review. Arthritis Care Res (Hoboken). 2020;72(2):243–55.CrossRef
22.
go back to reference Katon WJ, Simon G, Russo J, Von Korff M, Lin EH, Ludman E, et al. Quality of depression care in a population-based sample of patients with diabetes and major depression. Med Care. 2004;42(12):1222–9.CrossRef Katon WJ, Simon G, Russo J, Von Korff M, Lin EH, Ludman E, et al. Quality of depression care in a population-based sample of patients with diabetes and major depression. Med Care. 2004;42(12):1222–9.CrossRef
23.
go back to reference Trudel X, Gilbert-Ouimet M, Milot A, Duchaine CS, Vezina M, Laurin D, et al. Cohort profile: the PROspective Quebec (PROQ) study on work and health. Int J Epidemiol. 2018;47(3):693-i. Trudel X, Gilbert-Ouimet M, Milot A, Duchaine CS, Vezina M, Laurin D, et al. Cohort profile: the PROspective Quebec (PROQ) study on work and health. Int J Epidemiol. 2018;47(3):693-i.
24.
go back to reference Muntaner C, Eaton WW, Diala C, Kessler RC, Sorlie PD. Social class, assets, organizational control and the prevalence of common groups of psychiatric disorders. Soc Sci Med. 1998;47(12):2043–53.CrossRef Muntaner C, Eaton WW, Diala C, Kessler RC, Sorlie PD. Social class, assets, organizational control and the prevalence of common groups of psychiatric disorders. Soc Sci Med. 1998;47(12):2043–53.CrossRef
25.
go back to reference Gravel R, Beland Y. The Canadian community health survey: mental health and well-being. Can J Psychiatry. 2005;50(10):573–9.CrossRef Gravel R, Beland Y. The Canadian community health survey: mental health and well-being. Can J Psychiatry. 2005;50(10):573–9.CrossRef
26.
go back to reference Association AP. Statistical Manual of Mental Disorders 2013. Association AP. Statistical Manual of Mental Disorders 2013.
27.
go back to reference Patten SB. Performance of the composite international diagnostic interview short form for major depression in community and clinical samples. Chronic Dis Can. 1997;18(3):109–12.PubMed Patten SB. Performance of the composite international diagnostic interview short form for major depression in community and clinical samples. Chronic Dis Can. 1997;18(3):109–12.PubMed
28.
go back to reference Kessler RC, Berglund P, Demler O, Jin R, Koretz D, Merikangas KR, et al. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). Jama. 2003;289(23):3095–105.CrossRef Kessler RC, Berglund P, Demler O, Jin R, Koretz D, Merikangas KR, et al. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). Jama. 2003;289(23):3095–105.CrossRef
29.
go back to reference Plante C, Goudreau S, Jacques L, Tessier F. Agreement between survey data and Régie de l'assurance maladie du Québec (RAMQ) data with respect to the diagnosis of asthma and medical services use for asthma in children. Chronic Dis Inj Can. 2014;34(4):256–62.CrossRef Plante C, Goudreau S, Jacques L, Tessier F. Agreement between survey data and Régie de l'assurance maladie du Québec (RAMQ) data with respect to the diagnosis of asthma and medical services use for asthma in children. Chronic Dis Inj Can. 2014;34(4):256–62.CrossRef
30.
go back to reference Davidson M. Known-groups validity. In: Michalos AC, editor. Encyclopedia of quality of life and well-being research. Dordrecht: Springer Netherlands; 2014. p. 3481–2.CrossRef Davidson M. Known-groups validity. In: Michalos AC, editor. Encyclopedia of quality of life and well-being research. Dordrecht: Springer Netherlands; 2014. p. 3481–2.CrossRef
31.
go back to reference McCoach DB, Gable RK, Madura JP. Evidence Based on Relations to Other Variables: Bolstering the Empirical Validity Arguments for Constructs. In: Instrument Development in the Affective Domain: School and Corporate Applications. New York, NY: Springer New York; 2013. p. 209–48.CrossRef McCoach DB, Gable RK, Madura JP. Evidence Based on Relations to Other Variables: Bolstering the Empirical Validity Arguments for Constructs. In: Instrument Development in the Affective Domain: School and Corporate Applications. New York, NY: Springer New York; 2013. p. 209–48.CrossRef
32.
go back to reference Ilfeld F. Further validation of a psychiatric symptom index in a normal population. Psychol Rep. 1976;39:12015–1228. Ilfeld F. Further validation of a psychiatric symptom index in a normal population. Psychol Rep. 1976;39:12015–1228.
33.
go back to reference Perreault C. Les Mesures de Santé Mentale. Possibilités et Limites de la Mesure Utilisée (mental health measurements: strengths and limitation of the instrument used in Quebec). Québec, QC, Canada: Gouvernement du Québec; 1987. Perreault C. Les Mesures de Santé Mentale. Possibilités et Limites de la Mesure Utilisée (mental health measurements: strengths and limitation of the instrument used in Quebec). Québec, QC, Canada: Gouvernement du Québec; 1987.
34.
go back to reference Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.CrossRef Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.CrossRef
35.
go back to reference Schomaker M, Heumann C. Bootstrap inference when using multiple imputation. Stat Med. 2018;37(14):2252–66.CrossRef Schomaker M, Heumann C. Bootstrap inference when using multiple imputation. Stat Med. 2018;37(14):2252–66.CrossRef
36.
go back to reference Chen W, Qian L, Shi J, Franklin M. Comparing performance between log-binomial and robust Poisson regression models for estimating risk ratios under model misspecification. BMC Med Res Methodol. 2018;18(1):63.CrossRef Chen W, Qian L, Shi J, Franklin M. Comparing performance between log-binomial and robust Poisson regression models for estimating risk ratios under model misspecification. BMC Med Res Methodol. 2018;18(1):63.CrossRef
37.
go back to reference Stef Vb. Multiple imputation of multilevel data. 2011. Stef Vb. Multiple imputation of multilevel data. 2011.
38.
go back to reference Canty A, Ripley B. boot: Bootstrap R (S-Plus) functions. In: 1.3–22. Rpv, editor. 2017. Canty A, Ripley B. boot: Bootstrap R (S-Plus) functions. In: 1.3–22. Rpv, editor. 2017.
39.
go back to reference Eaton WW, Anthony JC, Gallo J, Cai GJ, Tien A, Romanoski A, et al. Natural history of diagnostic interview schedule DSM-IV major depression - the Baltimore epidemiologic catchment area follow-up. Arch Gen Psychiatry. 1997;54(11):993–9.CrossRef Eaton WW, Anthony JC, Gallo J, Cai GJ, Tien A, Romanoski A, et al. Natural history of diagnostic interview schedule DSM-IV major depression - the Baltimore epidemiologic catchment area follow-up. Arch Gen Psychiatry. 1997;54(11):993–9.CrossRef
41.
go back to reference Lasalvia A, Zoppei S, Van Bortel T, Bonetto C, Cristofalo D, Wahlbeck K, et al. Global pattern of experienced and anticipated discrimination reported by people with major depressive disorder: a cross-sectional survey. Lancet (London, England). 2013;381(9860):55–62.CrossRef Lasalvia A, Zoppei S, Van Bortel T, Bonetto C, Cristofalo D, Wahlbeck K, et al. Global pattern of experienced and anticipated discrimination reported by people with major depressive disorder: a cross-sectional survey. Lancet (London, England). 2013;381(9860):55–62.CrossRef
42.
go back to reference Lesage AD, Fournier L, Cyr M, Toupin J, Fabian J, Gaudette G, et al. The reliability of the community version of the MRC needs for care assessment. Psychol Med. 1996;26(2):237–43.CrossRef Lesage AD, Fournier L, Cyr M, Toupin J, Fabian J, Gaudette G, et al. The reliability of the community version of the MRC needs for care assessment. Psychol Med. 1996;26(2):237–43.CrossRef
43.
go back to reference Patten SB, Wang JL, Beck CA, Maxwell CJ. Measurement issues related to the evaluation and monitoring of major depression prevalence in Canada. Chronic Dis Can. 2005;26(4):100–6.PubMed Patten SB, Wang JL, Beck CA, Maxwell CJ. Measurement issues related to the evaluation and monitoring of major depression prevalence in Canada. Chronic Dis Can. 2005;26(4):100–6.PubMed
44.
go back to reference Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet (London, England). 2009;374(9690):609–19.CrossRef Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet (London, England). 2009;374(9690):609–19.CrossRef
45.
go back to reference Cepoiu M, McCusker J, Cole MG, Sewitch M, Belzile E, Ciampi A. Recognition of depression by non-psychiatric physicians--a systematic literature review and meta-analysis. J Gen Intern Med. 2008;23(1):25–36.CrossRef Cepoiu M, McCusker J, Cole MG, Sewitch M, Belzile E, Ciampi A. Recognition of depression by non-psychiatric physicians--a systematic literature review and meta-analysis. J Gen Intern Med. 2008;23(1):25–36.CrossRef
Metadata
Title
Validation of case definitions of depression derived from administrative data against the CIDI-SF as reference standard: results from the PROspective Québec (PROQ) study
Authors
Ana Paula Bruno Pena-Gralle
Denis Talbot
Xavier Trudel
Karine Aubé
Alain Lesage
Sophie Lauzier
Alain Milot
Chantal Brisson
Publication date
01-12-2021
Publisher
BioMed Central
Published in
BMC Psychiatry / Issue 1/2021
Electronic ISSN: 1471-244X
DOI
https://doi.org/10.1186/s12888-021-03501-x

Other articles of this Issue 1/2021

BMC Psychiatry 1/2021 Go to the issue