Top

BMC Psychiatry

Published in:

Open Access 01-12-2016 | Research article

External validation of the international risk prediction algorithm for major depressive episode in the US general population: the PredictD-US study

Authors: Yeshambel T. Nigatu, Yan Liu, JianLi Wang

Published in: BMC Psychiatry | Issue 1/2016

Abstract

Background

Multivariable risk prediction algorithms are useful for making clinical decisions and for health planning. While prediction algorithms for new onset of major depression in the primary care attendees in Europe and elsewhere have been developed, the performance of these algorithms in different populations is not known. The objective of this study was to validate the PredictD algorithm for new onset of major depressive episode (MDE) in the US general population.

Methods

Longitudinal study design was conducted with approximate 3-year follow-up data from a nationally representative sample of the US general population. A total of 29,621 individuals who participated in Wave 1 and 2 of the US National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) and who did not have an MDE in the past year at Wave 1 were included. The PredictD algorithm was directly applied to the selected participants. MDE was assessed by the Alcohol Use Disorder and Associated Disabilities Interview Schedule, based on the DSM-IV criteria.

Results

Among the participants, 8 % developed an MDE over three years. The PredictD algorithm had acceptable discriminative power (C-statistics = 0.708, 95 % CI: 0.696, 0.720), but poor calibration (p < 0.001) with the NESARC data. In the European primary care attendees, the algorithm had a C-statistics of 0.790 (95 % CI: 0.767, 0.813) with a perfect calibration.

Conclusions

The PredictD algorithm has acceptable discrimination, but the calibration capacity was poor in the US general population despite of re-calibration. Therefore, based on the results, at current stage, the use of PredictD in the US general population for predicting individual risk of MDE is not encouraged. More independent validation research is needed.

Kessler RC, Barber C, Beck A, Berglund P, Cleary PD, McKenas D, et al. The World Health Organization Health and Work Performance Questionnaire (HPQ). J Occup Environ Med. 2003;45:156–74.CrossRefPubMed

Mathers CD, Loncar D. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med. 2006;3:e442.CrossRefPubMedPubMedCentral

Bromet E, Andrade LH, Hwang I, Sampson NA, Alonso J, de Girolamo G, et al. Cross-national epidemiology of DSM-IV major depressive episode. BMC Med. 2011;9:90.CrossRefPubMedPubMedCentral

Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382:1575–86.CrossRefPubMed

Kessler RC, Bromet EJ. The epidemiology of depression across cultures. Annu Rev Public Health. 2013;34:119–38.CrossRefPubMedPubMedCentral

Bockting CL, Spinhoven P, Koeter MW, Wouters LF, Schene AH, Group DELTAS. Prediction of recurrence in recurrent depression and the influence of consecutive episodes on vulnerability for depression: a 2-year prospective study. J Clin Psychiatry. 2006;67:747–55.CrossRefPubMed

Wang JL, Patten S, Sareen J, Bolton J, Schmitz N, MacQueen G. Development and validation of a prediction algorithm for use by health professionals in prediction of recurrence of major depression. Depress Anxiety. 2014;31:451–7.CrossRefPubMed

Wang J, Sareen J, Patten S, Bolton J, Schmitz N, Birney A. A prediction algorithm for first onset of major depression in the general population: development and validation. J Epidemiol Community Health. 2014;68:418–24.CrossRefPubMed

Wang JL, Manuel D, Williams J, Schmitz N, Gilmour H, Patten S, et al. Development and validation of prediction algorithms for major depressive episode in the general population. J Affect Disord. 2013;151:39–45.CrossRefPubMed

10.

Djernes JK. Prevalence and predictors of depression in populations of elderly: a review. Acta Psychiatr Scand. 2006;113:372–87.CrossRefPubMed

11.

Anstey KJ, von Sanden C, Sargent-Cox K, Luszcz MA. Prevalence and risk factors for depression in a longitudinal, population-based study including individuals in the community and residential care. Am J Geriatr Psychiatry. 2007;15:497–505.CrossRefPubMed

12.

Patten SB, Wang JL, Williams JV, Lavorato DH, Khaled SM, Bulloch AG. Predictors of the longitudinal course of major depression in a Canadian population sample. Can J Psychiatry. 2010;55:669–76.CrossRefPubMed

13.

King M, Walker C, Levy G, Bottomley C, Royston P, Weich S, et al. Development and validation of an international risk prediction algorithm for episodes of major depression in general practice attendees: the PredictD study. Arch Gen Psychiatry. 2008;65:1368–76.CrossRefPubMed

14.

Chekroud AM, Zotti RJ, Shehzad Z, Gueorguieva R, Johnson MK, Trivedi MH, et al. Cross-trial prediction of treatment outcome in depression: a machine learning approach. Lancet Psychiatry. 2016;3:243–50.CrossRefPubMed

15.

Kessler RC, van Loo HM, Wardenaar KJ, Bossarte RM, Brenner LA, Cai T, et al. Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports. Mol Psychiatry 2016: doi: 10.1038/mp.2015.198. In press.

16.

Grant BF, Goldstein RB, Chou SP, Huang B, Stinson FS, Dawson DA, et al. Sociodemographic and psychopathologic predictors of first incidence of DSM-IV substance use, mood and anxiety disorders: results from the Wave 2 National Epidemiologic Survey on Alcohol and Related Conditions. Mol Psychiatry. 2009;14:1051–66.CrossRefPubMed

17.

Hasin DS, Goodwin RD, Stinson FS, Grant BF. Epidemiology of major depressive disorder: results from the National Epidemiologic Survey on Alcoholism and Related Conditions. Arch Gen Psychiatry. 2005;62:1097–106.CrossRefPubMed

18.

Ruan WJ, Goldstein RB, Chou SP, Smith SM, Saha TD, Pickering RP, et al. The alcohol use disorder and associated disabilities interview schedule-IV (AUDADIS-IV): reliability of new psychiatric diagnostic modules and risk factors in a general population sample. Drug Alcohol Depend. 2008;92:27–36.CrossRefPubMed

19.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed. Washington, DC: American Psychiatric Association; 1994.

20.

Jenkinson C, Layte R, Jenkinson D, Lawrence K, Petersen S, Paice C, et al. A shorter form health survey: can the SF-12 replicate results from the SF-36 in longitudinal studies? J Public Health Med. 1997;19:179–86.CrossRefPubMed

21.

Steyeberg E. Clinical prediction models. A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.

22.

Bellon JA, de Dios LJ, Moreno B, Monton-Franco C, GildeGomez-Barragan MJ, Sanchez-Celaya M, et al. Psychosocial and sociodemographic predictors of attrition in a longitudinal study of major depression in primary care: the predictD-Spain study. J Epidemiol Community Health. 2010;64:874–84.CrossRefPubMed

23.

King M, Bottomley C, Bellon-Saameno J, Torres-Gonzalez F, Svab I, Rotar D, et al. Predicting onset of major depression in general practice attendees in Europe: extending the application of the predictD risk algorithm from 12 to 24 months. Psychol Med. 2013;43:1929–39.CrossRefPubMed

24.

Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35:1925–31.CrossRefPubMedPubMedCentral

25.

Steyerberg EW, Van Calster B, Pencina MJ. Performance measures for prediction models and markers: evaluation of predictions and classifications. Rev Esp Cardiol. 2011;64:788–94.CrossRefPubMed

26.

Steyerberg EW, Harrell Jr FE. Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol. 2016;69:245–7.CrossRefPubMed

27.

Van Calster B, Steyerberg EW, Harrell FH. Risk Prediction for Individuals. JAMA. 2015;314:1875.CrossRefPubMed

Title: External validation of the international risk prediction algorithm for major depressive episode in the US general population: the PredictD-US study
Authors: Yeshambel T. Nigatu
Yan Liu
JianLi Wang
Publication date: 01-12-2016
Publisher: BioMed Central
Published in: BMC Psychiatry / Issue 1/2016
Electronic ISSN: 1471-244X
DOI: https://doi.org/10.1186/s12888-016-0971-x

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

External validation of the international risk prediction algorithm for major depressive episode in the US general population: the PredictD-US study

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2016

Erratum to: Readmission rates of South Korean psychiatric inpatients by inpatient volumes per psychiatrist

Associations between delayed completion of high school and educational attainment and symptom levels of anxiety and depression in adulthood

The experience of Greek-Cypriot individuals living with mental illness: preliminary results of a phenomenological study

Investigating the (cost-) effectiveness of attention bias modification (ABM) for outpatients with major depressive disorder (MDD): a randomized controlled trial protocol

Psychotic relapse and associated factors among patients attending health services in Southwest Ethiopia: a cross-sectional study

Rate, timing and predictors of relapse in patients with anorexia nervosa following a relapse prevention program: a cohort study