Skip to main content
Top
Published in: BMC Psychiatry 1/2021

Open Access 01-12-2021 | Mood Disorders | Research article

Simple action for depression detection: using kinect-recorded human kinematic skeletal data

Authors: Wentao Li, Qingxiang Wang, Xin Liu, Yanhong Yu

Published in: BMC Psychiatry | Issue 1/2021

Login to get access

Abstract

Background

Depression, a common worldwide mental disorder, which brings huge challenges to family and social burden around the world is different from fluctuant emotion and psychological pressure in their daily life. Although body signs have been shown to present manifestations of depression in general, few researches focus on whole body kinematic cues with the help of machine learning methods to aid depression recognition. Using the Kinect V2 device to record participants’ simple kinematic skeleton data of the participant’s body joints, the presented spatial features and low-level features is directly extracted from the record original Kinect-3D coordinates. This research aimed to constructed machine learning model with the preprocessed data importing, which could be used for depression automatic classification.

Methods

Considering some patients’ conditions and current status and refer to psychiatrists’ advices, simple and significant designed stimulus task will lead human skeleton data collection job. With original Kinect skeleton data extracting and preprocessing, the proposed experiment demonstrated four strong machine learning tools: Support Vector Machine, Logistic Regression, Random Forest and Gradient Boosting. Using the precision, recall, sensitivity, specificity, roc-curve, confusion matrix et.al, indicators were calculated as the measurement of methods, which were commonly used to evaluate classification methodologies.

Results

Across screened 64 pairs with age and gender totally matching in depression and control group, and Gradient Boosting achieved the best performance with the prediction accuracy of 76.92%. Sorted by female (54.69%) and male for the gender-based depression recognition, we applied best performance classifier Gradient Boosting got prediction accuracy of 66.67% in the male group, and 71.73% in the female group. Utilizing the best model Gradient Boosting for age-based classification, prediction accuracy got 76.92% in the older group (age >40, 50% of total) and 53.85% accuracy in the younger group (age <= 40).

Conclusion

The depression and non-depression individuals can be well classified by computational models using Kinect captured skeletal data. The Gradient Boosting, an excellent machine learning tool, get the performance in the four methods we demonstrated. Meanwhile, in the gender-based depression classification also gets reasonable accuracy. In particular, the recognition results of the old group are significantly better than that of the young group. All these findings suggest that kinematic skeletal data based depression recognition can be applied as an effective tool for assisting in depression analysis.
Literature
1.
go back to reference James SL, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, Abbastabar H, Abd-Allah F, Abdela J, Abdelalim A, et al.Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the global burden of disease study 2017. The Lancet. 2018; 392(10159):1789–858.CrossRef James SL, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, Abbastabar H, Abd-Allah F, Abdela J, Abdelalim A, et al.Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the global burden of disease study 2017. The Lancet. 2018; 392(10159):1789–858.CrossRef
2.
go back to reference Organization WH, et al.Depression and other common mental disorders: global health estimates. Technical report, World Health Organization. 2017. Organization WH, et al.Depression and other common mental disorders: global health estimates. Technical report, World Health Organization. 2017.
3.
go back to reference Kitchener BA, Jorm AF. Mental health first aid training: review of evaluation studies. Aust N Z J Psychiatr. 2006; 40(1):6–8.CrossRef Kitchener BA, Jorm AF. Mental health first aid training: review of evaluation studies. Aust N Z J Psychiatr. 2006; 40(1):6–8.CrossRef
4.
go back to reference Organization WH, et al.Made in viet nam vaccines: efforts to develop sustainable in-country manufacturing for seasonal and pandemic influenza vaccines: consultation held in viet nam, april-june 2016. Technical report, World Health Organization. 2017. Organization WH, et al.Made in viet nam vaccines: efforts to develop sustainable in-country manufacturing for seasonal and pandemic influenza vaccines: consultation held in viet nam, april-june 2016. Technical report, World Health Organization. 2017.
5.
go back to reference Giordano A, Granella F, Lugaresi A, Martinelli V, Trojano M, Confalonieri P, Radice D, Solari A, Group S-T, et al. Anxiety and depression in multiple sclerosis patients around diagnosis. J Neurol Sci. 2011; 307(1-2):86–91.PubMedCrossRef Giordano A, Granella F, Lugaresi A, Martinelli V, Trojano M, Confalonieri P, Radice D, Solari A, Group S-T, et al. Anxiety and depression in multiple sclerosis patients around diagnosis. J Neurol Sci. 2011; 307(1-2):86–91.PubMedCrossRef
6.
go back to reference Wilson RS, Boyle PA, Segawa E, Yu L, Begeny CT, Anagnos SE, Bennett DA. The influence of cognitive decline on well-being in old age. Psychol Aging. 2013; 28(2):304.PubMedPubMedCentralCrossRef Wilson RS, Boyle PA, Segawa E, Yu L, Begeny CT, Anagnos SE, Bennett DA. The influence of cognitive decline on well-being in old age. Psychol Aging. 2013; 28(2):304.PubMedPubMedCentralCrossRef
7.
go back to reference Bryant C. Anxiety and depression in old age: challenges in recognition and diagnosis. Int Psychogeriatr. 2010; 22(4):511–3.PubMedCrossRef Bryant C. Anxiety and depression in old age: challenges in recognition and diagnosis. Int Psychogeriatr. 2010; 22(4):511–3.PubMedCrossRef
8.
go back to reference Beck AT, Beamesderfer A. Assessment of depression: the depression inventory. Psychol Meas Psychopharmacol. 1974; 7:151–69. Beck AT, Beamesderfer A. Assessment of depression: the depression inventory. Psychol Meas Psychopharmacol. 1974; 7:151–69.
9.
go back to reference Craft LL, Landers DM. The effect of exercise on clinical depression and depression resulting from mental illness: A meta-analysis. J Sport Exerc Psychol. 1998; 20(4):339–57.CrossRef Craft LL, Landers DM. The effect of exercise on clinical depression and depression resulting from mental illness: A meta-analysis. J Sport Exerc Psychol. 1998; 20(4):339–57.CrossRef
10.
go back to reference Brosse AL, Sheets ES, Lett HS, Blumenthal JA. Exercise and the treatment of clinical depression in adults. Sports Med. 2002; 32:741–60.PubMedCrossRef Brosse AL, Sheets ES, Lett HS, Blumenthal JA. Exercise and the treatment of clinical depression in adults. Sports Med. 2002; 32:741–60.PubMedCrossRef
11.
go back to reference Roberts RE, Andrews JA, Lewinsohn PM, Hops H. Psychol Assess J Consult Clin Psychol. 1990; 2(2):122. Roberts RE, Andrews JA, Lewinsohn PM, Hops H. Psychol Assess J Consult Clin Psychol. 1990; 2(2):122.
12.
go back to reference Blatt SJ. Experiences of depression: Theoretical, clinical, and research perspectives. Washington, DC: American Psychological Association; 2004.CrossRef Blatt SJ. Experiences of depression: Theoretical, clinical, and research perspectives. Washington, DC: American Psychological Association; 2004.CrossRef
13.
go back to reference Nezu AM, Nezu CM, Lee M, Stern JB. Assessment of depression. New York: Guilford Press; 2014. Nezu AM, Nezu CM, Lee M, Stern JB. Assessment of depression. New York: Guilford Press; 2014.
14.
go back to reference Gao S, Calhoun VD, Sui J. Machine learning in major depression: From classification to treatment outcome prediction. CNS Neurosci Ther. 2018; 24(11):1037–52.PubMedPubMedCentralCrossRef Gao S, Calhoun VD, Sui J. Machine learning in major depression: From classification to treatment outcome prediction. CNS Neurosci Ther. 2018; 24(11):1037–52.PubMedPubMedCentralCrossRef
15.
go back to reference Pigoni A, Delvecchio G, Madonna D, Bressi C, Soares J, Brambilla P. Can machine learning help us in dealing with treatment resistant depression? a review. J Affect Disord. 2019; 259:21–6.PubMedCrossRef Pigoni A, Delvecchio G, Madonna D, Bressi C, Soares J, Brambilla P. Can machine learning help us in dealing with treatment resistant depression? a review. J Affect Disord. 2019; 259:21–6.PubMedCrossRef
16.
go back to reference Cohn JF, Kruez TS, Matthews I, Yang Y, Nguyen MH, Padilla MT, et al.Detecting depression from facial actions and vocal prosody. In: 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. Amsterdam: IEEE: 2009. p. 1–7. Cohn JF, Kruez TS, Matthews I, Yang Y, Nguyen MH, Padilla MT, et al.Detecting depression from facial actions and vocal prosody. In: 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. Amsterdam: IEEE: 2009. p. 1–7.
17.
go back to reference Kleinsmith A, Bianchi-Berthouze N. Affective body expression perception and recognition: A survey. IEEE Trans Affect Comput. 2012; 4(1):15–33.CrossRef Kleinsmith A, Bianchi-Berthouze N. Affective body expression perception and recognition: A survey. IEEE Trans Affect Comput. 2012; 4(1):15–33.CrossRef
18.
go back to reference Pastore LM, Patrie JT, Morris WL, Dalal P, Bray MJ. Depression symptoms and body dissatisfaction association among polycystic ovary syndrome women. J Psychosom Res. 2011; 71(4):270–6.PubMedPubMedCentralCrossRef Pastore LM, Patrie JT, Morris WL, Dalal P, Bray MJ. Depression symptoms and body dissatisfaction association among polycystic ovary syndrome women. J Psychosom Res. 2011; 71(4):270–6.PubMedPubMedCentralCrossRef
19.
go back to reference Dibeklioğlu H, Hammal Z, Yang Y, Cohn JF. Multimodal detection of depression in clinical interviews. In: Proceedings of the 2015 ACM on international conference on multimodal interaction. Seattle: 2015. p. 307–10. Dibeklioğlu H, Hammal Z, Yang Y, Cohn JF. Multimodal detection of depression in clinical interviews. In: Proceedings of the 2015 ACM on international conference on multimodal interaction. Seattle: 2015. p. 307–10.
20.
go back to reference Goldfield GS, Moore C, Henderson K, Buchholz A, Obeid N, Flament MF. Body dissatisfaction, dietary restraint, depression, and weight status in adolescents. J Sch Health. 2010; 80(4):186–92.PubMedCrossRef Goldfield GS, Moore C, Henderson K, Buchholz A, Obeid N, Flament MF. Body dissatisfaction, dietary restraint, depression, and weight status in adolescents. J Sch Health. 2010; 80(4):186–92.PubMedCrossRef
21.
go back to reference Robertson R, Robertson A, Jepson R, Maxwell M. Walking for depression or depressive symptoms: a systematic review and meta-analysis. Ment Health Phys Act. 2012; 5(1):66–75.CrossRef Robertson R, Robertson A, Jepson R, Maxwell M. Walking for depression or depressive symptoms: a systematic review and meta-analysis. Ment Health Phys Act. 2012; 5(1):66–75.CrossRef
22.
go back to reference Kim J-Y, Liu N, Tan H-X, Chu C-H. Unobtrusive monitoring to detect depression for elderly with chronic illnesses. IEEE Sensors J. 2017; 17(17):5694–704.CrossRef Kim J-Y, Liu N, Tan H-X, Chu C-H. Unobtrusive monitoring to detect depression for elderly with chronic illnesses. IEEE Sensors J. 2017; 17(17):5694–704.CrossRef
23.
go back to reference Pampouchidou A, Simos PG, Marias K, Meriaudeau F, Yang F, Pediaditis M, Tsiknakis M. Automatic assessment of depression based on visual cues: A systematic review. IEEE Trans Affect Comput. 2017; 10(4):445–70.CrossRef Pampouchidou A, Simos PG, Marias K, Meriaudeau F, Yang F, Pediaditis M, Tsiknakis M. Automatic assessment of depression based on visual cues: A systematic review. IEEE Trans Affect Comput. 2017; 10(4):445–70.CrossRef
24.
go back to reference Weng T-T, Hao J-H, Qian Q-W, Cao H, Fu J-L, Sun Y, Huang L, Tao F-B. Is there any relationship between dietary patterns and depression and anxiety in chinese adolescents?Public Health Nutr. 2012; 15(4):673–82.PubMedCrossRef Weng T-T, Hao J-H, Qian Q-W, Cao H, Fu J-L, Sun Y, Huang L, Tao F-B. Is there any relationship between dietary patterns and depression and anxiety in chinese adolescents?Public Health Nutr. 2012; 15(4):673–82.PubMedCrossRef
25.
go back to reference Zhang Z. Microsoft kinect sensor and its effect. IEEE Multimedia. 2012; 19(2):4–10.CrossRef Zhang Z. Microsoft kinect sensor and its effect. IEEE Multimedia. 2012; 19(2):4–10.CrossRef
26.
go back to reference Saini R, Kumar P, Kaur B, Roy PP, Dogra DP, Santosh K. Kinect sensor-based interaction monitoring system using the blstm neural network in healthcare. Int J Mach Learn Cybern. 2019; 10(9):2529–40.CrossRef Saini R, Kumar P, Kaur B, Roy PP, Dogra DP, Santosh K. Kinect sensor-based interaction monitoring system using the blstm neural network in healthcare. Int J Mach Learn Cybern. 2019; 10(9):2529–40.CrossRef
27.
go back to reference Müller K, Fröhlich S, Germano AM, Kondragunta J, Hurtado MFdCA, Rudisch J, Schmidt D, Hirtz G, Stollmann P, Voelcker-Rehage C. Sensor-based systems for early detection of dementia (senda): a study protocol for a prospective cohort sequential study. BMC Neurol. 2020; 20(1):1–15.CrossRef Müller K, Fröhlich S, Germano AM, Kondragunta J, Hurtado MFdCA, Rudisch J, Schmidt D, Hirtz G, Stollmann P, Voelcker-Rehage C. Sensor-based systems for early detection of dementia (senda): a study protocol for a prospective cohort sequential study. BMC Neurol. 2020; 20(1):1–15.CrossRef
28.
go back to reference Fang J, Wang T, Li C, Hu X, Ngai E, Seet B-C, Cheng J, Guo Y, Jiang X. Depression prevalence in postgraduate students and its association with gait abnormality. IEEE Access. 2019; 7:174425–37.CrossRef Fang J, Wang T, Li C, Hu X, Ngai E, Seet B-C, Cheng J, Guo Y, Jiang X. Depression prevalence in postgraduate students and its association with gait abnormality. IEEE Access. 2019; 7:174425–37.CrossRef
29.
go back to reference Rica RL, Shimojo GL, Gomes MC, Alonso AC, Pitta RM, Santa-Rosa FA, Pontes Junior FL, Ceschini F, Gobbo S, Bergamin M, et al. Effects of a kinect-based physical training program on body composition, functional fitness and depression in institutionalized older adults. Geriatr Gerontol Int. 2020; 20(3):195–200.PubMedCrossRef Rica RL, Shimojo GL, Gomes MC, Alonso AC, Pitta RM, Santa-Rosa FA, Pontes Junior FL, Ceschini F, Gobbo S, Bergamin M, et al. Effects of a kinect-based physical training program on body composition, functional fitness and depression in institutionalized older adults. Geriatr Gerontol Int. 2020; 20(3):195–200.PubMedCrossRef
30.
go back to reference Wang T, Li C, Wu C, Zhao C, Sun J, Peng H, Hu X, Hu B. A gait assessment framework for depression detection using kinect sensors. IEEE Sensors J. 2020; 21(3):3260–70.CrossRef Wang T, Li C, Wu C, Zhao C, Sun J, Peng H, Hu X, Hu B. A gait assessment framework for depression detection using kinect sensors. IEEE Sensors J. 2020; 21(3):3260–70.CrossRef
31.
go back to reference Kondragunta J, Hirtz G. Gait Parameter Estimation of Elderly People using 3D Human Pose Estimation in Early Detection of Dementia. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Montreal: IEEE: 2020. p. 5798–801. Kondragunta J, Hirtz G. Gait Parameter Estimation of Elderly People using 3D Human Pose Estimation in Early Detection of Dementia. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Montreal: IEEE: 2020. p. 5798–801.
32.
go back to reference Sun J, Wang Y, Li J, Wan W, Cheng D, Zhang H. View-invariant gait recognition based on kinect skeleton feature. Multimedia Tools Appl. 2018; 77(19):24909–35.CrossRef Sun J, Wang Y, Li J, Wan W, Cheng D, Zhang H. View-invariant gait recognition based on kinect skeleton feature. Multimedia Tools Appl. 2018; 77(19):24909–35.CrossRef
33.
go back to reference Joshi J, Dhall A, Goecke R, Cohn JF. Relative body parts movement for automatic depression analysis. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. Geneva: IEEE: 2013. p. 492–7. Joshi J, Dhall A, Goecke R, Cohn JF. Relative body parts movement for automatic depression analysis. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. Geneva: IEEE: 2013. p. 492–7.
34.
go back to reference Chebli R. Treatment of seasonal depression with d-fenfluramine. J Clin Psychiatry. 1989; 50:343–7.PubMed Chebli R. Treatment of seasonal depression with d-fenfluramine. J Clin Psychiatry. 1989; 50:343–7.PubMed
35.
go back to reference Buisine S, Courgeon M, Charles A, Clavel C, Martin J-C, Tan N, Grynszpan O. The role of body postures in the recognition of emotions in contextually rich scenarios. Int J Hum-Comput Interact. 2014; 30(1):52–62.CrossRef Buisine S, Courgeon M, Charles A, Clavel C, Martin J-C, Tan N, Grynszpan O. The role of body postures in the recognition of emotions in contextually rich scenarios. Int J Hum-Comput Interact. 2014; 30(1):52–62.CrossRef
36.
go back to reference Nummenmaa L, Glerean E, Hari R, Hietanen JK. Bodily maps of emotions. Proc Natl Acad Sci. 2014; 111(2):646–51.PubMedCrossRef Nummenmaa L, Glerean E, Hari R, Hietanen JK. Bodily maps of emotions. Proc Natl Acad Sci. 2014; 111(2):646–51.PubMedCrossRef
37.
go back to reference Dael N, Mortillaro M, Scherer KR. Emotion expression in body action and posture. Emotion. 2012; 12(5):1085.PubMedCrossRef Dael N, Mortillaro M, Scherer KR. Emotion expression in body action and posture. Emotion. 2012; 12(5):1085.PubMedCrossRef
38.
go back to reference Hamilton M. The Hamilton rating scale for depression. In: Assessment of depression, vol. 14. Berlin: Springer: 1986. p. 143–152. Hamilton M. The Hamilton rating scale for depression. In: Assessment of depression, vol. 14. Berlin: Springer: 1986. p. 143–152.
39.
go back to reference Tucker CS, Behoora I, Nembhard HB, Lewis M, Sterling NW, Huang X. Machine learning classification of medication adherence in patients with movement disorders using non-wearable sensors. Comput Biol Med. 2015; 66:120–34.PubMedPubMedCentralCrossRef Tucker CS, Behoora I, Nembhard HB, Lewis M, Sterling NW, Huang X. Machine learning classification of medication adherence in patients with movement disorders using non-wearable sensors. Comput Biol Med. 2015; 66:120–34.PubMedPubMedCentralCrossRef
40.
go back to reference Wu M-J, Mwangi B, Bauer IE, Passos IC, Sanches M, Zunta-Soares GB, Meyer TD, Hasan KM, Soares JC. Identification and individualized prediction of clinical phenotypes in bipolar disorders using neurocognitive data, neuroimaging scans and machine learning. Neuroimage. 2017; 145:254–64.PubMedCrossRef Wu M-J, Mwangi B, Bauer IE, Passos IC, Sanches M, Zunta-Soares GB, Meyer TD, Hasan KM, Soares JC. Identification and individualized prediction of clinical phenotypes in bipolar disorders using neurocognitive data, neuroimaging scans and machine learning. Neuroimage. 2017; 145:254–64.PubMedCrossRef
41.
go back to reference Aggarwal S, Aggarwal L, Rihal MS, Aggarwal S. EEG based participant independent emotion classification using gradient boosting machines. In: 2018 IEEE 8th International Advance Computing Conference (IACC). Greater Noida: IEEE: 2018. p. 266–71. Aggarwal S, Aggarwal L, Rihal MS, Aggarwal S. EEG based participant independent emotion classification using gradient boosting machines. In: 2018 IEEE 8th International Advance Computing Conference (IACC). Greater Noida: IEEE: 2018. p. 266–71.
42.
go back to reference Song S, Shen L, Valstar M. Human behaviour-based automatic depression analysis using hand-crafted statistics and deep learned spectral features. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). Xi’an: IEEE: 2018. p. 158–65. Song S, Shen L, Valstar M. Human behaviour-based automatic depression analysis using hand-crafted statistics and deep learned spectral features. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). Xi’an: IEEE: 2018. p. 158–65.
43.
go back to reference Alghowinem S, Goecke R, Wagner M, Parkerx G, Breakspear M. Head pose and movement analysis as an indicator of depression. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. Geneva: IEEE: 2013. p. 283–8. Alghowinem S, Goecke R, Wagner M, Parkerx G, Breakspear M. Head pose and movement analysis as an indicator of depression. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. Geneva: IEEE: 2013. p. 283–8.
44.
go back to reference Zhao N, Zhang Z, Wang Y, Wang J, Li B, Zhu T, Xiang Y. See your mental state from your walk: Recognizing anxiety and depression through kinect-recorded gait data. PLoS ONE. 2019; 14(5):0216591.CrossRef Zhao N, Zhang Z, Wang Y, Wang J, Li B, Zhu T, Xiang Y. See your mental state from your walk: Recognizing anxiety and depression through kinect-recorded gait data. PLoS ONE. 2019; 14(5):0216591.CrossRef
45.
go back to reference Jing C, Liu X, Zhao N, Zhu T. Different Performances of Speech and Natural Gait in Identifying Anxiety and Depression. In: International Conference on Human Centered Computing. Switzerland: Springer, Cham: 2019. p. 200–10. Jing C, Liu X, Zhao N, Zhu T. Different Performances of Speech and Natural Gait in Identifying Anxiety and Depression. In: International Conference on Human Centered Computing. Switzerland: Springer, Cham: 2019. p. 200–10.
Metadata
Title
Simple action for depression detection: using kinect-recorded human kinematic skeletal data
Authors
Wentao Li
Qingxiang Wang
Xin Liu
Yanhong Yu
Publication date
01-12-2021
Publisher
BioMed Central
Published in
BMC Psychiatry / Issue 1/2021
Electronic ISSN: 1471-244X
DOI
https://doi.org/10.1186/s12888-021-03184-4

Other articles of this Issue 1/2021

BMC Psychiatry 1/2021 Go to the issue