Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 3/2020

Open Access 01-07-2020 | Research

Enhanced character-level deep convolutional neural networks for cardiovascular disease prediction

Authors: Zhichang Zhang, Yanlong Qiu, Xiaoli Yang, Minyu Zhang

Published in: BMC Medical Informatics and Decision Making | Special Issue 3/2020

Login to get access

Abstract

Background

Electronic medical records contain a variety of valuable medical information for patients. So, when we are able to recognize and extract risk factors for disease from EMRs of patients with cardiovascular disease (CVD), and are able to use them to predict CVD, we have the ability to automatically process clinical texts, resulting in an improved accuracy of supporting doctors for the clinical diagnosis of CVD. In the case where CVD is becoming more worldwide, predictive CVD based on EMRs has been studied by many researchers to address this important aspect of improving diagnostic efficiency.

Methods

This paper proposes an Enhanced Character-level Deep Convolutional Neural Networks (EnDCNN) model for cardiovascular disease prediction.

Results

On the manually annotated Chinese EMRs corpus, our risk factor identification extraction model achieved 0.9073 of F-score, our prediction model achieved 0.9516 of F-score, and the prediction result is better than the most previous methods.

Conclusions

The character-level model based on text region embedding can well map risk factors and their labels as a unit into a vector, and downsampling plays a crucial role in improving the training efficiency of deep CNN. What’s more, the shortcut connections with pre-activation used in our model architecture implements dimension-matching free in training.
Literature
1.
go back to reference Yue-Hei Ng J, Hausknecht M, Vijayanarasimhan S, Vinyals O, Monga R, Toderici G. Beyond short snippets: Deep networks for video classification. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE: 2015. Yue-Hei Ng J, Hausknecht M, Vijayanarasimhan S, Vinyals O, Monga R, Toderici G. Beyond short snippets: Deep networks for video classification. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE: 2015.
2.
go back to reference Liang Z, Zhang G, Huang JX, Hu QV. Deep learning for healthcare decision making with EMRS. In: 2014 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2014, November 2-5, 2014. Belfast, United Kingdom: IEEE: 2014. p. 556–9. Liang Z, Zhang G, Huang JX, Hu QV. Deep learning for healthcare decision making with EMRS. In: 2014 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2014, November 2-5, 2014. Belfast, United Kingdom: IEEE: 2014. p. 556–9.
3.
go back to reference Wang J, Ding H, Bidgoli FA, Zhou B, Iribarren C, Molloi S, Baldi P. Detecting cardiovascular disease from mammograms with deep learning. IEEE Trans Med Imaging. 2017; 36(5):1172–81.CrossRef Wang J, Ding H, Bidgoli FA, Zhou B, Iribarren C, Molloi S, Baldi P. Detecting cardiovascular disease from mammograms with deep learning. IEEE Trans Med Imaging. 2017; 36(5):1172–81.CrossRef
4.
go back to reference He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, June 27-30, 2016. Las Vegas, NV, USA: IEEE: 2016. p. 770–8. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, June 27-30, 2016. Las Vegas, NV, USA: IEEE: 2016. p. 770–8.
5.
go back to reference Huang T, Shen G, Deng Z. Leap-lstm: Enhancing long short-term memory for text categorization. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, August 10-16, 2019. Macao, China: International Joint Conferences on Artificial Intelligence Organization: 2019. p. 5017–23. Huang T, Shen G, Deng Z. Leap-lstm: Enhancing long short-term memory for text categorization. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, August 10-16, 2019. Macao, China: International Joint Conferences on Artificial Intelligence Organization: 2019. p. 5017–23.
6.
go back to reference Qiao C, Huang B, Niu G, Li D, Dong D, He W, Yu D, Wu H. A new method of region embedding for text classification. In: 6th International Conference on Learning Representations, ICLR 2018, April 30 - May 3, 2018, Conference Track Proceedings. Vancouver: OpenReview.net: 2018. Qiao C, Huang B, Niu G, Li D, Dong D, He W, Yu D, Wu H. A new method of region embedding for text classification. In: 6th International Conference on Learning Representations, ICLR 2018, April 30 - May 3, 2018, Conference Track Proceedings. Vancouver: OpenReview.net: 2018.
7.
go back to reference Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. CoRR 1508.01991. 2015;abs/1508.01991. Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. CoRR 1508.01991. 2015;abs/1508.01991.
8.
go back to reference Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C. Neural architectures for named entity recognition. In: NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 12-17, 2016. San Diego California, USA: Association for Computational Linguistics: 2016. p. 260–70. Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C. Neural architectures for named entity recognition. In: NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 12-17, 2016. San Diego California, USA: Association for Computational Linguistics: 2016. p. 260–70.
9.
go back to reference Ma X, Hovy EH. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Volume 1: Long Papers. Berlin, Germany: Association for Computational Linguistics: 2016. Ma X, Hovy EH. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Volume 1: Long Papers. Berlin, Germany: Association for Computational Linguistics: 2016.
10.
go back to reference Viterbi AJ. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans Inf Theory. 1967; 13(2):260–9.CrossRef Viterbi AJ. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans Inf Theory. 1967; 13(2):260–9.CrossRef
11.
go back to reference He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. In: Computer Vision - ECCV 2016 - 14th European Conference, October 11-14, 2016, Proceedings, Part IV. Amsterdam, The Netherlands: Springer International Publishing: 2016. p. 630–45. He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. In: Computer Vision - ECCV 2016 - 14th European Conference, October 11-14, 2016, Proceedings, Part IV. Amsterdam, The Netherlands: Springer International Publishing: 2016. p. 630–45.
12.
go back to reference Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015, May 7-9, 2015, Conference Track Proceedings. San Diego: Arxiv.org: 2015. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015, May 7-9, 2015, Conference Track Proceedings. San Diego: Arxiv.org: 2015.
13.
go back to reference He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, June 27-30, 2016. Las Vegas, NV, USA: IEEE: 2016. p. 770–8. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, June 27-30, 2016. Las Vegas, NV, USA: IEEE: 2016. p. 770–8.
14.
go back to reference Conneau A, Schwenk H, Barrault L, LeCun Y. Very deep convolutional networks for natural language processing. CoRR 1606.01781. 2016;abs/1606.01781. Conneau A, Schwenk H, Barrault L, LeCun Y. Very deep convolutional networks for natural language processing. CoRR 1606.01781. 2016;abs/1606.01781.
15.
go back to reference Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8):1735–80.CrossRef Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8):1735–80.CrossRef
16.
go back to reference Johnson R, Zhang T. Semi-supervised convolutional neural networks for text categorization via region embedding. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015. Montreal: Curran Associates, Inc.: 2015. p. 919–27. Johnson R, Zhang T. Semi-supervised convolutional neural networks for text categorization via region embedding. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015. Montreal: Curran Associates, Inc.: 2015. p. 919–27.
17.
go back to reference Su J, He B, Guan Y, Jiang J, Yang J. Developing a cardiovascular disease risk factor annotated corpus of chinese electronic medical records. BMC Med Inf Decis Making. 2017; 17(1):117–1711,.CrossRef Su J, He B, Guan Y, Jiang J, Yang J. Developing a cardiovascular disease risk factor annotated corpus of chinese electronic medical records. BMC Med Inf Decis Making. 2017; 17(1):117–1711,.CrossRef
19.
go back to reference Guo J. Clinical Practical Cardiology. New Haven: Peking University Medical Press; 2015. Guo J. Clinical Practical Cardiology. New Haven: Peking University Medical Press; 2015.
20.
go back to reference Mao X, Li F, Duan Y, Wang H. Named entity recognition of electronic medical record in ophthalmology based on crf model. In: 2017 International Conference on Computer Technology, Electronics and Communication (ICCTEC). Dalian: IEEE: 2017. p. 785–8. Mao X, Li F, Duan Y, Wang H. Named entity recognition of electronic medical record in ophthalmology based on crf model. In: 2017 International Conference on Computer Technology, Electronics and Communication (ICCTEC). Dalian: IEEE: 2017. p. 785–8.
21.
go back to reference Li W, Song W, Jia X, Yang J, Wang Q, Lei Y, Huang K, Li J, Yang T. Drug specification named entity recognition base on BILSTM-CRF model. In: 43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019, July 15-19, 2019, Volume 2. Milwaukee, WI, USA: IEEE: 2019. p. 429–33. Li W, Song W, Jia X, Yang J, Wang Q, Lei Y, Huang K, Li J, Yang T. Drug specification named entity recognition base on BILSTM-CRF model. In: 43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019, July 15-19, 2019, Volume 2. Milwaukee, WI, USA: IEEE: 2019. p. 429–33.
23.
go back to reference Zhang X, Zhao JJ, LeCun Y. Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015. Montreal: Curran Associates, Inc.: 2015. p. 649–57. Zhang X, Zhao JJ, LeCun Y. Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015. Montreal: Curran Associates, Inc.: 2015. p. 649–57.
24.
go back to reference Hong X, Lin R, Yang C, Zeng N, Cai C, Gou J. Predicting alzheimer’s disease using lstm. IEEE Access. 2019; PP:1–1. Hong X, Lin R, Yang C, Zeng N, Cai C, Gou J. Predicting alzheimer’s disease using lstm. IEEE Access. 2019; PP:1–1.
Metadata
Title
Enhanced character-level deep convolutional neural networks for cardiovascular disease prediction
Authors
Zhichang Zhang
Yanlong Qiu
Xiaoli Yang
Minyu Zhang
Publication date
01-07-2020
Publisher
BioMed Central
DOI
https://doi.org/10.1186/s12911-020-1118-z

Other articles of this Special Issue 3/2020

BMC Medical Informatics and Decision Making 3/2020 Go to the issue