Skip to main content
Top
Published in: Journal of Digital Imaging 4/2017

Open Access 01-08-2017

Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network

Authors: Jianning Chi, Ekta Walia, Paul Babyn, Jimmy Wang, Gary Groot, Mark Eramian

Published in: Journal of Imaging Informatics in Medicine | Issue 4/2017

Login to get access

Abstract

With many thyroid nodules being incidentally detected, it is important to identify as many malignant nodules as possible while excluding those that are highly likely to be benign from fine needle aspiration (FNA) biopsies or surgeries. This paper presents a computer-aided diagnosis (CAD) system for classifying thyroid nodules in ultrasound images. We use deep learning approach to extract features from thyroid ultrasound images. Ultrasound images are pre-processed to calibrate their scale and remove the artifacts. A pre-trained GoogLeNet model is then fine-tuned using the pre-processed image samples which leads to superior feature extraction. The extracted features of the thyroid ultrasound images are sent to a Cost-sensitive Random Forest classifier to classify the images into “malignant” and “benign” cases. The experimental results show the proposed fine-tuned GoogLeNet model achieves excellent classification performance, attaining 98.29% classification accuracy, 99.10% sensitivity and 93.90% specificity for the images in an open access database (Pedraza et al. 16), while 96.34% classification accuracy, 86% sensitivity and 99% specificity for the images in our local health region database.
Literature
1.
go back to reference Diana Gaitini, Rhodri M Evans, and Gordana Ivanac, Chapter 16: thyroid ultrasound. EFSUMB Course Book, 2011. Diana Gaitini, Rhodri M Evans, and Gordana Ivanac, Chapter 16: thyroid ultrasound. EFSUMB Course Book, 2011.
2.
go back to reference Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, Choi JS, Kim BM, Kim E-K: Thyroid imaging reporting and data system for us features of nodules: a step in establishing better stratification of cancer risk. Radiology 260(3):892–899, 2011CrossRefPubMed Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, Choi JS, Kim BM, Kim E-K: Thyroid imaging reporting and data system for us features of nodules: a step in establishing better stratification of cancer risk. Radiology 260(3):892–899, 2011CrossRefPubMed
3.
go back to reference Lingam RK, Qarib MH, Tolley NS: Evaluating thyroid nodules: predicting and selecting malignant nodules for fine-needle aspiration (fna) cytology. Insights Imaging 4(5):617–624, 2013CrossRefPubMedPubMedCentral Lingam RK, Qarib MH, Tolley NS: Evaluating thyroid nodules: predicting and selecting malignant nodules for fine-needle aspiration (fna) cytology. Insights Imaging 4(5):617–624, 2013CrossRefPubMedPubMedCentral
4.
go back to reference Chen S-J, Chang C-Y, Chang K-Y, Tzeng J-E, Chen Y-T, Lin C-W, Hsu W-C, Wei C-K: Classification of the thyroid nodules based on characteristic sonographic textural feature and correlated histopathology using hierarchical support vector machines. Ultrasound Med Biol 36(12):2018–2026, 2010CrossRefPubMed Chen S-J, Chang C-Y, Chang K-Y, Tzeng J-E, Chen Y-T, Lin C-W, Hsu W-C, Wei C-K: Classification of the thyroid nodules based on characteristic sonographic textural feature and correlated histopathology using hierarchical support vector machines. Ultrasound Med Biol 36(12):2018–2026, 2010CrossRefPubMed
5.
go back to reference Chang C-Y, Liu H-Y, Tseng C-H, Shih S-R: Computer-aided diagnosis for thyroid Graves’ disease in ultrasound images. Biomed Eng: Appl Basis Commun 22(02):91–99, 2010 Chang C-Y, Liu H-Y, Tseng C-H, Shih S-R: Computer-aided diagnosis for thyroid Graves’ disease in ultrasound images. Biomed Eng: Appl Basis Commun 22(02):91–99, 2010
6.
go back to reference Katsigiannis S, Keramidas EG, Maroulis D: A contourlet transform feature extraction scheme for ultrasound thyroid texture classification. Int J Eng Intell Syst Electr Eng Commun 18(3):171, 2010 Katsigiannis S, Keramidas EG, Maroulis D: A contourlet transform feature extraction scheme for ultrasound thyroid texture classification. Int J Eng Intell Syst Electr Eng Commun 18(3):171, 2010
7.
go back to reference Rajendra Acharya U, Vinitha Sree S, Swapna G, Gupta S, Filippo M, Garberoglio R, Witkowska A, Suri JS: Effect of complex wavelet transform filter on thyroid tumor classification in three-dimensional ultrasound. Proc Inst Mech Eng H J Eng Med 227(3):284–292, 2013CrossRef Rajendra Acharya U, Vinitha Sree S, Swapna G, Gupta S, Filippo M, Garberoglio R, Witkowska A, Suri JS: Effect of complex wavelet transform filter on thyroid tumor classification in three-dimensional ultrasound. Proc Inst Mech Eng H J Eng Med 227(3):284–292, 2013CrossRef
8.
go back to reference Savelonas MA, Iakovidis DK, Dimitropoulos N, Maroulis D: Computational characterization of thyroid tissue in the radon domain. In Computer-Based Medical Systems, 2007. CBMS’07. Twentieth IEEE International Symposium on. IEEE, 2007, pp 189–192 Savelonas MA, Iakovidis DK, Dimitropoulos N, Maroulis D: Computational characterization of thyroid tissue in the radon domain. In Computer-Based Medical Systems, 2007. CBMS’07. Twentieth IEEE International Symposium on. IEEE, 2007, pp 189–192
9.
go back to reference Iakovidis DK, Keramidas EG, Maroulis D: Fuzzy local binary patterns for ultrasound texture characterization in International Conference Image Analysis and Recognition. Berlin: Springer, 2008, pp. 750–759 Iakovidis DK, Keramidas EG, Maroulis D: Fuzzy local binary patterns for ultrasound texture characterization in International Conference Image Analysis and Recognition. Berlin: Springer, 2008, pp. 750–759
10.
go back to reference Chang C-Y, Chen S-J, Tsai M-F: Application of support-vector-machine-based method for feature selection and classification of thyroid nodules in ultrasound images. Pattern Recogn 43(10):3494–3506, 2010CrossRef Chang C-Y, Chen S-J, Tsai M-F: Application of support-vector-machine-based method for feature selection and classification of thyroid nodules in ultrasound images. Pattern Recogn 43(10):3494–3506, 2010CrossRef
11.
go back to reference Rajendra Acharya U, Swapna G, Vinitha Sree S, Molinari F, Gupta S, Bardales RH, Witkowska A, Suri JS: A review on ultrasound based thyroid cancer tissue characterization and automated classification. Technol Cancer Res Treat 13(4):289–301, 2014CrossRef Rajendra Acharya U, Swapna G, Vinitha Sree S, Molinari F, Gupta S, Bardales RH, Witkowska A, Suri JS: A review on ultrasound based thyroid cancer tissue characterization and automated classification. Technol Cancer Res Treat 13(4):289–301, 2014CrossRef
12.
go back to reference Hijazi S, Kumar R, Rowen C: Using convolutional neural networks for image recognition. Technical Report, 2015, available online: http://ip.cadence.com/uploads/901/cnn-wp-pdf Hijazi S, Kumar R, Rowen C: Using convolutional neural networks for image recognition. Technical Report, 2015, available online: http://​ip.​cadence.​com/​uploads/​901/​cnn-wp-pdf
13.
go back to reference Spanhol FA, Oliveira LS, Petitjean C, Heutte L: Breast cancer histopathological image classification using convolutional neural networks. In Neural Networks (IJCNN), 2016 International Joint Conference on, pp. 2560–2567, Vancouver, BC, Canada, 2016 Spanhol FA, Oliveira LS, Petitjean C, Heutte L: Breast cancer histopathological image classification using convolutional neural networks. In Neural Networks (IJCNN), 2016 International Joint Conference on, pp. 2560–2567, Vancouver, BC, Canada, 2016
14.
go back to reference Li W, Cao P, Zhao D, Wang J: Pulmonary nodule classification with deep convolutional neural networks on computed tomography images. In Computational and Mathematical Methods in Medicine, 2016 Li W, Cao P, Zhao D, Wang J: Pulmonary nodule classification with deep convolutional neural networks on computed tomography images. In Computational and Mathematical Methods in Medicine, 2016
15.
go back to reference Holger R. Roth, Le Lu, Ari Seff, Kevin M. Cherry, Joanne Hoffman, Shijun Wang, Jiamin Liu and Evrim Turkbey, A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014: 17th International Conference, pp. 520–527, Boston, MA, USA, 2014. Holger R. Roth, Le Lu, Ari Seff, Kevin M. Cherry, Joanne Hoffman, Shijun Wang, Jiamin Liu and Evrim Turkbey, A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014: 17th International Conference, pp. 520–527, Boston, MA, USA, 2014.
16.
go back to reference Lina Pedraza, Carlos Vargas, Fabián Narváez, Oscar Durán, Emma Muñoz, and Eduardo Romero, An open access thyroid ultrasound image database, 2015 Lina Pedraza, Carlos Vargas, Fabián Narváez, Oscar Durán, Emma Muñoz, and Eduardo Romero, An open access thyroid ultrasound image database, 2015
17.
go back to reference Waqas Ahmed and Mark G. Eramian, Automated detection of grayscale bar and distance scale in ultrasound images, 2010. Waqas Ahmed and Mark G. Eramian, Automated detection of grayscale bar and distance scale in ultrasound images, 2010.
18.
go back to reference N. S. Narayan, P. Marziliano, and C. G. L. Hobbs, Automatic removal of manually induced artefacts in ultrasound images of thyroid gland. In 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2013, pp. 3399–3402. N. S. Narayan, P. Marziliano, and C. G. L. Hobbs, Automatic removal of manually induced artefacts in ultrasound images of thyroid gland. In 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2013, pp. 3399–3402.
19.
go back to reference Youla DC, Webb H: Image restoration by the method of convex projections: part 1 theory. IEEE Trans Med Imaging 1(2):81–94, 1982CrossRefPubMed Youla DC, Webb H: Image restoration by the method of convex projections: part 1 theory. IEEE Trans Med Imaging 1(2):81–94, 1982CrossRefPubMed
20.
go back to reference Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell, Caffe: convolutional architecture for fast feature embedding. In Proceedings of the 22Nd ACM International Conference on Multimedia, New York, NY, USA, 2014, MM ‘14, pp. 675–678, ACM. 25. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell, Caffe: convolutional architecture for fast feature embedding. In Proceedings of the 22Nd ACM International Conference on Multimedia, New York, NY, USA, 2014, MM ‘14, pp. 675–678, ACM. 25.
21.
go back to reference Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich, Going deeper with convolutions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich, Going deeper with convolutions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
22.
go back to reference Krizhevsky A, Sutskever I, Hinton GE: Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ Eds. Advances in neural information processing systems 25. New York City: Curran Associates Inc., 2012, pp. 1097–1105 Krizhevsky A, Sutskever I, Hinton GE: Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ Eds. Advances in neural information processing systems 25. New York City: Curran Associates Inc., 2012, pp. 1097–1105
23.
go back to reference Soekhoe D, van der Putten P, Plaat A: On the impact of data set size in transfer learning using deep neural networks. Cham: Springer International Publishing, 2016, pp. 50–60 Soekhoe D, van der Putten P, Plaat A: On the impact of data set size in transfer learning using deep neural networks. Cham: Springer International Publishing, 2016, pp. 50–60
24.
go back to reference Lim KJ, Choi CS, Yoon DY, Chang SK, Kim KK, Han H, Kim SS, Lee J, Jeon YH: Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography. Acad Radiol 15(7):853–858, 2008CrossRefPubMed Lim KJ, Choi CS, Yoon DY, Chang SK, Kim KK, Han H, Kim SS, Lee J, Jeon YH: Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography. Acad Radiol 15(7):853–858, 2008CrossRefPubMed
25.
go back to reference Michalis A. Savelonas, Dimitris E. Maroulis, Dimitris K. Iakovidis, Nikos Dimitropoulos, Computer-aided malignancy risk assessment of nodules in thyroid US images utilizing boundary descriptors. Informatics, 2008. PCI'08. Panhellenic Conference on. IEEE, 2008 Michalis A. Savelonas, Dimitris E. Maroulis, Dimitris K. Iakovidis, Nikos Dimitropoulos, Computer-aided malignancy risk assessment of nodules in thyroid US images utilizing boundary descriptors. Informatics, 2008. PCI'08. Panhellenic Conference on. IEEE, 2008
26.
go back to reference Iakovidis DK, Keramidas EG, Maroulis D: Fusion of fuzzy statistical distributions for classification of thyroid ultrasound patterns. Artif Intell Med 50(1):33–41, 2010CrossRefPubMed Iakovidis DK, Keramidas EG, Maroulis D: Fusion of fuzzy statistical distributions for classification of thyroid ultrasound patterns. Artif Intell Med 50(1):33–41, 2010CrossRefPubMed
27.
go back to reference Legakis I, Savelonas MA, Maroulis D, Iakovidis DK: Computer-based nodule malignancy risk assessment in thyroid ultrasound images. Int J Comput Appl 33(1):29–35, 2011 Legakis I, Savelonas MA, Maroulis D, Iakovidis DK: Computer-based nodule malignancy risk assessment in thyroid ultrasound images. Int J Comput Appl 33(1):29–35, 2011
28.
go back to reference Luo S, Kim EH, Dighe M, Kim Y: Thyroid nodule classification using ultrasound elastography via linear discriminant analysis. Ultrasonics 51(4):425–431, 2011CrossRefPubMed Luo S, Kim EH, Dighe M, Kim Y: Thyroid nodule classification using ultrasound elastography via linear discriminant analysis. Ultrasonics 51(4):425–431, 2011CrossRefPubMed
29.
go back to reference Wu H, Deng Z, Zhang B, Liu Q, Chen J: Classifier model based on machine learning algorithms: Application to different diagnosis of suspicious thyroid nodules via sonography. Am J Roentgenol 207(4):859–864, 2016CrossRef Wu H, Deng Z, Zhang B, Liu Q, Chen J: Classifier model based on machine learning algorithms: Application to different diagnosis of suspicious thyroid nodules via sonography. Am J Roentgenol 207(4):859–864, 2016CrossRef
Metadata
Title
Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network
Authors
Jianning Chi
Ekta Walia
Paul Babyn
Jimmy Wang
Gary Groot
Mark Eramian
Publication date
01-08-2017
Publisher
Springer International Publishing
Published in
Journal of Imaging Informatics in Medicine / Issue 4/2017
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI
https://doi.org/10.1007/s10278-017-9997-y

Other articles of this Issue 4/2017

Journal of Digital Imaging 4/2017 Go to the issue