Skip to main content
Top
Published in: European Radiology 10/2019

01-10-2019 | Computed Tomography | Imaging Informatics and Artificial Intelligence

Application of deep learning to the diagnosis of cervical lymph node metastasis from thyroid cancer with CT

Authors: Jeong Hoon Lee, Eun Ju Ha, Ju Han Kim

Published in: European Radiology | Issue 10/2019

Login to get access

Abstract

Purpose

To develop a deep learning–based computer-aided diagnosis (CAD) system for use in the CT diagnosis of cervical lymph node metastasis (LNM) in patients with thyroid cancer.

Methods

A total of 995 axial CT images that included benign (n = 647) and malignant (n = 348) lymph nodes were collected from 202 patients with thyroid cancer who underwent CT for surgical planning between July 2017 and January 2018. The datasets were randomly split into training (79.0%), validation (10.5%), and test (10.5%) datasets. Eight deep convolutional neural network (CNN) models were used to classify the images into metastatic or benign lymph nodes. Pretrained networks were used on the ImageNet and the best-performing algorithm was selected. Class-specific discriminative regions were visualized with attention heatmap using a global average pooling method.

Results

The area under the ROC curve (AUROC) for the tested algorithms ranged from 0.909 to 0.953. The sensitivity, specificity, and accuracy of the best-performing algorithm were all 90.4%, respectively. Attention heatmap highlighted important subregions for further clinical review.

Conclusion

A deep learning–based CAD system could accurately classify cervical LNM in patients with thyroid cancer on preoperative CT with an AUROC of 0.953. Whether this approach has clinical utility will require evaluation in a clinical setting.

Key Points

• A deep learning–based CAD system could accurately classify cervical lymph node metastasis. The AUROC for the eight tested algorithms ranged from 0.909 to 0.953.
• Of the eight models, the ResNet50 algorithm was the best-performing model for the validation dataset with 0.953 AUROC. The sensitivity, specificity, and accuracy of the ResNet50 model were all 90.4%, respectively, in the test dataset.
• Based on its high accuracy of 90.4%, we consider that this model may be useful in a clinical setting to detect LNM on preoperative CT in patients with thyroid cancer.
Appendix
Available only for authorised users
Literature
1.
go back to reference Davies L, Welch HG (2014) Current thyroid cancer trends in the United States. JAMA Otolaryngol Head Neck Surg 140:317–322CrossRefPubMed Davies L, Welch HG (2014) Current thyroid cancer trends in the United States. JAMA Otolaryngol Head Neck Surg 140:317–322CrossRefPubMed
2.
go back to reference Li N, Du XL, Reitzel LR, Xu L, Sturgis EM (2013) Impact of enhanced detection on the increase in thyroid cancer incidence in the United States: review of incidence trends by socioeconomic status within the surveillance, epidemiology, and end results registry, 1980-2008. Thyroid 23:103–110CrossRefPubMedPubMedCentral Li N, Du XL, Reitzel LR, Xu L, Sturgis EM (2013) Impact of enhanced detection on the increase in thyroid cancer incidence in the United States: review of incidence trends by socioeconomic status within the surveillance, epidemiology, and end results registry, 1980-2008. Thyroid 23:103–110CrossRefPubMedPubMedCentral
3.
go back to reference Haugen BR, Alexander EK, Bible KC et al (2016) 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid Cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26:1–133CrossRefPubMedPubMedCentral Haugen BR, Alexander EK, Bible KC et al (2016) 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid Cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26:1–133CrossRefPubMedPubMedCentral
4.
go back to reference Al-Saif O, Farrar WB, Bloomston M, Porter K, Ringel MD, Kloos RT (2010) Long-term efficacy of lymph node reoperation for persistent papillary thyroid cancer. J Clin Endocrinol Metab 95:2187–2194CrossRefPubMedPubMedCentral Al-Saif O, Farrar WB, Bloomston M, Porter K, Ringel MD, Kloos RT (2010) Long-term efficacy of lymph node reoperation for persistent papillary thyroid cancer. J Clin Endocrinol Metab 95:2187–2194CrossRefPubMedPubMedCentral
5.
go back to reference Stack BC Jr, Ferris RL, Goldenberg D et al (2012) American Thyroid Association consensus review and statement regarding the anatomy, terminology, and rationale for lateral neck dissection in differentiated thyroid cancer. Thyroid 22:501–508CrossRefPubMed Stack BC Jr, Ferris RL, Goldenberg D et al (2012) American Thyroid Association consensus review and statement regarding the anatomy, terminology, and rationale for lateral neck dissection in differentiated thyroid cancer. Thyroid 22:501–508CrossRefPubMed
6.
go back to reference Durante C, Montesano T, Torlontano M et al (2013) Papillary thyroid cancer: time course of recurrences during postsurgery surveillance. J Clin Endocrinol Metab 98:636–642CrossRefPubMed Durante C, Montesano T, Torlontano M et al (2013) Papillary thyroid cancer: time course of recurrences during postsurgery surveillance. J Clin Endocrinol Metab 98:636–642CrossRefPubMed
7.
go back to reference Tufano RP, Clayman G, Heller KS et al (2015) Management of recurrent/persistent nodal disease in patients with differentiated thyroid cancer: a critical review of the risks and benefits of surgical intervention versus active surveillance. Thyroid 25:15–27CrossRefPubMed Tufano RP, Clayman G, Heller KS et al (2015) Management of recurrent/persistent nodal disease in patients with differentiated thyroid cancer: a critical review of the risks and benefits of surgical intervention versus active surveillance. Thyroid 25:15–27CrossRefPubMed
8.
10.
go back to reference Shin JH, Baek JH, Chung J et al (2016) Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean Society of Thyroid Radiology consensus statement and recommendations. Korean J Radiol 17:370–395CrossRefPubMedPubMedCentral Shin JH, Baek JH, Chung J et al (2016) Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean Society of Thyroid Radiology consensus statement and recommendations. Korean J Radiol 17:370–395CrossRefPubMedPubMedCentral
11.
go back to reference Suh CH, Baek JH, Choi YJ, Lee JH (2017) Performance of CT in the preoperative diagnosis of cervical lymph node metastasis in patients with papillary thyroid cancer: a systematic review and meta-analysis. AJNR Am J Neuroradiol 38:154–161CrossRefPubMedPubMedCentral Suh CH, Baek JH, Choi YJ, Lee JH (2017) Performance of CT in the preoperative diagnosis of cervical lymph node metastasis in patients with papillary thyroid cancer: a systematic review and meta-analysis. AJNR Am J Neuroradiol 38:154–161CrossRefPubMedPubMedCentral
14.
go back to reference Miotto R, Wang F, Wang S, Jiang X, Dudley JT (2017) Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform Miotto R, Wang F, Wang S, Jiang X, Dudley JT (2017) Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform
16.
go back to reference Choi YJ, Baek JH, Park HS et al (2017) A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 27:546–552CrossRefPubMed Choi YJ, Baek JH, Park HS et al (2017) A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 27:546–552CrossRefPubMed
18.
go back to reference Jeong EY, Kim HL, Ha EJ et al (2018) Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol Jeong EY, Kim HL, Ha EJ et al (2018) Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol
19.
go back to reference Lee JH, Baek JH, Kim JH et al (2018) Deep learning-based computer-aided diagnosis system for localization and diagnosis of metastatic lymph nodes on ultrasound: a pilot study. Thyroid 28:1332–1338CrossRefPubMed Lee JH, Baek JH, Kim JH et al (2018) Deep learning-based computer-aided diagnosis system for localization and diagnosis of metastatic lymph nodes on ultrasound: a pilot study. Thyroid 28:1332–1338CrossRefPubMed
21.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognitionProceedings of the IEEE conference on computer vision and. Pattern Recogn:770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognitionProceedings of the IEEE conference on computer vision and. Pattern Recogn:770–778
26.
go back to reference Russakovsky O, Deng J, Su H et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252CrossRef Russakovsky O, Deng J, Su H et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252CrossRef
28.
go back to reference Mulla MG, Knoefel WT, Gilbert J, McGregor A, Schulte KM (2012) Lateral cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the lateral compartment. Clin Endocrinol 77:126–131CrossRef Mulla MG, Knoefel WT, Gilbert J, McGregor A, Schulte KM (2012) Lateral cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the lateral compartment. Clin Endocrinol 77:126–131CrossRef
29.
go back to reference Moon HJ, Kim EK, Yoon JH, Kwak JY (2012) Differences in the diagnostic performances of staging US for thyroid malignancy according to experience. Ultrasound Med Biol 38:568–573CrossRefPubMed Moon HJ, Kim EK, Yoon JH, Kwak JY (2012) Differences in the diagnostic performances of staging US for thyroid malignancy according to experience. Ultrasound Med Biol 38:568–573CrossRefPubMed
30.
go back to reference Mulla M, Schulte KM (2012) Central cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the central compartment. Clin Endocrinol 76:131–136CrossRef Mulla M, Schulte KM (2012) Central cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the central compartment. Clin Endocrinol 76:131–136CrossRef
31.
go back to reference Jeong HS, Baek CH, Son YI et al (2006) Integrated 18F-FDG PET/CT for the initial evaluation of cervical node level of patients with papillary thyroid carcinoma: comparison with ultrasound and contrast-enhanced CT. Clin Endocrinol 65:402–407CrossRef Jeong HS, Baek CH, Son YI et al (2006) Integrated 18F-FDG PET/CT for the initial evaluation of cervical node level of patients with papillary thyroid carcinoma: comparison with ultrasound and contrast-enhanced CT. Clin Endocrinol 65:402–407CrossRef
32.
go back to reference Ahn JE, Lee JH, Yi JS et al (2008) Diagnostic accuracy of CT and ultrasonography for evaluating metastatic cervical lymph nodes in patients with thyroid cancer. World J Surg 32:1552–1558CrossRefPubMed Ahn JE, Lee JH, Yi JS et al (2008) Diagnostic accuracy of CT and ultrasonography for evaluating metastatic cervical lymph nodes in patients with thyroid cancer. World J Surg 32:1552–1558CrossRefPubMed
33.
go back to reference Lee DW, Ji YB, Sung ES et al (2013) Roles of ultrasonography and computed tomography in the surgical management of cervical lymph node metastases in papillary thyroid carcinoma. Eur J Surg Oncol 39:191–196CrossRefPubMed Lee DW, Ji YB, Sung ES et al (2013) Roles of ultrasonography and computed tomography in the surgical management of cervical lymph node metastases in papillary thyroid carcinoma. Eur J Surg Oncol 39:191–196CrossRefPubMed
34.
go back to reference Ha EJ, Baek JH, Na DG (2017) Risk stratification of thyroid nodules on ultrasonography: current status and perspectives. Thyroid 27:1463–1468CrossRefPubMed Ha EJ, Baek JH, Na DG (2017) Risk stratification of thyroid nodules on ultrasonography: current status and perspectives. Thyroid 27:1463–1468CrossRefPubMed
36.
go back to reference Lee HJ, Yoon DY, Seo YL et al (2018) Intraobserver and interobserver variability in ultrasound measurements of thyroid nodules. J Ultrasound Med 37:173–178CrossRefPubMed Lee HJ, Yoon DY, Seo YL et al (2018) Intraobserver and interobserver variability in ultrasound measurements of thyroid nodules. J Ultrasound Med 37:173–178CrossRefPubMed
Metadata
Title
Application of deep learning to the diagnosis of cervical lymph node metastasis from thyroid cancer with CT
Authors
Jeong Hoon Lee
Eun Ju Ha
Ju Han Kim
Publication date
01-10-2019
Publisher
Springer Berlin Heidelberg
Published in
European Radiology / Issue 10/2019
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-019-06098-8

Other articles of this Issue 10/2019

European Radiology 10/2019 Go to the issue