Skip to main content
Top
Published in: BMC Cancer 1/2024

Open Access 01-12-2024 | Metastasis | Research

An integrated model incorporating deep learning, hand-crafted radiomics and clinical and US features to diagnose central lymph node metastasis in patients with papillary thyroid cancer

Authors: Yang Gao, Weizhen Wang, Yuan Yang, Ziting Xu, Yue Lin, Ting Lang, Shangtong Lei, Yisheng Xiao, Wei Yang, Weijun Huang, Yingjia Li

Published in: BMC Cancer | Issue 1/2024

Login to get access

Abstract

Objective

To evaluate the value of an integrated model incorporating deep learning (DL), hand-crafted radiomics and clinical and US imaging features for diagnosing central lymph node metastasis (CLNM) in patients with papillary thyroid cancer (PTC).

Methods

This retrospective study reviewed 613 patients with clinicopathologically confirmed PTC from two institutions. The DL model and hand-crafted radiomics model were developed using primary lesion images and then integrated with clinical and US features selected by multivariate analysis to generate an integrated model. The performance was compared with junior and senior radiologists on the independent test set. SHapley Additive exPlanations (SHAP) plot and Gradient-weighted Class Activation Mapping (Grad-CAM) were used for the visualized explanation of the model.

Results

The integrated model yielded the best performance with an AUC of 0.841. surpassing that of the hand-crafted radiomics model (0.706, p < 0.001) and the DL model (0.819, p = 0.26). Compared to junior and senior radiologists, the integrated model reduced the missed CLNM rate from 57.89% and 44.74–27.63%, and decreased the rate of unnecessary central lymph node dissection (CLND) from 29.87% and 27.27–18.18%, respectively. SHAP analysis revealed that the DL features played a primary role in the diagnosis of CLNM, while clinical and US features (such as extrathyroidal extension, tumour size, age, gender, and multifocality) provided additional support. Grad-CAM indicated that the model exhibited a stronger focus on thyroid capsule in patients with CLNM.

Conclusion

Integrated model can effectively decrease the incidence of missed CLNM and unnecessary CLND. The application of the integrated model can help improve the acceptance of AI-assisted US diagnosis among radiologists.
Appendix
Available only for authorised users
Literature
1.
go back to reference Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global Cancer statistics 2020: GLOBOCAN estimates of incidence and Mortality Worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71:209–49.CrossRefPubMed Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global Cancer statistics 2020: GLOBOCAN estimates of incidence and Mortality Worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71:209–49.CrossRefPubMed
2.
go back to reference White ML, Gauger PG, Doherty GM. Central lymph node dissection in differentiated thyroid cancer. World J Surg. 2007;31:895–904.CrossRefPubMed White ML, Gauger PG, Doherty GM. Central lymph node dissection in differentiated thyroid cancer. World J Surg. 2007;31:895–904.CrossRefPubMed
3.
go back to reference Wang TS, Sosa JA. Thyroid surgery for differentiated thyroid cancer — recent advances and future directions. Nat Rev Endocrinol. 2018;14:670–83.CrossRefPubMed Wang TS, Sosa JA. Thyroid surgery for differentiated thyroid cancer — recent advances and future directions. Nat Rev Endocrinol. 2018;14:670–83.CrossRefPubMed
4.
go back to reference Medas F, Canu GL, Cappellacci F, Anedda G, Conzo G, Erdas E, et al. Prophylactic central lymph node dissection improves disease-free survival in patients with intermediate and high risk differentiated thyroid carcinoma: a retrospective analysis on 399 patients. Cancers (Basel). 2020;12:1–12.CrossRef Medas F, Canu GL, Cappellacci F, Anedda G, Conzo G, Erdas E, et al. Prophylactic central lymph node dissection improves disease-free survival in patients with intermediate and high risk differentiated thyroid carcinoma: a retrospective analysis on 399 patients. Cancers (Basel). 2020;12:1–12.CrossRef
5.
go back to reference Adam MA, Pura J, Goffredo P, Dinan MA, Reed SD, Scheri RP, et al. Presence and number of lymph node metastases are associated with compromised survival for patients younger than age 45 years with papillary thyroid cancer. J Clin Oncol. 2015;33:2370–5.CrossRefPubMed Adam MA, Pura J, Goffredo P, Dinan MA, Reed SD, Scheri RP, et al. Presence and number of lymph node metastases are associated with compromised survival for patients younger than age 45 years with papillary thyroid cancer. J Clin Oncol. 2015;33:2370–5.CrossRefPubMed
6.
go back to reference Chinese Society of Endocrinology; Thyroid and Metabolism Surgery Group of the Chinese Society of Surgery;Chinese Society of Nuclear Medicine. Chinese Society of Ultrasound in Medicine. Guidelines for the diagnosis and management of thyroid nodules and differentiated thyroid cancer (second edition). Chin J Endocrinol Metab. 2023;39:181–226. Chinese Society of Endocrinology; Thyroid and Metabolism Surgery Group of the Chinese Society of Surgery;Chinese Society of Nuclear Medicine. Chinese Society of Ultrasound in Medicine. Guidelines for the diagnosis and management of thyroid nodules and differentiated thyroid cancer (second edition). Chin J Endocrinol Metab. 2023;39:181–226.
7.
go back to reference Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, et al. 2015 American Thyroid Association Management Guidelines for adult patients with thyroid nodules and differentiated thyroid Cancer: the American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid Cancer. Thyroid. 2016;26:1–133.CrossRefPubMedPubMedCentral Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, et al. 2015 American Thyroid Association Management Guidelines for adult patients with thyroid nodules and differentiated thyroid Cancer: the American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid Cancer. Thyroid. 2016;26:1–133.CrossRefPubMedPubMedCentral
8.
go back to reference Zhao H, Li H. Meta-analysis of ultrasound for cervical lymph nodes in papillary thyroid cancer: diagnosis of central and lateral compartment nodal metastases. Eur J Radiol. 2019;112:14–21.CrossRefPubMed Zhao H, Li H. Meta-analysis of ultrasound for cervical lymph nodes in papillary thyroid cancer: diagnosis of central and lateral compartment nodal metastases. Eur J Radiol. 2019;112:14–21.CrossRefPubMed
9.
go back to reference Meng Y, Yang Y, Hu M, Zhang Z, Zhou X. Artificial intelligence-based radiomics in bone tumors: technical advances and clinical application. Semin Cancer Biol. 2023;95:75–87.CrossRefPubMed Meng Y, Yang Y, Hu M, Zhang Z, Zhou X. Artificial intelligence-based radiomics in bone tumors: technical advances and clinical application. Semin Cancer Biol. 2023;95:75–87.CrossRefPubMed
10.
go back to reference Lambin P, Leijenaar RTH, Deist TM, Peerlings J, De Jong EEC, Van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:749–62.CrossRefPubMed Lambin P, Leijenaar RTH, Deist TM, Peerlings J, De Jong EEC, Van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:749–62.CrossRefPubMed
11.
go back to reference Liu S, Wang Y, Yang X, Lei B, Liu L, Li SX, et al. Deep Learn Med Ultrasound Analysis: Rev Eng. 2019;5:261–75. Liu S, Wang Y, Yang X, Lei B, Liu L, Li SX, et al. Deep Learn Med Ultrasound Analysis: Rev Eng. 2019;5:261–75.
12.
go back to reference Levine AB, Schlosser C, Grewal J, Coope R, Jones SJM, Yip S. Rise of the machines: advances in Deep Learning for Cancer diagnosis. Trends in cancer. 2019;5:157–69.CrossRefPubMed Levine AB, Schlosser C, Grewal J, Coope R, Jones SJM, Yip S. Rise of the machines: advances in Deep Learning for Cancer diagnosis. Trends in cancer. 2019;5:157–69.CrossRefPubMed
13.
go back to reference Wang W, Li Y. [Radiomics for prediction of central lymph node metastasis in the neck in patients with thyroid papillary carcinoma]. Nan Fang Yi Ke Da Xue Xue Bao. 2019;39:1094–8.PubMed Wang W, Li Y. [Radiomics for prediction of central lymph node metastasis in the neck in patients with thyroid papillary carcinoma]. Nan Fang Yi Ke Da Xue Xue Bao. 2019;39:1094–8.PubMed
14.
go back to reference Wang S, Hou Y, Li Z, Dong J, Tang C. Combining ConvNets with hand-crafted features for action recognition based on an HMM-SVM classifier. Multimed Tools Appl. 2018;77:18983–98.CrossRef Wang S, Hou Y, Li Z, Dong J, Tang C. Combining ConvNets with hand-crafted features for action recognition based on an HMM-SVM classifier. Multimed Tools Appl. 2018;77:18983–98.CrossRef
15.
go back to reference Antropova N, Huynh BQ, Giger ML. A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys. 2017;44:5162–71.CrossRefPubMedPubMedCentral Antropova N, Huynh BQ, Giger ML. A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys. 2017;44:5162–71.CrossRefPubMedPubMedCentral
16.
go back to reference Dai Q, Liu D, Tao Y, Ding C, Li S, Zhao C, et al. Nomograms based on preoperative multimodal ultrasound of papillary thyroid carcinoma for predicting central lymph node metastasis. Eur Radiol. 2022;32:4596–608.CrossRefPubMed Dai Q, Liu D, Tao Y, Ding C, Li S, Zhao C, et al. Nomograms based on preoperative multimodal ultrasound of papillary thyroid carcinoma for predicting central lymph node metastasis. Eur Radiol. 2022;32:4596–608.CrossRefPubMed
17.
go back to reference Wang J, Yuan C, Han C, Wen Y, Lu H, Liu C, et al. IMAL-Net: interpretable multi-task attention learning network for invasive lung adenocarcinoma screening in CT images. Med Phys. 2021;48:7913–29.CrossRefPubMed Wang J, Yuan C, Han C, Wen Y, Lu H, Liu C, et al. IMAL-Net: interpretable multi-task attention learning network for invasive lung adenocarcinoma screening in CT images. Med Phys. 2021;48:7913–29.CrossRefPubMed
18.
go back to reference Kocak B, Baessler B, Bakas S, Cuocolo R, Fedorov A, Maier-Hein L et al. CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII. Insights Imaging. 2023;14. Kocak B, Baessler B, Bakas S, Cuocolo R, Fedorov A, Maier-Hein L et al. CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII. Insights Imaging. 2023;14.
19.
go back to reference Mongan J, Moy L, Kahn CE. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell. 2020;2:e200029.CrossRefPubMedPubMedCentral Mongan J, Moy L, Kahn CE. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell. 2020;2:e200029.CrossRefPubMedPubMedCentral
20.
go back to reference Zhou J, Yin L, Wei X, Zhang S, Song Y, Luo B, et al. 2020 Chinese guidelines for ultrasound malignancy risk stratification of thyroid nodules: the C-TIRADS. Endocrine. 2020;70:256–79.CrossRefPubMed Zhou J, Yin L, Wei X, Zhang S, Song Y, Luo B, et al. 2020 Chinese guidelines for ultrasound malignancy risk stratification of thyroid nodules: the C-TIRADS. Endocrine. 2020;70:256–79.CrossRefPubMed
21.
go back to reference Adler DD, Carson PL, Rubin JM, Quinn-Reid D. Of breast Cancer: preliminary findings. Ultrasound Med Biol. 1990;16:553–9.CrossRefPubMed Adler DD, Carson PL, Rubin JM, Quinn-Reid D. Of breast Cancer: preliminary findings. Ultrasound Med Biol. 1990;16:553–9.CrossRefPubMed
22.
go back to reference Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, et al. ACR thyroid imaging, reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee. J Am Coll Radiol. 2017;14:587–95.CrossRefPubMed Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, et al. ACR thyroid imaging, reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee. J Am Coll Radiol. 2017;14:587–95.CrossRefPubMed
23.
go back to reference van Griethuysen JJM, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V, et al. Computational Radiomics System to Decode the Radiographic phenotype. Cancer Res. 2017;77:e104–7.CrossRefPubMedPubMedCentral van Griethuysen JJM, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V, et al. Computational Radiomics System to Decode the Radiographic phenotype. Cancer Res. 2017;77:e104–7.CrossRefPubMedPubMedCentral
24.
go back to reference Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009. p. 248–55. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009. p. 248–55.
25.
go back to reference Kolesnikov A, Beyer L, Zhai X, Puigcerver J, Yung J, Gelly S et al. Big Transfer (BiT): General Visual Representation Learning. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). 2020;12350 LNCS:491–507. Kolesnikov A, Beyer L, Zhai X, Puigcerver J, Yung J, Gelly S et al. Big Transfer (BiT): General Visual Representation Learning. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). 2020;12350 LNCS:491–507.
26.
go back to reference Hwang HS, Orloff LA. Efficacy of preoperative neck ultrasound in the detection of cervical lymph node metastasis from thyroid cancer. Laryngoscope. 2011;121:487–91.CrossRefPubMed Hwang HS, Orloff LA. Efficacy of preoperative neck ultrasound in the detection of cervical lymph node metastasis from thyroid cancer. Laryngoscope. 2011;121:487–91.CrossRefPubMed
27.
go back to reference Zhou SC, Liu TT, Zhou J, Huang YX, Guo Y, Yu JH, et al. An Ultrasound Radiomics Nomogram for Preoperative Prediction of Central Neck Lymph Node Metastasis in Papillary thyroid carcinoma. Front Oncol. 2020;10:1–13.CrossRef Zhou SC, Liu TT, Zhou J, Huang YX, Guo Y, Yu JH, et al. An Ultrasound Radiomics Nomogram for Preoperative Prediction of Central Neck Lymph Node Metastasis in Papillary thyroid carcinoma. Front Oncol. 2020;10:1–13.CrossRef
28.
go back to reference Wu Y, Rao K, Liu J, Han C, Gong L, Chong Y, et al. Machine learning algorithms for the prediction of Central Lymph Node Metastasis in patients with papillary thyroid Cancer. Front Endocrinol (Lausanne). 2020;11:1–9.CrossRef Wu Y, Rao K, Liu J, Han C, Gong L, Chong Y, et al. Machine learning algorithms for the prediction of Central Lymph Node Metastasis in patients with papillary thyroid Cancer. Front Endocrinol (Lausanne). 2020;11:1–9.CrossRef
29.
go back to reference Zheng X, Yao Z, Huang Y, Yu Y, Wang Y, Liu Y, et al. Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer. Nat Commun. 2020;11:1236.CrossRefPubMedPubMedCentral Zheng X, Yao Z, Huang Y, Yu Y, Wang Y, Liu Y, et al. Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer. Nat Commun. 2020;11:1236.CrossRefPubMedPubMedCentral
30.
go back to reference Zou Y, Shi Y, Liu J, Cui G, Yang Z, Liu M, et al. A comparative analysis of six machine learning models based on Ultrasound to Distinguish the Possibility of Central Cervical Lymph Node Metastasis in patients with papillary thyroid carcinoma. Front Oncol. 2021;11:1–10.CrossRef Zou Y, Shi Y, Liu J, Cui G, Yang Z, Liu M, et al. A comparative analysis of six machine learning models based on Ultrasound to Distinguish the Possibility of Central Cervical Lymph Node Metastasis in patients with papillary thyroid carcinoma. Front Oncol. 2021;11:1–10.CrossRef
31.
go back to reference Xia E, Chi Y, Jin L, Shen Y, Hirachan S, Bhandari A, et al. Preoperative prediction of lymph node metastasis in patients with papillary thyroid carcinoma by an artificial intelligence algorithm. Am J Transl Res. 2021;13:7695–704.PubMedPubMedCentral Xia E, Chi Y, Jin L, Shen Y, Hirachan S, Bhandari A, et al. Preoperative prediction of lymph node metastasis in patients with papillary thyroid carcinoma by an artificial intelligence algorithm. Am J Transl Res. 2021;13:7695–704.PubMedPubMedCentral
32.
go back to reference Paul R, Hawkins SH, Schabath MB, Gillies RJ, Hall LO, Goldgof DB. Predicting malignant nodules by fusing deep features with classical radiomics features. J Med Imaging. 2018;5:1.CrossRef Paul R, Hawkins SH, Schabath MB, Gillies RJ, Hall LO, Goldgof DB. Predicting malignant nodules by fusing deep features with classical radiomics features. J Med Imaging. 2018;5:1.CrossRef
33.
go back to reference Hu X, Gong J, Zhou W, Li H, Wang S, Wei M et al. Computer-aided diagnosis of ground glass pulmonary nodule by fusing deep learning and radiomics features. Phys Med Biol. 2021;66. Hu X, Gong J, Zhou W, Li H, Wang S, Wei M et al. Computer-aided diagnosis of ground glass pulmonary nodule by fusing deep learning and radiomics features. Phys Med Biol. 2021;66.
Metadata
Title
An integrated model incorporating deep learning, hand-crafted radiomics and clinical and US features to diagnose central lymph node metastasis in patients with papillary thyroid cancer
Authors
Yang Gao
Weizhen Wang
Yuan Yang
Ziting Xu
Yue Lin
Ting Lang
Shangtong Lei
Yisheng Xiao
Wei Yang
Weijun Huang
Yingjia Li
Publication date
01-12-2024
Publisher
BioMed Central
Published in
BMC Cancer / Issue 1/2024
Electronic ISSN: 1471-2407
DOI
https://doi.org/10.1186/s12885-024-11838-1

Other articles of this Issue 1/2024

BMC Cancer 1/2024 Go to the issue
Webinar | 19-02-2024 | 17:30 (CET)

Keynote webinar | Spotlight on antibody–drug conjugates in cancer

Antibody–drug conjugates (ADCs) are novel agents that have shown promise across multiple tumor types. Explore the current landscape of ADCs in breast and lung cancer with our experts, and gain insights into the mechanism of action, key clinical trials data, existing challenges, and future directions.

Dr. Véronique Diéras
Prof. Fabrice Barlesi
Developed by: Springer Medicine