Skip to main content
Top
Published in: Insights into Imaging 1/2023

Open Access 01-12-2023 | Pulmonary Nodule | Original Article

The diagnostic performance and clinical value of deep learning-based nodule detection system concerning influence of location of pulmonary nodule

Authors: Seulgi You, Ji Hyun Park, Bumhee Park, Han-Bit Shin, Taeyang Ha, Jae Sung Yun, Kyoung Joo Park, Yongjun Jung, You Na Kim, Minji Kim, Joo Sung Sun

Published in: Insights into Imaging | Issue 1/2023

Login to get access

Abstract

Background

The deep learning-based nodule detection (DLD) system improves nodule detection performance of observers on chest radiographs (CXRs). However, its performance in different pulmonary nodule (PN) locations remains unknown.

Methods

We divided the CXR intrathoracic region into non-danger zone (NDZ) and danger zone (DZ). The DZ included the lung apices, paramediastinal areas, and retrodiaphragmatic areas, where nodules could be missed. We used a dataset of 300 CXRs (100 normal and 200 abnormal images with 216 PNs [107 NDZ and 109 DZ nodules]). Eight observers (two thoracic radiologists [TRs], two non-thoracic radiologists [NTRs], and four radiology residents [RRs]) interpreted each radiograph with and without the DLD system. The metric of lesion localization fraction (LLF; the number of correctly localized lesions divided by the total number of true lesions) was used to evaluate the diagnostic performance according to the nodule location.

Results

The DLD system demonstrated a lower LLF for the detection of DZ nodules (64.2) than that of NDZ nodules (83.2, = 0.008). For DZ nodule detection, the LLF of the DLD system (64.2) was lower than that of TRs (81.7, p < 0.001), which was comparable to that of NTRs (56.4, = 0.531) and RRs (56.7, = 0.459). Nonetheless, the LLF of RRs significantly improved from 56.7 to 65.6 using the DLD system (= 0.021) for DZ nodule detection.

Conclusion

The performance of the DLD system was lower in the detection of DZ nodules compared to that of NDZ nodules. Nonetheless, RR performance in detecting DZ nodules improved upon using the DLD system.

Critical relevance statement

Despite the deep learning-based nodule detection system’s limitations in detecting danger zone nodules, it proves beneficial for less-experienced observers by providing valuable assistance in identifying these nodules, thereby advancing nodule detection in clinical practice.

Key points

• The deep learning-based nodule detection (DLD) system can improve the diagnostic performance of observers in nodule detection.
• The DLD system shows poor diagnostic performance in detecting danger zone nodules.
• For less-experienced observers, the DLD system is helpful in detecting danger zone nodules.

Graphical Abstract

Appendix
Available only for authorised users
Literature
1.
go back to reference Kligerman S, Cai L, White CS (2013) The effect of computer-aided detection on radiologist performance in the detection of lung cancers previously missed on a chest radiograph. J Thorac Imaging 28:244–252CrossRefPubMed Kligerman S, Cai L, White CS (2013) The effect of computer-aided detection on radiologist performance in the detection of lung cancers previously missed on a chest radiograph. J Thorac Imaging 28:244–252CrossRefPubMed
2.
go back to reference Nam JG, Park S, Hwang EJ et al (2019) Development and validation of deep learning-based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Radiology 290:218–228CrossRefPubMed Nam JG, Park S, Hwang EJ et al (2019) Development and validation of deep learning-based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Radiology 290:218–228CrossRefPubMed
3.
go back to reference de Hoop B, De Boo DW, Gietema HA et al (2010) Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology 257:532–540CrossRefPubMed de Hoop B, De Boo DW, Gietema HA et al (2010) Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology 257:532–540CrossRefPubMed
4.
go back to reference Lee KH, Goo JM, Park CM, Lee HJ, Jin KN (2012) Computer-aided detection of malignant lung nodules on chest radiographs: effect on observers’ performance. Korean J Radiol 13:564–571CrossRefPubMedPubMedCentral Lee KH, Goo JM, Park CM, Lee HJ, Jin KN (2012) Computer-aided detection of malignant lung nodules on chest radiographs: effect on observers’ performance. Korean J Radiol 13:564–571CrossRefPubMedPubMedCentral
5.
go back to reference Quekel LG, Kessels AG, Goei R, van Engelshoven JM (1999) Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 115:720–724CrossRefPubMed Quekel LG, Kessels AG, Goei R, van Engelshoven JM (1999) Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 115:720–724CrossRefPubMed
6.
go back to reference Shah PK, Austin JH, White CS et al (2003) Missed non-small cell lung cancer: radiographic findings of potentially resectable lesions evident only in retrospect. Radiology 226:235–241CrossRefPubMed Shah PK, Austin JH, White CS et al (2003) Missed non-small cell lung cancer: radiographic findings of potentially resectable lesions evident only in retrospect. Radiology 226:235–241CrossRefPubMed
7.
go back to reference Muhm JR, Miller WE, Fontana RS, Sanderson DR, Uhlenhopp MA (1983) Lung cancer detected during a screening program using four-month chest radiographs. Radiology 148:609–615CrossRefPubMed Muhm JR, Miller WE, Fontana RS, Sanderson DR, Uhlenhopp MA (1983) Lung cancer detected during a screening program using four-month chest radiographs. Radiology 148:609–615CrossRefPubMed
8.
go back to reference Monnier-Cholley L, Arrive L, Porcel A et al (2001) Characteristics of missed lung cancer on chest radiographs: a French experience. Eur Radiol 11:597–605CrossRefPubMed Monnier-Cholley L, Arrive L, Porcel A et al (2001) Characteristics of missed lung cancer on chest radiographs: a French experience. Eur Radiol 11:597–605CrossRefPubMed
9.
go back to reference de Groot PM, Carter BW, Abbott GF, Wu CC (2015) Pitfalls in chest radiographic interpretation: blind spots. Semin Roentgenol 50:197–209CrossRefPubMed de Groot PM, Carter BW, Abbott GF, Wu CC (2015) Pitfalls in chest radiographic interpretation: blind spots. Semin Roentgenol 50:197–209CrossRefPubMed
10.
go back to reference Goun Choi BDN, Hwang Jung Hwa, Kim Ki-Up, Kim Hyun Jo, Kim Dong Won (2020) Missed lung cancers on chest radiograph: an illustrative review of common blind spots on chest radiograph with emphasis on various radiologic presentations of lung cancers. J Korean Radiol Soc 81:351–364CrossRef Goun Choi BDN, Hwang Jung Hwa, Kim Ki-Up, Kim Hyun Jo, Kim Dong Won (2020) Missed lung cancers on chest radiograph: an illustrative review of common blind spots on chest radiograph with emphasis on various radiologic presentations of lung cancers. J Korean Radiol Soc 81:351–364CrossRef
11.
go back to reference Jang S, Song H, Shin YJ et al (2020) Deep learning-based automatic detection algorithm for reducing overlooked lung cancers on chest radiographs. Radiology 296:652–661CrossRefPubMed Jang S, Song H, Shin YJ et al (2020) Deep learning-based automatic detection algorithm for reducing overlooked lung cancers on chest radiographs. Radiology 296:652–661CrossRefPubMed
12.
go back to reference Sung J, Park S, Lee SM et al (2021) Added value of deep learning-based detection system for multiple major findings on chest radiographs: a randomized crossover study. Radiology 299:450–459CrossRefPubMed Sung J, Park S, Lee SM et al (2021) Added value of deep learning-based detection system for multiple major findings on chest radiographs: a randomized crossover study. Radiology 299:450–459CrossRefPubMed
13.
go back to reference Kim EY, Bista AB, Kim T et al (2017) The advantage of digital tomosynthesis for pulmonary nodule detection concerning influence of nodule location and size: a phantom study. Clin Radiol 72:796 e791-796 e798CrossRef Kim EY, Bista AB, Kim T et al (2017) The advantage of digital tomosynthesis for pulmonary nodule detection concerning influence of nodule location and size: a phantom study. Clin Radiol 72:796 e791-796 e798CrossRef
14.
go back to reference Park S, Lee SM, Lee KH et al (2020) Deep learning-based detection system for multiclass lesions on chest radiographs: comparison with observer readings. Eur Radiol 30:1359–1368CrossRefPubMed Park S, Lee SM, Lee KH et al (2020) Deep learning-based detection system for multiclass lesions on chest radiographs: comparison with observer readings. Eur Radiol 30:1359–1368CrossRefPubMed
15.
go back to reference Chakraborty DP (2005) Recent advances in observer performance methodology: jackknife free-response ROC (JAFROC). Radiat Prot Dosimetry 114:26–31CrossRefPubMed Chakraborty DP (2005) Recent advances in observer performance methodology: jackknife free-response ROC (JAFROC). Radiat Prot Dosimetry 114:26–31CrossRefPubMed
16.
go back to reference van Leeuwen KG, Schalekamp S, Rutten M, van Ginneken B, de Rooij M (2021) Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol 31:3797–3804CrossRefPubMedPubMedCentral van Leeuwen KG, Schalekamp S, Rutten M, van Ginneken B, de Rooij M (2021) Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol 31:3797–3804CrossRefPubMedPubMedCentral
17.
go back to reference Hwang EJ, Goo JM, Yoon SH et al (2021) Use of artificial intelligence-based software as medical devices for chest radiography: a position paper from the Korean Society of Thoracic Radiology. Korean J Radiol 22:1743–1748CrossRefPubMedPubMedCentral Hwang EJ, Goo JM, Yoon SH et al (2021) Use of artificial intelligence-based software as medical devices for chest radiography: a position paper from the Korean Society of Thoracic Radiology. Korean J Radiol 22:1743–1748CrossRefPubMedPubMedCentral
Metadata
Title
The diagnostic performance and clinical value of deep learning-based nodule detection system concerning influence of location of pulmonary nodule
Authors
Seulgi You
Ji Hyun Park
Bumhee Park
Han-Bit Shin
Taeyang Ha
Jae Sung Yun
Kyoung Joo Park
Yongjun Jung
You Na Kim
Minji Kim
Joo Sung Sun
Publication date
01-12-2023
Publisher
Springer Vienna
Published in
Insights into Imaging / Issue 1/2023
Electronic ISSN: 1869-4101
DOI
https://doi.org/10.1186/s13244-023-01497-4

Other articles of this Issue 1/2023

Insights into Imaging 1/2023 Go to the issue