Skip to main content

Advertisement

SpringerLink
Log in

Deep learning–based algorithm improved radiologists’ performance in bone metastases detection on CT

  • Imaging Informatics and Artificial Intelligence
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To develop and evaluate a deep learning–based algorithm (DLA) for automatic detection of bone metastases on CT.

Methods

This retrospective study included CT scans acquired at a single institution between 2009 and 2019. Positive scans with bone metastases and negative scans without bone metastasis were collected to train the DLA. Another 50 positive and 50 negative scans were collected separately from the training dataset and were divided into validation and test datasets at a 2:3 ratio. The clinical efficacy of the DLA was evaluated in an observer study with board-certified radiologists. Jackknife alternative free-response receiver operating characteristic analysis was used to evaluate observer performance.

Results

A total of 269 positive scans including 1375 bone metastases and 463 negative scans were collected for the training dataset. The number of lesions identified in the validation and test datasets was 49 and 75, respectively. The DLA achieved a sensitivity of 89.8% (44 of 49) with 0.775 false positives per case for the validation dataset and 82.7% (62 of 75) with 0.617 false positives per case for the test dataset. With the DLA, the overall performance of nine radiologists with reference to the weighted alternative free-response receiver operating characteristic figure of merit improved from 0.746 to 0.899 (p < .001). Furthermore, the mean interpretation time per case decreased from 168 to 85 s (p = .004).

Conclusion

With the aid of the algorithm, the overall performance of radiologists in bone metastases detection improved, and the interpretation time decreased at the same time.

Key Points

• A deep learning–based algorithm for automatic detection of bone metastases on CT was developed.

• In the observer study, overall performance of radiologists in bone metastases detection improved significantly with the aid of the algorithm.

• Radiologists’ interpretation time decreased at the same time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

CAD:

Computer-aided detection

CNN:

Convolutional neural network

DLA:

Deep learning–based algorithm

DSC:

Dice similarity coefficient

FN:

False-negative

FP:

False-positive

JAFROC:

Jackknife free-response receiver operating characteristic

wAFROC-FOM:

Weighted alternative free-response receiver operating characteristic figure of merit

References

  1. Coleman RE (2001) Metastatic bone disease: clinical features, pathophysiology and treatment strategies. Cancer Treat Rev 27:165–176

    Article  CAS  Google Scholar 

  2. Macedo F, Ladeira K, Pinho F et al (2017) Bone metastases: an overview. Oncol Rev 11:321

    Google Scholar 

  3. D’Oronzo S, Coleman R, Brown J, Silvestris F (2019) Metastatic bone disease: pathogenesis and therapeutic options: up-date on bone metastasis management. J Bone Oncol 15:100205

    Article  Google Scholar 

  4. O’Sullivan GJ, Carty FL, Cronin CG (2015) Imaging of bone metastasis: an update. World J Radiol 7:202–211

    Article  Google Scholar 

  5. Heindel W, Gübitz R, Vieth V, Weckesser M, Schober O, Schäfers M (2014) The diagnostic imaging of bone metastases. Dtsch Arztebl Int 111:741–747

    Google Scholar 

  6. Kalogeropoulou C, Karachaliou A, Zampakis P (2009) Radiologic evaluation of skeletal metastases: role of plain radiographs and computed tomography. In: Cancer metastasis – biology and treatment, 12:119–136. Springer, Dordrecht

  7. Groves AM, Beadsmoore CJ, Cheow HK et al (2006) Can 16-detector multislice CT exclude skeletal lesions during tumour staging? Implications for the cancer patient. Eur Radiol 16:1066–1073

    Article  Google Scholar 

  8. Chmelik J, Jakubicek R, Walek P et al (2018) Deep convolutional neural network-based segmentation and classification of difficult to define metastatic spinal lesions in 3D CT data. Med Image Anal 49:76–88

    Article  Google Scholar 

  9. Hammon M, Dankerl P, Tsymbal A et al (2013) Automatic detection of lytic and blastic thoracolumbar spine metastases on computed tomography. Eur Radiol 23:1862–1870

    Article  Google Scholar 

  10. Vandemark RM, Shpall EJ, Lou AM (1992) Bone metastases from breast cancer: value of CT bone windows. J Comput Assist Tomogr 16:608–614

    Article  CAS  Google Scholar 

  11. Pomerantz SM, White CS, Krebs TL et al (2000) Liver and bone window settings for soft-copy interpretation of chest and abdominal CT. AJR Am J Roentgenol 174:311–314

  12. Burns JE, Yao J, Wiese TS, Muñoz HE, Jones EC, Summers RM (2013) Automated detection of sclerotic metastases in the thoracolumbar spine at CT. Radiology 268:69–78

    Article  Google Scholar 

  13. Choy G, Khalilzadeh O, Michalski M et al (2018) Current applications and future impact of machine learning in radiology. Radiology 288:318–328

    Article  Google Scholar 

  14. Roth HR, Lu L, Liu J et al (2016) Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 35:1170–1181

    Article  Google Scholar 

  15. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: MICCAI 2015. Lecture Notes in Computer Science, 9351:234–241. Springer, Cham

  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) p770–778. Las Vegas

  17. Noguchi S, Nishio M, Yakami M, Nakagomi K, Togashi K (2020) Bone segmentation on whole-body CT using convolutional neural network with novel data augmentation techniques. Comput Biol Med 121:103767

    Article  Google Scholar 

  18. Zou KH, Warfield SK, Bharatha A et al (2004) Statistical validation of image segmentation quality based on a spatial overlap index. Acad Radiol 11:178–189

    Article  Google Scholar 

  19. Chakraborty DP, Zhai X (2016) On the meaning of the weighted alternative free-response operating characteristic figure of merit. Med Phys 43:2548–2557

    Article  Google Scholar 

  20. Chakraborty DP (2017) Observer performance methods for diagnostic imaging: foundations, modeling, and applications with R-based examples. CRC Press, Boca Raton

    Book  Google Scholar 

  21. Chakraborty DP (2021) The RJafroc book. Available via https://dpc10ster.github.io/RJafrocBook/. Accessed 24 Dec 2021

  22. Sakamoto R, Yakami M, Fujimoto K et al (2017) Temporal subtraction of serial CT images with large deformation diffeomorphic metric mapping in the identification of bone metastases. Radiology 285:629–639

    Article  Google Scholar 

  23. Nakamoto Y, Osman M, Wahl RL (2003) Prevalence and patterns of bone metastases detected with positron emission tomography using F-18 FDG. Clin Nucl Med 28:302–307

    Article  Google Scholar 

  24. Kakhki VRD, Anvari K, Sadeghi R, Mahmoudian AS, Torabian-Kakhki M (2013) Pattern and distribution of bone metastases in common malignant tumors. Nucl Med Rev 16:66–69

    Article  Google Scholar 

  25. Kobatake H (2007) Future CAD in multi-dimensional medical images: - project on multi-organ, multi-disease CAD system -. Comput Med Imaging Graph 31:258–266

  26. Liu K, Li Q, Ma J et al (2019) Evaluating a fully automated pulmonary nodule detection approach and its impact on radiologist performance. Radiol Artif Intell 1:e180084

    Article  Google Scholar 

  27. Xie H, Yang D, Sun N, Chen Z, Zhang Y (2019) Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognit 85:109–119

    Article  Google Scholar 

  28. Pehrson LM, Nielsen MB, Lauridsen CA (2019) Automatic pulmonary nodule detection applying deep learning or machine learning algorithms to the LIDC-IDRI database: a systematic review. Diagnostics 9:29

    Article  Google Scholar 

  29. Chlebus G, Schenk A, Moltz JH, van Ginneken B, Hahn HK, Meine H (2018) Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing. Sci Rep 8:15497

    Article  Google Scholar 

  30. Vorontsov E, Cerny M, Régnier P et al (2019) Deep learning for automated segmentation of liver lesions at ct in patients with colorectal cancer liver metastases. Radiol Artif Intell 1:e180014

    Article  Google Scholar 

  31. Azer SA (2019) Deep learning with convolutional neural networks for identification of liver masses and hepatocellular carcinoma: a systematic review. World J Gastrointest Oncol 11:1218–1230

    Article  Google Scholar 

  32. van Leeuwen KG, Schalekamp S, Rutten MJCM, van Ginneken B, de Rooij M (2021) Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol 31:3797–3804

    Article  Google Scholar 

  33. Çiray I, Åström G, Sundström C, Hagberg H, Ahlström H (1997) Assessment of suspected bone metastases: CT with and without clinical information compared to CT-guided bone biopsy. Acta Radiol 38:890–895

    Google Scholar 

Download references

Acknowledgements

The authors are grateful to Editage (http://www.editage.com) for their assistance in language editing.

Funding

The authors state that this work has not received any funding.

Author information

Authors and Affiliations

  1. Department of Diagnostic Imaging and Nuclear Medicine, Kyoto University Graduate School of Medicine, 54 Shogoin-kawahara-cho, Sakyo-ku, Kyoto, 606-8507, Japan

    Shunjiro Noguchi, Mizuho Nishio, Ryo Sakamoto, Masahiro Yakami, Koji Fujimoto, Yutaka Emoto, Takeshi Kubo & Yuji Nakamoto

  2. Preemptive Medicine and Lifestyle-Related Disease Research Center, Kyoto University Hospital, 54 Shogoin-kawahara-cho, Sakyo-ku, Kyoto, 606-8507, Japan

    Ryo Sakamoto & Masahiro Yakami

  3. Department of Real World Data Research and Development, Kyoto University Graduate School of Medicine, 54 Shogoin-kawahara-cho, Sakyo-ku, Kyoto, 606-8507, Japan

    Koji Fujimoto

  4. Kyoto College of Medical Science, 1-3 Imakita, Koyamahigashi-machi, Sonobe-cho, Nantan, Kyoto, 622-0041, Japan

    Yutaka Emoto

  5. Department of Radiology, Tenri Hospital, 200 Mishima, Tenri, Nara, 632-8552, Japan

    Takeshi Kubo

  6. Medical Products Technology Development Center, R&D Headquarters, Canon Inc., 3-30-2, Shimomaruko, Ohta-ku, Tokyo, 146-8501, Japan

    Yoshio Iizuka, Keita Nakagomi, Kazuhiro Miyasa & Kiyohide Satoh

Authors
  1. Shunjiro Noguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mizuho Nishio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ryo Sakamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masahiro Yakami
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Koji Fujimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yutaka Emoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Takeshi Kubo
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yoshio Iizuka
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Keita Nakagomi
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Kazuhiro Miyasa
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Kiyohide Satoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Yuji Nakamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shunjiro Noguchi.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Yuji Nakamoto.

Conflict of interest

Yuji Nakamoto, Ryo Sakamoto, and Koji Fujimoto received a research grant from Canon Medical Systems Corporation. Iizuka Yoshio, Keita Nakagomi, Kazuhiro Miyasa, and Kiyohide Satoh are employees of Canon Inc. The other authors declare that they have no conflicts of interest related to this study.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Ethics Committee.

Ethical approval

Ethical approval was obtained from the Institutional Ethics Committee.

Methodology

• retrospective

• diagnostic or prognostic study/experimental

• performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(DOCX 1217 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Noguchi, S., Nishio, M., Sakamoto, R. et al. Deep learning–based algorithm improved radiologists’ performance in bone metastases detection on CT. Eur Radiol 32, 7976–7987 (2022). https://doi.org/10.1007/s00330-022-08741-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-022-08741-3

Keywords

Search

Navigation