Top

Published in:

Open Access 01-12-2023 | Prostate Cancer | Original Article

Does deep learning software improve the consistency and performance of radiologists with various levels of experience in assessing bi-parametric prostate MRI?

Authors: Aydan Arslan, Deniz Alis, Servet Erdemli, Mustafa Ege Seker, Gokberk Zeybel, Sabri Sirolu, Serpil Kurtcan, Ercan Karaarslan

Published in: Insights into Imaging | Issue 1/2023

Abstract

Objective

To investigate whether commercially available deep learning (DL) software improves the Prostate Imaging-Reporting and Data System (PI-RADS) scoring consistency on bi-parametric MRI among radiologists with various levels of experience; to assess whether the DL software improves the performance of the radiologists in identifying clinically significant prostate cancer (csPCa).

Methods

We retrospectively enrolled consecutive men who underwent bi-parametric prostate MRI at a 3 T scanner due to suspicion of PCa. Four radiologists with 2, 3, 5, and > 20 years of experience evaluated the bi-parametric prostate MRI scans with and without the DL software. Whole-mount pathology or MRI/ultrasound fusion-guided biopsy was the reference. The area under the receiver operating curve (AUROC) was calculated for each radiologist with and without the DL software and compared using De Long’s test. In addition, the inter-rater agreement was investigated using kappa statistics.

Results

In all, 153 men with a mean age of 63.59 ± 7.56 years (range 53–80) were enrolled in the study. In the study sample, 45 men (29.80%) had clinically significant PCa. During the reading with the DL software, the radiologists changed their initial scores in 1/153 (0.65%), 2/153 (1.3%), 0/153 (0%), and 3/153 (1.9%) of the patients, yielding no significant increase in the AUROC (p > 0.05). Fleiss’ kappa scores among the radiologists were 0.39 and 0.40 with and without the DL software (p = 0.56).

Conclusions

The commercially available DL software does not increase the consistency of the bi-parametric PI-RADS scoring or csPCa detection performance of radiologists with varying levels of experience.

Ahdoot M, Wilbur AR, Reese SE et al (2020) MRI-targeted, systematic, and combined biopsy for prostate cancer diagnosis. N Engl J Med 382:917–928. https://doi.org/10.1056/NEJMoa1910038CrossRefPubMedPubMedCentral

Ahmed HU, El-Shater Bosaily A, Brown LC et al (2017) Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. Lancet 389:815–822. https://doi.org/10.1016/S0140-6736(16)32401-1CrossRefPubMed

Kasivisvanathan V, Rannikko AS, Borghi M et al (2018) MRI-targeted or standard biopsy for prostate-cancer diagnosis. N Engl J Med 378:1767–1777. https://doi.org/10.1056/NEJMoa1801993CrossRefPubMedPubMedCentral

Turkbey B, Rosenkrantz AB, Haider MA et al (2019) Prostate imaging reporting and Data System Version 2.1: 2019 update of prostate imaging reporting and Data System Version 2. Eur Urol 76:340–351. https://doi.org/10.1016/j.eururo.2019.02.033CrossRefPubMed

Westphalen AC, McCulloch CE, Anaokar JM et al (2020) Variability of the positive predictive value of PI-RADS for prostate MRI across 26 centers: experience of the society of abdominal radiology prostate cancer disease-focused panel. Radiology 296:76–84. https://doi.org/10.1148/radiol.2020190646CrossRefPubMed

Smith CP, Harmon SA, Barrett T et al (2019) Intra- and interreader reproducibility of PI-RADSv2: a multireader study. J Magn Reson Imaging 49:1694–1703. https://doi.org/10.1002/jmri.26555CrossRefPubMed

Yang X, Liu C, Wang Z et al (2017) Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI. Med Image Anal 42:212–227. https://doi.org/10.1016/j.media.2017.08.006CrossRefPubMed

Wang X, Yang W, Weinreb J et al (2017) Searching for prostate cancer by fully automated magnetic resonance imaging classification: deep learning versus non-deep learning. Sci Rep 7:15415. https://doi.org/10.1038/s41598-017-15720-yCrossRefPubMedPubMedCentral

Ishioka J, Matsuoka Y, Uehara S et al (2018) Computer-aided diagnosis of prostate cancer on magnetic resonance imaging using a convolutional neural network algorithm. BJU Int 122:411–417. https://doi.org/10.1111/bju.14397CrossRefPubMed

10.

Winkel DJ, Wetterauer C, Matthias MO et al (2020) Autonomous detection and classification of pi-rads lesions in an MRI screening population incorporating multicenter-labeled deep learning and biparametric imaging: proof of concept. Diagnostics. https://doi.org/10.3390/diagnostics10110951CrossRefPubMedPubMedCentral

11.

Winkel DJ, Tong A, Lou B et al (2021) A novel deep learning based computer-aided diagnosis system improves the accuracy and efficiency of radiologists in reading biparametric magnetic resonance images of the prostate. Invest Radiol 56:605–613. https://doi.org/10.1097/RLI.0000000000000780CrossRefPubMed

12.

Sanford T, Harmon SA, Turkbey EB et al (2020) Deep-learning-based artificial intelligence for PI-RADS classification to assist multiparametric prostate MRI interpretation: a development study. J Magn Reson Imaging 52:1499–1507. https://doi.org/10.1002/jmri.27204CrossRefPubMedPubMedCentral

13.

The PI-CAI Challlenge. https://pi-cai.grand-challenge.org. Accessed 8 Aug 2022

14.

Yang D, Xu D, Zhou KS et al (2017) Automatic liver segmentation using an adversarial image-to-image network. In: Franz A, Jannin P et al (eds) Maier-Hein lena. Springer, Quebec City, pp 507–515

15.

Yu X, Lou B, Shi B et al (2020) False positive reduction using multi-scale contextual features for prostate cancer detection in multi-parametric MRI scans. In: 2020 IEEE 17th international symposium on biomedical imaging (ISBI). IEEE, pp 1355–1359

16.

American Urologic Association (2022) Optimal techniques of prostate biopsy and specimen handling. http://www.auanet.org/guidelines/prostate-biopsy-and-specimen-handling

17.

Samaratunga H, Montironi R, True L et al (2011) International Society of Urological Pathology (ISUP) consensus conference on handling and staging of radical prostatectomy specimens. Working group 1: specimen handling. Mod Pathol 24:6–15. https://doi.org/10.1038/modpathol.2010.178

18.

Fleiss JL, Levin B, Paik MC (2013) Statistical methods for rates and proportions. Wiley

19.

Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46. https://doi.org/10.1177/001316446002000104CrossRef

20.

Vanbelle S (2017) Comparing dependent kappa coefficients obtained on multilevel data. Biom J 59:1016–1034. https://doi.org/10.1002/bimj.201600093CrossRefPubMedPubMedCentral

21.

Youn SY, Choi MH, Kim DH et al (2021) Detection and PI-RADS classification of focal lesions in prostate MRI: performance comparison between a deep learning-based algorithm (DLA) and radiologists with various levels of experience. Eur J Radiol 142:109894. https://doi.org/10.1016/j.ejrad.2021.109894CrossRefPubMed

22.

Park KJ, Choi SH, Kim M et al (2021) Performance of prostate imaging reporting and Data System Version 2.1 for diagnosis of prostate cancer: a systematic review and meta-analysis. J Magn Reson Imaging 54:103–112. https://doi.org/10.1002/jmri.27546CrossRefPubMed

23.

Cuocolo R, Cipullo MB, Stanzione A et al (2020) Machine learning for the identification of clinically significant prostate cancer on MRI: a meta-analysis. Eur Radiol 30:6877–6887. https://doi.org/10.1007/s00330-020-07027-wCrossRefPubMed

24.

Schelb P, Kohl S, Radtke JP et al (2019) Classification of cancer at prostate MRI: deep learning versus clinical PI-RADS assessment. Radiology 293:607–617. https://doi.org/10.1148/radiol.2019190938CrossRefPubMed

25.

Schelb P, Wang X, Radtke JP et al (2021) Simulated clinical deployment of fully automatic deep learning for clinical prostate MRI assessment. Eur Radiol 31:302–313. https://doi.org/10.1007/s00330-020-07086-zCrossRefPubMed

26.

Aldoj N, Lukas S, Dewey M, Penzkofer T (2020) Semi-automatic classification of prostate cancer on multi-parametric MR imaging using a multi-channel 3D convolutional neural network. Eur Radiol 30:1243–1253. https://doi.org/10.1007/s00330-019-06417-zCrossRefPubMed

27.

Hosseinzadeh M, Saha A, Brand P et al (2022) Deep learning–assisted prostate cancer detection on bi-parametric MRI: minimum training data size requirements and effect of prior knowledge. Eur Radiol 32:2224–2234. https://doi.org/10.1007/s00330-021-08320-yCrossRefPubMed

28.

Ardila D, Kiraly AP, Bharadwaj S et al (2019) End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat Med 25:954–961. https://doi.org/10.1038/s41591-019-0447-xCrossRefPubMed

29.

McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nat 577:89–94. https://doi.org/10.1038/s41586-019-1799-6CrossRef

30.

Padhani AR, Turkbey B (2019) Detecting prostate cancer with deep learning for MRI: a small step forward. Radiol 293:618–619. https://doi.org/10.1148/radiol.2019192012CrossRef

31.

Choi MH, Kim CK, Lee YJ, Jung SE (2019) Prebiopsy biparametric MRI for clinically significant prostate cancer detection with PI-RADS Version 2: a multicenter study. AJR Am J Roentgenol 212:839–846. https://doi.org/10.2214/AJR.18.20498CrossRefPubMed

32.

London: National Institute for Health and Care Excellence (NICE) (2021) 2021 exceptional surveillance of prostate cancer: diagnosis and management (NICE guideline NG131)

33.

Brown J, Haider MA, Chin JLK et al (2022) Evidence-based guideline recommendations on multiparametric magnetic resonance imaging in the diagnosis of clinically significant prostate cancer: a Cancer Care Ontario updated clinical practice guideline. Can Urol Assoc J 16:16–23. https://doi.org/10.5489/cuaj.7425CrossRefPubMedPubMedCentral

34.

de Rooij M, Israël B, Tummers M et al (2020) ESUR/ESUI consensus statements on multi-parametric MRI for the detection of clinically significant prostate cancer: quality requirements for image acquisition, interpretation and radiologists’ training. Eur Radiol 30:5404–5416. https://doi.org/10.1007/s00330-020-06929-zCrossRefPubMedPubMedCentral

Title: Does deep learning software improve the consistency and performance of radiologists with various levels of experience in assessing bi-parametric prostate MRI?
Authors: Aydan Arslan
Deniz Alis
Servet Erdemli
Mustafa Ege Seker
Gokberk Zeybel
Sabri Sirolu
Serpil Kurtcan
Ercan Karaarslan
Publication date: 01-12-2023
Publisher: Springer Vienna
Keywords: Prostate Cancer
Magnetic Resonance Imaging
Magnetic Resonance Imaging
Prostate Cancer
Published in: Insights into Imaging / Issue 1/2023
Electronic ISSN: 1869-4101
DOI: https://doi.org/10.1186/s13244-023-01386-w

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Does deep learning software improve the consistency and performance of radiologists with various levels of experience in assessing bi-parametric prostate MRI?

Abstract

Objective

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Objective

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2023

Stiffness on shear wave elastography as a potential microenvironment biomarker for predicting tumor recurrence in HBV-related hepatocellular carcinoma

Pretreatment synthetic magnetic resonance imaging predicts disease progression in nonmetastatic nasopharyngeal carcinoma after intensity modulation radiation therapy

Evaluation of optimal monoenergetic images acquired by dual-energy CT in the diagnosis of T staging of thoracic esophageal cancer

An active learning approach to train a deep learning algorithm for tumor segmentation from brain MR images

Dual-layer spectral-detector CT for predicting microsatellite instability status and prognosis in locally advanced gastric cancer

Chyluria: non-enhanced MR lymphography