Skip to main content
Top
Published in: European Radiology 12/2020

01-12-2020 | Prostate Cancer | Imaging Informatics and Artificial Intelligence

Machine learning for the identification of clinically significant prostate cancer on MRI: a meta-analysis

Authors: Renato Cuocolo, Maria Brunella Cipullo, Arnaldo Stanzione, Valeria Romeo, Roberta Green, Valeria Cantoni, Andrea Ponsiglione, Lorenzo Ugga, Massimo Imbriaco

Published in: European Radiology | Issue 12/2020

Login to get access

Abstract

Objectives

The aim of this study was to systematically review the literature and perform a meta-analysis of machine learning (ML) diagnostic accuracy studies focused on clinically significant prostate cancer (csPCa) identification on MRI.

Methods

Multiple medical databases were systematically searched for studies on ML applications in csPCa identification up to July 31, 2019. Two reviewers screened all papers independently for eligibility. The area under the receiver operating characteristic curves (AUC) was pooled to quantify predictive accuracy. A random-effects model estimated overall effect size while statistical heterogeneity was assessed with the I2 value. A funnel plot was used to investigate publication bias. Subgroup analyses were performed based on reference standard (biopsy or radical prostatectomy) and ML type (deep and non-deep).

Results

After the final revision, 12 studies were included in the analysis. Statistical heterogeneity was high both in overall and in subgroup analyses. The overall pooled AUC for ML in csPCa identification was 0.86, with 0.81–0.91 95% confidence intervals (95%CI). The biopsy subgroup (n = 9) had a pooled AUC of 0.85 (95%CI = 0.79–0.91) while the radical prostatectomy one (n = 3) of 0.88 (95%CI = 0.76–0.99). Deep learning ML (n = 4) had a 0.78 AUC (95%CI = 0.69–0.86) while the remaining 8 had AUC = 0.90 (95%CI = 0.85–0.94).

Conclusions

ML pipelines using prostate MRI to identify csPCa showed good accuracy and should be further investigated, possibly with better standardisation in design and reporting of results.

Key Points

• Overall pooled AUC was 0.86 with 0.81–0.91 95% confidence intervals.
• In the reference standard subgroup analysis, algorithm accuracy was similar with pooled AUCs of 0.85 (0.79–0.91 95% confidence intervals) and 0.88 (0.76–0.99 95% confidence intervals) for studies employing biopsies and radical prostatectomy, respectively.
• Deep learning pipelines performed worse (AUC = 0.78, 0.69–0.86 95% confidence intervals) than other approaches (AUC = 0.90, 0.85–0.94 95% confidence intervals).
Appendix
Available only for authorised users
Literature
4.
go back to reference Barkovich EJ, Shankar PR, Westphalen AC (2019) A systematic review of the existing prostate imaging reporting and data system version 2 (PI-RADSv2) literature and subset meta-analysis of PI-RADSv2 categories stratified by Gleason scores. AJR Am J Roentgenol 212:847–854. https://doi.org/10.2214/AJR.18.20571CrossRef Barkovich EJ, Shankar PR, Westphalen AC (2019) A systematic review of the existing prostate imaging reporting and data system version 2 (PI-RADSv2) literature and subset meta-analysis of PI-RADSv2 categories stratified by Gleason scores. AJR Am J Roentgenol 212:847–854. https://​doi.​org/​10.​2214/​AJR.​18.​20571CrossRef
9.
go back to reference Zhang L, Tang M, Chen S, Lei X, Zhang X, Huan Y (2017) A meta-analysis of use of prostate imaging reporting and data system version 2 (PI-RADS V2) with multiparametric MR imaging for the detection of prostate cancer. Eur Radiol 27:5204–5214. https://doi.org/10.1007/s00330-017-4843-7 Zhang L, Tang M, Chen S, Lei X, Zhang X, Huan Y (2017) A meta-analysis of use of prostate imaging reporting and data system version 2 (PI-RADS V2) with multiparametric MR imaging for the detection of prostate cancer. Eur Radiol 27:5204–5214. https://​doi.​org/​10.​1007/​s00330-017-4843-7
18.
go back to reference R Core Team (2020) R: a language and environment for statistical computing R Core Team (2020) R: a language and environment for statistical computing
22.
go back to reference Sobecki P, Życka-Malesa D, Mykhalevych I, Sklinda K, Przelaskowski A (2018) MRI imaging texture features in prostate lesions classification. In: Eskola H, Väisänen O, Viik J, Hyttinen J (eds) EMBEC & NBC 2017. EMBEC 2017, NBC 2017, IFMBE proceedings, vol 65. Springer, Singapore, pp 827–830 Sobecki P, Życka-Malesa D, Mykhalevych I, Sklinda K, Przelaskowski A (2018) MRI imaging texture features in prostate lesions classification. In: Eskola H, Väisänen O, Viik J, Hyttinen J (eds) EMBEC & NBC 2017. EMBEC 2017, NBC 2017, IFMBE proceedings, vol 65. Springer, Singapore, pp 827–830
33.
go back to reference Strang B, van der Putten P, van Rijn JN, Hutter F (2018) Don’t rule out simple models prematurely: a large scale benchmark comparing linear and non-linear classifiers in OpenML. In: Duivesteijn W, Siebes A, Ukkonen A (eds) Advances in intelligent data analysis XVII. IDA 2018, Lecture notes in computer science, vol 11191. Springer, Cham, pp 303–315CrossRef Strang B, van der Putten P, van Rijn JN, Hutter F (2018) Don’t rule out simple models prematurely: a large scale benchmark comparing linear and non-linear classifiers in OpenML. In: Duivesteijn W, Siebes A, Ukkonen A (eds) Advances in intelligent data analysis XVII. IDA 2018, Lecture notes in computer science, vol 11191. Springer, Cham, pp 303–315CrossRef
34.
go back to reference Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) Self-normalizing neural networks. In: Proceedings of the 31st international conference on neural information processing systems (NIPS’17). Curran Associates Inc., Red Hook, pp 972–981 Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) Self-normalizing neural networks. In: Proceedings of the 31st international conference on neural information processing systems (NIPS’17). Curran Associates Inc., Red Hook, pp 972–981
44.
go back to reference Alabousi M, Salameh J-P, Gusenbauer K et al (2019) Biparametric vs multiparametric prostate magnetic resonance imaging for the detection of prostate cancer in treatment-naïve patients: a diagnostic test accuracy systematic review and meta-analysis. BJU Int 124:209–220. https://doi.org/10.1111/bju.14759CrossRef Alabousi M, Salameh J-P, Gusenbauer K et al (2019) Biparametric vs multiparametric prostate magnetic resonance imaging for the detection of prostate cancer in treatment-naïve patients: a diagnostic test accuracy systematic review and meta-analysis. BJU Int 124:209–220. https://​doi.​org/​10.​1111/​bju.​14759CrossRef
45.
go back to reference Zwanenburg A, Leger S, Vallières M, Löck S (2016) Image biomarker standardisation initiative. arXiv:1612.07003 Zwanenburg A, Leger S, Vallières M, Löck S (2016) Image biomarker standardisation initiative. arXiv:1612.07003
50.
go back to reference Recht B, Roelofs R, Schmidt L, Shankar V (2019) Do ImageNet classifiers generalize to ImageNet? arXiv:1902.10811 Recht B, Roelofs R, Schmidt L, Shankar V (2019) Do ImageNet classifiers generalize to ImageNet? arXiv:1902.10811
51.
go back to reference Recht B, Roelofs R, Schmidt L, Shankar V (2018) Do CIFAR-10 classifiers generalize to CIFAR-10? arXiv:1806.00451 Recht B, Roelofs R, Schmidt L, Shankar V (2018) Do CIFAR-10 classifiers generalize to CIFAR-10? arXiv:1806.00451
52.
go back to reference Yadav S, Shukla S (2016) Analysis of k-fold cross-validation over hold-out validation on colossal datasets for quality classification. In: 2016 IEEE 6th international conference on advanced computing (IACC). IEEE, pp 78–83 Yadav S, Shukla S (2016) Analysis of k-fold cross-validation over hold-out validation on colossal datasets for quality classification. In: 2016 IEEE 6th international conference on advanced computing (IACC). IEEE, pp 78–83
53.
go back to reference Claesen M, De Moor B (2015) Hyperparameter search in machine learning. arXiv:1502.02127 Claesen M, De Moor B (2015) Hyperparameter search in machine learning. arXiv:1502.02127
54.
go back to reference Rao RB, Fung G, Rosales R (2008) On the dangers of cross-validation. An experimental evaluation. In: Proceedings of the 2008 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, Philadelphia, pp 588–596 Rao RB, Fung G, Rosales R (2008) On the dangers of cross-validation. An experimental evaluation. In: Proceedings of the 2008 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, Philadelphia, pp 588–596
Metadata
Title
Machine learning for the identification of clinically significant prostate cancer on MRI: a meta-analysis
Authors
Renato Cuocolo
Maria Brunella Cipullo
Arnaldo Stanzione
Valeria Romeo
Roberta Green
Valeria Cantoni
Andrea Ponsiglione
Lorenzo Ugga
Massimo Imbriaco
Publication date
01-12-2020
Publisher
Springer Berlin Heidelberg
Published in
European Radiology / Issue 12/2020
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-020-07027-w

Other articles of this Issue 12/2020

European Radiology 12/2020 Go to the issue