Skip to main content
Top

Open Access 09-05-2024 | Magnetic Resonance Imaging | Imaging Informatics and Artificial Intelligence

Assessing deep learning reconstruction for faster prostate MRI: visual vs. diagnostic performance metrics

Authors: Quintin van Lohuizen, Christian Roest, Frank F. J. Simonis, Stefan J. Fransen, Thomas C. Kwee, Derya Yakar, Henkjan Huisman

Published in: European Radiology

Login to get access

Abstract

Objective

Deep learning (DL) MRI reconstruction enables fast scan acquisition with good visual quality, but the diagnostic impact is often not assessed because of large reader study requirements. This study used existing diagnostic DL to assess the diagnostic quality of reconstructed images.

Materials and methods

A retrospective multisite study of 1535 patients assessed biparametric prostate MRI between 2016 and 2020. Likely clinically significant prostate cancer (csPCa) lesions (PI-RADS \(\ge\) 4) were delineated by expert radiologists. T2-weighted scans were retrospectively undersampled, simulating accelerated protocols. DL reconstruction (DLRecon) and diagnostic DL detection (DLDetect) were developed. The effect on the partial area under (pAUC), the Free-Response Operating Characteristic (FROC) curve, and the structural similarity (SSIM) were compared as metrics for diagnostic and visual quality, respectively. DLDetect was validated with a reader concordance analysis. Statistical analysis included Wilcoxon, permutation, and Cohen’s kappa tests for visual quality, diagnostic performance, and reader concordance.

Results

DLRecon improved visual quality at 4- and 8-fold (R4, R8) subsampling rates, with SSIM (range: −1 to 1) improved to 0.78 ± 0.02 (p < 0.001) and 0.67 ± 0.03 (p < 0.001) from 0.68 ± 0.03 and 0.51 ± 0.03, respectively. However, diagnostic performance at R4 showed a pAUC FROC of 1.33 (CI 1.28–1.39) for DL and 1.29 (CI 1.23–1.35) for naive reconstructions, both significantly lower than fully sampled pAUC of 1.58 (DL: p = 0.024, naïve: p = 0.02). Similar trends were noted for R8.

Conclusion

DL reconstruction produces visually appealing images but may reduce diagnostic accuracy. Incorporating diagnostic AI into the assessment framework offers a clinically relevant metric essential for adopting reconstruction models into clinical practice.

Clinical relevance statement

In clinical settings, caution is warranted when using DL reconstruction for MRI scans. While it recovered visual quality, it failed to match the prostate cancer detection rates observed in scans not subjected to acceleration and DL reconstruction.
Appendix
Available only for authorised users
Literature
1.
go back to reference Smith-Bindman R, Miglioretti DL, Larson EB (2008) Rising use of diagnostic medical imaging in a large integrated health system. Health Aff (Millwood) 27:1491–1502CrossRefPubMed Smith-Bindman R, Miglioretti DL, Larson EB (2008) Rising use of diagnostic medical imaging in a large integrated health system. Health Aff (Millwood) 27:1491–1502CrossRefPubMed
2.
go back to reference Nam RK, Wallis CJD, Stojcic-Bendavid J et al (2016) A pilot study to evaluate the role of magnetic resonance imaging for prostate cancer screening in the general population. J Urol 196:361–366CrossRefPubMed Nam RK, Wallis CJD, Stojcic-Bendavid J et al (2016) A pilot study to evaluate the role of magnetic resonance imaging for prostate cancer screening in the general population. J Urol 196:361–366CrossRefPubMed
3.
go back to reference Mason A, Rioux J, Clarke SE et al (2020) Comparison of objective image quality metrics to expert radiologists’ scoring of diagnostic quality of MR images. IEEE Trans Med Imaging 39:1064–1072CrossRefPubMed Mason A, Rioux J, Clarke SE et al (2020) Comparison of objective image quality metrics to expert radiologists’ scoring of diagnostic quality of MR images. IEEE Trans Med Imaging 39:1064–1072CrossRefPubMed
4.
go back to reference Muckley MJ, Riemenschneider B, Radmanesh A et al (2021) Results of the 2020 fastMRI challenge for machine learning MR image reconstruction. IEEE Trans Med Imaging 40:2306–2317CrossRefPubMedPubMedCentral Muckley MJ, Riemenschneider B, Radmanesh A et al (2021) Results of the 2020 fastMRI challenge for machine learning MR image reconstruction. IEEE Trans Med Imaging 40:2306–2317CrossRefPubMedPubMedCentral
5.
go back to reference Stabile A, Giganti F, Kasivisvanathan V et al (2020) Factors influencing variability in the performance of multiparametric magnetic resonance imaging in detecting clinically significant prostate cancer: a systematic literature review. Eur Urol Oncol 3:145–167CrossRefPubMedPubMedCentral Stabile A, Giganti F, Kasivisvanathan V et al (2020) Factors influencing variability in the performance of multiparametric magnetic resonance imaging in detecting clinically significant prostate cancer: a systematic literature review. Eur Urol Oncol 3:145–167CrossRefPubMedPubMedCentral
6.
go back to reference Weinreb JC, Barentsz JO, Choyke PL et al (2016) PI-RADS prostate imaging - reporting and data system: 2015, version 2. Eur Urol 69:16–40CrossRefPubMed Weinreb JC, Barentsz JO, Choyke PL et al (2016) PI-RADS prostate imaging - reporting and data system: 2015, version 2. Eur Urol 69:16–40CrossRefPubMed
7.
go back to reference Hosseinzadeh M, Saha A, Brand P, Slootweg I, de Rooij M, Huisman H (2022) Deep learning–assisted prostate cancer detection on bi-parametric MRI: minimum training data size requirements and effect of prior knowledge. Eur Radiol 32:2224–2234CrossRefPubMed Hosseinzadeh M, Saha A, Brand P, Slootweg I, de Rooij M, Huisman H (2022) Deep learning–assisted prostate cancer detection on bi-parametric MRI: minimum training data size requirements and effect of prior knowledge. Eur Radiol 32:2224–2234CrossRefPubMed
8.
go back to reference Schelb P, Kohl S, Radtke JP et al (2019) Classification of cancer at prostate MRI: deep learning versus clinical PI-RADS assessment. Radiology 293:607–617CrossRefPubMed Schelb P, Kohl S, Radtke JP et al (2019) Classification of cancer at prostate MRI: deep learning versus clinical PI-RADS assessment. Radiology 293:607–617CrossRefPubMed
9.
go back to reference Saha A, Hosseinzadeh M, Huisman H (2021) End-to-end prostate cancer detection in bpMRI via 3D CNNs: effects of attention mechanisms, clinical priori and decoupled false positive reduction. Med Image Anal 73:102155 Saha A, Hosseinzadeh M, Huisman H (2021) End-to-end prostate cancer detection in bpMRI via 3D CNNs: effects of attention mechanisms, clinical priori and decoupled false positive reduction. Med Image Anal 73:102155
10.
go back to reference Kojima S, Shinohara H, Hashimoto T, Suzuki S (2018) Undersampling patterns in k-space for compressed sensing MRI using two-dimensional Cartesian sampling. Radiol Phys Technol 11:303–319CrossRefPubMed Kojima S, Shinohara H, Hashimoto T, Suzuki S (2018) Undersampling patterns in k-space for compressed sensing MRI using two-dimensional Cartesian sampling. Radiol Phys Technol 11:303–319CrossRefPubMed
12.
go back to reference Hu J, Shen L, Sun G (2018) Squeeze-and-Excitation Networks. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, IEEE Computer Society, Salt Lake City, 7132–7141 Hu J, Shen L, Sun G (2018) Squeeze-and-Excitation Networks. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, IEEE Computer Society, Salt Lake City, 7132–7141
13.
go back to reference McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94CrossRefPubMed McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94CrossRefPubMed
15.
go back to reference Yiasemis G, Sonke JJ, Sanchez C, Teuwen J (2022) Recurrent variational network: a deep learning inverse problem solver applied to the task of accelerated MRI reconstruction. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, IEEE Computer Society, 722–731 2022 Yiasemis G, Sonke JJ, Sanchez C, Teuwen J (2022) Recurrent variational network: a deep learning inverse problem solver applied to the task of accelerated MRI reconstruction. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, IEEE Computer Society, 722–731 2022
16.
go back to reference Lønning K, Putzky P, Caan M, Welling M (2019) Recurrent inference machines for accelerated MRI reconstruction. Med Image Anal. 53:64–78 Lønning K, Putzky P, Caan M, Welling M (2019) Recurrent inference machines for accelerated MRI reconstruction. Med Image Anal. 53:64–78
17.
go back to reference Sriram A, Zbontar J, Defazio A et al (2020) End-to-end variational networks for accelerated MRI reconstruction. Springer International Publishing, Cham, Switzerland, p 64–73 Sriram A, Zbontar J, Defazio A et al (2020) End-to-end variational networks for accelerated MRI reconstruction. Springer International Publishing, Cham, Switzerland, p 64–73
19.
go back to reference Wang S, Su Z, Ying L et al (2016) Accelerating magnetic resonance imaging via deep learning. Proc IEEE Int Symp Biomed Imaging 2016:514–517 Wang S, Su Z, Ying L et al (2016) Accelerating magnetic resonance imaging via deep learning. Proc IEEE Int Symp Biomed Imaging 2016:514–517
20.
go back to reference Schlemper J, Caballero J, Hajnal JV, Price AN, Rueckert D (2018) A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans Med Imaging 37:491–503CrossRefPubMed Schlemper J, Caballero J, Hajnal JV, Price AN, Rueckert D (2018) A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans Med Imaging 37:491–503CrossRefPubMed
21.
go back to reference Lee D, Yoo J, Tak S, Ye JC (2018) Deep residual learning for accelerated MRI using magnitude and phase networks. IEEE Trans Biomed Eng 65:1985–1995CrossRefPubMed Lee D, Yoo J, Tak S, Ye JC (2018) Deep residual learning for accelerated MRI using magnitude and phase networks. IEEE Trans Biomed Eng 65:1985–1995CrossRefPubMed
23.
go back to reference Hyun CM, Kim HP, Lee SM, Lee S, Seo JK (2018) Deep learning for undersampled MRI reconstruction. Phys Med Biol 63:135007 Hyun CM, Kim HP, Lee SM, Lee S, Seo JK (2018) Deep learning for undersampled MRI reconstruction. Phys Med Biol 63:135007
24.
go back to reference Aggarwal R, Sounderajah V, Martin G et al (2021) Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digit Med 4:65 Aggarwal R, Sounderajah V, Martin G et al (2021) Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digit Med 4:65
25.
go back to reference Lång K, Josefsson V, Larsson AM et al (2023) Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study. Lancet Oncol 24:936–944 Lång K, Josefsson V, Larsson AM et al (2023) Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study. Lancet Oncol 24:936–944
Metadata
Title
Assessing deep learning reconstruction for faster prostate MRI: visual vs. diagnostic performance metrics
Authors
Quintin van Lohuizen
Christian Roest
Frank F. J. Simonis
Stefan J. Fransen
Thomas C. Kwee
Derya Yakar
Henkjan Huisman
Publication date
09-05-2024
Publisher
Springer Berlin Heidelberg
Published in
European Radiology
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-024-10771-y