Top

Insights into Imaging

Published in:

Open Access 01-12-2023 | Prostate Cancer | Original Article

Label-set impact on deep learning-based prostate segmentation on MRI

Authors: Jakob Meglič, Mohammed R. S. Sunoqrot, Tone Frost Bathen, Mattijs Elschot

Published in: Insights into Imaging | Issue 1/2023

Abstract

Background

Prostate segmentation is an essential step in computer-aided detection and diagnosis systems for prostate cancer. Deep learning (DL)-based methods provide good performance for prostate gland and zones segmentation, but little is known about the impact of manual segmentation (that is, label) selection on their performance. In this work, we investigated these effects by obtaining two different expert label-sets for the PROSTATEx I challenge training dataset (n = 198) and using them, in addition to an in-house dataset (n = 233), to assess the effect on segmentation performance. The automatic segmentation method we used was nnU-Net.

Results

The selection of training/testing label-set had a significant (p < 0.001) impact on model performance. Furthermore, it was found that model performance was significantly (p < 0.001) higher when the model was trained and tested with the same label-set. Moreover, the results showed that agreement between automatic segmentations was significantly (p < 0.0001) higher than agreement between manual segmentations and that the models were able to outperform the human label-sets used to train them.

Conclusions

We investigated the impact of label-set selection on the performance of a DL-based prostate segmentation model. We found that the use of different sets of manual prostate gland and zone segmentations has a measurable impact on model performance. Nevertheless, DL-based segmentation appeared to have a greater inter-reader agreement than manual segmentation. More thought should be given to the label-set, with a focus on multicenter manual segmentation and agreement on common procedures.

Critical relevance statement

Label-set selection significantly impacts the performance of a deep learning-based prostate segmentation model. Models using different label-set showed higher agreement than manual segmentations.

Key points

• Label-set selection has a significant impact on the performance of automatic segmentation models.

• Deep learning-based models demonstrated true learning rather than simply mimicking the label-set.

• Automatic segmentation appears to have a greater inter-reader agreement than manual segmentation.

Graphical Abstract

Available only for authorised users

Sung H, Ferlay J, Siegel RL et al (2021) Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer JClin 71(3):209–249. https://doi.org/10.3322/caac.21660CrossRef

Mottet N, Bellmunt J, Bolla M, et al. (2017) EAU-ESTRO-SIOG guidelines on prostate cancer. Part 1: screening, diagnosis, and local treatment with curative intent. Eur Urol 71(4):618–629. https://doi.org/10.1016/j.eururo.2016.08.003CrossRefPubMed

Weinreb JC, Barentsz JO, Choyke PL, et al. (2016) PI-RADS prostate imaging – reporting and data system: 2015, version 2. Eur Urol 69(1):16–40. https://doi.org/10.1016/j.eururo.2015.08.052CrossRefPubMed

Litjens G, Debats O, Barentsz J, Karssemeijer N, Huisman H (2014) Computer-aided detection of prostate cancer in MRI. IEEE Trans Med Imaging 33(5):1083–1092. https://doi.org/10.1109/TMI.2014.2303821CrossRefPubMed

Greer MD, Shih JH, Lay N, et al (2019) Interreader variability of prostate imaging reporting and data system version 2 in detecting and assessing prostate cancer lesions at prostate MRI. AJR. Am J Roentgenol 1. https://doi.org/10.2214/AJR.18.20536

Sunoqrot MR, Saha A, Hosseinzadeh M, Elschot M, Huisman H (2022) Artificial intelligence for prostate MRI: open datasets, available applications, and grand challenges. Eur Radiol Exp 6(1):35. https://doi.org/10.1186/s41747-022-00288-8CrossRefPubMedPubMedCentral

Khan Z, Yahya N, Alsaih K, Al-Hiyali MI, Meriaudeau F (2021) Recent automatic segmentation algorithms of MRI prostate regions: a review. IEEE Access. https://doi.org/10.1109/ACCESS.2021.3090825CrossRefPubMed

Li H, Lee CH, Chia D, Lin Z, Huang W, Tan CH (2022) Machine learning in prostate MRI for prostate cancer: current status and future opportunities. Diagnostics 12(2):289. https://doi.org/10.3390/diagnostics12020289CrossRefPubMedPubMedCentral

Twilt JJ, van Leeuwen KG, Huisman HJ, Fütterer JJ, de Rooij M (2021) Artificial intelligence based algorithms for prostate cancer classification and detection on magnetic resonance imaging: a narrative review. Diagnostics 11(6):959. https://doi.org/10.3390/diagnostics11060959CrossRefPubMedPubMedCentral

10.

Becker AS, Chaitanya K, Schawkat K, et al. (2019) Variability of manual segmentation of the prostate in axial t2-weighted MRI: A multi-reader study. Eur J Radiol 121:108716. https://doi.org/10.1016/j.ejrad.2019.108716CrossRefPubMed

11.

Armato SG III, Huisman H, Drukker K et al (2018) Prostatex challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images. J Med Imaging (Bellingham) 5(4):044501–044501. https://doi.org/10.1117/1.JMI.5.4.044501CrossRefPubMed

12.

Krüger-Stokke B, Bertilsson H, Langørgen S, Sjøbakk TAE, Bathen TF, Selnæs KM (2021) Multiparametric prostate MRI in biopsy-naïve men: a prospective evaluation of performance and biopsy strategies. Front Oncol 11:745657. https://doi.org/10.3389/fonc.2021.745657CrossRefPubMedPubMedCentral

13.

Yushkevich PA, Piven J, Hazlett HC, et al. (2006) User-guided 3d active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3):1116–1128. https://doi.org/10.1016/j.neuroimage.2006.01.015CrossRefPubMed

14.

Cuocolo R, Stanzione A, Castaldo A, De Lucia DR, Imbriaco M (2021) Quality control and whole-gland, zonal and lesion annotations for the PROSTATEx challenge public dataset. Eur J Radiol 138:109647. https://doi.org/10.1016/j.ejrad.2021.109647CrossRefPubMed

15.

Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH (2021) nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 18(2):203–211. https://doi.org/10.1038/s41592-020-01008-zCrossRefPubMed

16.

Paszke A, Gross S, Massa F et al (2019) PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems, vol 32. Curran Associates, Inc., Red Hook

17.

Klein S, Van Der Heide UA, Lips IM, Van Vulpen M, Staring M, Pluim JP (2008) Automatic segmentation of the prostate in 3d mr images by atlas matching using localized mutual information. Med Phys 35(4):1407–1417. https://doi.org/10.1118/1.2842076CrossRefPubMed

18.

Chandra SS, Dowling JA, Shen K-K, et al. (2012) Patient specific prostate segmentation in 3-d magnetic resonance images. IEEE Trans Med Imaging 31(10):1955–1964. https://doi.org/10.1109/TMI.2012.2211377CrossRefPubMed

19.

Heimann T, van Ginneken B, Styner MA et al (2009) Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Trans Med Imaging 28(8):1251–12652. https://doi.org/10.1109/TMI.2009.2013851CrossRefPubMed

20.

Montagne S, Hamzaoui D, Allera A, et al. (2021) Challenge of prostate MRI segmentation on T2-weighted images: inter-observer variability and impact of prostate morphology. Insights Imaging 12:71. https://doi.org/10.1186/s13244-021-01010-9CrossRefPubMedPubMedCentral

21.

Sunoqrot MRS, Selnæs KM, Sandsmark E, et al. (2021) The reproducibility of deep learning-based segmentation of the prostate gland and zones on T2-weighted MR images. Diagnostics 11(9):1690. https://doi.org/10.3390/diagnostics11091690CrossRefPubMedPubMedCentral

22.

Wong T, Schieda N, Sathiadoss P, Haroon M, Abreu-Gomez J, Ukwatta E (2021) Fully automated detection of prostate transition zone tumors on T2-weighted and apparent diffusion coefficient (ADC) map MR images using U-Net ensemble. Med Phys 48(11):6889–6900. https://doi.org/10.1002/mp.15181

23.

Warfield SK, Zou KH, Wells WM (2004) Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans Med Imaging 23(7):903–921. https://doi.org/10.1109/TMI.2004.828354CrossRefPubMedPubMedCentral

24.

Panch T, Pollard TJ, Mattie H, Lindemer E, Keane PA, Celi LA (2020) “Yes, but will it work for my patients?” Driving clinically relevant research with benchmark datasets. NPJ Digital Med 3(1):1–4. https://doi.org/10.1038/s41746-020-0295-6CrossRef

25.

Ellis RJ, Sander RM, Limon A (2022) Twelve key challenges in medical machine learning and solutions. Intelligence-Based Med 6:100068. https://doi.org/10.1016/j.ibmed.2022.100068CrossRef

26.

Ballestar LM, Vilaplana V (2021) MRI Brain Tumor Segmentation and Uncertainty Estimation Using 3D-UNet Architectures. In: Crimi A, Bakas S (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture Notes in Computer Science. Springer International Publishing, New York, pp 376–90. https://doi.org/10.1007/978-3-030-72084-134CrossRef

27.

Chahal ES, Patel A, Gupta A, Purwar A, G, D (2022) Unet based Xception model for prostate cancer segmentation from MRI images. Multimed Tools Appl 81(26):37333–37349. https://doi.org/10.1007/s11042-021-11334-9

28.

Zyuzin V, Chumarnaya T (2019) Comparison of Unet architectures for segmentation of the left ventricle endocardial border on two-dimensional ultrasound images. 2019 Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT). pp 110–13. https://doi.org/10.1109/USBEREIT.2019.8736616CrossRef

Title: Label-set impact on deep learning-based prostate segmentation on MRI
Authors: Jakob Meglič
Mohammed R. S. Sunoqrot
Tone Frost Bathen
Mattijs Elschot
Publication date: 01-12-2023
Publisher: Springer Vienna
Keywords: Prostate Cancer
Prostate Cancer
Published in: Insights into Imaging / Issue 1/2023
Electronic ISSN: 1869-4101
DOI: https://doi.org/10.1186/s13244-023-01502-w

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Label-set impact on deep learning-based prostate segmentation on MRI

Abstract

Background

Results

Conclusions

Critical relevance statement

Key points

Graphical Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Results

Conclusions

Critical relevance statement

Key points

Graphical Abstract

Please log in to get access to this content

Other articles of this Issue 1/2023

Diagnostic performance of prediction models for extraprostatic extension in prostate cancer: a systematic review and meta-analysis

CT and MRI features of hepatic epithelioid haemangioendothelioma: a multi-institutional retrospective analysis of 15 cases and a literature review

Impact of glycemic control on biventricular function in patients with type 2 diabetes mellitus: a cardiac magnetic resonance tissue tracking study

Validity of a multiphase CT-based radiomics model in predicting the Leibovich risk groups for localized clear cell renal cell carcinoma: an exploratory study

Development and validation of core entrustable professional activities for abdominal radiology

An interpretable MRI-based radiomics model predicting the prognosis of high-intensity focused ultrasound ablation of uterine fibroids