Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 2/2023

Open Access 01-09-2022 | Original Article

Efficient contour-based annotation by iterative deep learning for organ segmentation from volumetric medical images

Authors: Mingrui Zhuang, Zhonghua Chen, Hongkai Wang, Hong Tang, Jiang He, Bobo Qin, Yuxin Yang, Xiaoxian Jin, Mengzhu Yu, Baitao Jin, Taijing Li, Lauri Kettunen

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 2/2023

Login to get access

Abstract

Purpose

Training deep neural networks usually require a large number of human-annotated data. For organ segmentation from volumetric medical images, human annotation is tedious and inefficient. To save human labour and to accelerate the training process, the strategy of annotation by iterative deep learning recently becomes popular in the research community. However, due to the lack of domain knowledge or efficient human-interaction tools, the current AID methods still suffer from long training time and high annotation burden.

Methods

We develop a contour-based annotation by iterative deep learning (AID) algorithm which uses boundary representation instead of voxel labels to incorporate high-level organ shape knowledge. We propose a contour segmentation network with a multi-scale feature extraction backbone to improve the boundary detection accuracy. We also developed a contour-based human-intervention method to facilitate easy adjustments of organ boundaries. By combining the contour-based segmentation network and the contour-adjustment intervention method, our algorithm achieves fast few-shot learning and efficient human proofreading.

Results

For validation, two human operators independently annotated four abdominal organs in computed tomography (CT) images using our method and two compared methods, i.e. a traditional contour-interpolation method and a state-of-the-art (SOTA) convolutional network (CNN) method based on voxel label representation. Compared to these methods, our approach considerably saved annotation time and reduced inter-rater variabilities. Our contour detection network also outperforms the SOTA nnU-Net in producing anatomically plausible organ shape with only a small training set.

Conclusion

Taking advantage of the boundary shape prior and the contour representation, our method is more efficient, more accurate and less prone to inter-operator variability than the SOTA AID methods for organ segmentation from volumetric medical images. The good shape learning ability and flexible boundary adjustment function make it suitable for fast annotation of organ structures with regular shape.
Appendix
Available only for authorised users
Literature
2.
go back to reference Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th international conference on machine learning. Sydney, Australia Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th international conference on machine learning. Sydney, Australia
3.
go back to reference Finn C, Xu K, Levine S (2018) Probabilistic model-agnostic meta-learning. In: Proceedings of the 32nd conference on neural information processing systems (NIPS). Montreal, Canada Finn C, Xu K, Levine S (2018) Probabilistic model-agnostic meta-learning. In: Proceedings of the 32nd conference on neural information processing systems (NIPS). Montreal, Canada
4.
go back to reference Liyan S, Chenxin L, Xinghao D, Yue H, Guisheng W, Yizhou Y (2020) Few-shot medical image segmentation using a global correlation network with discriminative embedding. arXiv. arXiv (USA). pp 10 Liyan S, Chenxin L, Xinghao D, Yue H, Guisheng W, Yizhou Y (2020) Few-shot medical image segmentation using a global correlation network with discriminative embedding. arXiv. arXiv (USA). pp 10
5.
go back to reference Bai W, Oktay O, Sinclair M, Suzuki H, Rajchl M, Tarroni G, Glocker B, King A, Matthews PM, Rueckert D (2017) Semi-supervised learning for network-based cardiac MR image segmentation. Springer, ChamCrossRef Bai W, Oktay O, Sinclair M, Suzuki H, Rajchl M, Tarroni G, Glocker B, King A, Matthews PM, Rueckert D (2017) Semi-supervised learning for network-based cardiac MR image segmentation. Springer, ChamCrossRef
6.
go back to reference Li S, Zhang C, He X (2020) Shape-aware semi-supervised 3D semantic segmentation for medical images. In: Medical image computing and computer assisted intervention – MICCAI 2020; Lecture Notes in Computer Science. pp 552–561 Li S, Zhang C, He X (2020) Shape-aware semi-supervised 3D semantic segmentation for medical images. In: Medical image computing and computer assisted intervention – MICCAI 2020; Lecture Notes in Computer Science. pp 552–561
7.
go back to reference Li X, Yu L, Chen H, Fu CW, Xing L, Heng PA (2020) Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Trans Neural Netw Learn Syst 32(2):523–534CrossRef Li X, Yu L, Chen H, Fu CW, Xing L, Heng PA (2020) Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Trans Neural Netw Learn Syst 32(2):523–534CrossRef
8.
go back to reference Hu X, Zeng D, Xu X, Shi Y (2021) Semi-supervised contrastive learning for label-efficient medical image segmentation. In: Proceedings of the medical image computing and computer assisted intervention – MICCAI 2021. Cham. pp 481–490 Hu X, Zeng D, Xu X, Shi Y (2021) Semi-supervised contrastive learning for label-efficient medical image segmentation. In: Proceedings of the medical image computing and computer assisted intervention – MICCAI 2021. Cham. pp 481–490
9.
go back to reference Lin D, Dai J, Jia J, He K, Sun J (2016) ScribbleSup: scribble-supervised convolutional networks for semantic segmentation. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), 27–30 June 2016. pp 3159–3167 Lin D, Dai J, Jia J, He K, Sun J (2016) ScribbleSup: scribble-supervised convolutional networks for semantic segmentation. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), 27–30 June 2016. pp 3159–3167
12.
14.
15.
go back to reference Poon K, Hamarneh G, Abugharbieh R (2007) Segmentation of complex objects with non-spherical topologies from volumetric medical images using 3D livewire. In: SPIE, Vol. 6512 Poon K, Hamarneh G, Abugharbieh R (2007) Segmentation of complex objects with non-spherical topologies from volumetric medical images using 3D livewire. In: SPIE, Vol. 6512
16.
go back to reference Institute, S.C.A.I. "Seg3D" Volumetric image segmentation and visualization. Scientific computing and imaging institute (SCI). Available online: http://www.seg3d.org Institute, S.C.A.I. "Seg3D" Volumetric image segmentation and visualization. Scientific computing and imaging institute (SCI). Available online: http://​www.​seg3d.​org
17.
go back to reference Top A, Hamarneh G, Abugharbieh R (2010) Spotlight: automated confidence-based user guidance for increasing efficiency in interactive 3D image segmentation. In: Proceedings of the international MICCAI workshop on medical computer vision Top A, Hamarneh G, Abugharbieh R (2010) Spotlight: automated confidence-based user guidance for increasing efficiency in interactive 3D image segmentation. In: Proceedings of the international MICCAI workshop on medical computer vision
18.
go back to reference Sowell RT, Liu L, Ju T, Grimm CM (2009) VolumeViewer: an interactive tool for fitting surfaces to volume data. In: Proceedings of the international conference on computer graphics and interactive techniques, SIGGRAPH 2008, Los Angeles, California, USA, August 11–15, 2008 Sowell RT, Liu L, Ju T, Grimm CM (2009) VolumeViewer: an interactive tool for fitting surfaces to volume data. In: Proceedings of the international conference on computer graphics and interactive techniques, SIGGRAPH 2008, Los Angeles, California, USA, August 11–15, 2008
24.
go back to reference Sakinis T, Milletari F, Roth H, Korfiatis P, Kostandy P, Philbrick K, Akkus Z, Xu Z, Xu D, Erickson BJ (2018) Interactive segmentation of medical images through fully convolutional neural networks. arXiv preprint arXiv:1903.08205 Sakinis T, Milletari F, Roth H, Korfiatis P, Kostandy P, Philbrick K, Akkus Z, Xu Z, Xu D, Erickson BJ (2018) Interactive segmentation of medical images through fully convolutional neural networks. arXiv preprint arXiv:​1903.​08205
26.
go back to reference Ling H, Gao J, Kar A, Chen W, Fidler S (2019) Fast interactive object annotation with curve-GCN. In: Proceedings of the 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR) Ling H, Gao J, Kar A, Chen W, Fidler S (2019) Fast interactive object annotation with curve-GCN. In: Proceedings of the 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR)
27.
go back to reference Liu ZC, Liew JH, Chen XY, Feng JS (2021) Ieee. DANCE: a deep attentive contour model for efficient instance segmentation. In: Proceedings of the IEEE winter conference on applications of computer vision (WACV), Electr Network, Jan 05–09, 2021. pp 345–354 Liu ZC, Liew JH, Chen XY, Feng JS (2021) Ieee. DANCE: a deep attentive contour model for efficient instance segmentation. In: Proceedings of the IEEE winter conference on applications of computer vision (WACV), Electr Network, Jan 05–09, 2021. pp 345–354
28.
go back to reference Peng S, Jiang W, Pi H, Li X, Bao H, Zhou X (2020) Deep snake for real-time instance segmentation. 8530–8539 Peng S, Jiang W, Pi H, Li X, Bao H, Zhou X (2020) Deep snake for real-time instance segmentation. 8530–8539
30.
go back to reference Yu F, Wang D, Shelhamer E, Darrell T (2018) Deep layer aggregation. In: Proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2403–2412 Yu F, Wang D, Shelhamer E, Darrell T (2018) Deep layer aggregation. In: Proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2403–2412
31.
go back to reference Girshick R (2015) Fast r-cnn. In: Proceedings of the proceedings of the IEEE international conference on computer vision. pp 1440–1448 Girshick R (2015) Fast r-cnn. In: Proceedings of the proceedings of the IEEE international conference on computer vision. pp 1440–1448
32.
go back to reference Sederberg TW, Parry SR (1986) Free-form deformation of solid geometric models. In Proceedings of the proceedings of the 13th annual conference on Computer graphics and interactive techniques. pp 151–160 Sederberg TW, Parry SR (1986) Free-form deformation of solid geometric models. In Proceedings of the proceedings of the 13th annual conference on Computer graphics and interactive techniques. pp 151–160
33.
go back to reference Menzel S, Olhofer M, Sendhoff B (2006) Direct manipulation of free form deformation in evolutionary design optimisation. Parallel problem solving from nature-PPSN IX. Springer, Cham, pp 352–361CrossRef Menzel S, Olhofer M, Sendhoff B (2006) Direct manipulation of free form deformation in evolutionary design optimisation. Parallel problem solving from nature-PPSN IX. Springer, Cham, pp 352–361CrossRef
34.
go back to reference Lorensen WE, Cline HE (1987) Marching cubes: A high resolution 3D surface construction algorithm. ACM Siggraph Comput Graphics 21:163–169CrossRef Lorensen WE, Cline HE (1987) Marching cubes: A high resolution 3D surface construction algorithm. ACM Siggraph Comput Graphics 21:163–169CrossRef
37.
go back to reference Jaeger S, Karargyris A, Candemir S, Folio L, Siegelman J, Callaghan F, Xue Z, Palaniappan K, Singh RK, Antani S (2013) Automatic tuberculosis screening using chest radiographs. IEEE Trans Med Imaging 33:233–245CrossRef Jaeger S, Karargyris A, Candemir S, Folio L, Siegelman J, Callaghan F, Xue Z, Palaniappan K, Singh RK, Antani S (2013) Automatic tuberculosis screening using chest radiographs. IEEE Trans Med Imaging 33:233–245CrossRef
38.
go back to reference Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Xue Z, Karargyris A, Antani S, Thoma G, McDonald CJ (2013) Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging 33:577–590CrossRef Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Xue Z, Karargyris A, Antani S, Thoma G, McDonald CJ (2013) Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging 33:577–590CrossRef
39.
go back to reference Antonelli M, Reinke A, Bakas S, Farahani K, AnnetteKopp-Schneider, Landman BA, Litjens G, Menze B, Ronneberger O, Summers RM, van Ginneken B, Bilello M, Bilic P, Christ PF, Do RKG, Gollub MJ, Heckers SH, Huisman H, Jarnagin WR, McHugo MK, Napel S, Goli Pernicka JS, Rhode K, Tobon-Gomez C, Vorontsov E, Huisman H, Meakin JA, Ourselin S, Wiesenfarth M, Arbelaez P, Bae B, Chen S, Daza L Feng J, He B, Isensee F, Ji Y, Jia F, Kim N, Kim I, Merhof D, Pai A, Park B, Perslev M, Rezaiifar R, Rippel O, Sarasua I, Shen W, Son J, Wachinger C, Wang L, Wang Y, Xia Y, Xu D, Xu Z, Zheng Y, Simpson AL, Maier-Hein L, Cardoso MJ (2021) The medical segmentation decathlon. arXiv:2106.05735 Antonelli M, Reinke A, Bakas S, Farahani K, AnnetteKopp-Schneider, Landman BA, Litjens G, Menze B, Ronneberger O, Summers RM, van Ginneken B, Bilello M, Bilic P, Christ PF, Do RKG, Gollub MJ, Heckers SH, Huisman H, Jarnagin WR, McHugo MK, Napel S, Goli Pernicka JS, Rhode K, Tobon-Gomez C, Vorontsov E, Huisman H, Meakin JA, Ourselin S, Wiesenfarth M, Arbelaez P, Bae B, Chen S, Daza L Feng J, He B, Isensee F, Ji Y, Jia F, Kim N, Kim I, Merhof D, Pai A, Park B, Perslev M, Rezaiifar R, Rippel O, Sarasua I, Shen W, Son J, Wachinger C, Wang L, Wang Y, Xia Y, Xu D, Xu Z, Zheng Y, Simpson AL, Maier-Hein L, Cardoso MJ (2021) The medical segmentation decathlon. arXiv:​2106.​05735
40.
go back to reference Kervadec H, Bouchtiba J, Desrosiers C, Granger E. Dolz J, Ayed IB (2019) Boundary loss for highly unbalanced segmentation. In Proceedings of the International conference on medical imaging with deep learning. pp 285–296 Kervadec H, Bouchtiba J, Desrosiers C, Granger E. Dolz J, Ayed IB (2019) Boundary loss for highly unbalanced segmentation. In Proceedings of the International conference on medical imaging with deep learning. pp 285–296
Metadata
Title
Efficient contour-based annotation by iterative deep learning for organ segmentation from volumetric medical images
Authors
Mingrui Zhuang
Zhonghua Chen
Hongkai Wang
Hong Tang
Jiang He
Bobo Qin
Yuxin Yang
Xiaoxian Jin
Mengzhu Yu
Baitao Jin
Taijing Li
Lauri Kettunen
Publication date
01-09-2022
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 2/2023
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-022-02730-z

Other articles of this Issue 2/2023

International Journal of Computer Assisted Radiology and Surgery 2/2023 Go to the issue