Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-03-2017 | Original Article

Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets

Authors: Peijun Hu, Fa Wu, Jialin Peng, Yuanyuan Bao, Feng Chen, Dexing Kong

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 3/2017

Abstract

Purpose

Multi-organ segmentation from CT images is an essential step for computer-aided diagnosis and surgery planning. However, manual delineation of the organs by radiologists is tedious, time-consuming and poorly reproducible. Therefore, we propose a fully automatic method for the segmentation of multiple organs from three-dimensional abdominal CT images.

Methods

The proposed method employs deep fully convolutional neural networks (CNNs) for organ detection and segmentation, which is further refined by a time-implicit multi-phase evolution method. Firstly, a 3D CNN is trained to automatically localize and delineate the organs of interest with a probability prediction map. The learned probability map provides both subject-specific spatial priors and initialization for subsequent fine segmentation. Then, for the refinement of the multi-organ segmentation, image intensity models, probability priors as well as a disjoint region constraint are incorporated into an unified energy functional. Finally, a novel time-implicit multi-phase level-set algorithm is utilized to efficiently optimize the proposed energy functional model.

Results

Our method has been evaluated on 140 abdominal CT scans for the segmentation of four organs (liver, spleen and both kidneys). With respect to the ground truth, average Dice overlap ratios for the liver, spleen and both kidneys are 96.0, 94.2 and 95.4%, respectively, and average symmetric surface distance is less than 1.3 mm for all the segmented organs. The computation time for a CT volume is 125 s in average. The achieved accuracy compares well to state-of-the-art methods with much higher efficiency.

Conclusion

A fully automatic method for multi-organ segmentation from abdominal CT images was developed and evaluated. The results demonstrated its potential in clinical usage with high effectiveness, robustness and efficiency.

https://code.google.com/p/cuda-convnet/.

http://www.visceral.eu/benchmarks/anatomy3-open/.

Brosch T, Tang LY, Yoo Y, Li DK, Traboulsee A, Tam R (2016) Deep 3d convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans Med Imaging 35(5):1229–1239CrossRefPubMed

Cha KH, Hadjiiski L, Samala RK, Chan HP, Caoili EM, Cohan RH (2016) Urinary bladder segmentation in ct urography using deep-learning convolutional neural network and level sets. Med Phys 43(4):1882–1896CrossRefPubMed

Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2015) Semantic image segmentation with deep convolutional nets and fully connected crfs. In: International conference on learning representations (ICLR)

Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3d u-net: learning dense volumetric segmentation from sparse annotation. In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W (eds) International conference on medical image computing and computer-assisted intervention, (MICCAI), Springer, pp 424–432

Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in neural information processing systems, pp 2843–2851

Criminisi A, Robertson D, Konukoglu E, Shotton J, Pathak S, White S, Siddiqui K (2013) Regression forests for efficient anatomy detection and localization in computed tomography scans. Med Image Anal 17(8):1293–1303CrossRefPubMed

Dou Q, Chen H, Jin Y, Yu L, Qin J, Heng PA (2016) 3d deeply supervised network for automatic liver segmentation from ct volumes. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 149–157

Gauriau R, Cuingnet R, Lesage D, Bloch I (2015) Multi-organ localization with cascaded global-to-local regression and shape prior. Med Image Anal 23(1):70–83CrossRefPubMed

He B, Huang C, Jia F (2015) Fully automatic multi-organ segmentation based on multi-boost learning and statistical shape model search. VISCERAL@ ISBI 2015 VISCERAL Anatomy3 Organ Segmentation Challenge, p 18

10.

He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

11.

Heimann T, van Ginneken B, Styner M, Arzhaeva Y, Aurich V, Bauer C, Beck A, Becker C, Beichel R, Bekes G, Bello F, Binnig G, Bischof H, Bornik A, Cashman P, Chi Y, Cordova A, Dawant B, Fidrich M, Furst J, Furukawa D, Grenacher L, Hornegger J, Kainmuller D, Kitney R, Kobatake H, Lamecker H, Lange T, Lee J, Lennon B, Li R, Li S, Meinzer HP, Nemeth G, Raicu D, Rau AM, vanRikxoort E, Rousson M, Rusko L, Saddi K, Schmidt G, Seghers D, Shimizu A, Slagmolen P, Sorantin E, Soza G, Susomboon R, Waite J, Wimmer A, Wolf I (2009) Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Trans Med Imaging 28(8):1251–1265

12.

Hu P, Wu F, Peng J, Liang P, Kong D (2016) Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution. Phys Med Biol 61:8676–8698. doi:10.1088/1361-6560/61/24/8676

13.

Jawarneh MS, Mandava R, Ramachandram D, Shuaib IL (2010) Automatic initialization of contour for level set algorithms guided by integration of multiple views to segment abdominal ct scans. In: 2010 Second international conference on computational intelligence, modelling and simulation (CIMSiM). IEEE, pp 315–320

14.

Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

15.

Lai M (2015) Deep learning for medical image segmentation. arXiv preprint arXiv:1505.02000

16.

Lee CY, Xie S, Gallagher P, Zhang Z, Tu Z (2015) Deeply-supervised nets. In: AISTATS, vol 2, p 6

17.

Linguraru MG, Pura JA, Pamulapati V, Summers RM (2012) Statistical 4d graphs for multi-organ abdominal segmentation from multiphase ct. Med Image Anal 16(4):904–914CrossRefPubMedPubMedCentral

18.

Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

19.

Lu F, Wu F, Hu P, Peng Z, Kong D. Automatic 3d liver location and segmentation via convolutional neural networks and graph cut. Int J Comput Assist Radiol Surg. doi:10.1007/s11548-016-1467-3

20.

Milletari F, Navab N, Ahmadi SA (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. arXiv preprint arXiv:1606.04797

21.

Okada T, Linguraru MG, Hori M, Summers RM, Tomiyama N, Sato Y (2015) Abdominal multi-organ segmentation from ct images using conditional shape-location and unsupervised intensity priors. Med Image Anal 26(1):1–18CrossRefPubMedPubMedCentral

22.

Peng J, Dong F, Chen Y, Kong D (2014) A region-appearance-based adaptive variational model for 3d liver segmentation. Med Phys 41(4):043,502CrossRef

23.

Peng J, Hu P, Lu F, Peng Z, Kong D, Zhang H (2015) 3d liver segmentation using multiple region appearances and graph cuts. Med Phys 42(12):6840–6852CrossRefPubMed

24.

Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell 12(7):629–639CrossRef

25.

Potts RB (1952) Some generalized order-disorder transformations. In: Mathematical proceedings of the Cambridge philosophical society, vol 48. Cambridge University Press, pp 106–109

26.

Rajchl M, Baxter JS, Bae E, Tai XC, Fenster A, Peters TM, Yuan J (2015) Variational time-implicit multiphase level-sets. In: Energy minimization methods in computer vision and pattern recognition. Springer, pp 278–291

27.

Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

28.

Roth HR, Lu L, Farag A, Shin HC, Liu J, Turkbey EB, Summers RM (2015) Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation. In: Medical image computing and computer-assisted intervention. Springer, pp 556–564

29.

Saito A, Nawano S, Shimizu A (2016) Joint optimization of segmentation and shape prior from level-set-based statistical shape model, and its application to the automated segmentation of abdominal organs. Med Image Anal 28:46–65CrossRefPubMed

30.

Selver MA (2014) Segmentation of abdominal organs from ct using a multi-level, hierarchical neural network strategy. Comput Methods Programs Biomed 113(3):830–852CrossRefPubMed

31.

Tong T, Wolz R, Wang Z, Gao Q, Misawa K, Fujiwara M, Mori K, Hajnal JV, Rueckert D (2015) Discriminative dictionary learning for abdominal multi-organ segmentation. Med Image Anal 23(1):92–104CrossRefPubMed

32.

Vese LA, Chan TF (2002) A multiphase level set framework for image segmentation using the mumford and shah model. Int J Comput Vis 50(3):271–293CrossRef

33.

Wang C, Smedby O (2014) Automatic multi-organ segmentation using fast model based level set method and hierarchical shape priors. Proc VISCERAL Chall ISBI 1194:25–31

34.

Wolz R, Chu C, Misawa K, Fujiwara M, Mori K, Rueckert D (2013) Automated abdominal multi-organ segmentation with subject-specific atlas generation. IEEE Trans Med Imaging 32(9):1723–1730CrossRefPubMed

35.

Xu Z, Burke RP, Lee CP, Baucom RB, Poulose BK, Abramson RG, Landman BA (2015) Efficient multi-atlas abdominal segmentation on clinically acquired ct with simple context learning. Med Image Anal 24(1):18–27CrossRefPubMedPubMedCentral

36.

Yuan J, Bae E, Tai XC, Boykov Y (2010) A continuous max-flow approach to potts model. In: European conference on computer vision. Springer, pp 379–392

37.

Yuan J, Ukwatta E, Tai X, Fenster A, Schnoerr C (2012) A fast global optimization-based approach to evolving contours with generic shape prior. University of California, Los Angeles. Technical report CAM 12, vol 38

38.

Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, Shen D (2015) Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108:214–224CrossRefPubMedPubMedCentral

Title: Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets
Authors: Peijun Hu
Fa Wu
Jialin Peng
Yuanyuan Bao
Feng Chen
Dexing Kong
Publication date: 01-03-2017
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 3/2017
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-016-1501-5

At a glance: The STEP trials

Springer Medicine

Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets

Abstract

Purpose

Methods

Results

Conclusion

At a glance: The STEP trials

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 3/2017

A new method for the automatic retrieval of medical cases based on the RadLex ontology

Semiautomatic classification of acetabular shape from three-dimensional ultrasound for diagnosis of infant hip dysplasia using geometric features

Erratum to: GRAPE: a graphical pipeline environment for image analysis in adaptive magnetic resonance imaging

IBIS: an OR ready open-source platform for image-guided neurosurgery

MITK-OpenIGTLink for combining open-source toolkits in real-time computer-assisted interventions

Haustral loop extraction for CT colonography using geodesics