Skip to main content
Top
Published in: Journal of Digital Imaging 4/2019

Open Access 01-08-2019

Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges

Authors: Mohammad Hesam Hesamian, Wenjing Jia, Xiangjian He, Paul Kennedy

Published in: Journal of Imaging Informatics in Medicine | Issue 4/2019

Login to get access

Abstract

Deep learning-based image segmentation is by now firmly established as a robust tool in image segmentation. It has been widely used to separate homogeneous areas as the first and critical component of diagnosis and treatment pipeline. In this article, we present a critical appraisal of popular methods that have employed deep-learning techniques for medical image segmentation. Moreover, we summarize the most common challenges incurred and suggest possible solutions.
Literature
1.
go back to reference Alakwaa W, Nassef M, Badr A: Lung cancer detection and classification with 3D convolutional neural network (3D-CNN). Lung Cancer 8(8): 409, 2017 Alakwaa W, Nassef M, Badr A: Lung cancer detection and classification with 3D convolutional neural network (3D-CNN). Lung Cancer 8(8): 409, 2017
2.
go back to reference Anirudh R, Thiagarajan JJ, Bremer T, Kim H: Lung nodule detection using 3D convolutional neural networks trained on weakly labeled data.. In: Medical Imaging 2016: Computer-Aided Diagnosis, vol 9785, 2016, p 978532. International Society for Optics and Photonics Anirudh R, Thiagarajan JJ, Bremer T, Kim H: Lung nodule detection using 3D convolutional neural networks trained on weakly labeled data.. In: Medical Imaging 2016: Computer-Aided Diagnosis, vol 9785, 2016, p 978532. International Society for Optics and Photonics
3.
go back to reference Armato SG III, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA, et al: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 38(2): 915–931, 2011CrossRefPubMedPubMedCentral Armato SG III, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA, et al: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 38(2): 915–931, 2011CrossRefPubMedPubMedCentral
4.
go back to reference Bar Y, Diamant I, Wolf L, Greenspan H: Deep learning with non-medical training used for chest pathology identification.. In: Medical Imaging 2015: Computer-Aided Diagnosis, vol 9414, 2015, p 94140v. International Society for Optics and Photonics Bar Y, Diamant I, Wolf L, Greenspan H: Deep learning with non-medical training used for chest pathology identification.. In: Medical Imaging 2015: Computer-Aided Diagnosis, vol 9414, 2015, p 94140v. International Society for Optics and Photonics
5.
go back to reference Baumgartner CF, Koch LM, Pollefeys M, Konukoglu E: An exploration of 2D and 3D deep learning techniques for cardiac mr image segmentation.. In: International Workshop on Statistical Atlases and Computational Models of the Heart. Springer, 2017, pp 111–119 Baumgartner CF, Koch LM, Pollefeys M, Konukoglu E: An exploration of 2D and 3D deep learning techniques for cardiac mr image segmentation.. In: International Workshop on Statistical Atlases and Computational Models of the Heart. Springer, 2017, pp 111–119
6.
go back to reference Bergamo A, Torresani L, Fitzgibbon AW: Picodes: Learning a compact code for novel-category recognition.. In: Advances in Neural Information Processing Systems, 2011, pp 2088–2096 Bergamo A, Torresani L, Fitzgibbon AW: Picodes: Learning a compact code for novel-category recognition.. In: Advances in Neural Information Processing Systems, 2011, pp 2088–2096
7.
go back to reference Cai J, Lu L, Xie Y, Xing F, Yang L (2017) Improving deep pancreas segmentation in CT and MRI images via recurrent neural contextual learning and direct loss function, arXiv:1707.04912 Cai J, Lu L, Xie Y, Xing F, Yang L (2017) Improving deep pancreas segmentation in CT and MRI images via recurrent neural contextual learning and direct loss function, arXiv:1707.​04912
8.
go back to reference Chen H, Dou Q, Yu L, Qin J, Heng PA: Voxresnet: deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage 170: 446–455, 2017CrossRefPubMed Chen H, Dou Q, Yu L, Qin J, Heng PA: Voxresnet: deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage 170: 446–455, 2017CrossRefPubMed
9.
go back to reference Chen H, Ni D, Qin J, Li S, Yang X, Wang T, Heng PA: Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE J Biomed Health Inform 19(5): 1627–1636, 2015CrossRefPubMed Chen H, Ni D, Qin J, Li S, Yang X, Wang T, Heng PA: Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE J Biomed Health Inform 19(5): 1627–1636, 2015CrossRefPubMed
10.
go back to reference Chen H, Qi X, Cheng JZ, Heng PA, et al.: Deep contextual networks for neuronal structure segmentation.. In: AAAI, 2016, pp 1167–1173 Chen H, Qi X, Cheng JZ, Heng PA, et al.: Deep contextual networks for neuronal structure segmentation.. In: AAAI, 2016, pp 1167–1173
11.
go back to reference Chen H, Qi X, Yu L, Heng PA: DCAN: deep contour-aware networks for accurate gland segmentation.. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2016, pp 2487–2496 Chen H, Qi X, Yu L, Heng PA: DCAN: deep contour-aware networks for accurate gland segmentation.. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2016, pp 2487–2496
12.
go back to reference Chen J, Yang L, Zhang Y, Alber M, Chen DZ: Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation.. In: Advances in Neural Information Processing Systems, 2016, pp 3036–3044 Chen J, Yang L, Zhang Y, Alber M, Chen DZ: Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation.. In: Advances in Neural Information Processing Systems, 2016, pp 3036–3044
13.
go back to reference Cheng D, Liu M: Combining convolutional and recurrent neural networks for Alzheimer’s disease diagnosis using pet images.. In: 2017 IEEE International Conference on Imaging Systems And Techniques (IST). IEEE, 2017, pp 1–5 Cheng D, Liu M: Combining convolutional and recurrent neural networks for Alzheimer’s disease diagnosis using pet images.. In: 2017 IEEE International Conference on Imaging Systems And Techniques (IST). IEEE, 2017, pp 1–5
14.
go back to reference Cheng JZ, Ni D, Chou YH, Qin J, Tiu CM, Chang YC, Huang CS, Shen D, Chen CM: Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 6: 24454, 2016CrossRefPubMedPubMedCentral Cheng JZ, Ni D, Chou YH, Qin J, Tiu CM, Chang YC, Huang CS, Shen D, Chen CM: Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 6: 24454, 2016CrossRefPubMedPubMedCentral
15.
go back to reference Christ PF, Elshaer MEA, Ettlinger F, Tatavarty S, Bickel M, Bilic P, Rempfler M, Armbruster M, Hofmann F, D’Anastasi M, et al: Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 415–423 Christ PF, Elshaer MEA, Ettlinger F, Tatavarty S, Bickel M, Bilic P, Rempfler M, Armbruster M, Hofmann F, D’Anastasi M, et al: Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 415–423
16.
go back to reference Christ PF, Ettlinger F, Grün F, Elshaera MEA, Lipkova J, Schlecht S, Ahmaddy F, Tatavarty S, Bickel M, Bilic P, et al (2017) Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. arXiv:1702.05970 Christ PF, Ettlinger F, Grün F, Elshaera MEA, Lipkova J, Schlecht S, Ahmaddy F, Tatavarty S, Bickel M, Bilic P, et al (2017) Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. arXiv:1702.​05970
17.
go back to reference Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O: 3F U-Net: learning dense volumetric segmentation from sparse annotation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 424–432 Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O: 3F U-Net: learning dense volumetric segmentation from sparse annotation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 424–432
18.
go back to reference Ciresan D, Giusti A, Gambardella LM, Schmidhuber J: Deep neural networks segment neuronal membranes in electron microscopy images.. In: Advances in Neural Information Processing Systems, 2012, pp 2843–2851 Ciresan D, Giusti A, Gambardella LM, Schmidhuber J: Deep neural networks segment neuronal membranes in electron microscopy images.. In: Advances in Neural Information Processing Systems, 2012, pp 2843–2851
19.
go back to reference Codella NC, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H, et al: Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC).. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018, pp 168–172 Codella NC, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H, et al: Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC).. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018, pp 168–172
20.
go back to reference Commandeur F, Goeller M, Betancur J, Cadet S, Doris M, Chen X, Berman DS, Slomka PJ, Tamarappoo BK, Dey D: Deep learning for quantification of epicardial and thoracic adipose tissue from non-contrast CT. IEEE Trans Med Imaging 37(8): 1835–1846, 2018CrossRefPubMedPubMedCentral Commandeur F, Goeller M, Betancur J, Cadet S, Doris M, Chen X, Berman DS, Slomka PJ, Tamarappoo BK, Dey D: Deep learning for quantification of epicardial and thoracic adipose tissue from non-contrast CT. IEEE Trans Med Imaging 37(8): 1835–1846, 2018CrossRefPubMedPubMedCentral
21.
go back to reference Dhungel N, Carneiro G, Bradley AP: Deep learning and structured prediction for the segmentation of mass in mammograms.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, pp 605–612 Dhungel N, Carneiro G, Bradley AP: Deep learning and structured prediction for the segmentation of mass in mammograms.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, pp 605–612
22.
go back to reference Dou Q, Chen H, Jin Y, Yu L, Qin J, Heng PA: 3D deeply supervised network for automatic liver segmentation from CT volumes.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 149–157 Dou Q, Chen H, Jin Y, Yu L, Qin J, Heng PA: 3D deeply supervised network for automatic liver segmentation from CT volumes.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 149–157
23.
go back to reference Dou Q, Yu L, Chen H, Jin Y, Yang X, Qin J, Heng PA: 3D deeply supervised network for automated segmentation of volumetric medical images. Med Image Anal 41: 40–54, 2017CrossRefPubMed Dou Q, Yu L, Chen H, Jin Y, Yang X, Qin J, Heng PA: 3D deeply supervised network for automated segmentation of volumetric medical images. Med Image Anal 41: 40–54, 2017CrossRefPubMed
24.
go back to reference Fakoor R, Ladhak F, Nazi A, Huber M: Using deep learning to enhance cancer diagnosis and classification.. In: Proceedings of the International Conference on Machine Learning, vol 28, 2013 Fakoor R, Ladhak F, Nazi A, Huber M: Using deep learning to enhance cancer diagnosis and classification.. In: Proceedings of the International Conference on Machine Learning, vol 28, 2013
25.
go back to reference Feng X, Yang J, Laine AF, Angelini ED: Discriminative localization in CNNS for weakly-supervised segmentation of pulmonary nodules.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2017, pp 568–576 Feng X, Yang J, Laine AF, Angelini ED: Discriminative localization in CNNS for weakly-supervised segmentation of pulmonary nodules.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2017, pp 568–576
27.
go back to reference Gibson E, Giganti F, Hu Y, Bonmati E, Bandula S, Gurusamy K, Davidson BR, Pereira SP, Clarkson MJ, Barratt DC: Towards image-guided pancreas and biliary endoscopy: automatic multi-organ segmentation on abdominal CT with dense dilated networks.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2017, pp 728–736 Gibson E, Giganti F, Hu Y, Bonmati E, Bandula S, Gurusamy K, Davidson BR, Pereira SP, Clarkson MJ, Barratt DC: Towards image-guided pancreas and biliary endoscopy: automatic multi-organ segmentation on abdominal CT with dense dilated networks.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2017, pp 728–736
28.
go back to reference Gibson E, Robu MR, Thompson S, Edwards PE, Schneider C, Gurusamy K, Davidson B, Hawkes DJ, Barratt DC, Clarkson MJ: Deep residual networks for automatic segmentation of laparoscopic videos of the liver.. In: Medical Imaging 2017: Image-Guided Procedures, Robotic Interventions, and Modeling, vol 10135, 2017, p 101351m. International society for optics and photonics Gibson E, Robu MR, Thompson S, Edwards PE, Schneider C, Gurusamy K, Davidson B, Hawkes DJ, Barratt DC, Clarkson MJ: Deep residual networks for automatic segmentation of laparoscopic videos of the liver.. In: Medical Imaging 2017: Image-Guided Procedures, Robotic Interventions, and Modeling, vol 10135, 2017, p 101351m. International society for optics and photonics
29.
go back to reference Girshick R, Donahue J, Darrell T, Malik J: Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 38(1): 142–158, 2016CrossRefPubMed Girshick R, Donahue J, Darrell T, Malik J: Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 38(1): 142–158, 2016CrossRefPubMed
30.
go back to reference Golan R, Jacob C, Denzinger J: Lung nodule detection in CT images using deep convolutional neural networks.. In: 2016 International Joint Conference on Neural Networks (IJCNN). IEEE, 2016, pp 243–250 Golan R, Jacob C, Denzinger J: Lung nodule detection in CT images using deep convolutional neural networks.. In: 2016 International Joint Conference on Neural Networks (IJCNN). IEEE, 2016, pp 243–250
31.
go back to reference Gordienko Y, Gang P, Hui J, Zeng W, Kochura Y, Alienin O, Rokovyi O, Stirenko S: Deep learning with lung segmentation and bone shadow exclusion techniques for chest X-ray analysis of lung cancer.. In: International Conference on Theory and Applications of Fuzzy Systems and Soft Computing. Springer, 2018, pp 638–647 Gordienko Y, Gang P, Hui J, Zeng W, Kochura Y, Alienin O, Rokovyi O, Stirenko S: Deep learning with lung segmentation and bone shadow exclusion techniques for chest X-ray analysis of lung cancer.. In: International Conference on Theory and Applications of Fuzzy Systems and Soft Computing. Springer, 2018, pp 638–647
32.
go back to reference Hamidian S, Sahiner B, Petrick N, Pezeshk A: 3D convolutional neural network for automatic detection of lung nodules in chest CT.. In: Medical Imaging 2017: Computer-Aided Diagnosis, vol 10134, 2017, p 1013409. International Society for Optics and Photonics Hamidian S, Sahiner B, Petrick N, Pezeshk A: 3D convolutional neural network for automatic detection of lung nodules in chest CT.. In: Medical Imaging 2017: Computer-Aided Diagnosis, vol 10134, 2017, p 1013409. International Society for Optics and Photonics
33.
go back to reference He K, Zhang X, Ren S, Sun J: Deep residual learning for image recognition.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp 770–778 He K, Zhang X, Ren S, Sun J: Deep residual learning for image recognition.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp 770–778
34.
go back to reference He K, Zhang X, Ren S, Sun J: Identity mappings in deep residual networks.. In: European Conference on Computer Vision. Springer, 2016, pp 630–645 He K, Zhang X, Ren S, Sun J: Identity mappings in deep residual networks.. In: European Conference on Computer Vision. Springer, 2016, pp 630–645
35.
36.
go back to reference Hoo-Chang S, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5): 1285, 2016CrossRef Hoo-Chang S, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5): 1285, 2016CrossRef
37.
go back to reference Hu P, Wu F, Peng J, Bao Y, Chen F, Kong D: Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets. Int J Comput Assist Radiol Surg 12(3): 399–411, 2017CrossRefPubMed Hu P, Wu F, Peng J, Bao Y, Chen F, Kong D: Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets. Int J Comput Assist Radiol Surg 12(3): 399–411, 2017CrossRefPubMed
38.
go back to reference Huyskens DP, Maingon P, Vanuytsel L, Remouchamps V, Roques T, Dubray B, Haas B, Kunz P, Coradi T, Bühlman R, et al: A qualitative and a quantitative analysis of an auto-segmentation module for prostate cancer. Acta Radiol Oncol 90(3): 337–345, 2009CrossRef Huyskens DP, Maingon P, Vanuytsel L, Remouchamps V, Roques T, Dubray B, Haas B, Kunz P, Coradi T, Bühlman R, et al: A qualitative and a quantitative analysis of an auto-segmentation module for prostate cancer. Acta Radiol Oncol 90(3): 337–345, 2009CrossRef
39.
go back to reference Hwang S, Kim HE: Self-transfer learning for weakly supervised lesion localization.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 239–246 Hwang S, Kim HE: Self-transfer learning for weakly supervised lesion localization.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 239–246
40.
go back to reference Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167 Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.​03167
41.
go back to reference Kamnitsas K, Chen L, Ledig C, Rueckert D, Glocker B: Multi-scale 3D convolutional neural networks for lesion segmentation in brain mri. Ischemic Stroke Lesion Segmentation 13: 46, 2015 Kamnitsas K, Chen L, Ledig C, Rueckert D, Glocker B: Multi-scale 3D convolutional neural networks for lesion segmentation in brain mri. Ischemic Stroke Lesion Segmentation 13: 46, 2015
42.
go back to reference Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36: 61–78, 2017CrossRefPubMed Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36: 61–78, 2017CrossRefPubMed
43.
go back to reference Kawahara J, BenTaieb A, Hamarneh G: Deep features to classify skin lesions.. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE, 2016, pp 1397–1400 Kawahara J, BenTaieb A, Hamarneh G: Deep features to classify skin lesions.. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE, 2016, pp 1397–1400
44.
go back to reference Kleesiek J, Urban G, Hubert A, Schwarz D, Maier-Hein K, Bendszus M, Biller A: Deep MRI brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage 129: 460–469, 2016CrossRefPubMed Kleesiek J, Urban G, Hubert A, Schwarz D, Maier-Hein K, Bendszus M, Biller A: Deep MRI brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage 129: 460–469, 2016CrossRefPubMed
45.
46.
go back to reference Kronman A, Joskowicz L: A geometric method for the detection and correction of segmentation leaks of anatomical structures in volumetric medical images. Int J Comput Assist Radiol Surg 11(3): 369–380, 2016CrossRefPubMed Kronman A, Joskowicz L: A geometric method for the detection and correction of segmentation leaks of anatomical structures in volumetric medical images. Int J Comput Assist Radiol Surg 11(3): 369–380, 2016CrossRefPubMed
47.
go back to reference Lee CY, Xie S, Gallagher P, Zhang Z, Tu Z: Deeply-supervised nets.. In: Artificial Intelligence and Statistics, 2015, pp 562–570 Lee CY, Xie S, Gallagher P, Zhang Z, Tu Z: Deeply-supervised nets.. In: Artificial Intelligence and Statistics, 2015, pp 562–570
48.
go back to reference Lewandowski AJ, Augustine D, Lamata P, Davis EF, Lazdam M, Francis J, McCormick K, Wilkinson AR, Singhal A, Lucas A, et al: Preterm heart in adult life: cardiovascular magnetic resonance reveals distinct differences in left ventricular mass, geometry, and function. Circulation 127(2): 197–206, 2013CrossRefPubMed Lewandowski AJ, Augustine D, Lamata P, Davis EF, Lazdam M, Francis J, McCormick K, Wilkinson AR, Singhal A, Lucas A, et al: Preterm heart in adult life: cardiovascular magnetic resonance reveals distinct differences in left ventricular mass, geometry, and function. Circulation 127(2): 197–206, 2013CrossRefPubMed
49.
go back to reference Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI: A survey on deep learning in medical image analysis. Med Image Anal 42: 60–88, 2017CrossRefPubMed Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI: A survey on deep learning in medical image analysis. Med Image Anal 42: 60–88, 2017CrossRefPubMed
50.
go back to reference Long J, Shelhamer E, Darrell T: Fully convolutional networks for semantic segmentation.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp 3431–3440 Long J, Shelhamer E, Darrell T: Fully convolutional networks for semantic segmentation.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp 3431–3440
51.
go back to reference Merkow J, Marsden A, Kriegman D, Tu Z: Dense volume-to-volume vascular boundary detection.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 371–379 Merkow J, Marsden A, Kriegman D, Tu Z: Dense volume-to-volume vascular boundary detection.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 371–379
52.
go back to reference Milletari F, Ahmadi SA, Kroll C, Plate A, Rozanski V, Maiostre J, Levin J, Dietrich O, Ertl-Wagner B, Bötzel K, et al: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound. Comp Vision Image Underst 164: 92–102, 2017CrossRef Milletari F, Ahmadi SA, Kroll C, Plate A, Rozanski V, Maiostre J, Levin J, Dietrich O, Ertl-Wagner B, Bötzel K, et al: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound. Comp Vision Image Underst 164: 92–102, 2017CrossRef
53.
go back to reference Milletari F, Navab N, Ahmadi SA: V-net: fully convolutional neural networks for volumetric medical image segmentation.. In: 2016 Fourth International Conference on 3D Vision (3DV). IEEE, 2016, pp 565–571 Milletari F, Navab N, Ahmadi SA: V-net: fully convolutional neural networks for volumetric medical image segmentation.. In: 2016 Fourth International Conference on 3D Vision (3DV). IEEE, 2016, pp 565–571
54.
go back to reference Moeskops P, Wolterink JM, van der Velden BH, Gilhuijs KG, Leiner T, Viergever MA, Išgum I: Deep learning for multi-task medical image segmentation in multiple modalities.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 478–486 Moeskops P, Wolterink JM, van der Velden BH, Gilhuijs KG, Leiner T, Viergever MA, Išgum I: Deep learning for multi-task medical image segmentation in multiple modalities.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 478–486
55.
go back to reference Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS: Inbreast: toward a full-field digital mammographic database. Acad Radiol 19(2): 236–248, 2012CrossRefPubMed Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS: Inbreast: toward a full-field digital mammographic database. Acad Radiol 19(2): 236–248, 2012CrossRefPubMed
56.
go back to reference Nam CM, Kim J, Lee KJ (2018) Lung nodule segmentation with convolutional neural network trained by simple diameter information Nam CM, Kim J, Lee KJ (2018) Lung nodule segmentation with convolutional neural network trained by simple diameter information
57.
go back to reference Nie D, Wang L, Gao Y, Sken D: Fully convolutional networks for multi-modality isointense infant brain image segmentation.. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE, 2016, pp 1342–1345 Nie D, Wang L, Gao Y, Sken D: Fully convolutional networks for multi-modality isointense infant brain image segmentation.. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE, 2016, pp 1342–1345
58.
go back to reference Perez L, Wang J (2017) The effectiveness of data augmentation in image classification using deep learning. arXiv:1712.04621 Perez L, Wang J (2017) The effectiveness of data augmentation in image classification using deep learning. arXiv:1712.​04621
59.
go back to reference Poudel RP, Lamata P, Montana G: Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation.. In: Reconstruction, Segmentation, and Analysis of Medical Images. Springer, 2016, pp 83–94 Poudel RP, Lamata P, Montana G: Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation.. In: Reconstruction, Segmentation, and Analysis of Medical Images. Springer, 2016, pp 83–94
60.
go back to reference Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M: Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2013, pp 246–253 Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M: Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2013, pp 246–253
61.
go back to reference Ravishankar H, Sudhakar P, Venkataramani R, Thiruvenkadam S, Annangi P, Babu N, Vaidya V: Understanding the mechanisms of deep transfer learning for medical images.. In: Deep Learning and Data Labeling for Medical Applications. Springer, 2016, pp 188–196 Ravishankar H, Sudhakar P, Venkataramani R, Thiruvenkadam S, Annangi P, Babu N, Vaidya V: Understanding the mechanisms of deep transfer learning for medical images.. In: Deep Learning and Data Labeling for Medical Applications. Springer, 2016, pp 188–196
62.
go back to reference Ronneberger O, Fischer P, Brox T: U-net: convolutional networks for biomedical image segmentation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, pp 234–241 Ronneberger O, Fischer P, Brox T: U-net: convolutional networks for biomedical image segmentation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, pp 234–241
63.
go back to reference Roth HR, Lu L, Farag A, Shin HC, Liu J, Turkbey EB, Summers RM: Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, pp 556–564 Roth HR, Lu L, Farag A, Shin HC, Liu J, Turkbey EB, Summers RM: Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, pp 556–564
64.
go back to reference Roth HR, Lu L, Farag A, Sohn A, Summers RM: Spatial aggregation of holistically-nested networks for automated pancreas segmentation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 451–459 Roth HR, Lu L, Farag A, Sohn A, Summers RM: Spatial aggregation of holistically-nested networks for automated pancreas segmentation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 451–459
65.
go back to reference Roth HR, Lu L, Seff A, Cherry KM, Hoffman J, Wang S, Liu J, Turkbey E, Summers RM: A new 2.5 D representation for lymph node detection using random sets of deep convolutional neural network observations.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2014, pp 520–527 Roth HR, Lu L, Seff A, Cherry KM, Hoffman J, Wang S, Liu J, Turkbey E, Summers RM: A new 2.5 D representation for lymph node detection using random sets of deep convolutional neural network observations.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2014, pp 520–527
66.
go back to reference Roth HR, Oda H, Hayashi Y, Oda M, Shimizu N, Fujiwara M, Misawa K, Mori K (2017) Hierarchical 3D fully convolutional networks for multi-organ segmentation. arXiv:1704.06382 Roth HR, Oda H, Hayashi Y, Oda M, Shimizu N, Fujiwara M, Misawa K, Mori K (2017) Hierarchical 3D fully convolutional networks for multi-organ segmentation. arXiv:1704.​06382
68.
go back to reference Shie CK, Chuang CH, Chou CN, Wu MH, Chang EY: Transfer representation learning for medical image analysis.. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2015, pp 711–714 Shie CK, Chuang CH, Chou CN, Wu MH, Chang EY: Transfer representation learning for medical image analysis.. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2015, pp 711–714
69.
go back to reference Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.​1556
70.
go back to reference Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R: Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1): 1929–1958, 2014 Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R: Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1): 1929–1958, 2014
71.
go back to reference Srivastava N, Mansimov E, Salakhudinov R: Unsupervised learning of video representations using lstms.. In: International Conference on Machine Learning, 2015, pp 843–852 Srivastava N, Mansimov E, Salakhudinov R: Unsupervised learning of video representations using lstms.. In: International Conference on Machine Learning, 2015, pp 843–852
72.
go back to reference Stollenga MF, Byeon W, Liwicki M, Schmidhuber J: Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation.. In: Advances in Neural Information Processing Systems, 2015, pp 2998–3006 Stollenga MF, Byeon W, Liwicki M, Schmidhuber J: Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation.. In: Advances in Neural Information Processing Systems, 2015, pp 2998–3006
73.
go back to reference Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A: Going deeper with convolutions.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp 1–9 Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A: Going deeper with convolutions.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp 1–9
74.
go back to reference Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5): 1299–1312, 2016CrossRefPubMed Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5): 1299–1312, 2016CrossRefPubMed
75.
go back to reference Tran D, Bourdev L, Fergus R, Torresani L, Paluri M: Deep end2end voxel2voxel prediction.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016, pp 17–24 Tran D, Bourdev L, Fergus R, Torresani L, Paluri M: Deep end2end voxel2voxel prediction.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016, pp 17–24
76.
go back to reference Urban G, Bendszus M, Hamprecht F, Kleesiek J (2014) Multi-modal brain tumor segmentation using deep convolutional neural networks. MICCAI braTS (Brain Tumor Segmentation) Challenge. Proceedings, winning contribution Urban G, Bendszus M, Hamprecht F, Kleesiek J (2014) Multi-modal brain tumor segmentation using deep convolutional neural networks. MICCAI braTS (Brain Tumor Segmentation) Challenge. Proceedings, winning contribution
77.
go back to reference Wang J, MacKenzie JD, Ramachandran R, Chen DZ: A deep learning approach for semantic segmentation in histology tissue images.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 176–184 Wang J, MacKenzie JD, Ramachandran R, Chen DZ: A deep learning approach for semantic segmentation in histology tissue images.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 176–184
78.
go back to reference Wu L, Xin Y, Li S, Wang T, Heng PA, Ni D: Cascaded fully convolutional networks for automatic prenatal ultrasound image segmentation.. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE, 2017, pp 663–666 Wu L, Xin Y, Li S, Wang T, Heng PA, Ni D: Cascaded fully convolutional networks for automatic prenatal ultrasound image segmentation.. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE, 2017, pp 663–666
79.
go back to reference Xie S, Tu Z: Holistically-nested edge detection.. In: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp 1395–1403 Xie S, Tu Z: Holistically-nested edge detection.. In: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp 1395–1403
80.
go back to reference Xie Y, Zhang Z, Sapkota M, Yang L: Spatial clockwork recurrent neural network for muscle perimysium segmentation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 185–193 Xie Y, Zhang Z, Sapkota M, Yang L: Spatial clockwork recurrent neural network for muscle perimysium segmentation.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2016, pp 185–193
81.
go back to reference Xingjian S, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC: Convolutional LSTM network: a machine learning approach for precipitation nowcasting.. In: Advances in Neural Information Processing Systems, 2015, pp 802–810 Xingjian S, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC: Convolutional LSTM network: a machine learning approach for precipitation nowcasting.. In: Advances in Neural Information Processing Systems, 2015, pp 802–810
82.
go back to reference Yosinski J, Clune J, Bengio Y, Lipson H: How transferable are features in deep neural networks?.. In: Advances in Neural Information Processing Systems, 2014, pp 3320–3328 Yosinski J, Clune J, Bengio Y, Lipson H: How transferable are features in deep neural networks?.. In: Advances in Neural Information Processing Systems, 2014, pp 3320–3328
83.
go back to reference Yu L, Chen H, Dou Q, Qin J, Heng PA: Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4): 994–1004, 2017CrossRefPubMed Yu L, Chen H, Dou Q, Qin J, Heng PA: Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4): 994–1004, 2017CrossRefPubMed
85.
go back to reference Zeiler MD, Fergus R: Visualizing and understanding convolutional networks.. In: European Conference on Computer Vision. Springer, 2014, pp 818–833 Zeiler MD, Fergus R: Visualizing and understanding convolutional networks.. In: European Conference on Computer Vision. Springer, 2014, pp 818–833
86.
go back to reference Zeng G, Yang X, Li J, Yu L, Heng PA, Zheng G: 3D U-net with multi-level deep supervision: fully automatic segmentation of proximal femur in 3D MR images.. In: International Workshop on Machine Learning in Medical Imaging. Springer, 2017, pp 274–282 Zeng G, Yang X, Li J, Yu L, Heng PA, Zheng G: 3D U-net with multi-level deep supervision: fully automatic segmentation of proximal femur in 3D MR images.. In: International Workshop on Machine Learning in Medical Imaging. Springer, 2017, pp 274–282
87.
go back to reference Zeng G, Zheng G: Multi-stream 3D FCN with multi-scale deep supervision for multi-modality isointense infant brain MR image segmentation.. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018, pp 136–140 Zeng G, Zheng G: Multi-stream 3D FCN with multi-scale deep supervision for multi-modality isointense infant brain MR image segmentation.. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018, pp 136–140
88.
go back to reference Zhang H, Kyaw Z, Yu J, Chang SF (2017) PPR-FCN: weakly supervised visual relation detection via parallel pairwise R-FCN. arXiv:1708.01956 Zhang H, Kyaw Z, Yu J, Chang SF (2017) PPR-FCN: weakly supervised visual relation detection via parallel pairwise R-FCN. arXiv:1708.​01956
89.
go back to reference Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, Shen D: Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108: 214–224, 2015CrossRefPubMed Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, Shen D: Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108: 214–224, 2015CrossRefPubMed
90.
go back to reference Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A: Learning deep features for discriminative localization.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp 2921–2929 Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A: Learning deep features for discriminative localization.. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp 2921–2929
91.
go back to reference Zhou X, Ito T, Takayama R, Wang S, Hara T, Fujita H: Three-dimensional ct image segmentation by combining 2D fully convolutional network with 3D majority voting.. In: Deep Learning and Data Labeling for Medical Applications. Springer, 2016, pp 111–120 Zhou X, Ito T, Takayama R, Wang S, Hara T, Fujita H: Three-dimensional ct image segmentation by combining 2D fully convolutional network with 3D majority voting.. In: Deep Learning and Data Labeling for Medical Applications. Springer, 2016, pp 111–120
92.
go back to reference Zhou X, Takayama R, Wang S, Hara T, Fujita H: Deep learning of the sectional appearances of 3D CT images for anatomical structure segmentation based on an FCN voting method. Med Phys 44(10): 5221–5233, 2017CrossRefPubMed Zhou X, Takayama R, Wang S, Hara T, Fujita H: Deep learning of the sectional appearances of 3D CT images for anatomical structure segmentation based on an FCN voting method. Med Phys 44(10): 5221–5233, 2017CrossRefPubMed
93.
go back to reference Zhou XY, Shen M, Riga C, Yang GZ, Lee SL (2017) Focal FCN: towards small object segmentation with limited training data. arXiv:1711.01506 Zhou XY, Shen M, Riga C, Yang GZ, Lee SL (2017) Focal FCN: towards small object segmentation with limited training data. arXiv:1711.​01506
94.
go back to reference Zhou Y, Xie L, Shen W, Fishman E, Yuille A (2016) Pancreas segmentation in abdominal CT scan: a coarse-to-fine approach. CoRR arXiv:1612.08230 Zhou Y, Xie L, Shen W, Fishman E, Yuille A (2016) Pancreas segmentation in abdominal CT scan: a coarse-to-fine approach. CoRR arXiv:1612.​08230
95.
go back to reference Zhou Y, Xie L, Shen W, Wang Y, Fishman EK, Yuille AL: A fixed-point model for pancreas segmentation in abdominal CT scans.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2017, pp 693–701 Zhou Y, Xie L, Shen W, Wang Y, Fishman EK, Yuille AL: A fixed-point model for pancreas segmentation in abdominal CT scans.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2017, pp 693–701
Metadata
Title
Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges
Authors
Mohammad Hesam Hesamian
Wenjing Jia
Xiangjian He
Paul Kennedy
Publication date
01-08-2019
Publisher
Springer International Publishing
Published in
Journal of Imaging Informatics in Medicine / Issue 4/2019
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI
https://doi.org/10.1007/s10278-019-00227-x

Other articles of this Issue 4/2019

Journal of Digital Imaging 4/2019 Go to the issue