Skip to main content
Top
Published in: Journal of Digital Imaging 1/2023

22-08-2022

Inter-Slice Resolution Improvement Using Convolutional Neural Network with Orbital Bone Edge-Aware in Facial CT Images

Authors: Hee Rim Yun, Min Jin Lee, Helen Hong, Kyu Won Shim

Published in: Journal of Imaging Informatics in Medicine | Issue 1/2023

Login to get access

Abstract

The 3D modeling of orbital bones in facial CT images is essential to provide a customized implant for reconstructions of orbit and related structures during surgery. However, 3D models of the orbital bone show an aliasing effect and disconnected thin bone in the inter-slice direction because the slice thickness is two to three times larger than the pixel spacing. To improve the inter-slice resolution of facial CT images, we propose a method based on a 2D convolutional neural network (CNN) that uses the spatial information on the sagittal and axial planes and the orbital bone edge-aware (OBE) loss. First, intermediate slices are generated on the sagittal plane. Second, the generated intermediate slices are transformed to an axial image, which is then compared with the original axial image. To generate intermediate slices with an accurate orbital bone structure, the OBE loss considering the orbital bone structure on the sagittal and axial planes is used. To improve the perceptual quality of the generated intermediate slices, the feature map difference loss is additionally used on the axial plane. In the experiment, the proposed method showed the best performance among bilinear and bicubic interpolations, 3D SRGAN, and a 2D CNN-based method. Experimental results confirmed that the proposed method can generate intermediate slices with clear edges of thin bones as well as cortical bones on both the sagittal and the axial plane.
Literature
1.
go back to reference Burnstine MA: Clinical recommendations for repair of orbital facial fractures. Current opinion in ophthalmology 14(5): 236-240, 2003CrossRefPubMed Burnstine MA: Clinical recommendations for repair of orbital facial fractures. Current opinion in ophthalmology 14(5): 236-240, 2003CrossRefPubMed
3.
go back to reference Keys R: Cubic convolution interpolation for digital image processing. IEEE transactions on acoustics, speech, and signal processing 29(6): 1153-1160, 1981CrossRef Keys R: Cubic convolution interpolation for digital image processing. IEEE transactions on acoustics, speech, and signal processing 29(6): 1153-1160, 1981CrossRef
4.
go back to reference Bae W, Lee S, Park G, Park H., Jung KH: Residual CNN-based image super-resolution for CT slice thickness reduction using paired CT scans: preliminary validation study. Medical Imaging with Deep Learning, 2018 Bae W, Lee S, Park G, Park H., Jung KH: Residual CNN-based image super-resolution for CT slice thickness reduction using paired CT scans: preliminary validation study. Medical Imaging with Deep Learning, 2018
5.
go back to reference Georgescu, MI., Ionescu RT, Verga N: Convolutional neural networks with intermediate loss for 3D super-resolution of CT and MRI scans. IEEE Access 8: 49112-49124, 2020CrossRef Georgescu, MI., Ionescu RT, Verga N: Convolutional neural networks with intermediate loss for 3D super-resolution of CT and MRI scans. IEEE Access 8: 49112-49124, 2020CrossRef
6.
go back to reference Jurek J, Kociński M, Materka A, Elgalal M, Majos A: CNN-based superresolution reconstruction of 3D MR images using thick-slice scans. Biocybernetics and Biomedical Engineering 40(1): 111-125, 2020CrossRef Jurek J, Kociński M, Materka A, Elgalal M, Majos A: CNN-based superresolution reconstruction of 3D MR images using thick-slice scans. Biocybernetics and Biomedical Engineering 40(1): 111-125, 2020CrossRef
7.
go back to reference Yun, H. R., Lee, M. J., Hong, H., Shim, K. W., & Jeon, J: Improvement of inter-slice resolution based on 2D CNN with thin bone structure-aware on head-and-neck CT images. International Society for Optics and Photonics, 2021. Yun, H. R., Lee, M. J., Hong, H., Shim, K. W., & Jeon, J: Improvement of inter-slice resolution based on 2D CNN with thin bone structure-aware on head-and-neck CT images. International Society for Optics and Photonics, 2021.
8.
go back to reference Li Z, Wang Y, Yu J: Reconstruction of thin-slice medical images using generative adversarial network. International Workshop on Machine Learning in Medical Imaging, 2017 Li Z, Wang Y, Yu J: Reconstruction of thin-slice medical images using generative adversarial network. International Workshop on Machine Learning in Medical Imaging, 2017
9.
go back to reference Delannoy Q, Pham CH, Cazorla C, Tor-Díez C, Dollé G, Meunier H, Rousseau F, et al: SegSRGAN: Super-resolution and segmentation using generative adversarial networks—Application to neonatal brain MRI. Computers in Biology and Medicine 120: 103755, 2020CrossRefPubMed Delannoy Q, Pham CH, Cazorla C, Tor-Díez C, Dollé G, Meunier H, Rousseau F, et al: SegSRGAN: Super-resolution and segmentation using generative adversarial networks—Application to neonatal brain MRI. Computers in Biology and Medicine 120: 103755, 2020CrossRefPubMed
10.
go back to reference van der Ouderaa TF, Worrall DE, van Ginneken B: Chest CT Super-resolution and Domain-adaptation using Memory-efficient 3D Reversible GANs. arXiv preprint arXiv:1908.00295. 2019 van der Ouderaa TF, Worrall DE, van Ginneken B: Chest CT Super-resolution and Domain-adaptation using Memory-efficient 3D Reversible GANs. arXiv preprint arXiv:​1908.​00295. 2019
11.
go back to reference Kudo A, Kitamura Y, Li Y, Iizuka S, Simo-Serra E: Virtual thin slice: 3D conditional GAN-based super-resolution for CT slice interval. International Workshop on Machine Learning for Medical Image Reconstruction pp.91–100, 2019 Kudo A, Kitamura Y, Li Y, Iizuka S, Simo-Serra E: Virtual thin slice: 3D conditional GAN-based super-resolution for CT slice interval. International Workshop on Machine Learning for Medical Image Reconstruction pp.91–100, 2019
12.
go back to reference Ledig C, Theis L, Husz´ar F., Caballero J, Cunningham A, Acosta A, Aitken A P, Tejani A, Totz J, Wang Z, et al: Photo-realistic single image super-resolution using a generative adversarial network. CVPR vol 2: p 4, 2017 Ledig C, Theis L, Husz´ar F., Caballero J, Cunningham A, Acosta A, Aitken A P, Tejani A, Totz J, Wang Z, et al: Photo-realistic single image super-resolution using a generative adversarial network. CVPR vol 2: p 4, 2017
13.
go back to reference Liu J, Chen F, Wang X, Liao H: An edge enhanced srgan for mri super resolution in slice-selection direction. Multimodal Brain Image Analysis and Mathematical Foundations of Computational Anatomy pp. 12–20, 2019 Liu J, Chen F, Wang X, Liao H: An edge enhanced srgan for mri super resolution in slice-selection direction. Multimodal Brain Image Analysis and Mathematical Foundations of Computational Anatomy pp. 12–20, 2019
15.
go back to reference Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Wang, Z, et al: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proceedings of the IEEE conference on computer vision and pattern recognition pp. 1874–1883, 2016 Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Wang, Z, et al: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proceedings of the IEEE conference on computer vision and pattern recognition pp. 1874–1883, 2016
16.
17.
go back to reference Deng J, Dong W, Socher R, Li L J, Li K, Fei-Fei L: Imagenet: a large-scale hierarchical image database. CVPR09, 2009 Deng J, Dong W, Socher R, Li L J, Li K, Fei-Fei L: Imagenet: a large-scale hierarchical image database. CVPR09, 2009
18.
19.
go back to reference Huynh-Thu Q, Ghanbari M: Scope of validity of PSNR in image/video quality assessment. Electronics letters 44(13): 800-801, 2008CrossRef Huynh-Thu Q, Ghanbari M: Scope of validity of PSNR in image/video quality assessment. Electronics letters 44(13): 800-801, 2008CrossRef
20.
go back to reference Wang Z., Bovik AC, Sheikh HR, Simoncelli EP: Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4): 600-612, 2004CrossRefPubMed Wang Z., Bovik AC, Sheikh HR, Simoncelli EP: Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4): 600-612, 2004CrossRefPubMed
21.
go back to reference Sheikh HR, Bovik AC: Image information and visual quality. IEEE Transactions on image processing 15(2): 430-444, 2006CrossRefPubMed Sheikh HR, Bovik AC: Image information and visual quality. IEEE Transactions on image processing 15(2): 430-444, 2006CrossRefPubMed
Metadata
Title
Inter-Slice Resolution Improvement Using Convolutional Neural Network with Orbital Bone Edge-Aware in Facial CT Images
Authors
Hee Rim Yun
Min Jin Lee
Helen Hong
Kyu Won Shim
Publication date
22-08-2022
Publisher
Springer International Publishing
Published in
Journal of Imaging Informatics in Medicine / Issue 1/2023
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI
https://doi.org/10.1007/s10278-022-00686-9

Other articles of this Issue 1/2023

Journal of Digital Imaging 1/2023 Go to the issue