Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 7/2020

Open Access 01-07-2020 | Original Article

Light source calibration for multispectral imaging in surgery

Authors: Leonardo Ayala, Silvia Seidlitz, Anant Vemuri, Sebastian J. Wirkert, Thomas Kirchner, Tim J. Adler, Christina Engels, Dogu Teber, Lena Maier-Hein

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 7/2020

Login to get access

Abstract

Purpose

Live intra-operative functional imaging has multiple potential clinical applications, such as localization of ischemia, assessment of organ transplantation success and perfusion monitoring. Recent research has shown that live monitoring of functional tissue properties, such as tissue oxygenation and blood volume fraction, is possible using multispectral imaging in laparoscopic surgery. While the illuminant spectrum is typically kept constant in laparoscopic surgery and can thus be estimated from preoperative calibration images, a key challenge in open surgery originates from the dynamic changes of lighting conditions.

Methods

The present paper addresses this challenge with a novel approach to light source calibration based on specular highlight analysis. It involves the acquisition of low-exposure time images serving as a basis for recovering the illuminant spectrum from pixels that contain a dominant specular reflectance component.

Results

Comprehensive in silico and in vivo experiments with a range of different light sources demonstrate that our approach enables an accurate and robust recovery of the illuminant spectrum in the field of view of the camera, which results in reduced errors with respect to the estimation of functional tissue properties. Our approach further outperforms state-of-the-art methods proposed in the field of computer vision.

Conclusion

Our results suggest that low-exposure multispectral images are well suited for light source calibration via specular highlight analysis. This work thus provides an important first step toward live functional imaging in open surgery.
Literature
1.
go back to reference Agarwal V, Abidi B, Koschan A, Abidi MA (2006) An overview of color constancy algorithms. J Pattern Recognit Res 1(1):42–54CrossRef Agarwal V, Abidi B, Koschan A, Abidi MA (2006) An overview of color constancy algorithms. J Pattern Recognit Res 1(1):42–54CrossRef
2.
go back to reference Antonio RK, Cong PH (2013) Imaging spectroscopy for scene analysis. chap. 3.2.2, pp 24–25 Antonio RK, Cong PH (2013) Imaging spectroscopy for scene analysis. chap. 3.2.2, pp 24–25
3.
go back to reference Bianco S, Cusano C, Schettini R (2015) Color constancy using CNNs. In: IEEE computer society conference on computer vision and pattern recognition workshops 2015(2), pp 81–89 Bianco S, Cusano C, Schettini R (2015) Color constancy using CNNs. In: IEEE computer society conference on computer vision and pattern recognition workshops 2015(2), pp 81–89
4.
go back to reference Bianco S, Cusano C, Schettini R (2017) Single and multiple illuminant estimation using convolutional neural networks. IEEE Trans Image Process 26(9):4347–4362CrossRef Bianco S, Cusano C, Schettini R (2017) Single and multiple illuminant estimation using convolutional neural networks. IEEE Trans Image Process 26(9):4347–4362CrossRef
5.
go back to reference Finlayson G, Trezzi E (2004) Shades of gray and colour constancy. In: The twelfth color imaging conference: color science and engineering systems, technologies, applications, pp 37–41 Finlayson G, Trezzi E (2004) Shades of gray and colour constancy. In: The twelfth color imaging conference: color science and engineering systems, technologies, applications, pp 37–41
6.
go back to reference Finlayson GD, Schaefer G (2001) Solving for colour constancy using a constrained dichromatic reflection model. Int J Comput Vis 42(3):127–144CrossRef Finlayson GD, Schaefer G (2001) Solving for colour constancy using a constrained dichromatic reflection model. Int J Comput Vis 42(3):127–144CrossRef
7.
go back to reference Healey GE, Kondepudy R (1994) Radiometric CCD camera calibration and noise estimation. IEEE Trans Pattern Anal Mach Intell 16(3):267–276CrossRef Healey GE, Kondepudy R (1994) Radiometric CCD camera calibration and noise estimation. IEEE Trans Pattern Anal Mach Intell 16(3):267–276CrossRef
8.
go back to reference Holmer A, Marotz J, Wahl P, Dau M, Kämmerer PW (2018) Hyperspectral imaging in perfusion and wound diagnostics - Methods and algorithms for the determination of tissue parameters. Biomedizinische Technik 63(5):547–556CrossRef Holmer A, Marotz J, Wahl P, Dau M, Kämmerer PW (2018) Hyperspectral imaging in perfusion and wound diagnostics - Methods and algorithms for the determination of tissue parameters. Biomedizinische Technik 63(5):547–556CrossRef
9.
go back to reference Hu Y, Wang B, Lin S (2017) FC 4 : Fully convolutional color constancy with confidence-weighted pooling. In: IEEE conference on computer vision and pattern recognition, pp 330–339 Hu Y, Wang B, Lin S (2017) FC 4 : Fully convolutional color constancy with confidence-weighted pooling. In: IEEE conference on computer vision and pattern recognition, pp 330–339
10.
go back to reference Imai Y, Kato Y, Kadoi H, Horiuchi T, Tominaga S (2011) Estimation of multiple illuminants based on specular highlight detection. In: International workshop on computational color imaging, pp 85–98 Imai Y, Kato Y, Kadoi H, Horiuchi T, Tominaga S (2011) Estimation of multiple illuminants based on specular highlight detection. In: International workshop on computational color imaging, pp 85–98
11.
go back to reference Kaneko E, Aoki H, Tsukada M (2016) Daylight spectrum estimation from hyper-and multispectral image without area extraction of uniform materials. In: Proceedings—11th international conference on signal-image technology and internet-based systems, pp 53–60 Kaneko E, Aoki H, Tsukada M (2016) Daylight spectrum estimation from hyper-and multispectral image without area extraction of uniform materials. In: Proceedings—11th international conference on signal-image technology and internet-based systems, pp 53–60
12.
go back to reference Kato Y, Horiuchi T, Tominaga S (2012) Estimation of multiple light sources from specular highlights. In: International conference on pattern recognition, pp 2083–2086 Kato Y, Horiuchi T, Tominaga S (2012) Estimation of multiple light sources from specular highlights. In: International conference on pattern recognition, pp 2083–2086
13.
go back to reference Khan HA, Thomas J-B, Hardeberg JY, Laligant O (2017) Illuminant estimation in multispectral imaging. J Opt Soc Am A 34(7):1085–1098CrossRef Khan HA, Thomas J-B, Hardeberg JY, Laligant O (2017) Illuminant estimation in multispectral imaging. J Opt Soc Am A 34(7):1085–1098CrossRef
14.
go back to reference Kulcke A, Holmer A, Wahl P, Siemers F, Wild T, Daeschlein G (2018) A compact hyperspectral camera for measurement of perfusion parameters in medicine. Biomedizinische Technik 63(5):519–527CrossRef Kulcke A, Holmer A, Wahl P, Siemers F, Wild T, Daeschlein G (2018) A compact hyperspectral camera for measurement of perfusion parameters in medicine. Biomedizinische Technik 63(5):519–527CrossRef
15.
16.
go back to reference Shi W, Loy CC, Tang X (2016) Deep specialized network for illuminant estimation. In: European conference on computer vision, pp 371–387 Shi W, Loy CC, Tang X (2016) Deep specialized network for illuminant estimation. In: European conference on computer vision, pp 371–387
17.
go back to reference Wiesenfarth M, Reinke A, Landman BA, Cardoso MJ, Maier-Hein L, Kopp-Schneider A (2019) Methods and open-source toolkit for analyzing and visualizing challenge results. ArXiv preprint arXiv:1910.05121 Wiesenfarth M, Reinke A, Landman BA, Cardoso MJ, Maier-Hein L, Kopp-Schneider A (2019) Methods and open-source toolkit for analyzing and visualizing challenge results. ArXiv preprint arXiv:​1910.​05121
18.
go back to reference Wirkert SJ, Kenngott H, Mayer B, Mietkowski P, Wagner M, Sauer P, Clancy NT, Elson DS, Maier-Hein L (2016) Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression. Int J Comput Assist Radiol Surg 11(6):909–917CrossRef Wirkert SJ, Kenngott H, Mayer B, Mietkowski P, Wagner M, Sauer P, Clancy NT, Elson DS, Maier-Hein L (2016) Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression. Int J Comput Assist Radiol Surg 11(6):909–917CrossRef
19.
go back to reference Wirkert SJ, Vemuri AS, Kenngott HG, Moccia S, Götz M, Mayer BFB, Maier-Hein Klaus, H, Elson, Daniel S, Maier-Hein L (2017) Physiological parameter estimation from multispectral images unleashed. In: Medical image computing and computer assisted interventions, pp 134–141 Wirkert SJ, Vemuri AS, Kenngott HG, Moccia S, Götz M, Mayer BFB, Maier-Hein Klaus, H, Elson, Daniel S, Maier-Hein L (2017) Physiological parameter estimation from multispectral images unleashed. In: Medical image computing and computer assisted interventions, pp 134–141
Metadata
Title
Light source calibration for multispectral imaging in surgery
Authors
Leonardo Ayala
Silvia Seidlitz
Anant Vemuri
Sebastian J. Wirkert
Thomas Kirchner
Tim J. Adler
Christina Engels
Dogu Teber
Lena Maier-Hein
Publication date
01-07-2020
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 7/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-020-02195-y

Other articles of this Issue 7/2020

International Journal of Computer Assisted Radiology and Surgery 7/2020 Go to the issue