Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 2/2020

01-02-2020 | Original Article

Automatic cancer tissue detection using multispectral photoacoustic imaging

Authors: Kamal Jnawali, Bhargava Chinni, Vikram Dogra, Navalgund Rao

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 2/2020

Login to get access

Abstract

Purpose

In the case of multispecimen study to locate cancer regions, such as in thyroidectomy and prostatectomy, a significant labor-intensive processing is required at a high cost. Pathology diagnosis is usually done by a pathologist observing tissue-stained glass slide under a microscope.

Method

Multispectral photoacoustic (MPA) specimen imaging has proven successful in differentiating photoacoustic (PA) signal characteristics between a histopathology-defined cancer region and normal tissue. This is mainly due to its ability to efficiently map oxyhemoglobin and deoxyhemoglobin contents from MPA images and key features for cancer detection. A fully automated deep learning algorithm is purposed, which learns to detect the presence of malignant tissue in freshly excised ex vivo human thyroid and prostate tissue specimens using the three-dimensional MPA dataset. The proposed automated deep learning model consisted of the convolutional neural network architecture, which extracts spatially colocated features, and a softmax function, which detects thyroid and prostate cancer tissue at once. This is one of the first deep learning models, to the best of our knowledge, to detect the presence of cancer in excised thyroid and prostate tissue of humans at once based on PA imaging.

Result

The area under the curve (AUC) was used as a metric to evaluate the predictive performance of the classifier. The proposed model detected the cancer tissue with the AUC of 0.96, which is very promising.

Conclusion

This model is an improvement over the previous work using machine learning and deep learning algorithms. This model may have immediate application in cancer screening of the numerous sliced specimens that result from thyroidectomy and prostatectomy. Since the instrument that was used to capture the ex vivo PA images is now being developed for in vivo use, this model may also prove to be a starting point for in vivo PA image analysis for cancer diagnosis.
Literature
1.
go back to reference Siegel RL, Miller KD, Jemal A (2016) Cancer statistics, 2016. CA Cancer J Clin 66(1):7–30CrossRef Siegel RL, Miller KD, Jemal A (2016) Cancer statistics, 2016. CA Cancer J Clin 66(1):7–30CrossRef
2.
go back to reference Shinohara K, Wheeler TM, Scardino PT (1989) The appearance of prostate cancer on transrectal ultrasonography: correlation of imaging and pathological examinations. J Urol 142(1):76–82CrossRef Shinohara K, Wheeler TM, Scardino PT (1989) The appearance of prostate cancer on transrectal ultrasonography: correlation of imaging and pathological examinations. J Urol 142(1):76–82CrossRef
3.
go back to reference Dogra VS, Chinni BK, Valluru KS, Joseph JV, Ghazi A, Yao JL, Evans K, Messing EM, Rao NA (2013) Multispectral photoacoustic imaging of prostate cancer: preliminary ex-vivo results. J Clin Imaging Sci 3:41CrossRef Dogra VS, Chinni BK, Valluru KS, Joseph JV, Ghazi A, Yao JL, Evans K, Messing EM, Rao NA (2013) Multispectral photoacoustic imaging of prostate cancer: preliminary ex-vivo results. J Clin Imaging Sci 3:41CrossRef
4.
go back to reference Valluru KS, Chinni BK, Rao NA, Shweta B, Dogra VS (2009) Basics and clinical applications of photoacoustic imaging. Ultrasound Clinics 4(3):403–429CrossRef Valluru KS, Chinni BK, Rao NA, Shweta B, Dogra VS (2009) Basics and clinical applications of photoacoustic imaging. Ultrasound Clinics 4(3):403–429CrossRef
5.
go back to reference Dogra VS, Chinni BK, Valluru KS, Moalem J, Giampoli EJ, Evans K, Rao NA (2014) Preliminary results of ex vivo multispectral photoacoustic imaging in the management of thyroid cancer. Am J Roentgenol 202(6):W552–W558CrossRef Dogra VS, Chinni BK, Valluru KS, Moalem J, Giampoli EJ, Evans K, Rao NA (2014) Preliminary results of ex vivo multispectral photoacoustic imaging in the management of thyroid cancer. Am J Roentgenol 202(6):W552–W558CrossRef
6.
go back to reference Liu S, Wang Y, Yang X, Lei B, Liu L, Li SX, Ni D, Wang T (2019) Deep learning in medical ultrasound analysis: a review. Engineering 5:261–275CrossRef Liu S, Wang Y, Yang X, Lei B, Liu L, Li SX, Ni D, Wang T (2019) Deep learning in medical ultrasound analysis: a review. Engineering 5:261–275CrossRef
7.
go back to reference Kuligowska E, Barish MA, Fenlon HM, Blake M (2001) Predictors of prostate carcinoma: accuracy of gray-scale and color doppler us and serum markers. Radiology 220(3):757–764CrossRef Kuligowska E, Barish MA, Fenlon HM, Blake M (2001) Predictors of prostate carcinoma: accuracy of gray-scale and color doppler us and serum markers. Radiology 220(3):757–764CrossRef
8.
go back to reference Valluru KS, Chinni BK, Rao NA (2011) Photoacoustic imaging: opening new frontiers in medical imaging. J Clin Imaging Sci 1:24CrossRef Valluru KS, Chinni BK, Rao NA (2011) Photoacoustic imaging: opening new frontiers in medical imaging. J Clin Imaging Sci 1:24CrossRef
9.
go back to reference Jnawali K, Chinni B, Dogra V, Rao N (2017) Photoacoustic simulation study of chirp excitation response from different size absorbers. In: Medical imaging 2017: ultrasonic imaging and tomography, vol 10139. International Society for Optics and Photonics, p 101391L Jnawali K, Chinni B, Dogra V, Rao N (2017) Photoacoustic simulation study of chirp excitation response from different size absorbers. In: Medical imaging 2017: ultrasonic imaging and tomography, vol 10139. International Society for Optics and Photonics, p 101391L
10.
go back to reference Lashkari B (2011) Photoacoustic imaging using chirp technique: comparison with pulsed laser photoacoustics. PhD thesis Lashkari B (2011) Photoacoustic imaging using chirp technique: comparison with pulsed laser photoacoustics. PhD thesis
11.
go back to reference Sinha S (2018) Photoacoustic image analysis for cancer detection and building a novel ultrasound imaging system. PhD thesis Sinha S (2018) Photoacoustic image analysis for cancer detection and building a novel ultrasound imaging system. PhD thesis
12.
go back to reference Hou AH, Swanson D, Barqawi AB (2009) Modalities for imaging of prostate cancer. Adv Urol 2009 Hou AH, Swanson D, Barqawi AB (2009) Modalities for imaging of prostate cancer. Adv Urol 2009
13.
go back to reference Ruiz J, Nouizi F, Cho J, Zheng J, Li Y, Chen J-H, Su M-Y, Gulsen G (2017) Breast density quantification using structured-light-based diffuse optical tomography simulations. Appl Opt 56(25):7146–7157CrossRef Ruiz J, Nouizi F, Cho J, Zheng J, Li Y, Chen J-H, Su M-Y, Gulsen G (2017) Breast density quantification using structured-light-based diffuse optical tomography simulations. Appl Opt 56(25):7146–7157CrossRef
14.
go back to reference Xu M, Wang LV (2006) Photoacoustic imaging in biomedicine. Rev Sci Instrum 77(4):041101CrossRef Xu M, Wang LV (2006) Photoacoustic imaging in biomedicine. Rev Sci Instrum 77(4):041101CrossRef
15.
go back to reference Sinha S, Dogra VS, Chinni BK, Rao NA (2017) Frequency domain analysis of multiwavelength photoacoustic signals for differentiating among malignant, benign, and normal thyroids in an ex vivo study with human thyroids. J Ultrasound Med 36:2047–2059CrossRef Sinha S, Dogra VS, Chinni BK, Rao NA (2017) Frequency domain analysis of multiwavelength photoacoustic signals for differentiating among malignant, benign, and normal thyroids in an ex vivo study with human thyroids. J Ultrasound Med 36:2047–2059CrossRef
16.
go back to reference Lashkari B, Mandelis A (2011) Linear frequency modulation photoacoustic radar: optimal bandwidth and signal-to-noise ratio for frequency-domain imaging of turbid media. J Acoust Soc Am 130(3):1313–1324CrossRef Lashkari B, Mandelis A (2011) Linear frequency modulation photoacoustic radar: optimal bandwidth and signal-to-noise ratio for frequency-domain imaging of turbid media. J Acoust Soc Am 130(3):1313–1324CrossRef
17.
go back to reference Agarwal A, Huang SW, O’donnell M, Day KC, Day M, Kotov N, Ashkenazi S (2007) Targeted gold nanorod contrast agent for prostate cancer detection by photoacoustic imaging. J Appl Phys 102(6):064701CrossRef Agarwal A, Huang SW, O’donnell M, Day KC, Day M, Kotov N, Ashkenazi S (2007) Targeted gold nanorod contrast agent for prostate cancer detection by photoacoustic imaging. J Appl Phys 102(6):064701CrossRef
18.
go back to reference Beard P (2011) Biomedical photoacoustic imaging. Interface Focus 1:602–631CrossRef Beard P (2011) Biomedical photoacoustic imaging. Interface Focus 1:602–631CrossRef
19.
go back to reference Mallidi S, Luke GP, Emelianov S (2011) Photoacoustic imaging in cancer detection, diagnosis, and treatment guidance. Trends Biotechnol 29(5):213–221CrossRef Mallidi S, Luke GP, Emelianov S (2011) Photoacoustic imaging in cancer detection, diagnosis, and treatment guidance. Trends Biotechnol 29(5):213–221CrossRef
20.
go back to reference Oraevsky AA, Savateeva EV, Solomatin SV, Karabutov AA, Andreev VG, Gatalica Z, Khamapirad T, Henrichs PM (2002) Optoacoustic imaging of blood for visualization and diagnostics of breast cancer. In: Biomedical optoacoustics III, vol 4618. International Society for Optics and Photonics, pp 81–95 Oraevsky AA, Savateeva EV, Solomatin SV, Karabutov AA, Andreev VG, Gatalica Z, Khamapirad T, Henrichs PM (2002) Optoacoustic imaging of blood for visualization and diagnostics of breast cancer. In: Biomedical optoacoustics III, vol 4618. International Society for Optics and Photonics, pp 81–95
21.
go back to reference Jnawali K, Chinni B, Dogra V, Rao N (2019) Transfer learning for automatic cancer tissue detection using multispectral photoacoustic imaging. In: Medical imaging 2019: computer-aided diagnosis, vol 10950. International Society for Optics and Photonics, p 109503W Jnawali K, Chinni B, Dogra V, Rao N (2019) Transfer learning for automatic cancer tissue detection using multispectral photoacoustic imaging. In: Medical imaging 2019: computer-aided diagnosis, vol 10950. International Society for Optics and Photonics, p 109503W
22.
go back to reference Jnawali K, Chinni B, Dogra V, Sinha S, Rao N (2019) Deep 3D convolutional neural network for automatic cancer tissue detection using multispectral photoacoustic imaging. In: Medical imaging 2019: ultrasonic imaging and tomography, vol 10955. International Society for Optics and Photonics, p 109551D Jnawali K, Chinni B, Dogra V, Sinha S, Rao N (2019) Deep 3D convolutional neural network for automatic cancer tissue detection using multispectral photoacoustic imaging. In: Medical imaging 2019: ultrasonic imaging and tomography, vol 10955. International Society for Optics and Photonics, p 109551D
23.
go back to reference Jnawali K, Arbabshirani MR, Rao N, Patel AA (2018) Deep 3D convolution neural network for CT brain hemorrhage classification. In: Medical imaging 2018: computer-aided diagnosis, vol 10575. International Society for Optics and Photonics, p 105751C Jnawali K, Arbabshirani MR, Rao N, Patel AA (2018) Deep 3D convolution neural network for CT brain hemorrhage classification. In: Medical imaging 2018: computer-aided diagnosis, vol 10575. International Society for Optics and Photonics, p 105751C
24.
go back to reference He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034 He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034
25.
go back to reference Martín A, Paul B, Jianmin C, Zhifeng C, Andy D, Jeffrey D, Matthieu D, Sanjay G, Geoffrey I, Michael I et al (2016) Tensorflow: a system for large-scale machine learning. In: OSDI, vol 16, pp 265–283 Martín A, Paul B, Jianmin C, Zhifeng C, Andy D, Jeffrey D, Matthieu D, Sanjay G, Geoffrey I, Michael I et al (2016) Tensorflow: a system for large-scale machine learning. In: OSDI, vol 16, pp 265–283
26.
go back to reference Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456 Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456
27.
go back to reference Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958 Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
28.
go back to reference Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
30.
go back to reference Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin
31.
go back to reference Friedman J, Hastie T, Tibshirani R (2001) The elements of statistical learning, vol 1. Springer series in statistics. Springer, New York Friedman J, Hastie T, Tibshirani R (2001) The elements of statistical learning, vol 1. Springer series in statistics. Springer, New York
32.
go back to reference Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9 Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
33.
go back to reference Zhou Z-H, Liu X-Y (2006) Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans Knowl Data Eng 18(1):63–77CrossRef Zhou Z-H, Liu X-Y (2006) Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans Knowl Data Eng 18(1):63–77CrossRef
34.
go back to reference Jnawali K, Arbabshirani MR, Ulloa AE, Rao N, Patel AA (2019) Automatic classification of radiological report for intracranial hemorrhage. In: 2019 IEEE 13th international conference on semantic computing (ICSC). IEEE, pp 187–190 Jnawali K, Arbabshirani MR, Ulloa AE, Rao N, Patel AA (2019) Automatic classification of radiological report for intracranial hemorrhage. In: 2019 IEEE 13th international conference on semantic computing (ICSC). IEEE, pp 187–190
35.
go back to reference Bezdek JC, Hall LO, Clarke L-P (1993) Review of MR image segmentation techniques using pattern recognition. Med Phys 20(4):1033–1048CrossRef Bezdek JC, Hall LO, Clarke L-P (1993) Review of MR image segmentation techniques using pattern recognition. Med Phys 20(4):1033–1048CrossRef
36.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
37.
go back to reference Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, vol 4, p 12 Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, vol 4, p 12
38.
go back to reference Wong SC, Gatt A, Stamatescu V, McDonnell MD (2016) Understanding data augmentation for classification: when to warp? arXiv preprint arXiv:1609.08764 Wong SC, Gatt A, Stamatescu V, McDonnell MD (2016) Understanding data augmentation for classification: when to warp? arXiv preprint arXiv:​1609.​08764
39.
go back to reference Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305 Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305
Metadata
Title
Automatic cancer tissue detection using multispectral photoacoustic imaging
Authors
Kamal Jnawali
Bhargava Chinni
Vikram Dogra
Navalgund Rao
Publication date
01-02-2020
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 2/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-019-02101-1

Other articles of this Issue 2/2020

International Journal of Computer Assisted Radiology and Surgery 2/2020 Go to the issue