Skip to main content
Top
Published in: Molecular Imaging and Biology 5/2020

Open Access 01-10-2020 | Research Article

Deep Learning for Virtual Histological Staining of Bright-Field Microscopic Images of Unlabeled Carotid Artery Tissue

Authors: Dan Li, Hui Hui, Yingqian Zhang, Wei Tong, Feng Tian, Xin Yang, Jie Liu, Yundai Chen, Jie Tian

Published in: Molecular Imaging and Biology | Issue 5/2020

Login to get access

Abstract

Purpose

Histological analysis of artery tissue samples is a widely used method for diagnosis and quantification of cardiovascular diseases. However, the variable and labor-intensive tissue staining procedures hinder efficient and informative histological image analysis.

Procedures

In this study, we developed a deep learning-based method to transfer bright-field microscopic images of unlabeled tissue sections into equivalent bright-field images of histologically stained versions of the same samples. We trained a convolutional neural network to build maps between the unstained images and histologically stained images using a conditional generative adversarial network model.

Results

The results of a blind evaluation by board-certified pathologists illustrate that the virtual staining and standard histological staining images of rat carotid artery tissue sections and those involving different types of stains showed no major differences. Quantification of virtual and histological H&E staining in carotid artery tissue sections showed that the relative errors of intima thickness, intima area, and media area were lower than 1.6 %, 5.6 %, and 12.7 %, respectively. The training time of deep learning network was 12.857 h with 1800 training patches and 200 epoches.

Conclusions

This virtual staining method significantly mitigates the typically laborious and time-consuming histological staining procedures and could be augmented with other label-free microscopic imaging modalities.
Appendix
Available only for authorised users
Literature
1.
go back to reference Purushothaman KR, Purushothaman M, Muntner P, Lento PA, O'Connor WN, Sharma SK, Fuster V, Moreno PR (2011) Inflammation, neovascularization and intra-plaque hemorrhage are associated with increased reparative collagen content: implication for plaque progression in diabetic atherosclerosis. Vasc Med 16:103–108CrossRef Purushothaman KR, Purushothaman M, Muntner P, Lento PA, O'Connor WN, Sharma SK, Fuster V, Moreno PR (2011) Inflammation, neovascularization and intra-plaque hemorrhage are associated with increased reparative collagen content: implication for plaque progression in diabetic atherosclerosis. Vasc Med 16:103–108CrossRef
2.
go back to reference Bentzon JF, Otsuka F, Virmani R, Falk E (2014) Mechanisms of plaque formation and rupture. Circ Res 114:1852–1866CrossRef Bentzon JF, Otsuka F, Virmani R, Falk E (2014) Mechanisms of plaque formation and rupture. Circ Res 114:1852–1866CrossRef
3.
go back to reference Virmani R, Kolodgie FD, Burke AP, Farb A, Schwartz SM (2000) Lessons from sudden coronary death: a comprehensive morphological classification scheme for atherosclerotic lesions. Arterioscler Thromb Vasc Biol 20:1262–1275CrossRef Virmani R, Kolodgie FD, Burke AP, Farb A, Schwartz SM (2000) Lessons from sudden coronary death: a comprehensive morphological classification scheme for atherosclerotic lesions. Arterioscler Thromb Vasc Biol 20:1262–1275CrossRef
4.
go back to reference Virmani R, Kolodgie FD, Burke AP, Finn AV, Gold HK, Tulenko TN, Wrenn SP, Narula J (2005) Atherosclerotic plaque progression and vulnerability to rupture: angiogenesis as a source of intraplaque hemorrhage. Arterioscler Thromb Vasc Biol 25:2054–2061CrossRef Virmani R, Kolodgie FD, Burke AP, Finn AV, Gold HK, Tulenko TN, Wrenn SP, Narula J (2005) Atherosclerotic plaque progression and vulnerability to rupture: angiogenesis as a source of intraplaque hemorrhage. Arterioscler Thromb Vasc Biol 25:2054–2061CrossRef
5.
go back to reference McCann MT, Ozolek JA, Castro CA, Parvin B, Kovacevic J (2015) Automated histology analysis [opportunities for signal processing]. IEEE Signal Process Mag 32:78–87CrossRef McCann MT, Ozolek JA, Castro CA, Parvin B, Kovacevic J (2015) Automated histology analysis [opportunities for signal processing]. IEEE Signal Process Mag 32:78–87CrossRef
6.
go back to reference Yu W, Braz JC, Dutton AM, Prusakov P, Rekhter M (2007) In vivo imaging of atherosclerotic plaques in apolipoprotein E deficient mice using nonlinear microscopy. J Biomed Opt 12:054008CrossRef Yu W, Braz JC, Dutton AM, Prusakov P, Rekhter M (2007) In vivo imaging of atherosclerotic plaques in apolipoprotein E deficient mice using nonlinear microscopy. J Biomed Opt 12:054008CrossRef
7.
go back to reference van Zandvoort M, Engels W, Douma K, Beckers L, oude Egbrink M, Daemen M, Slaaf DW (2004) Two-photon microscopy for imaging of the (atherosclerotic) vascular wall: a proof of concept study. J Vasc Res 41:54–63CrossRef van Zandvoort M, Engels W, Douma K, Beckers L, oude Egbrink M, Daemen M, Slaaf DW (2004) Two-photon microscopy for imaging of the (atherosclerotic) vascular wall: a proof of concept study. J Vasc Res 41:54–63CrossRef
8.
go back to reference Croce AC, Bottiroli G (2014) Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur J Histochem 58:2461PubMedPubMedCentral Croce AC, Bottiroli G (2014) Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur J Histochem 58:2461PubMedPubMedCentral
9.
go back to reference Jamme F, Kascakova S, Villette S, Allouche F, Pallu S, Rouam V, Réfrégiers M (2013) Deep UV autofluorescence microscopy for cell biology and tissue histology. Biol Cell 105:277–288CrossRef Jamme F, Kascakova S, Villette S, Allouche F, Pallu S, Rouam V, Réfrégiers M (2013) Deep UV autofluorescence microscopy for cell biology and tissue histology. Biol Cell 105:277–288CrossRef
10.
go back to reference Le TT, Langohr IM, Locker MJ, Sturek M, Cheng JX (2007) Label-free molecular imaging of atherosclerotic lesions using multimodal nonlinear optical microscopy. J Biomed Opt 12:054007CrossRef Le TT, Langohr IM, Locker MJ, Sturek M, Cheng JX (2007) Label-free molecular imaging of atherosclerotic lesions using multimodal nonlinear optical microscopy. J Biomed Opt 12:054007CrossRef
11.
go back to reference Zoumi A, Yeh A, Tromberg BJ (2002) Imaging cells and extracellular matrix in vivo by using second-harmonic generation and two-photon excited fluorescence. Proc Natl Acad Sci U S A 99:11014–11019CrossRef Zoumi A, Yeh A, Tromberg BJ (2002) Imaging cells and extracellular matrix in vivo by using second-harmonic generation and two-photon excited fluorescence. Proc Natl Acad Sci U S A 99:11014–11019CrossRef
12.
go back to reference Witte S, Negrean A, Lodder JC, de Kock CPJ, Testa Silva G, Mansvelder HD, Louise Groot M (2011) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy. Proc Natl Acad Sci U S A 108:5970–5975CrossRef Witte S, Negrean A, Lodder JC, de Kock CPJ, Testa Silva G, Mansvelder HD, Louise Groot M (2011) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy. Proc Natl Acad Sci U S A 108:5970–5975CrossRef
13.
go back to reference Ji M, Orringer DA, Freudiger CW et al (2013) Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med 5:201ra119CrossRef Ji M, Orringer DA, Freudiger CW et al (2013) Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med 5:201ra119CrossRef
14.
go back to reference Orringer DA, Pandian B, Niknafs YS, et al. (2017) Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng 1 Orringer DA, Pandian B, Niknafs YS, et al. (2017) Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng 1
15.
go back to reference Jansen K, Wu M, van der Steen AF, van Soest G (2014) Photoacoustic imaging of human coronary atherosclerosis in two spectral bands. Photoacoustics 2:12–20CrossRef Jansen K, Wu M, van der Steen AF, van Soest G (2014) Photoacoustic imaging of human coronary atherosclerosis in two spectral bands. Photoacoustics 2:12–20CrossRef
16.
go back to reference Wang B, Su JL, Amirian J, Litovsky SH, Smalling R, Emelianov S (2010) Detection of lipid in atherosclerotic vessels using ultrasound-guided spectroscopic intravascular photoacoustic imaging. Opt Express 18:4889–4897CrossRef Wang B, Su JL, Amirian J, Litovsky SH, Smalling R, Emelianov S (2010) Detection of lipid in atherosclerotic vessels using ultrasound-guided spectroscopic intravascular photoacoustic imaging. Opt Express 18:4889–4897CrossRef
17.
go back to reference Seeger M, Karlas A, Soliman D, Pelisek J, Ntziachristos V (2016) Multimodal optoacoustic and multiphoton microscopy of human carotid atheroma. Photoacoustics 4:102–111CrossRef Seeger M, Karlas A, Soliman D, Pelisek J, Ntziachristos V (2016) Multimodal optoacoustic and multiphoton microscopy of human carotid atheroma. Photoacoustics 4:102–111CrossRef
18.
go back to reference Rivenson Y, Wang H, Wei Z, de Haan K, Zhang Y, Wu Y, Günaydın H, Zuckerman JE, Chong T, Sisk AE, Westbrook LM, Wallace WD, Ozcan A (2019) Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng 3:466–477CrossRef Rivenson Y, Wang H, Wei Z, de Haan K, Zhang Y, Wu Y, Günaydın H, Zuckerman JE, Chong T, Sisk AE, Westbrook LM, Wallace WD, Ozcan A (2019) Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng 3:466–477CrossRef
19.
go back to reference Bayramoglu N, Kaakinen M, Eklund L, Heikkila J (2017) Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks. 2017 Ieee international conference on computer vision workshops (Iccvw 2017):64-71 Bayramoglu N, Kaakinen M, Eklund L, Heikkila J (2017) Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks. 2017 Ieee international conference on computer vision workshops (Iccvw 2017):64-71
20.
go back to reference Shaban MT, Baur C, Navab N, Albarqouni S (2019) Staingan: stain style transfer for digital histological images. 2019 Ieee 16th international symposium on biomedical imaging (Isbi 2019):953-956 Shaban MT, Baur C, Navab N, Albarqouni S (2019) Staingan: stain style transfer for digital histological images. 2019 Ieee 16th international symposium on biomedical imaging (Isbi 2019):953-956
21.
go back to reference Bautista PA, Yagi Y (2012) Digital simulation of staining in histopathology multispectral images: enhancement and linear transformation of spectral transmittance. J Biomed Opt 17:056013CrossRef Bautista PA, Yagi Y (2012) Digital simulation of staining in histopathology multispectral images: enhancement and linear transformation of spectral transmittance. J Biomed Opt 17:056013CrossRef
22.
go back to reference Clowes AW, Reidy MA, Clowes MM (1983) Kinetics of cellular proliferation after arterial injury. I. Smooth muscle growth in the absence of endothelium. Lab Investig 49:327–333PubMed Clowes AW, Reidy MA, Clowes MM (1983) Kinetics of cellular proliferation after arterial injury. I. Smooth muscle growth in the absence of endothelium. Lab Investig 49:327–333PubMed
23.
go back to reference Isola P, Zhu JY, Zhou TH, Efros AA (2017) Image-to-image translation with conditional adversarial networks. 30th Ieee conference on computer vision and pattern recognition (Cvpr 2017):5967-5976 Isola P, Zhu JY, Zhou TH, Efros AA (2017) Image-to-image translation with conditional adversarial networks. 30th Ieee conference on computer vision and pattern recognition (Cvpr 2017):5967-5976
24.
go back to reference Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. (2014) Generative adversarial nets. Advances in neural information processing systems 27 (nips 2014) 27 Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. (2014) Generative adversarial nets. Advances in neural information processing systems 27 (nips 2014) 27
25.
go back to reference Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention, Pt Iii 9351:234–241 Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention, Pt Iii 9351:234–241
26.
go back to reference Klages P, Benslimane I, Riyahi S et al (2019) Patch-based generative adversarial neural network models for head and neck MR-only planning. Med Phys Klages P, Benslimane I, Riyahi S et al (2019) Patch-based generative adversarial neural network models for head and neck MR-only planning. Med Phys
27.
go back to reference Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: feature learning by inpainting. 2016 Ieee conference on computer vision and pattern recognition (Cvpr):2536–2544 Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: feature learning by inpainting. 2016 Ieee conference on computer vision and pattern recognition (Cvpr):2536–2544
28.
go back to reference Schneider CA, Rasband WS, Eliceiri KW (2012) NIH image to ImageJ: 25 years of image analysis. Nat Methods 9:671–675CrossRef Schneider CA, Rasband WS, Eliceiri KW (2012) NIH image to ImageJ: 25 years of image analysis. Nat Methods 9:671–675CrossRef
29.
go back to reference Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. 2018 Ieee/Cvf conference on computer vision and pattern recognition (Cvpr):8789-8797 Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. 2018 Ieee/Cvf conference on computer vision and pattern recognition (Cvpr):8789-8797
Metadata
Title
Deep Learning for Virtual Histological Staining of Bright-Field Microscopic Images of Unlabeled Carotid Artery Tissue
Authors
Dan Li
Hui Hui
Yingqian Zhang
Wei Tong
Feng Tian
Xin Yang
Jie Liu
Yundai Chen
Jie Tian
Publication date
01-10-2020
Publisher
Springer International Publishing
Published in
Molecular Imaging and Biology / Issue 5/2020
Print ISSN: 1536-1632
Electronic ISSN: 1860-2002
DOI
https://doi.org/10.1007/s11307-020-01508-6

Other articles of this Issue 5/2020

Molecular Imaging and Biology 5/2020 Go to the issue