Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
Molecular Imaging and Biology
Published in: Molecular Imaging and Biology 1/2022

01-02-2022 | Research Article

Image-to-Images Translation for Multiple Virtual Histological Staining of Unlabeled Human Carotid Atherosclerotic Tissue

Authors: Guanghao Zhang, Bin Ning, Hui Hui, Tengfei Yu, Xin Yang, Hongxia Zhang, Jie Tian, Wen He

Published in: Molecular Imaging and Biology | Issue 1/2022

Login to get access

Abstract

Purpose

Histological analysis of human carotid atherosclerotic plaques is critical in understanding atherosclerosis biology and developing effective plaque prevention and treatment for ischemic stroke. However, the histological staining process is laborious, tedious, variable, and destructive to the highly valuable atheroma tissue obtained from patients.

Procedures

We proposed a deep learning-based method to simultaneously transfer bright-field microscopic images of unlabeled tissue sections into equivalent multiple sections of the same samples that are virtually stained. Using a pix2pix model, we trained a generative adversarial neural network to achieve image-to-images translation of multiple stains, including hematoxylin and eosin (H&E), picrosirius red (PSR), and Verhoeff van Gieson (EVG) stains.

Results

The quantification of evaluation metrics indicated that the proposed approach achieved the best performance in comparison with other state-of-the-art methods. Further blind evaluation by board-certified pathologists demonstrated that the multiple virtual stains have high consistency with standard histological stains. The proposed approach also indicated that the generated histopathological features of atherosclerotic plaques, such as the necrotic core, neovascularization, cholesterol crystals, collagen, and elastic fibers, are optimally matched with those of standard histological stains.

Conclusions

The proposed approach allows for the virtual staining of unlabeled human carotid plaque tissue images with multiple types of stains. In addition, it identifies the histopathological features of atherosclerotic plaques in the same tissue sample, which could facilitate the development of personalized prevention and other interventional treatments for carotid atherosclerosis.
Appendix
Available only for authorised users
Literature
1.
2.
Pelisek J, Well G, Reeps C et al (2012) Neovascularization and angiogenic factors in advanced human carotid artery stenosis. Circ J 76:1274–1282CrossRef
3.
Pelisek J, Pongratz J, Deutsch L, Reeps C, Stadlbauer T, Eckstein HH (2012) Expression and cellular localization of metalloproteases ADAMs in high graded carotid artery lesions. Scand J Clin Lab Inv 72:648–656CrossRef
4.
Zhong XY, Ma ZC, Su YS et al (2020) Flavin adenine dinucleotide ameliorates hypertensive vascular remodeling via activating short chain acyl-CoA dehydrogenase. Life Sci 258:118156CrossRef
5.
Rivenson Y, de Haan K, Wallace WD, Ozcan A (2020) Emerging advances to transform histopathology using virtual staining. BME Frontiers 2020:1–11CrossRef
6.
Croce AC, Bottiroli G (2014) Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur J Histochem 58:2461PubMedPubMedCentral
7.
Jamme F, Kascakova S, Villette S et al (2013) Deep UV autofluorescence microscopy for cell biology and tissue histology. Biol Cell 105:277–288CrossRef
8.
Le TT, Langohr IM, Locker MJ, Sturek M, Cheng JX (2007) Label-free molecular imaging of atherosclerotic lesions using multimodal nonlinear optical microscopy. J Biomed Opt 12:54007CrossRef
9.
Zoumi A, Yeh A, Tromberg BJ (2002) Imaging cells and extracellular matrix In vivo by using second-harmonic generation and two-photon excited fluorescence. P Natl Acad Sci USA 99:11014–11019CrossRef
10.
Witte S, Negrean A, Lodder JC et al (2011) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy. Proc Natl Acad Sci U S A 108:5970–5975CrossRef
11.
Ji M, Orringer DA, Freudiger CW et al (2013) Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med 5:201ra119CrossRef
12.
Orringer DA, Pandian B, Niknafs YS et al (2017) Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng 1:27CrossRef
13.
Seeger M, Karlas A, Soliman D, Pelisek J, Ntziachristos V (2016) Multimodal optoacoustic and multiphoton microscopy of human carotid atheroma. Photoacoustics 4:102–111CrossRef
14.
Bayramoglu N, Kaakinen M, Eklund L, Heikkila J (2017) Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks. Ieee Int Conf Comp V:64–71.
15.
Rivenson Y, Wang H, Wei Z et al (2019) Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng 3:466–477CrossRef
16.
Rivenson Y, Liu TR, Wei ZS, Zhang Y, de Haan K, Ozcan A (2019) PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light-Sci Appl 8:23CrossRef
17.
Christiansen EM, Yang SJ, Ando DM et al (2018) In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173:792CrossRef
18.
Liu Y, Yuan H, Wang ZY, Ji SW (2020) Global pixel transformers for virtual staining of microscopy images. Ieee T Med Imaging 39:2256–2266CrossRef
19.
Li D, Hui H, Zhang YQ et al (2020) Deep learning for virtual histological staining of bright-field microscopic images of unlabeled carotid artery tissue. Mol Imaging Biol 22:1301–1309CrossRef
20.
Zhang Y, de Haan K, Rivenson Y, Li J, Delis A, Ozcan A (2020) Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci Appl 9:78CrossRef
21.
Zhou NY, Cai D, Han X, Yao JH (2019) Enhanced cycle-consistent generative adversarial network for color normalization of H&E stained images. Lect Notes Comput Sc 11764:694–702CrossRef
22.
Gupta L, Klinkhammer BM, Boor P, Merhof D, Gadermayr M (2019) GAN-based image enrichment in digital pathology boosts segmentation accuracy. Lect Notes Comput Sc 11764:631–639CrossRef
23.
Isola P, Zhu JY, Zhou TH, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Proc Cvpr Ieee:5967–5976.
24.
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. Med Imag Comput Comput Assist Interv Pt Iii 9351:234–241
25.
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. Ieee T Image Process 13:600–612CrossRef
26.
Zhang R, Isola P, Efros AA, Shechtman E, Wang O (2018) The unreasonable effectiveness of deep features as a perceptual metric. 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr):586–595.
27.
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun Acm 60:84–90CrossRef
28.
Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr):8789–8797.
29.
He ZL, Zuo WM, Kan MN, Shan SG, Chen XL (2019) AttGAN: facial attribute editing by only changing what you want. Ieee T Image Process 28:5464–5478CrossRef
30.
Liu M, Ding YK, Xia M, et al. (2019) STGAN: a unified selective transfer network for arbitrary image attribute editing. 2019 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr 2019):3668–3677.
31.
Wang W, Zhang Y, Hui H et al (2021) The effect of endothelial progenitor cell transplantation on neointimal hyperplasia and reendothelialisation after balloon catheter injury in rat carotid arteries. Stem Cell Res Ther 12:99CrossRef
32.
Tong W, Hui H, Shang W et al (2021) Highly sensitive magnetic particle imaging of vulnerable atherosclerotic plaque with active myeloperoxidase-targeted nanoparticles. Theranostics 11:506–521CrossRef
Metadata
Title
Image-to-Images Translation for Multiple Virtual Histological Staining of Unlabeled Human Carotid Atherosclerotic Tissue
Authors
Guanghao Zhang
Bin Ning
Hui Hui
Tengfei Yu
Xin Yang
Hongxia Zhang
Jie Tian
Wen He
Publication date
01-02-2022
Publisher
Springer International Publishing
Published in
Molecular Imaging and Biology / Issue 1/2022
Print ISSN: 1536-1632
Electronic ISSN: 1860-2002
DOI
https://doi.org/10.1007/s11307-021-01641-w

Other articles of this Issue 1/2022

Molecular Imaging and Biology 1/2022 Go to the issue

Research Article

Direct Comparison of [18F]F-DPA with [18F]DPA-714 and [11C]PBR28 for Neuroinflammation Imaging in the same Alzheimer’s Disease Model Mice and Healthy Controls

Research Article

Incremental Impact of [68 Ga]Ga-PSMA-11 PET/CT in Primary N and M Staging of Prostate Cancer Prior to Curative-Intent Surgery: a Prospective Clinical Trial in Comparison with mpMRI

Research Article

GlucoCEST MRI for the Evaluation Response to Chemotherapeutic and Metabolic Treatments in a Murine Triple-Negative Breast Cancer: A Comparison with[18F]F-FDG-PET

Research Article

PET/MRI and PET/CT Radiomics in Primary Cervical Cancer: A Pilot Study on the Correlation of Pelvic PET, MRI, and CT Derived Image Features

Research Article

Application of a Flexible PET Scanner Combined with 3 T MRI Using Non-local Means Reconstruction: Qualitative and Quantitative Comparison with Whole-Body PET/CT

Commentary

Development and Validation of a PET/SPECT Radiopharmaceutical in Oncology