Skip to main content
ADVANCED SEARCH
Log in
Top
Journal of Imaging Informatics in Medicine
Published in:

08-07-2024 | Pneumonia

TransMVAN: Multi-view Aggregation Network with Transformer for Pneumonia Diagnosis

Authors: Xiaohong Wang, Zhongkang Lu, Su Huang, Yonghan Ting, Jordan Sim Zheng Ting, Wenxiang Chen, Cher Heng Tan, Weimin Huang

Published in: Journal of Imaging Informatics in Medicine | Issue 1/2025

Login to get access

Abstract

Automated and accurate classification of pneumonia plays a crucial role in improving the performance of computer-aided diagnosis systems for chest X-ray images. Nevertheless, it is a challenging task due to the difficulty of learning the complex structure information of lung abnormality from chest X-ray images. In this paper, we propose a multi-view aggregation network with Transformer (TransMVAN) for pneumonia classification in chest X-ray images. Specifically, we propose to incorporate the knowledge from glance and focus views to enrich the feature representation of lung abnormality. Moreover, to capture the complex relationships among different lung regions, we propose a bi-directional multi-scale vision Transformer (biMSVT), with which the informative messages between different lung regions are propagated through two directions. In addition, we also propose a gated multi-view aggregation (GMVA) to adaptively select the feature information from glance and focus views for further performance enhancement of pneumonia diagnosis. Our proposed method achieves AUCs of 0.9645 and 0.9550 for pneumonia classification on two different chest X-ray image datasets. In addition, it achieves an AUC of 0.9761 for evaluating positive and negative polymerase chain reaction (PCR). Furthermore, our proposed method also attains an AUC of 0.9741 for classifying non-COVID-19 pneumonia, COVID-19 pneumonia, and normal cases. Experimental results demonstrate the effectiveness of our method over other methods used for comparison in pneumonia diagnosis from chest X-ray images.
Literature
1.
Wei, Q., Hu, Y., Gelfand, G., MacGregor, J.: Segmentation of lung lobes in high-resolution isotropic CT images. IEEE Trans. Biomed. Eng. 56(5), 1383–1393 (2009)
2.
Vagliano, I., Kingma, M., Dongelmans, D., Lange, D., Keizer, N., Schut, M., Arbous, M., Verbiest, D., Velde, L., Driel, E., et al: Automated identification of patient subgroups: A case-study on mortality of COVID-19 patients admitted to the ICU. Comput. Biol. Med. 163, 107146 (2023)
3.
Wang, P., Zhang, S., Qi, C., Wang, C., Zhu, Z., Shi, L., Cheng, L., Zhang, X.: Blood microbial analyses reveal long-term effects of SARS-CoV-2 infection on patients who recovered from COVID-19. Comput. Biol. Med. 168, 107721 (2024)
4.
Alhudhaif, A., Polat, K., Karaman, O.: Determination of COVID-19 pneumonia based on generalized convolutional neural network model from chest x-ray images. Expert Syst. Appl. 180, 115141 (2021)
5.
Minaee, S., Kafieh, R., Sonka, M., Yazdani, S., Soufi, G.: Deep-COVID: Predicting COVID-19 from chest x-ray images using deep transfer learning. Med. Image Anal. 65, 101794 (2020)
6.
Shamsi, A., Asgharnezhad, H., Jokandan, S., Khosravi, A., Kebria, P., Nahavandi, D., Nahavandi, S., Srinivasan, D.: An uncertainty-aware transfer learning-based framework for COVID-19 diagnosis. IEEE Trans. Neural Netw. Learn. Syst. 32(4), 1408–1417 (2021)
7.
Oh, Y., Park, S., Ye, J.: Deep learning COVID-19 features on cxr using limited training data sets. IEEE Trans. Med. Imag. 39(8), 2688–2700 (2020)
8.
Brunese, L., Mercaldo, F., Reginelli, A., Santone, A.: Explainable deep learning for pulmonary disease and coronavirus COVID-19 detection from x-rays. Comput. Methods Programs Biomed. 196, 105608 (2020)
9.
Pereira, R., Bertolini, D., Teixeira, L., Silla Jr, C., Costa, Y.: COVID-19 identification in chest x-ray images on flat and hierarchical classification scenarios. Comput. Methods Programs Biomed. 194, 105532 (2020)
10.
Dey, N., Zhang, Y., Rajinikanth, V., Pugalenthi, R., Raja, N.: Customized vgg19 architecture for pneumonia detection in chest x-rays. Pattern Recognit. Lett. 143, 67–74 (2021)
11.
Rajaraman, S., Siegelman, J., Alderson, P., Folio, L., Folio, L., Antani, S.: Iteratively pruned deep learning ensembles for COVID-19 detection in chest x-rays. IEEE Access 8, 115041–115050 (2020)
12.
Guarrasi, V., D’Amico, N., Sicilia, R., Cordelli, E., Soda, P.: Pareto optimization of deep networks for COVID-19 diagnosis from chest x-rays. Pattern Recognit. 121, 108242 (2022)
13.
Sirazitdinov, I., Kholiavchenko, M., Mustafaev, T., Yixuan, Y., Kuleev, R., Ibragimov, B.: Deep neural network ensemble for pneumonia localization from a large-scale chest x-ray database. Comput. Electr. Eng. 78, 388–399 (2019)
14.
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556 (2014)
15.
Huang, G., Liu, Z., Van D., L., Weinberger, K.: Densely connected convolutional networks. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 4700–4708 (2017)
16.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 770–778 (2016)
17.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V.t., Rabinovich, A.: Going deeper with convolutions. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 1–9 (2015)
18.
Wang, X., Jiang, X., Ding, H., Liu, J.: Bi-directional dermoscopic feature learning and multi-scale consistent decision fusion for skin lesion segmentation. IEEE Trans. Image Process. 29, 3039–3051 (2020)
19.
Wang, X., Jiang, X., Ding, H., Zhao, Y., Liu, J.: Knowledge-aware deep framework for collaborative skin lesion segmentation and melanoma recognition. Pattern Recognit., 108075 (2021)
20.
Xu, W., Nie, L., Chen, B., Ding, W.: Dual-stream efficientnet with adversarial sample augmentation for covid-19 computer aided diagnosis. Comput. Biol. Med. 165, 107451 (2023)
21.
Ozturk, T., Talo, M., Yildirim, E., Baloglu, U., Yildirim, O., Acharya, U.: Automated detection of COVID-19 cases using deep neural networks with x-ray images. Comput. Biol. Med. 121, 103792 (2020)
22.
Li, J., Wang, Y., Wang, S., Wang, J., Liu, J., Jin, Q., Sun, L.: Multiscale attention guided network for COVID-19 diagnosis using chest x-ray images. IEEE J. Biomed. Health Inform. 25(5), 1336–1346 (2021)
23.
Veluchamy, S., Sudharson, S., Annamalai, R., Bassfar, Z., Aljaedi, A., Jamal, S.: Automated detection of covid-19 from multimodal imaging data using optimized convolutional neural network model. J. Imaging Infor. Med., 1–15 (2024)
24.
Ren, H., Jing, F., Chen, Z., He, S., Zhou, J., Liu, L., Jing, R., Lian, W., Tian, J., Zhang, Q., et al: CheXMed: A multimodal learning algorithm for pneumonia detection in the elderly. Inf. Sci. 654, 119854 (2024)
25.
Zhang, J., Xie, Y., Pang, G., Liao, Z., Verjans, J., Li, W., Sun, Z., He, J., Li, Y., Shen, C., et al: Viral pneumonia screening on chest x-rays using confidence-aware anomaly detection. IEEE Trans. Med. Imag. 40(3), 879–890 (2020)
26.
Hussain, A., Amin, S.U., Lee, H., Khan, A., Khan, N.F., Seo, S.: An automated chest x-ray image analysis for covid-19 and pneumonia diagnosis using deep ensemble strategy. IEEE Access 11, 97207–97220 (2023)
27.
Rifai, A., Raharjo, S., Utami, E., Ariatmanto, D.: Analysis for diagnosis of pneumonia symptoms using chest x-ray based on mobilenetv2 models with image enhancement using white balance and contrast limited adaptive histogram equalization (CLAHE). Biomed. Signal Process. Control 90, 105857 (2024)
28.
Ali, M., Shahroz, M., Akram, U., Mushtaq, M., Altamiranda, S., Obregon, S., Díez, I., Ashraf, I.: Pneumonia detection using chest radiographs with novel efficientnetv2l model. IEEE Access 12, 34691–34707 (2024)
29.
Madan, S., Chaudhury, S., Gandhi, T.: Explainable few-shot learning with visual explanations on a low resource pneumonia dataset. Pattern Recognit. Lett. 176, 109–116 (2023)
30.
Sanchez, K., Hinojosa, C., Arguello, H., Kouamé, D., Meyrignac, O., Basarab, A.: CX-DaGAN: Domain adaptation for pneumonia diagnosis on a small chest X-ray dataset. IEEE Trans. Med. Imag. 41(11), 3278–3288 (2022)
31.
Liu, W., Ni, Z., Chen, Q., Ni, L.: Attention-guided partial domain adaptation for automated pneumonia diagnosis from chest X-ray images. IEEE J. Biomed. Health Inform. 27(12), 5848–5859 (2023)
32.
Feng, Y., Yang, X., Qiu, D., Zhang, H., Wei, D., Liu, J.: Pcxrnet: Pneumonia diagnosis from chest X-ray images using condense attention block and multiconvolution attention block. IEEE J. Biomed. Health Inform. 26(4), 1484–1495 (2022)
33.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Proc. Adv. Neural Inf. Process. Syst. (NIPS), pp. 5998–6008 (2017)
34.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:​2010.​11929 (2020)
35.
Zou, C., Wang, B., Hu, Y., Liu, J., Wu, Q., Zhao, Y., Li, B., Zhang, C., Zhang, C., Wei, Y., et al: End-to-end human object interaction detection with hoi transformer. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 11825–11834 (2021)
36.
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P., et al: Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 6881–6890 (2021)
37.
Song, D., Fu, B., Li, F., Xiong, J., He, J., Zhang, X., Qiao, Y.: Deep relation transformer for diagnosing glaucoma with optical coherence tomography and visual field function. IEEE Trans. Med. Imag. 40(9), 2392–2402 (2021)
38.
Yang, J., Deng, H., Huang, X., Ni, B., Xu, Y.: Relational learning between multiple pulmonary nodules via deep set attention transformers. In: Proc. IEEE Int. Symp. Biomed. Imaging (ISBI), pp. 1875–1878 (2020). IEEE
39.
Valanarasu, J., Oza, P., Hacihaliloglu, I., Patel, V.: Medical transformer: gated axial-attention for medical image segmentation. In: Proc. Med. Image Comput. Comput. Assist. Interv. (MICCAI), pp. 36–46 (2021)
40.
Gao, Y., Zhou, M., Metaxas, D.: UTNet: A hybrid transformer architecture for medical image segmentation. In: Proc. Med. Image Comput. Comput. Assist. Interv. (MICCAI), pp. 61–71 (2021)
41.
Zhang, Y., Higashita, R., Fu, H., Xu, Y., Zhang, Y., Liu, H., Zhang, J., Liu, J.: A multi-branch hybrid transformer networkfor corneal endothelial cell segmentation. In: Proc. Med. Image Comput. Comput. Assist. Interv. (MICCAI), pp. 99–108 (2021)
42.
Ji, Y., Zhang, R., Wang, H., Li, Z., Wu, L., Zhang, S., Luo, P.: Multi-compound transformer for accurate biomedical image segmentation. In: Proc. Med. Image Comput. Comput. Assist. Interv. (MICCAI), pp. 326–336 (2021)
43.
Signoroni, A., Savardi, M., Benini, S., Adami, N., Leonardi, R., Gibellini, P., Vaccher, F., Ravanelli, M., Borghesi, A., Maroldi, R., et al: BS-Net: learning COVID-19 pneumonia severity on a large chest x-ray dataset. Med. Image Anal. 71, 102046 (2021)
44.
Johnson, A., Pollard, T., Berkowitz, S., Greenbaum, N., Lungren, M., Deng, C., Mark, R., Horng, S.: MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Sci. Data 6(1), 1–8 (2019)
45.
46.
47.
Hui, T., Khoo, H., Young, B., Mohideen, S., Lee, Y., Lim, C., Leo, Y., Kaw, G., Lye, D., Tan, C.: Clinical utility of chest radiography for severe COVID-19. Quantitative imaging in medicine and surgery 10(7), 1540 (2020)
48.
Kermany, D., Goldbaum, M., Cai, W., Valentim, C., Liang, H., Baxter, S., McKeown, A., Yang, G., Wu, X., Yan, F., et al: Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5), 1122–1131 (2018)
Metadata
Title
TransMVAN: Multi-view Aggregation Network with Transformer for Pneumonia Diagnosis
Authors
Xiaohong Wang
Zhongkang Lu
Su Huang
Yonghan Ting
Jordan Sim Zheng Ting
Wenxiang Chen
Cher Heng Tan
Weimin Huang
Publication date
08-07-2024
Publisher
Springer International Publishing
Published in
Journal of Imaging Informatics in Medicine / Issue 1/2025
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI
https://doi.org/10.1007/s10278-024-01169-9

Other articles of this Issue 1/2025

The Use of fMRI Regional Analysis to Automatically Detect ADHD Through a 3D CNN-Based Approach

Concomitant Prediction of the Ki67 and PIT-1 Expression in Pituitary Adenoma Using Different Radiomics Models

A Cloud-Based System for Automated AI Image Analysis and Reporting

Heatmap-Based Active Shape Model for Landmark Detection in Lumbar X-ray Images

A Novel Radiology Communication Tool to Reduce Workflow Interruptions: Clinical Evaluation of RadConnect

TransDiffSeg: Transformer-Based Conditional Diffusion Segmentation Model for Abdominal Multi-Objective