Skip to main content
Top
Published in: Insights into Imaging 1/2021

Open Access 01-12-2021 | Magnetic Resonance Imaging | Original Article

Saliency-based 3D convolutional neural network for categorising common focal liver lesions on multisequence MRI

Authors: Shu-Hui Wang, Xin-Jun Han, Jing Du, Zhen-Chang Wang, Chunwang Yuan, Yinan Chen, Yajing Zhu, Xin Dou, Xiao-Wei Xu, Hui Xu, Zheng-Han Yang

Published in: Insights into Imaging | Issue 1/2021

Login to get access

Abstract

Background

The imaging features of focal liver lesions (FLLs) are diverse and complex. Diagnosing FLLs with imaging alone remains challenging. We developed and validated an interpretable deep learning model for the classification of seven categories of FLLs on multisequence MRI and compared the differential diagnosis between the proposed model and radiologists.

Methods

In all, 557 lesions examined by multisequence MRI were utilised in this retrospective study and divided into training–validation (n = 444) and test (n = 113) datasets. The area under the receiver operating characteristic curve (AUC) was calculated to evaluate the performance of the model. The accuracy and confusion matrix of the model and individual radiologists were compared. Saliency maps were generated to highlight the activation region based on the model perspective.

Results

The AUC of the two- and seven-way classifications of the model were 0.969 (95% CI 0.944–0.994) and from 0.919 (95% CI 0.857–0.980) to 0.999 (95% CI 0.996–1.000), respectively. The model accuracy (79.6%) of the seven-way classification was higher than that of the radiology residents (66.4%, p = 0.035) and general radiologists (73.5%, p = 0.346) but lower than that of the academic radiologists (85.4%, p = 0.291). Confusion matrices showed the sources of diagnostic errors for the model and individual radiologists for each disease. Saliency maps detected the activation regions associated with each predicted class.

Conclusion

This interpretable deep learning model showed high diagnostic performance in the differentiation of FLLs on multisequence MRI. The analysis principle contributing to the predictions can be explained via saliency maps.
Appendix
Available only for authorised users
Literature
1.
go back to reference Matos AP, Velloni F, Ramalho M, AlObaidy M, Rajapaksha A, Semelka RC (2015) Focal liver lesions: practical magnetic resonance imaging approach. World J Hepatol 7:1987–2008CrossRef Matos AP, Velloni F, Ramalho M, AlObaidy M, Rajapaksha A, Semelka RC (2015) Focal liver lesions: practical magnetic resonance imaging approach. World J Hepatol 7:1987–2008CrossRef
2.
go back to reference van den Bos IC, Hussain SM, de Man RA et al (2008) Magnetic resonance imaging of liver lesions: exceptions and atypical lesions. Curr Probl Diagn Radiol 37:95–103CrossRef van den Bos IC, Hussain SM, de Man RA et al (2008) Magnetic resonance imaging of liver lesions: exceptions and atypical lesions. Curr Probl Diagn Radiol 37:95–103CrossRef
3.
go back to reference Strassburg CP, Manns MP (2006) Approaches to liver biopsy techniques–revisited. Semin Liver Dis 26:318–327CrossRef Strassburg CP, Manns MP (2006) Approaches to liver biopsy techniques–revisited. Semin Liver Dis 26:318–327CrossRef
4.
go back to reference Zhou LQ, Wang JY, Yu SY et al (2019) Artificial intelligence in medical imaging of the liver. World J Gastroenterol 25:672–682CrossRef Zhou LQ, Wang JY, Yu SY et al (2019) Artificial intelligence in medical imaging of the liver. World J Gastroenterol 25:672–682CrossRef
5.
go back to reference Jacobson FL (2020) Medical image perception research in the emerging age of artificial intelligence. Radiology 294:210–211CrossRef Jacobson FL (2020) Medical image perception research in the emerging age of artificial intelligence. Radiology 294:210–211CrossRef
6.
go back to reference To MNN, Vu DQ, Turkbey B, Choyke PL, Kwak JT (2018) Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging. Int J Comput Assist Radiol Surg 13:1687–1696CrossRef To MNN, Vu DQ, Turkbey B, Choyke PL, Kwak JT (2018) Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging. Int J Comput Assist Radiol Surg 13:1687–1696CrossRef
7.
8.
go back to reference Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK (2018) Medical image analysis using convolutional neural networks: a review. J Med Syst 42:226CrossRef Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK (2018) Medical image analysis using convolutional neural networks: a review. J Med Syst 42:226CrossRef
9.
go back to reference Zhen SH, Cheng M, Tao YB et al (2020) deep learning for accurate diagnosis of liver tumor based on magnetic resonance imaging and clinical data. Front Oncol 10:680CrossRef Zhen SH, Cheng M, Tao YB et al (2020) deep learning for accurate diagnosis of liver tumor based on magnetic resonance imaging and clinical data. Front Oncol 10:680CrossRef
10.
go back to reference Yasaka K, Akai H, Abe O, Kiryu S (2018) Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286:887–896CrossRef Yasaka K, Akai H, Abe O, Kiryu S (2018) Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286:887–896CrossRef
11.
go back to reference Zhou J, Wang W, Lei B et al (2020) Automatic detection and classification of focal liver lesions based on deep convolutional neural networks: a preliminary study. Front Oncol 10:581210CrossRef Zhou J, Wang W, Lei B et al (2020) Automatic detection and classification of focal liver lesions based on deep convolutional neural networks: a preliminary study. Front Oncol 10:581210CrossRef
12.
go back to reference Ortiz-Ramon R, Larroza A, Arana E, Moratal D (2017) A radiomics evaluation of 2D and 3D MRI texture features to classify brain metastases from lung cancer and melanoma. Annu Int Conf IEEE Eng Med Biol Soc 2017:493–496PubMed Ortiz-Ramon R, Larroza A, Arana E, Moratal D (2017) A radiomics evaluation of 2D and 3D MRI texture features to classify brain metastases from lung cancer and melanoma. Annu Int Conf IEEE Eng Med Biol Soc 2017:493–496PubMed
13.
go back to reference Depeursinge A, Foncubierta-Rodriguez A, Van De Ville D, Muller H (2014) Three-dimensional solid texture analysis in biomedical imaging: review and opportunities. Med Image Anal 18:176–196CrossRef Depeursinge A, Foncubierta-Rodriguez A, Van De Ville D, Muller H (2014) Three-dimensional solid texture analysis in biomedical imaging: review and opportunities. Med Image Anal 18:176–196CrossRef
14.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Las Vegas, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Las Vegas, pp 770–778
15.
go back to reference Erkan B, Meier J, Clark TJ, Kaplan J, Lambert JR, Chang S (2019) Non-invasive diagnostic criteria of hepatocellular carcinoma: comparison of diagnostic accuracy of updated LI-RADS with clinical practice guidelines of OPTN-UNOS, AASLD, NCCN, EASL-EORTC, and KLSCG-NCC. PLoS One 14:e0226291CrossRef Erkan B, Meier J, Clark TJ, Kaplan J, Lambert JR, Chang S (2019) Non-invasive diagnostic criteria of hepatocellular carcinoma: comparison of diagnostic accuracy of updated LI-RADS with clinical practice guidelines of OPTN-UNOS, AASLD, NCCN, EASL-EORTC, and KLSCG-NCC. PLoS One 14:e0226291CrossRef
16.
go back to reference Jiang HY, Chen J, Xia CC, Cao LK, Duan T, Song B (2018) Noninvasive imaging of hepatocellular carcinoma: from diagnosis to prognosis. World J Gastroenterol 24:2348–2362CrossRef Jiang HY, Chen J, Xia CC, Cao LK, Duan T, Song B (2018) Noninvasive imaging of hepatocellular carcinoma: from diagnosis to prognosis. World J Gastroenterol 24:2348–2362CrossRef
17.
go back to reference Mitchell DG, Bruix J, Sherman M, Sirlin CB (2015) LI-RADS (liver imaging reporting and data system): summary, discussion, and consensus of the LI-RADS management working group and future directions. Hepatology 61:1056–1065CrossRef Mitchell DG, Bruix J, Sherman M, Sirlin CB (2015) LI-RADS (liver imaging reporting and data system): summary, discussion, and consensus of the LI-RADS management working group and future directions. Hepatology 61:1056–1065CrossRef
18.
go back to reference Roberts LR, Sirlin CB, Zaiem F et al (2018) Imaging for the diagnosis of hepatocellular carcinoma: a systematic review and meta-analysis. Hepatology 67:401–421CrossRef Roberts LR, Sirlin CB, Zaiem F et al (2018) Imaging for the diagnosis of hepatocellular carcinoma: a systematic review and meta-analysis. Hepatology 67:401–421CrossRef
19.
go back to reference Burns PN, Wilson SR (2007) Focal liver masses: enhancement patterns on contrast-enhanced images-concordance of US scans with CT scans and MR images. Radiology 242:162–174CrossRef Burns PN, Wilson SR (2007) Focal liver masses: enhancement patterns on contrast-enhanced images-concordance of US scans with CT scans and MR images. Radiology 242:162–174CrossRef
20.
go back to reference Elsayes K, Hooker J, Agrons M et al (2017) 2017 version of LI-RADS for CT and MR imaging: an update. Radiographics 37:1994–2017CrossRef Elsayes K, Hooker J, Agrons M et al (2017) 2017 version of LI-RADS for CT and MR imaging: an update. Radiographics 37:1994–2017CrossRef
21.
go back to reference Barth BK, Donati OF, Fischer MA et al (2016) Reliability, validity, and reader acceptance of LI-RADS-an in-depth analysis. Acad Radiol 23:1145–1153CrossRef Barth BK, Donati OF, Fischer MA et al (2016) Reliability, validity, and reader acceptance of LI-RADS-an in-depth analysis. Acad Radiol 23:1145–1153CrossRef
22.
go back to reference Ehman EC, Behr SC, Umetsu SE et al (2016) Rate of observation and inter-observer agreement for LI-RADS major features at CT and MRI in 184 pathology proven hepatocellular carcinomas. Abdom Radiol (NY) 41:963–969CrossRef Ehman EC, Behr SC, Umetsu SE et al (2016) Rate of observation and inter-observer agreement for LI-RADS major features at CT and MRI in 184 pathology proven hepatocellular carcinomas. Abdom Radiol (NY) 41:963–969CrossRef
23.
go back to reference Becker AS, Barth BK, Marquez PH et al (2017) Increased interreader agreement in diagnosis of hepatocellular carcinoma using an adapted LI-RADS algorithm. Eur J Radiol 86:33–40CrossRef Becker AS, Barth BK, Marquez PH et al (2017) Increased interreader agreement in diagnosis of hepatocellular carcinoma using an adapted LI-RADS algorithm. Eur J Radiol 86:33–40CrossRef
24.
go back to reference Kierans AS, Makkar J, Guniganti P et al (2019) Validation of liver imaging reporting and data system 2017 (LI-RADS) criteria for imaging diagnosis of hepatocellular carcinoma. J Magn Reson Imaging 49:e205–e215CrossRef Kierans AS, Makkar J, Guniganti P et al (2019) Validation of liver imaging reporting and data system 2017 (LI-RADS) criteria for imaging diagnosis of hepatocellular carcinoma. J Magn Reson Imaging 49:e205–e215CrossRef
25.
go back to reference Park HJ, Kim SH, Jang KM, Lee SJ, Park MJ, Choi D (2013) Differentiating hepatic abscess from malignant mimickers: value of diffusion-weighted imaging with an emphasis on the periphery of the lesion. J Magn Reson Imaging 38:1333–1341CrossRef Park HJ, Kim SH, Jang KM, Lee SJ, Park MJ, Choi D (2013) Differentiating hepatic abscess from malignant mimickers: value of diffusion-weighted imaging with an emphasis on the periphery of the lesion. J Magn Reson Imaging 38:1333–1341CrossRef
26.
go back to reference Dănilă M, Popescu A, Sirli R, Sporea I, Martie A, Sendroiu M (2010) Contrast enhanced ultrasound (CEUS) in the evaluation of liver metastases. Med Ultrason 12:233–237PubMed Dănilă M, Popescu A, Sirli R, Sporea I, Martie A, Sendroiu M (2010) Contrast enhanced ultrasound (CEUS) in the evaluation of liver metastases. Med Ultrason 12:233–237PubMed
27.
go back to reference Kong WT, Ji ZB, Wang WP, Cai H, Huang BJ, Ding H (2016) Evaluation of liver metastases using contrast-enhanced ultrasound: enhancement patterns and influencing factors. Gut Liver 10:283–287CrossRef Kong WT, Ji ZB, Wang WP, Cai H, Huang BJ, Ding H (2016) Evaluation of liver metastases using contrast-enhanced ultrasound: enhancement patterns and influencing factors. Gut Liver 10:283–287CrossRef
28.
go back to reference Mantatzis M, Kakolyris S, Amarantidis K, Karayiannakis A, Prassopoulos P (2009) Treatment response classification of liver metastatic disease evaluated on imaging. Are RECIST unidimensional measurements accurate? Eur Radiol 19:1809–1816CrossRef Mantatzis M, Kakolyris S, Amarantidis K, Karayiannakis A, Prassopoulos P (2009) Treatment response classification of liver metastatic disease evaluated on imaging. Are RECIST unidimensional measurements accurate? Eur Radiol 19:1809–1816CrossRef
29.
go back to reference Chapiro J, Lin M, Duran R, Schernthaner RE, Geschwind JF (2015) Assessing tumor response after loco-regional liver cancer therapies: the role of 3D MRI. Expert Rev Anticancer Ther 15:199–205CrossRef Chapiro J, Lin M, Duran R, Schernthaner RE, Geschwind JF (2015) Assessing tumor response after loco-regional liver cancer therapies: the role of 3D MRI. Expert Rev Anticancer Ther 15:199–205CrossRef
30.
go back to reference Chapiro J, Wood LD, Lin M et al (2014) Radiologic-pathologic analysis of contrast-enhanced and diffusion-weighted MR imaging in patients with HCC after TACE: diagnostic accuracy of 3D quantitative image analysis. Radiology 273:746–758CrossRef Chapiro J, Wood LD, Lin M et al (2014) Radiologic-pathologic analysis of contrast-enhanced and diffusion-weighted MR imaging in patients with HCC after TACE: diagnostic accuracy of 3D quantitative image analysis. Radiology 273:746–758CrossRef
31.
go back to reference Fowler KJ, Brown JJ, Narra VR (2011) Magnetic resonance imaging of focal liver lesions: approach to imaging diagnosis. Hepatology 54:2227–2237CrossRef Fowler KJ, Brown JJ, Narra VR (2011) Magnetic resonance imaging of focal liver lesions: approach to imaging diagnosis. Hepatology 54:2227–2237CrossRef
32.
go back to reference Donato H, Franca M, Candelaria I, Caseiro-Alves F (2017) Liver MRI: from basic protocol to advanced techniques. Eur J Radiol 93:30–39CrossRef Donato H, Franca M, Candelaria I, Caseiro-Alves F (2017) Liver MRI: from basic protocol to advanced techniques. Eur J Radiol 93:30–39CrossRef
33.
go back to reference Qayyum A (2009) Diffusion-weighted imaging in the abdomen and pelvis: concepts and applications. Radiographics 29:1797–1810CrossRef Qayyum A (2009) Diffusion-weighted imaging in the abdomen and pelvis: concepts and applications. Radiographics 29:1797–1810CrossRef
34.
go back to reference Lamba R, Fananapazir G, Corwin MT, Khatri VP (2014) Diagnostic imaging of hepatic lesions in adults. Surg Oncol Clin N Am 23:789–820CrossRef Lamba R, Fananapazir G, Corwin MT, Khatri VP (2014) Diagnostic imaging of hepatic lesions in adults. Surg Oncol Clin N Am 23:789–820CrossRef
35.
go back to reference Bluemke DA, Moy L, Bredella MA et al (2020) Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers-from the radiology editorial board. Radiology 294:487–489CrossRef Bluemke DA, Moy L, Bredella MA et al (2020) Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers-from the radiology editorial board. Radiology 294:487–489CrossRef
36.
go back to reference Wong SH, Al-Hasani H, Alam Z, Alam A (2019) Artificial intelligence in radiology: How will we be affected? Eur Radiol 29:141–143CrossRef Wong SH, Al-Hasani H, Alam Z, Alam A (2019) Artificial intelligence in radiology: How will we be affected? Eur Radiol 29:141–143CrossRef
Metadata
Title
Saliency-based 3D convolutional neural network for categorising common focal liver lesions on multisequence MRI
Authors
Shu-Hui Wang
Xin-Jun Han
Jing Du
Zhen-Chang Wang
Chunwang Yuan
Yinan Chen
Yajing Zhu
Xin Dou
Xiao-Wei Xu
Hui Xu
Zheng-Han Yang
Publication date
01-12-2021
Publisher
Springer International Publishing
Published in
Insights into Imaging / Issue 1/2021
Electronic ISSN: 1869-4101
DOI
https://doi.org/10.1186/s13244-021-01117-z

Other articles of this Issue 1/2021

Insights into Imaging 1/2021 Go to the issue