Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

Open Access 01-12-2018 | Original Article

Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images

Authors: Michał Byra, Grzegorz Styczynski, Cezary Szmigielski, Piotr Kalinowski, Łukasz Michałowski, Rafał Paluszkiewicz, Bogna Ziarkiewicz-Wróblewska, Krzysztof Zieniewicz, Piotr Sobieraj, Andrzej Nowicki

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 12/2018

Abstract

Purpose

The nonalcoholic fatty liver disease is the most common liver abnormality. Up to date, liver biopsy is the reference standard for direct liver steatosis quantification in hepatic tissue samples. In this paper we propose a neural network-based approach for nonalcoholic fatty liver disease assessment in ultrasound.

Methods

We used the Inception-ResNet-v2 deep convolutional neural network pre-trained on the ImageNet dataset to extract high-level features in liver B-mode ultrasound image sequences. The steatosis level of each liver was graded by wedge biopsy. The proposed approach was compared with the hepatorenal index technique and the gray-level co-occurrence matrix algorithm. After the feature extraction, we applied the support vector machine algorithm to classify images containing fatty liver. Based on liver biopsy, the fatty liver was defined to have more than 5% of hepatocytes with steatosis. Next, we used the features and the Lasso regression method to assess the steatosis level.

Results

The area under the receiver operating characteristics curve obtained using the proposed approach was equal to 0.977, being higher than the one obtained with the hepatorenal index method, 0.959, and much higher than in the case of the gray-level co-occurrence matrix algorithm, 0.893. For regression the Spearman correlation coefficients between the steatosis level and the proposed approach, the hepatorenal index and the gray-level co-occurrence matrix algorithm were equal to 0.78, 0.80 and 0.39, respectively.

Conclusions

The proposed approach may help the sonographers automatically diagnose the amount of fat in the liver. The presented approach is efficient and in comparison with other methods does not require the sonographers to select the region of interest.

Beeman SC, Garbow JR (2018) Imaging and metabolism. Springer, New York

Adams LA, Scott Harmsen JL, Charatcharoenwitthaya P, Enders FB, Therneau T, Angulo P (2010) Nonalcoholic fatty liver disease increases risk of death among patients with diabetes: a community-based cohort study. Am J Gastroenterol 105:1567–1573CrossRef

Adams LA, Anstee QM, Tilg H, Targher G (2017) Non-alcoholic fatty liver disease and its relationship with cardiovascular disease and other extrahepatic diseases. Gut 66:1138–1153CrossRef

Tapper EB, Lok ASF (2017) Use of liver imaging and biopsy in clinical practice. N Engl J Med 377:756–768CrossRef

Luapuadat AM, Jianu IR, Ungureanu BS, Florescu LM, Gheonea DI, Sovaila S, Gheonea IA (2017) Non-invasive imaging techniques in assessing non-alcoholic fatty liver disease: a current status of available methods. J Med Life 10:19–26

Bharti P, Mittal D, Ananthasivan R (2017) Computer-aided characterization and diagnosis of diffuse liver diseases based on ultrasound imaging: a review. Ultrason Imaging 39:33–61CrossRef

Marshall RH, Eissa M, Bluth EI, Gulotta PM, Davis NK (2012) Hepatorenal index as an accurate, simple, and effective tool in screening for steatosis. Am J Roentgenol 199:997–1002CrossRef

Webb M, Yeshua H, Zelber-Sagi S, Santo E, Brazowski E, Halpern Z, Oren R (2009) Diagnostic value of a computerized hepatorenal index for sonographic quantification of liver steatosis. Am J Roentgenol 192:909–914CrossRef

Cengiz M, Senturk S, Cetin B, Bayrak AH, Bilek SU (2014) Sonographic assessment of fatty liver: intraobserver and interobserver variability. Int J Clin Exp Med 7:5453–5460PubMedPubMedCentral

10.

Strauss S, Gavish E, Gottlieb P, Katsnelson L (2007) Interobserver and intraobserver variability in the sonographic assessment of fatty liver. Am J Roentgenol 189:320–323CrossRef

11.

Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern 6:610–621CrossRef

12.

Andrade A, Silva JS, Santos J, Belo-Soares P (2012) Classifier approaches for liver steatosis using ultrasound images. Procedia Technol 5:763–770CrossRef

13.

Gaitini D, Baruch Y, Ghersin E, Veitsman E, Kerner H, Shalem B, Yaniv G, Sarfaty C, Azhari H (2004) Feasibility study of ultrasonic fatty liver biopsy: texture vs. attenuation and backscatter. Ultrasound Med Biol 30:1321–1327CrossRef

14.

Pavlopoulos S, Kyriacou E, Koutsouris D, Blekas K, Stafylopatis A, Zoumpoulis P (2000) Fuzzy neural network-based texture analysis of ultrasonic images. IEEE Eng Med Biol Mag 19:39–47CrossRef

15.

Rivas EC, Moreno F, Benitez A, Morocho V, Vanegas P, Medina R (2015) Hepatic Steatosis detection using the co-occurrence matrix in tomography and ultrasound images. In: 20th symposium on signal processing, Images and Computer Vision (STSIVA), Bogota, pp 1–7. https://doi.org/10.1109/STSIVA.2015.7330417

16.

Litjens GJS, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sanchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRef

17.

Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35:1285–1298CrossRef

18.

Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big Data 3:9CrossRef

19.

Byra M (2018) Discriminant analysis of neural style representations for breast lesion classification in ultrasound. Biocybern Biomed Eng 38:684–690CrossRef

20.

Cheng JZ, Ni D, Chou YH, Qin J, Tiu CM, Chang YC, Huang CS, Shen D, Chen CM (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 6:244–254

21.

Huynh BQ, Li H, Giger ML (2016) Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. J Med Imaging 3:034501CrossRef

22.

Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35:1299–1312CrossRef

23.

Kalinowski P, Paluszkiewicz R, Ziarkiewicz-Wroblewska B, Wroblewski T, Remiszewski P, Grodzicki M, Krawczyk M (2017) Liver function in patients with nonalcoholic fatty liver disease randomized to roux-en-y gastric bypass versus sleeve gastrectomy: a secondary analysis of a randomized clinical trial. Ann Surg 266:738–745CrossRef

24.

Kleiner DE, Brunt EM, Van Natta M, Behling C, Contos MJ, Cummings OW, Ferrell LD, Liu YC, Torbenson MS, Unalp-Arida A, Yeh M, McCullough AJ, Sanya AJ (2005) Design and validation of a histological scoring system for nonalcoholic fatty liver disease. Hepatology 41:1313–1321CrossRef

25.

Styczynski G, Milewska A, Marczewska M, Sobieraj P, Sobczynska M, Dabrowski M, Kuch-Wocial A, Szmigielski C (2016) Echocardiographic correlates of abnormal liver tests in patients with exacerbation of chronic heart failure. J Am Soc Echocardiogr 29:132–139CrossRef

26.

Clausi DA (2002) An analysis of co-occurrence texture statistics as a function of grey level quantization. Can J Remote Sens 28:45–62CrossRef

27.

Chollet F (2015) Keras. https://github.com/fchollet/keras

28.

Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the thirty-first AAAI conference on artificial intelligence, vol 4

29.

Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 248–255

30.

Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology 27:1–27CrossRef

31.

DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845CrossRef

32.

Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, Muller M (2011) proc: an open-source package for r and s + to analyze and compare roc curves. BMC Bioinform 12:17CrossRef

33.

Diedenhofen B, Musch J (2015) cocor: a comprehensive solution for the statistical comparison of correlations. PLoS ONE 10:e0131499CrossRef

34.

Meng XL, Rosenthal R, Rubin DB (1992) Comparing correlated correlation coefficients. Psychol Bull 111:172–175CrossRef

35.

Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874CrossRef

36.

Wang JH, Hung CH, Kuo FY, Eng HL, Chen CH, Lee CM, Lu SN, Hu TH (2013) Ultrasonographic quantification of hepatic–renal echogenicity difference in hepatic steatosis diagnosis. Dig Dis Sci 58:2993–3000CrossRef

37.

Ballestri S, Romagnoli D, Nascimbeni F, Francica G, Lonardo A (2015) Role of ultrasound in the diagnosis and treatment of nonalcoholic fatty liver disease and its complications. Expert Rev Gastroenterol Hepatol 9:603–627CrossRef

38.

Chauhan A, Sultan LR, Furth EE, Jones LP, Khungar V, Sehgal CM (2016) Diagnostic accuracy of hepatorenal index in the detection and grading of hepatic steatosis. J Clin Ultrasound 44:580–586CrossRef

39.

Mancini M, Prinster A, Annuzzi G, Liuzzi R, Giacco R, Medagli C, Cremone M, Clemente G, Maurea S, Riccardi G, Rivellese AA, Salvatore M (2009) Sonographic hepatic-renal ratio as indicator of hepatic steatosis: comparison with 1 h magnetic resonance spectroscopy. Metab Clin Exp 58:1724–1730CrossRef

40.

Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 427–436

41.

Oelze ML, Mamou J (2016) Review of quantitative ultrasound: envelope statistics and backscatter coefficient imaging and contributions to diagnostic ultrasound. IEEE Trans Ultrason Ferroelectr Freq Control 63:336–351CrossRef

42.

Sadeghi-Naini A, Suraweera H, Tran WT, Hadizad F, Bruni G, Rastegar RF, Curpen B, Czarnota GJ (2017) Breast-lesion characterization using textural features of quantitative ultrasound parametric maps. Sci Rep 7:13638CrossRef

43.

Byra M, Nowicki A, Wroblewska-Piotrzkowska H, Dobruch-Sobczak K (2016) Classification of breast lesions using segmented quantitative ultrasound maps of homodyned K distribution parameters. Med Phys 43:5561–5569CrossRef

44.

Lin SC, Heba E, Wolfson T, Ang B, Gamst A, Han A, Erdman JW, O’Brien WD, Andre MP, Sirlin CB, Loomba R (2015) Noninvasive diagnosis of nonalcoholic fatty liver disease and quantification of liver fat using a new quantitative ultrasound technique. Clin Gastroenterol Hepatol 13:1337–1345CrossRef

45.

Paige JS, Bernstein GS, Heba E, Costa EAC, Fereirra M, Wolfson T, Gamst AC, Valasek MA, Lin GY, Han A, Erdman JW, O’Brien WD, Andre MP, Loomba R, Sirlin CB (2017) A pilot comparative study of quantitative ultrasound, conventional ultrasound, and MRI for predicting histology determined steatosis grade in adult nonalcoholic fatty liver disease. Am J Roentgenol 208:168–177CrossRef

Title: Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images
Authors: Michał Byra
Grzegorz Styczynski
Cezary Szmigielski
Piotr Kalinowski
Łukasz Michałowski
Rafał Paluszkiewicz
Bogna Ziarkiewicz-Wróblewska
Krzysztof Zieniewicz
Piotr Sobieraj
Andrzej Nowicki
Publication date: 01-12-2018
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 12/2018
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-018-1843-2

2024 ASCO Annual Meeting

Springer Medicine

Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images

Abstract

Purpose

Methods

Results

Conclusions

2024 ASCO Annual Meeting

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 12/2018

Reliability and correlation analysis of computed methods to convert conventional 2D radiological hindfoot measurements to a 3D setting using weightbearing CT

ImFEATbox: a toolbox for extraction and analysis of medical image features

Accuracy assessment of wireless transponder tracking in the operating room environment

Can tumor coverage evaluated 24 h post-radiofrequency ablation predict local tumor progression of liver metastases?

Stereotactically navigated percutaneous microwave ablation (MWA) compared to conventional MWA: a matched pair analysis

Using the variogram for vector outlier screening: application to feature-based image registration