Skip to main content
Top
Published in: BMC Medical Imaging 1/2022

Open Access 01-12-2022 | Computed Tomography | Research

Development and validation of an ensemble artificial intelligence model for comprehensive imaging quality check to classify body parts and contrast enhancement

Authors: Seongwon Na, Yu Sub Sung, Yousun Ko, Youngbin Shin, Junghyun Lee, Jiyeon Ha, Su Jung Ham, Kyoungro Yoon, Kyung Won Kim

Published in: BMC Medical Imaging | Issue 1/2022

Login to get access

Abstract

Background

Despite the dramatic increase in the use of medical imaging in various therapeutic fields of clinical trials, the first step of image quality check (image QC), which aims to check whether images are uploaded appropriately according to the predefined rules, is still performed manually by image analysts, which requires a lot of manpower and time.

Methods

In this retrospective study, 1669 computed tomography (CT) images with five specific anatomical locations were collected from Asan Medical Center and Kangdong Sacred Heart Hospital. To generate the ground truth, two radiologists reviewed the anatomical locations and presence of contrast enhancement using the collected data. The individual deep learning model is developed through InceptionResNetv2 and transfer learning, and we propose Image Quality Check-Net (Image QC-Net), an ensemble AI model that utilizes it. To evaluate their clinical effectiveness, the overall accuracy and time spent on image quality check of a conventional model and ImageQC-net were compared.

Results

ImageQC-net body part classification showed excellent performance in both internal (precision, 100%; recall, 100% accuracy, 100%) and external verification sets (precision, 99.8%; recovery rate, 99.8%, accuracy, 99.8%). In addition, contrast enhancement classification performance achieved 100% precision, recall, and accuracy in the internal verification set and achieved (precision, 100%; recall, 100%; accuracy 100%) in the external dataset. In the case of clinical effects, the reduction of time by checking the quality of artificial intelligence (AI) support by analysts 1 and 2 (49.7% and 48.3%, respectively) was statistically significant (p < 0.001).

Conclusions

Comprehensive AI techniques to identify body parts and contrast enhancement on CT images are highly accurate and can significantly reduce the time spent on image quality checks.
Appendix
Available only for authorised users
Literature
1.
go back to reference Lee AJ, Kim KW, Shin Y, Lee J, Park HJ, Cho YC, Ko Y, Sung YS, Yoon BS. CDISC-compliant clinical trial imaging management system with automatic verification and data transformation: focusing on tumor response assessment data in clinical trials. J Biomed Inform. 2021;117: 103782.CrossRef Lee AJ, Kim KW, Shin Y, Lee J, Park HJ, Cho YC, Ko Y, Sung YS, Yoon BS. CDISC-compliant clinical trial imaging management system with automatic verification and data transformation: focusing on tumor response assessment data in clinical trials. J Biomed Inform. 2021;117: 103782.CrossRef
2.
go back to reference Park HY, Kim KW, Yoon MA, Lee MH, Chae EJ, Lee JH, Chung HW, Yoon DH. Role of whole-body MRI for treatment response assessment in multiple myeloma: comparison between clinical response and imaging response. Cancer Imaging. 2020;20(1):14.CrossRef Park HY, Kim KW, Yoon MA, Lee MH, Chae EJ, Lee JH, Chung HW, Yoon DH. Role of whole-body MRI for treatment response assessment in multiple myeloma: comparison between clinical response and imaging response. Cancer Imaging. 2020;20(1):14.CrossRef
3.
go back to reference Yankeelov TE, Mankoff DA, Schwartz LH, Lieberman FS, Buatti JM, Mountz JM, Erickson BJ, Fennessy FM, Huang W, Kalpathy-Cramer J, et al. Quantitative Imaging in Cancer Clinical Trials. Clin Cancer Res. 2016;22(2):284–90.CrossRef Yankeelov TE, Mankoff DA, Schwartz LH, Lieberman FS, Buatti JM, Mountz JM, Erickson BJ, Fennessy FM, Huang W, Kalpathy-Cramer J, et al. Quantitative Imaging in Cancer Clinical Trials. Clin Cancer Res. 2016;22(2):284–90.CrossRef
4.
go back to reference US, Food, and, Drug, Administration: clinical trial imaging endpoint process standards guidance for industry; 2018. US, Food, and, Drug, Administration: clinical trial imaging endpoint process standards guidance for industry; 2018.
5.
go back to reference Gierada DS, Garg K, Nath H, Strollo DC, Fagerstrom RM, Ford MB. CT quality assurance in the lung screening study component of the National Lung Screening Trial: implications for multicenter imaging trials. AJR Am J Roentgenol. 2009;193(2):419–24.CrossRef Gierada DS, Garg K, Nath H, Strollo DC, Fagerstrom RM, Ford MB. CT quality assurance in the lung screening study component of the National Lung Screening Trial: implications for multicenter imaging trials. AJR Am J Roentgenol. 2009;193(2):419–24.CrossRef
6.
go back to reference Kim DW, Jang HY, Kim KW, Shin Y, Park SH. Design characteristics of studies reporting the performance of artificial intelligence algorithms for diagnostic analysis of medical images: results from recently published papers. Korean J Radiol. 2019;20(3):405–10.CrossRef Kim DW, Jang HY, Kim KW, Shin Y, Park SH. Design characteristics of studies reporting the performance of artificial intelligence algorithms for diagnostic analysis of medical images: results from recently published papers. Korean J Radiol. 2019;20(3):405–10.CrossRef
7.
go back to reference Philbrick KA, Yoshida K, Inoue D, Akkus Z, Kline TL, Weston AD, Korfiatis P, Takahashi N, Erickson BJ. What does deep learning see? Insights from a classifier trained to predict contrast enhancement phase from CT images. Am J Roentgenol. 2018;211(6):1184–93.CrossRef Philbrick KA, Yoshida K, Inoue D, Akkus Z, Kline TL, Weston AD, Korfiatis P, Takahashi N, Erickson BJ. What does deep learning see? Insights from a classifier trained to predict contrast enhancement phase from CT images. Am J Roentgenol. 2018;211(6):1184–93.CrossRef
8.
go back to reference Sugimori H. Classification of computed tomography images in different slice positions using deep learning. J Healthc Eng. 2018;2018:1–9.CrossRef Sugimori H. Classification of computed tomography images in different slice positions using deep learning. J Healthc Eng. 2018;2018:1–9.CrossRef
9.
go back to reference Liu Y, Chen PC, Krause J, Peng L. How to read articles that use machine learning: users’ guides to the medical literature. JAMA. 2019;322(18):1806–16.CrossRef Liu Y, Chen PC, Krause J, Peng L. How to read articles that use machine learning: users’ guides to the medical literature. JAMA. 2019;322(18):1806–16.CrossRef
10.
go back to reference Sato Y, Shiraga N, Nakajima S, Tamura S, Kikinis R. Local maximum intensity projection (LMIP: a new rendering method for vascular visualization. J Comput Assist Tomogr. 1998;22(6):912–7.CrossRef Sato Y, Shiraga N, Nakajima S, Tamura S, Kikinis R. Local maximum intensity projection (LMIP: a new rendering method for vascular visualization. J Comput Assist Tomogr. 1998;22(6):912–7.CrossRef
11.
go back to reference Park HJ, Shin Y, Park J, Kim H, Lee IS, Seo D-W, Huh J, Lee TY, Park T, Lee J, et al. Development and validation of a deep learning system for segmentation of abdominal muscle and fat on computed tomography. Korean J Radiol. 2020;21(1):88–100.CrossRef Park HJ, Shin Y, Park J, Kim H, Lee IS, Seo D-W, Huh J, Lee TY, Park T, Lee J, et al. Development and validation of a deep learning system for segmentation of abdominal muscle and fat on computed tomography. Korean J Radiol. 2020;21(1):88–100.CrossRef
12.
go back to reference Schneider U, Pedroni E, Lomax A. The calibration of CT Hounsfield units for radiotherapy treatment planning. Phys Med Biol. 1996;41(1):111–24.CrossRef Schneider U, Pedroni E, Lomax A. The calibration of CT Hounsfield units for radiotherapy treatment planning. Phys Med Biol. 1996;41(1):111–24.CrossRef
13.
go back to reference Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the AAAI conference on artificial intelligence; 2017. Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the AAAI conference on artificial intelligence; 2017.
14.
go back to reference Shin Y, Kim KW, Lee AJ, Sung YS, Ahn S, Koo JH, Choi CG, Ko Y, Kim HS, Park SH. A good practice-compliant clinical trial imaging management system for multicenter clinical trials: development and validation study. JMIR Med Inform. 2019;7(3): e14310.CrossRef Shin Y, Kim KW, Lee AJ, Sung YS, Ahn S, Koo JH, Choi CG, Ko Y, Kim HS, Park SH. A good practice-compliant clinical trial imaging management system for multicenter clinical trials: development and validation study. JMIR Med Inform. 2019;7(3): e14310.CrossRef
15.
go back to reference Kornblith S, Shlens J, Le QV. Do better imagenet models transfer better? In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2019. p. 2661–71. Kornblith S, Shlens J, Le QV. Do better imagenet models transfer better? In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2019. p. 2661–71.
Metadata
Title
Development and validation of an ensemble artificial intelligence model for comprehensive imaging quality check to classify body parts and contrast enhancement
Authors
Seongwon Na
Yu Sub Sung
Yousun Ko
Youngbin Shin
Junghyun Lee
Jiyeon Ha
Su Jung Ham
Kyoungro Yoon
Kyung Won Kim
Publication date
01-12-2022
Publisher
BioMed Central
Published in
BMC Medical Imaging / Issue 1/2022
Electronic ISSN: 1471-2342
DOI
https://doi.org/10.1186/s12880-022-00815-4

Other articles of this Issue 1/2022

BMC Medical Imaging 1/2022 Go to the issue