Top

Published in:

01-12-2020 | Computed Tomography

Automated detection of pulmonary embolism in CT pulmonary angiograms using an AI-powered algorithm

Authors: Thomas Weikert, David J. Winkel, Jens Bremerich, Bram Stieltjes, Victor Parmar, Alexander W. Sauter, Gregor Sommer

Published in: European Radiology | Issue 12/2020

Abstract

Objectives

To evaluate the performance of an AI-powered algorithm for the automatic detection of pulmonary embolism (PE) on chest computed tomography pulmonary angiograms (CTPAs) on a large dataset.

Methods

We retrospectively identified all CTPAs conducted at our institution in 2017 (n = 1499). Exams with clinical questions other than PE were excluded from the analysis (n = 34). The remaining exams were classified into positive (n = 232) and negative (n = 1233) for PE based on the final written reports, which defined the reference standard. The fully anonymized 1-mm series in soft tissue reconstruction served as input for the PE detection prototype algorithm that was based on a deep convolutional neural network comprising a Resnet architecture. It was trained and validated on 28,000 CTPAs acquired at other institutions. The result series were reviewed using a web-based feedback platform. Measures of diagnostic performance were calculated on a per patient and a per finding level.

Results

The algorithm correctly identified 215 of 232 exams positive for pulmonary embolism (sensitivity 92.7%; 95% confidence interval [CI] 88.3–95.5%) and 1178 of 1233 exams negative for pulmonary embolism (specificity 95.5%; 95% CI 94.2–96.6%). On a per finding level, 1174 of 1352 findings marked as embolus by the algorithm were true emboli. Most of the false positive findings were due to contrast agent–related flow artifacts, pulmonary veins, and lymph nodes.

Conclusion

The AI prototype algorithm we tested has a high degree of diagnostic accuracy for the detection of PE on CTPAs. Sensitivity and specificity are balanced, which is a prerequisite for its clinical usefulness.

Key Points

• An AI-based prototype algorithm showed a high degree of diagnostic accuracy for the detection of pulmonary embolism on CTPAs.

• It can therefore help clinicians to automatically prioritize exams with a high suspection of pulmonary embolism and serve as secondary reading tool.

• By complementing traditional ways of worklist prioritization in radiology departments, this can speed up the diagnostic and therapeutic workup of patients with pulmonary embolism and help to avoid false negative calls.

Brown J, Shesser R (2004) Computed tomography scan use is rising faster than other investigational modalities in the emergency department evaluation of patients. Ann Emerg Med 44:S33. https://doi.org/10.1016/j.annemergmed.2004.07.109CrossRef

Kocher KE, Meurer WJ, Fazel R, Scott PA, Krumholz HM, Nallamothu BK (2011) National trends in use of computed tomography in the emergency department. Ann Emerg Med 58:452–462.e3. https://doi.org/10.1016/J.ANNEMERGMED.2011.05.020

Estrada-Y-Martin RM, Oldham SA (2011) CTPA as the gold standard for the diagnosis of pulmonary embolism. Int J Comput Assist Radiol Surg 6:557–563. https://doi.org/10.1007/s11548-010-0526-4CrossRef

Pernod G, Caterino J, Maignan M et al (2017) D-dimer use and pulmonary embolism diagnosis in emergency units: why is there such a difference in pulmonary embolism prevalence between the United States of America and countries outside USA? PLoS One 12:e0169268. https://doi.org/10.1371/journal.pone.0169268CrossRef

Smith SB, Geske JB, Maguire JM, Zane NA, Carter RE, Morgenthaler TI (2010) Early anticoagulation is associated with reduced mortality for acute pulmonary embolism. Chest 137:1382–1390

Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86:2278–2324. https://doi.org/10.1109/5.726791CrossRef

Prevedello LM, Erdal BS, Ryu JL et al (2017) Automated critical test findings identification and online notification system using artificial intelligence in imaging. Radiology 285:923–931. https://doi.org/10.1148/radiol.2017162664CrossRef

Nagel S, Sinha D, Day D et al (2017) e-ASPECTS software is non-inferior to neuroradiologists in applying the ASPECT score to computed tomography scans of acute ischemic stroke patients. Int J Stroke 12:615–622. https://doi.org/10.1177/1747493016681020CrossRef

Winkel DJ, Heye T, Weikert TJ, Boll DT, Stieltjes B (2018) Evaluation of an AI-based detection software for acute findings in abdominal computed tomography scans. Invest Radiol:1. https://doi.org/10.1097/RLI.0000000000000509

10.

Liu K, Li Q, Ma J et al (2019) Evaluating a fully automated pulmonary nodule detection approach and its impact on radiologist performance. Radiol Artif Intell 1:e180084. https://doi.org/10.1148/ryai.2019180084CrossRef

11.

Vorontsov E, Cerny M, Régnier P et al (2019) Deep learning for automated segmentation of liver lesions at CT in patients with colorectal cancer liver metastases. Radiol Artif Intell 1:180014. https://doi.org/10.1148/ryai.2019180014CrossRef

12.

Thian YL, Li Y, Jagmohan P, Sia D, Chan VE, Tan RT (2019) Convolutional neural networks for automated fracture detection and localization on wrist radiographs. Radiol Artif Intell 1:e180001. https://doi.org/10.1148/ryai.2019180001

13.

Li L, Liu Z, Huang H, Lin M, Luo D (2018) Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: comparison with the performance of double reading by radiologists. Thorac Cancer 10:1759–7714.12931. https://doi.org/10.1111/1759-7714.12931

14.

Ekert T, Krois J, Meinhold L et al (2019) Deep learning for the radiographic detection of apical lesions. J Endod. https://doi.org/10.1016/j.joen.2019.03.016

15.

Cheng C-T, Ho T-Y, Lee T-Y et al (2019) Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs. Eur Radiol. https://doi.org/10.1007/s00330-019-06167-y

16.

Ye W, Gu W, Guo X et al (2019) Detection of pulmonary ground-glass opacity based on deep learning computer artificial intelligence. Biomed Eng Online 18:6. https://doi.org/10.1186/s12938-019-0627-4CrossRef

17.

Maizlin ZV, Vos PM, Godoy MB, Cooperberg PL (2007) Computer-aided detection of pulmonary embolism on CT angiography. J Thorac Imaging 22:324–329. https://doi.org/10.1097/RTI.0b013e31815b89ca

18.

Engelke C, Schmidt S, Bakai A, Auer F, Marten K (2008) Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists. Eur Radiol 18:298–307. https://doi.org/10.1007/s00330-007-0770-3

19.

Bouma H, Sonnemans JJ, Vilanova A, Gerritsen FA (2009) Automatic detection of pulmonary embolism in CTA images. IEEE Trans Med Imaging 28:1223–1230. https://doi.org/10.1109/TMI.2009.2013618CrossRef

20.

Lee CW, Seo JB, Song J-W et al (2011) Evaluation of computer-aided detection and dual energy software in detection of peripheral pulmonary embolism on dual-energy pulmonary CT angiography. Eur Radiol 21:54–62. https://doi.org/10.1007/s00330-010-1903-7CrossRef

21.

Kligerman SJ, Lahiji K, Galvin JR, Stokum C, White CS (2014) Missed pulmonary emboli on CT angiography: assessment with pulmonary embolism–computer-aided detection. Am J Roentgenol 202:65–73. https://doi.org/10.2214/AJR.13.11049

22.

Masutani Y, MacMahon H, Doi K (2002) Computerized detection of pulmonary embolism in spiral CT angiography based on volumetric image analysis. IEEE Trans Med Imaging 21:1517–1523. https://doi.org/10.1109/TMI.2002.806586CrossRef

23.

Pichon E, Novak CL, Kiraly AP, Naidich DP (2004) A novel method for pulmonary emboli visualization from high-resolution CT images. Proceedings of the SPIE, Volume 5367, p 161-170 (2004). p 161

24.

Liang J, Bi J (2007) Computer aided detection of pulmonary embolism with tobogganing and mutiple instance classification in CT pulmonary angiography. Inf Process Med Imaging 20:630–641

25.

Zhou C, Chan H-P, Sahiner B et al (2009) Computer-aided detection of pulmonary embolism in computed tomographic pulmonary angiography (CTPA): performance evaluation with independent data sets. Med Phys 36:3385–3396. https://doi.org/10.1118/1.3157102CrossRef

26.

Özkan H, Osman O, Şahin S, Boz AF (2014) A novel method for pulmonary embolism detection in CTA images. Comput Methods Programs Biomed 113:757–766. https://doi.org/10.1016/j.cmpb.2013.12.014CrossRef

27.

Digumarthy S, Kagay C, Legasto A, Muse V, Wittram C, Shepard J (2006) Computer-aided detection (CAD) of acute pulmonary emboli: evaluation in patients without significant pulmonary disease. Radiological Society of North America 2006 Scientific Assembly and Annual Meeting, November 26 - December 1, 2006, Chicago IL

28.

Schoepf UJ, Schneider AC, Das M, Wood SA, Cheema JI, Philip Costello P (2007) Pulmonary embolism: computer-aided detection at multidetector row spiral computed tomography. J Thorac Imaging 22:319–323. https://doi.org/10.1097/RTI.0b013e31815842a9

29.

Wittenberg R, Peters JF, Sonnemans JJ, Prokop M, Schaefer-Prokop CM (2010) Computer-assisted detection of pulmonary embolism: evaluation of pulmonary CT angiograms performed in an on-call setting. Eur Radiol 20:801–806. https://doi.org/10.1007/s00330-009-1628-7

30.

Blackmon KN, Florin C, Bogoni L et al (2011) Computer-aided detection of pulmonary embolism at CT pulmonary angiography: can it improve performance of inexperienced readers? Eur Radiol 21:1214–1223. https://doi.org/10.1007/s00330-010-2050-xCrossRef

31.

Wittenberg R, Berger FH, Peters JF et al (2012) Acute pulmonary embolism: effect of a computer-assisted detection prototype on diagnosis—an observer study. Radiology 262:305–313. https://doi.org/10.1148/radiol.11110372CrossRef

32.

Sang Cheol Park SC, Chapman BE, Bin Zheng B (2011) A multistage approach to improve performance of computer-aided detection of pulmonary embolisms depicted on CT images: preliminary investigation. IEEE Trans Biomed Eng 58:1519–1527. https://doi.org/10.1109/TBME.2010.2063702CrossRef

33.

Das M, Mühlenbruch G, Helm A et al (2008) Computer-aided detection of pulmonary embolism: influence on radiologists’ detection performance with respect to vessel segments. Eur Radiol 18:1350–1355. https://doi.org/10.1007/s00330-008-0889-xCrossRef

34.

Lee G, Lee HY, Park H et al (2017) Radiomics and its emerging role in lung cancer research, imaging biomarkers and clinical management: state of the art. Eur J Radiol 86:297–307. https://doi.org/10.1016/J.EJRAD.2016.09.005CrossRef

35.

Lahiji K, Kligerman S, Jeudy J, White C (2014) Improved accuracy of pulmonary embolism computer-aided detection using iterative reconstruction compared with filtered back projection. AJR Am J Roentgenol 203:763–771. https://doi.org/10.2214/AJR.13.11838CrossRef

36.

He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. Proceedings of the IEEE Conference on Cumputer Vision and Pattern Recognition: 770-778. https://doi.org/10.1109/CVPR.2016.90

37.

Wilson EB (1927) Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 22:209. https://doi.org/10.2307/2276774CrossRef

38.

Wittenberg R, Peters JF, van den Berk IAH et al (2013) Computed tomography pulmonary angiography in acute pulmonary embolism. J Thorac Imaging 28:315–321. https://doi.org/10.1097/RTI.0b013e3182870b97CrossRef

39.

Bhargavan M, Kaye AH, Forman HP, Sunshine JH (2009) Workload of radiologists in United States in 2006-2007 and trends since 1991-1992. Radiology 252:458–467. https://doi.org/10.1148/radiol.2522081895CrossRef

40.

Tajbakhsh N, Gotway MB, Liang J (2015) Computer-aided pulmonary embolism detection using a novel vessel-aligned multiplanar image representation and convolutional neural networks. In: Navab N, Hornegger J, Wells W, Frangi A (eds) Medical image computing and computer-assisted intervention --MICCAI 2015. MICCAI 2015. Lecture notes in computer science, vol 9350. Springer, Cham

41.

Tajbakhsh N, Shin JY, Gotway MB, Liang J (2019) Computer-aided detection and visualization of pulmonary embolism using a novel, compact, and discriminative image representation. Med Image Anal 58:101541. https://doi.org/10.1016/J.MEDIA.2019.101541CrossRef

42.

Beydilli İ, Yılmaz F, Sönmez BM et al (2016) Thrombolytic therapy delay is independent predictor of mortality in acute pulmonary embolism at emergency service. Kaohsiung J Med Sci 32:572–578. https://doi.org/10.1016/j.kjms.2016.09.004CrossRef

43.

Annarumma M, Withey SJ, Bakewell RJ, Pesce E, Goh V, Montana G (2019) Automated triaging of adult chest radiographs with deep artificial neural networks. Radiology:291

44.

Rayan JC, Reddy N, Kan JH, Zhang W, Annapragada A (2019) Binomial classification of pediatric elbow fractures using a deep learning multiview approach emulating radiologist decision making. Radiol Artif Intell 1:e180015. https://doi.org/10.1148/ryai.2019180015

45.

Penaloza A, Kline J, Verschuren F et al (2012) European and American suspected and confirmed pulmonary embolism populations: comparison and analysis. J Thromb Haemost 10:375–381. https://doi.org/10.1111/j.1538-7836.2012.04631.xCrossRef

Title: Automated detection of pulmonary embolism in CT pulmonary angiograms using an AI-powered algorithm
Authors: Thomas Weikert
David J. Winkel
Jens Bremerich
Bram Stieltjes
Victor Parmar
Alexander W. Sauter
Gregor Sommer
Publication date: 01-12-2020
Publisher: Springer Berlin Heidelberg
Published in: European Radiology / Issue 12/2020
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI: https://doi.org/10.1007/s00330-020-06998-0

At a glance: The STEP trials

Springer Medicine

Automated detection of pulmonary embolism in CT pulmonary angiograms using an AI-powered algorithm

Abstract

Objectives

Methods

Results

Conclusion

Key Points

At a glance: The STEP trials

Springer Medicine

Abstract

Objectives

Methods

Results

Conclusion

Key Points

Please log in to get access to this content

Other articles of this Issue 12/2020

COVID-19 impact assessment on the French radiological centers: a nationwide survey

MRI in acute muscle tears in athletes: can quantitative T2 and DTI predict return to play better than visual assessment?

Longitudinal study of interventional radiology activity in a large metropolitan Italian tertiary care hospital: how the COVID-19 pandemic emergency has changed our activity

Feed-forward neural networks using cerebral MR spectroscopy and DTI might predict neurodevelopmental outcome in preterm neonates

Machine learning–based CT texture analysis to predict HPV status in oropharyngeal squamous cell carcinoma: comparison of 2D and 3D segmentation

Emphysema quantification using low-dose computed tomography with deep learning–based kernel conversion comparison