Skip to main content
Top
Published in: European Radiology 9/2022

22-03-2022 | Artificial Intelligence | Imaging Informatics and Artificial Intelligence

How artificial intelligence improves radiological interpretation in suspected pulmonary embolism

Authors: Alexandre Ben Cheikh, Guillaume Gorincour, Hubert Nivet, Julien May, Mylene Seux, Paul Calame, Vivien Thomson, Eric Delabrousse, Amandine Crombé

Published in: European Radiology | Issue 9/2022

Login to get access

Abstract

Objectives

To evaluate and compare the diagnostic performances of a commercialized artificial intelligence (AI) algorithm for diagnosing pulmonary embolism (PE) on CT pulmonary angiogram (CTPA) with those of emergency radiologists in routine clinical practice.

Methods

This was an IRB-approved retrospective multicentric study including patients with suspected PE from September to December 2019 (i.e., during a preliminary evaluation period of an approved AI algorithm). CTPA quality and conclusions by emergency radiologists were retrieved from radiological reports. The gold standard was a retrospective review of CTPA, radiological and clinical reports, AI outputs, and patient outcomes. Diagnostic performance metrics for AI and radiologists were assessed in the entire cohort and depending on CTPA quality.

Results

Overall, 1202 patients were included (median age: 66.2 years). PE prevalence was 15.8% (190/1202). The AI algorithm detected 219 suspicious PEs, of which 176 were true PEs, including 19 true PEs missed by radiologists. In the cohort, the highest sensitivity and negative predictive values (NPVs) were obtained with AI (92.6% versus 90% and 98.6% versus 98.1%, respectively), while the highest specificity and positive predictive value (PPV) were found with radiologists (99.1% versus 95.8% and 95% versus 80.4%, respectively). Accuracy, specificity, and PPV were significantly higher for radiologists except in subcohorts with poor-to-average injection quality. Radiologists positively evaluated the AI algorithm to improve their diagnostic comfort (55/79 [69.6%]).

Conclusion

Instead of replacing radiologists, AI for PE detection appears to be a safety net in emergency radiology practice due to high sensitivity and NPV, thereby increasing the self-confidence of radiologists.

Key Points

• Both the AI algorithm and emergency radiologists showed excellent performance in diagnosing PE on CTPA (sensitivity and specificity ≥ 90%; accuracy ≥ 95%).
• The AI algorithm for PE detection can help increase the sensitivity and NPV of emergency radiologists in clinical practice, especially in cases of poor-to-moderate injection quality.
• Emergency radiologists recommended the use of AI for PE detection in satisfaction surveys to increase their confidence and comfort in their final diagnosis.
Appendix
Available only for authorised users
Literature
1.
go back to reference Smith SB, Geske JB, Maguire JM et al (2010) Early anticoagulation is associated with reduced mortality for acute pulmonary embolism. Chest 137:1382–1390CrossRef Smith SB, Geske JB, Maguire JM et al (2010) Early anticoagulation is associated with reduced mortality for acute pulmonary embolism. Chest 137:1382–1390CrossRef
2.
go back to reference Konstantinides SV, Meyer G, Becattini C et al (2019) 2019 ESC Guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the European Respiratory Society (ERS). Eur Heart J 41:543–603 Konstantinides SV, Meyer G, Becattini C et al (2019) 2019 ESC Guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the European Respiratory Society (ERS). Eur Heart J 41:543–603
3.
go back to reference Essien EO, Rali P, Mathai SC (2019) Pulmonary Embolism. Med Clin North Am 103:549–564CrossRef Essien EO, Rali P, Mathai SC (2019) Pulmonary Embolism. Med Clin North Am 103:549–564CrossRef
4.
go back to reference Suhail Akhter M, Hamali HA, Mobarki AA et al (2021) Clinical medicine SARS-CoV-2 infection: modulator of pulmonary embolism paradigm. J Clin Med 10:1064CrossRef Suhail Akhter M, Hamali HA, Mobarki AA et al (2021) Clinical medicine SARS-CoV-2 infection: modulator of pulmonary embolism paradigm. J Clin Med 10:1064CrossRef
5.
go back to reference Barragán-Montero A, Javaid U, Valdés G et al (2021) Artificial intelligence and machine learning for medical imaging: a technology review. Phys Medica 83:242–256CrossRef Barragán-Montero A, Javaid U, Valdés G et al (2021) Artificial intelligence and machine learning for medical imaging: a technology review. Phys Medica 83:242–256CrossRef
6.
go back to reference Prevedello LM, Little KJ, Qian S, White RD (2017) Artificial intelligence in imaging 1. Radiology 285:923–931CrossRef Prevedello LM, Little KJ, Qian S, White RD (2017) Artificial intelligence in imaging 1. Radiology 285:923–931CrossRef
10.
go back to reference Winkel DJ, Heye T, Weikert TJ et al (2019) Evaluation of an AI-based detection software for acute findings in abdominal computed tomography scans: toward an automated work list prioritization of routine CT examinations. Invest Radiol 54:55–59CrossRef Winkel DJ, Heye T, Weikert TJ et al (2019) Evaluation of an AI-based detection software for acute findings in abdominal computed tomography scans: toward an automated work list prioritization of routine CT examinations. Invest Radiol 54:55–59CrossRef
13.
go back to reference Prokop M, van Everdingen W, van Rees VT et al (2020) CO-RADS: a categorical CT assessment scheme for patients suspected of having COVID-19-definition and evaluation. Radiology 296(2):E97–E104CrossRef Prokop M, van Everdingen W, van Rees VT et al (2020) CO-RADS: a categorical CT assessment scheme for patients suspected of having COVID-19-definition and evaluation. Radiology 296(2):E97–E104CrossRef
19.
go back to reference Lee CW, Seo JB, Song J et al (2011) Evaluation of computer-aided detection and dual energy software in detection of peripheral pulmonary embolism on dual-energy pulmonary CT angiography. Eur Radiol:54–62 Lee CW, Seo JB, Song J et al (2011) Evaluation of computer-aided detection and dual energy software in detection of peripheral pulmonary embolism on dual-energy pulmonary CT angiography. Eur Radiol:54–62
22.
go back to reference Pichon E, Novak CL, Kiraly AP, Naidich DP (2004) A novel method for pulmonary emboli visualization from high-resolution CT images. Proc. SPIE 5367, Medical Imaging 2004: Visualization, Image-Guided Procedures, and Display, (5 May 2004); Robert L. Galloway Jr., Editor(s). https://doi.org/10.1117/12.532892 Pichon E, Novak CL, Kiraly AP, Naidich DP (2004) A novel method for pulmonary emboli visualization from high-resolution CT images. Proc. SPIE 5367, Medical Imaging 2004: Visualization, Image-Guided Procedures, and Display, (5 May 2004); Robert L. Galloway Jr., Editor(s). https://​doi.​org/​10.​1117/​12.​532892
23.
go back to reference Liang J, Bi J (2007) Computer aided detection of pulmonary embolism with tobogganing and multiple instance classification in CT pulmonary angiography. Inf Process Med Imaging 20:630–641PubMed Liang J, Bi J (2007) Computer aided detection of pulmonary embolism with tobogganing and multiple instance classification in CT pulmonary angiography. Inf Process Med Imaging 20:630–641PubMed
24.
go back to reference Digumarthy SR, Kagay CR, Legasto AC, et al (2006) Computer-aided detection (CAD) of acute pulmonary emboli: evaluation in patients without significant pulmonary disease. In: Radiological Society of North America 2006 Scientific Assembly and Annual Meeting. Chicago IL Digumarthy SR, Kagay CR, Legasto AC, et al (2006) Computer-aided detection (CAD) of acute pulmonary emboli: evaluation in patients without significant pulmonary disease. In: Radiological Society of North America 2006 Scientific Assembly and Annual Meeting. Chicago IL
25.
go back to reference Schoepf UJ, Schneider AC, Das M et al (2007) Pulmonary embolism : computer-aided detection at multidetector row spiral computed tomography. J Thorac Imaging 22:319–323CrossRef Schoepf UJ, Schneider AC, Das M et al (2007) Pulmonary embolism : computer-aided detection at multidetector row spiral computed tomography. J Thorac Imaging 22:319–323CrossRef
26.
go back to reference Wittenberg R, Peters JF, Sonnemans JJ et al (2010) Computer-assisted detection of pulmonary embolism : evaluation of pulmonary CT angiograms performed in an on-call setting. Eur Radiol 20:801–806CrossRef Wittenberg R, Peters JF, Sonnemans JJ et al (2010) Computer-assisted detection of pulmonary embolism : evaluation of pulmonary CT angiograms performed in an on-call setting. Eur Radiol 20:801–806CrossRef
27.
go back to reference Blackmon KN, Florin C, Bogoni L et al (2011) Computer-aided detection of pulmonary embolism at CT pulmonary angiography : can it improve performance of inexperienced readers ? Eur Radiol 21:1214–1223CrossRef Blackmon KN, Florin C, Bogoni L et al (2011) Computer-aided detection of pulmonary embolism at CT pulmonary angiography : can it improve performance of inexperienced readers ? Eur Radiol 21:1214–1223CrossRef
29.
go back to reference Lahiji K, Kligerman S, Jeudy J, White C (2014) Improved Accuracy of pulmonary embolism computer-aided detection using iterative reconstruction compared with filtered back projection. AJR Am J Roentgenol:763–771 Lahiji K, Kligerman S, Jeudy J, White C (2014) Improved Accuracy of pulmonary embolism computer-aided detection using iterative reconstruction compared with filtered back projection. AJR Am J Roentgenol:763–771
30.
go back to reference Wittenberg R, Peters JF, Van Den BIAH et al (2013) Computed tomography pulmonary angiography in acute pulmonary embolism -the effect of a computer-assisted detection prototype used as a concurrent reader. J Thorac Imaging 28:315–321CrossRef Wittenberg R, Peters JF, Van Den BIAH et al (2013) Computed tomography pulmonary angiography in acute pulmonary embolism -the effect of a computer-assisted detection prototype used as a concurrent reader. J Thorac Imaging 28:315–321CrossRef
32.
go back to reference Calder KK, Herbert M, Henderson SO (2005) The mortality of untreated pulmonary embolism in emergency department patients. Ann Emerg Med 45:302–310CrossRef Calder KK, Herbert M, Henderson SO (2005) The mortality of untreated pulmonary embolism in emergency department patients. Ann Emerg Med 45:302–310CrossRef
33.
go back to reference Das M, Mühlenbruch G, Helm A et al (2008) Computer-aided detection of pulmonary embolism : influence on radiologists ’ detection performance with respect to vessel segments. Eur Radiol:1350–1355 Das M, Mühlenbruch G, Helm A et al (2008) Computer-aided detection of pulmonary embolism : influence on radiologists ’ detection performance with respect to vessel segments. Eur Radiol:1350–1355
35.
go back to reference Leithner D, Gruber-Rouh T, Beeres M et al (2018) 90-kVp low-tube-voltage CT pulmonary angiography in combination with advanced modeled iterative reconstruction algorithm: Effects on radiation dose, image quality and diagnostic accuracy for the detection of pulmonary embolism. Br J Radiol 91. https://doi.org/10.1259/bjr.20180269 Leithner D, Gruber-Rouh T, Beeres M et al (2018) 90-kVp low-tube-voltage CT pulmonary angiography in combination with advanced modeled iterative reconstruction algorithm: Effects on radiation dose, image quality and diagnostic accuracy for the detection of pulmonary embolism. Br J Radiol 91. https://​doi.​org/​10.​1259/​bjr.​20180269
37.
go back to reference Crombé A, Lecomte J-C, Banaste N et al (2021) Emergency teleradiological activity is an epidemiological estimator and predictor of the COVID-10 pandemic in mainland France. Insights Imaging 12(1):103CrossRef Crombé A, Lecomte J-C, Banaste N et al (2021) Emergency teleradiological activity is an epidemiological estimator and predictor of the COVID-10 pandemic in mainland France. Insights Imaging 12(1):103CrossRef
38.
go back to reference Grillet F, Busse-Coté A, Calame P et al (2020) COVID-19 pneumonia: microvascular disease revealed on pulmonary dual-energy computed tomography angiography. Quant Imaging Med Surg 10:1852–1862CrossRef Grillet F, Busse-Coté A, Calame P et al (2020) COVID-19 pneumonia: microvascular disease revealed on pulmonary dual-energy computed tomography angiography. Quant Imaging Med Surg 10:1852–1862CrossRef
Metadata
Title
How artificial intelligence improves radiological interpretation in suspected pulmonary embolism
Authors
Alexandre Ben Cheikh
Guillaume Gorincour
Hubert Nivet
Julien May
Mylene Seux
Paul Calame
Vivien Thomson
Eric Delabrousse
Amandine Crombé
Publication date
22-03-2022
Publisher
Springer Berlin Heidelberg
Published in
European Radiology / Issue 9/2022
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-022-08645-2

Other articles of this Issue 9/2022

European Radiology 9/2022 Go to the issue