Top

Published in:

01-02-2008 | Chest

Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists

Authors: Christoph Engelke, Stephan Schmidt, Annemarie Bakai, Florian Auer, Katharina Marten

Published in: European Radiology | Issue 2/2008

Abstract

The value of a computer-aided detection tool (CAD) as second reader in combination with experienced and inexperienced radiologists for the diagnosis of acute pulmonary embolism (PE) was assessed prospectively. Computed tomographic angiography (CTA) scans (64 × 0.6 mm collimation; 61.4 mm/rot table feed) of 56 patients (31 women, 34–89 years, mean = 66 years) with suspected PE were analysed by two experienced (R1, R2) and two inexperienced (R3, R4) radiologists for the presence and distribution of emboli using a five-point confidence rating, and by CAD. Informed consent was obtained from all patients. Results were compared with an independent reference standard. Inter-observer agreement was calculated by kappa, confidence assessed by ROC analysis. A total of 1,116 emboli [within mediastinal (n = 72), lobar (n = 133), segmental (n = 465) and subsegmental arteries (n = 455)] were included. CAD detected 343 emboli (sensitivity = 30.74%, correct-positive rate = 6.13/patient; false-positive rate = 4.1/patient). Inter-observer agreement was good (R1, R2: κ = 0.84, 95% CI = 0.81–0.87; R3, R4: κ = 0.79, 95% CI = 0.76–0.81). Extended inter-observer agreement was higher in mediastinal and lobar than in segmental and subsegmental arteries (κ = 0.84–0.86 and κ = 0.51–0.58 for mediastinal/lobar and segmental/subsegmental arteries, respectively P < 0.05). Agreement between experienced and inexperienced readers was improved by CAD (κ = 0.60–0.62 and κ = 0.69–0.72 before and after CAD consensus, respectively P < 0.05). The experienced outperformed the inexperienced readers (Az = 0.95, 0.93, 0.89 and 0.86 for R1–4, respectively, P < 0.05). CAD significantly improved overall performances of readers 3 and 4 (Az = 0.86 for R3, R4 and Az = 0.89 for R3, R4 with CAD, P < 0.05), by enhancing sensitivities in segmental/subsegmental arteries. CAD improved experienced readers’ sensitivities in segmental/subsegmental arteries (sens. = 0.93 and 0.90 for R1, R2 before and 0.97 and 0.94 for R1, R2 after CAD consensus, P < 0.05), without significant improvement of their overall performances (P > 0.05). Particularly inexperienced readers benefit from consensus with CAD data, greatly improving detection of segmental and subsegmental emboli. This system is advocated as a second reader.

Dalen J, Alpert J (1975) Natural history of pulmonary embolism. Progr Cardiovasc Dis 17:257–270CrossRef

The PIOPED Investigators (1990) Value of the ventilation/perfusion scan in acute pulmonary embolism. Results of the Prospective Investigation of Pulmonary Embolism Diagnosis (PIOPED). JAMA 263:2753–2759CrossRef

Price D (1976) Pulmonary embolism. Prophylaxis diagnosis and treatment. Anaesthesia 31:925–932PubMedCrossRef

Engelke C, Rummeny EJ, Marten K (2006) Pulmonary embolism at multi-detector row CT of chest: one-year survival of treated and untreated patients. Radiology 239:563–575PubMedCrossRef

Stein PD, Fowler SE, Goodman LR et al (2006) Multidetector computed tomography for acute pulmonary embolism. N Engl J Med 354:2317–2327PubMedCrossRef

Remy-Jardin M, Remy J, Wattinne L, Giraud F (1992) Central pulmonary thromboembolism diagnosis with spiral volumetric CT with the single-breath-hold technique-comparison with pulmonary angiography. Radiology 185:381–387PubMed

McCollough C, Zink F (1999) Performance evaluation of a multi-slice CT system. Med Phys 26:2223–2230PubMedCrossRef

Rubin GD, Paik DS, Johnston PC, Napel S (1998) Measurements of the aorta and its branches with helical CT. Radiology 206:823–829PubMed

Stein PD (1999) Reassessment of pulmonary angiography for the diagnosis of pulmonary embolism: relation of interpreter agreement to the order of the involved pulmonary arterial branch. Radiology 210:689–691PubMed

10.

Ghaye B, Szapiro D, Mastora I et al (2001) Peripheral pulmonary arteries: how far in the lung does multi-detector row spiral CT allow analysis? Radiology 219:629–636PubMed

11.

Raptopoulos V, Boiselle P (2001) Multi-detector row spiral CT pulmonary angiography comparison with single-detector row spiral CT. Radiology 221:606–613PubMedCrossRef

12.

Schoepf U, Holzknecht M, Helmberger TK et al (2002) Subsegmental pulmonary emboli improved detection with thin-collimation multi-detector row spiral CT. Radiology 222:482–490CrossRef

13.

Diffin D, Leyendecker J, Johnson S et al (1998) Effect of anatomic distribution of pulmonary emboli on interobserver agreement in the interpretation of pulmonary angiography, AJR Am J Roentgenol 171:1085–1089PubMed

14.

Mayo J, Remy-Jardin M, Muller N (1997) Pulmonary embolism: prospective comparison of spiral CT with ventilation perfusion scintigraphy. Radiology 205:447–452PubMed

15.

Qanadli S, Hajjam ME, Mesurille B (2000) Pulmonary embolism detection: prospective evaluation of dual-section helical CT versus selective pulmonary arteriography in 157 patients. Radiology 217:447–455PubMed

16.

Goodman LR, Curtin J, Mewissen M (1995) Detection of pulmonary embolism in patients with unsolved clinical and scintigraphic diagnosis. Helical CT versus angiography. AJR Am J Roentgenol 164:1369–1374PubMed

17.

Senac J, Verhnet H, Bousquet C (1995) Embolie pulmonaire: apport de la tomodensitometrie helicoidale. J Radiol 76:339–345PubMed

18.

van Rossum AB, Pattynama PM, Ton ER, Treurniet FE, Arndt JW, van Eck B, Kieft GJ (1996) Pulmonary embolism Validation of spiral CT angiography in 149 patients. Radiology 201:467–470PubMed

19.

Remy-Jardin M, Remy J, Petyt L, Duhamel A (1996). Diagnosis of acute pulmonary embolism with spiral CT Comparison with pulmonary angiography and scintigraphy. Radiology 200:699–706PubMed

20.

Ferretti G, Bosson J, Buffaz P (1997) Acute pulmonary embolism role of helical CT in 164 patients with intermediate probability at ventilation-perfusion scintigraphy and normal results at duplex US of the legs. Radiology 205:453–458PubMed

21.

van Rossum AB, Pattynama PM, Mallens WM, Hermans J, Heijerman HG (1998) Can helical CT replace scintigraphy in the diagnostic process in suspected pulmonary embolism? A retrolective-prolective cohort study focusing on total diagnostic yield. Eur Radiol 8:90–96PubMedCrossRef

22.

Drucker E, Rivitz S, Shepard J (1998) Acute pulmonary embolism: assessment of helical CT for diagnosis. Radiology 209:235–241PubMed

23.

Herold C, Remy-Jardin M, Grenier P (1998) Prospective evaluation of pulmonary embolism initial results of the European multicenter trial (ESTIPEP) [abstract]. Radiology 209:299

24.

Garg K, Welsh C, Feyerabend A (1998) Pulmonary embolism diagnosis with spiral CT and ventilation-perfusion scanning-correlation with pulmonary angiographic results or clinical outcome. Radiology 208:201–208PubMed

25.

Baghaie F, Remy-Jardin M, Remy J, Fribourg M (1998) Diagnosis of peripheral acute pulmonary emboli: optimization of the spiral CT acquisition protocol [abstract]. Radiology 209:299

26.

Kim K, Muller N, Mayo J (1999) Clinically suspected pulmonary embolism: utility of spiral CT. Radiology 210:693–697PubMed

27.

Brunot S, Corneloup O, Latrabe V, Montaudon M, Laurent F (2005) Reproducibility of multi-detector spiral computed tomography in detection of sub-segmental acute pulmonary embolism. Eur Radiol 15:2057–2063PubMedCrossRef

28.

Le Gal G, Righini M, Parent F, van Strijen M, Couturaud F (2006) Diagnosis and management of subsegmental pulmonary embolism. J Thromb Haemost 4:724–731PubMedCrossRef

29.

Wood KE (2002) Major pulmonary embolism: review of a pathophysiologic approach to the golden hour of hemodynamically significant pulmonary embolism. Chest 121:877–905PubMedCrossRef

30.

Egermayer P (2001) Differences in the rate of anticoagulant complications between routine practice and controlled prospective trials [letter]. J Intern Med 250:258–261PubMedCrossRef

31.

Zhou C, Chan HP, Patel S, Cascade PN, Sahiner B, Hadjiiski LM, Kazerooni EA (2005) Preliminary investigation of computer-aided detection of pulmonary embolism in three-dimensional computed tomography pulmonary angiography images. Acad Radiol 12:782–792PubMedCrossRef

32.

Masutani Y, MacMahon H, Doi K (2002) Computerized detection of pulmonary embolism in spiral CT angiography based on volumetric image analysis. IEEE Trans Med Imaging 21:1517–1523PubMedCrossRef

33.

Liang J, Bi J (2007) Computer aided detection of pulmonary embolism with tobogganing and multiple instance classification in CT pulmonary angiography. Inf Process Med Imaging 20:630–641PubMedCrossRef

34.

Armitage P, Berry G, Matthews JNS (2002) Statistical methods in medical research (4th edn). Blackwell, Oxford

35.

Press WH et al (1992) Numerical recipes: the art of scientific computing (2nd edn). Cambridge University Press, Cambridge

36.

Hanley JA, McNeil BJ (1982) The meaning and use of area under a Receiver Operating Characteristic (ROC) curve. Radiology 143:29–36PubMed

37.

DeLong ER, DeLong DM, Clark-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating curves: a nonparametric approach. Biometrics 44:837–845PubMedCrossRef

38.

Fleiss JL (1981) Statistical methods for rates and proportions (2nd edn). Wiley, New York

39.

Fleiss JL, Nee CM, Landis JR (1979) Large sample variance of kappa in the case of different sets of raters. Psychol Bull 86:974–977CrossRef

40.

Marten K, Grillhösl A, Seyfarth T, Obenauer S, Rummeny EJ, Engelke C (2005) Computer-assisted detection of pulmonary nodules: evaluation of diagnostic performance using an expert knowledge-based detection system with variable reconstruction slice thickness settings. Eur Radiol 15:203–212PubMedCrossRef

41.

Croisille P, Souto M, Cova M, Wood S, Afework Y, Kuhlman J, Zerhouni E (1995) Pulmonary nodules: improved detection with vascular segmentation and extraction with spiral CT. Radiology 197:397–401PubMed

42.

Wormanns D, Fiebich M, Saidi M, Diederich S, Heindel W (2002) Automatic detection of pulmonary nodules at spiral CT: clinical application of a computer-aided diagnosis system. Eur Radiol 12:1052–1057PubMedCrossRef

43.

Marten K, Engelke C (2007) Computer-aided detection and automated CT volumetry of pulmonary nodules. Eur Radiol 17:888–901PubMedCrossRef

44.

Armato SG III, Giger ML, Moran CJ, Blackburn JT, Doi K, MacMahon H (1999) Computerized detection of pulmonary nodules on CT scans. Radiographics 19:1303–1311PubMed

45.

Armato SG III, Li F, Giger ML, MacMahon H, Sone S, Doi K (2002) Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology 225:685–692PubMedCrossRef

46.

Rubin G, Naidich D, Sherbondy A, Lyo J, Napel S (2003) Inadequacy of lung nodule reference standard based upon standard methods of expert consensus review using cine-paging of transverse thin-section MDCT lung scans [abstract]. Radiology 229:441–442

47.

Marten K, Seyfarth T, Auer F, Wiener E, Grillhösl A, Obenauer S, Rummeny EJ, Engelke C (2004) Computer-assisted detection of pulmonary nodules: performance evaluation of an expert knowledge-based detection system in consensus reading with experienced and inexperienced chest radiologists. Eur Radiol 14:1930–1938PubMedCrossRef

48.

Awai K, Murao K, Ozawa A et al (2004) Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists´detection performance. Radiology 230:347–352PubMedCrossRef

49.

Schoepf UJ, Das M, Schneider AC et al (2002) Computer-aided detection (CAD) of segmental and subsegmental pulmonary embolism on 1-mm multidetector-row CT (MDCT) studies [abstract]. Radiology 225:384

50.

Zhou C, Hadjiiski LM, Sahiner B et al (2003) Computerized detection of pulmonary embolism in 3D computed tomographic (CT) images: vessel tracking and segmentation techniques. Proc SPIE 5032:1613–1620CrossRef

51.

Buhmann S, Herzog P, Liang J, Wolf M, Salganicoff M, Kirchhoff C, Reiser M, Becker CH (2007) Clinical evaluation of a computer-aided diagnosis (CAD) prototype for the detection of pulmonary embolism. Acad Radiol 14:651–658PubMedCrossRef

52.

Quiroz R, Kucher N, Schoepf UJ et al (2004) Right ventricular enlargement on chest computed tomography: prognostic role in acute pulmonary embolism. Circulation 109:2401–2404PubMedCrossRef

53.

Engelke C, Rummeny EJ, Marten K (2006) Acute pulmonary embolism on multidetector-row CT of the chest: prediction of cor pulmonale and short-term patient survival from morphological embolus burden. AJR Am J Roentgenol 186:1265–1271PubMedCrossRef

54.

Mastora I, Remy-Jardin M, Masson P et al (2003) Severity of acute pulmonary embolism: evaluation of a new spiral CT angiographic score in correlation with echocardiographic data. Eur Radiol 13:29–35PubMed

55.

Pech M, Wieners G, Dul P, Fischbach F, Dudeck O, Hänninen EL, Ricke J (2007) Computed tomography pulmonary embolism index for the assessment of survival in patients with pulmonary embolism. Eur Radiol 17:1954–1959PubMedCrossRef

Title: Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists
Authors: Christoph Engelke
Stephan Schmidt
Annemarie Bakai
Florian Auer
Katharina Marten
Publication date: 01-02-2008
Publisher: Springer-Verlag
Published in: European Radiology / Issue 2/2008
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI: https://doi.org/10.1007/s00330-007-0770-3

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 2/2008

Valvular heart disease: what does cardiovascular MRI add?

ECG-gated chest CT angiography with 64-MDCT and tri-phasic IV contrast administration regimen in patients with acute non-specific chest pain

High-resolution myocardial stress perfusion at 3 T in patients with suspected coronary artery disease

New method for 3D parametric visualization of contrast-enhanced pulmonary perfusion MRI data

Stereotactic large-core needle breast biopsy: analysis of pain and discomfort related to the biopsy procedure

Modern concepts of ductal carcinoma in situ (DCIS) and its diagnosis through percutaneous biopsy