Skip to main content
Top
Published in: European Radiology 8/2012

Open Access 01-08-2012 | Chest

Observer training for computer-aided detection of pulmonary nodules in chest radiography

Authors: Diederick W. De Boo, François van Hoorn, Joost van Schuppen, Laura Schijf, Maeke J. Scheerder, Nicole J. Freling, Onno Mets, Michael Weber, Cornelia M. Schaefer-Prokop

Published in: European Radiology | Issue 8/2012

Login to get access

Abstract

Objectives

To assess whether short-term feedback helps readers to increase their performance using computer-aided detection (CAD) for nodule detection in chest radiography.

Methods

The 140 CXRs (56 with a solitary CT-proven nodules and 84 negative controls) were divided into four subsets of 35; each were read in a different order by six readers. Lesion presence, location and diagnostic confidence were scored without and with CAD (IQQA-Chest, EDDA Technology) as second reader. Readers received individual feedback after each subset. Sensitivity, specificity and area under the receiver-operating characteristics curve (AUC) were calculated for readings with and without CAD with respect to change over time and impact of CAD.

Results

CAD stand-alone sensitivity was 59 % with 1.9 false-positives per image. Mean AUC slightly increased over time with and without CAD (0.78 vs. 0.84 with and 0.76 vs. 0.82 without CAD) but differences did not reach significance. The sensitivity increased (65 % vs. 70 % and 66 % vs. 70 %) and specificity decreased over time (79 % vs. 74 % and 80 % vs. 77 %) but no significant impact of CAD was found.

Conclusion

Short-term feedback does not increase the ability of readers to differentiate true- from false-positive candidate lesions and to use CAD more effectively.

Key Points

Computer-aided detection (CAD) is increasingly used as an adjunct for many radiological techniques.
Short-term feedback does not improve reader performance with CAD in chest radiography.
Differentiation between true- and false-positive CAD for low conspicious possible lesions proves difficult.
CAD can potentially increase reader performance for nodule detection in chest radiography.
Literature
1.
go back to reference Li F, Engelmann R, Metz CE, Doi K, MacMahon H (2008) Lung cancers missed on chest radiographs: results obtained with a commercial computer-aided detection program. Radiology 246:273–280PubMed Li F, Engelmann R, Metz CE, Doi K, MacMahon H (2008) Lung cancers missed on chest radiographs: results obtained with a commercial computer-aided detection program. Radiology 246:273–280PubMed
2.
go back to reference Van Beek EJ, Mullan B, Thompson B (2008) Evaluation of a real-time interactive pulmonary nodule analysis system on chest digital radiographic images: a prospective study. Acad Radiol 15:571–575PubMedCrossRef Van Beek EJ, Mullan B, Thompson B (2008) Evaluation of a real-time interactive pulmonary nodule analysis system on chest digital radiographic images: a prospective study. Acad Radiol 15:571–575PubMedCrossRef
3.
go back to reference Bley TA, Baumann T, Saueressig U et al (2008) Comparison of radiologist and CAD performance in the detection of CT-confirmed subtle pulmonary nodules on digital chest radiographs. Invest Radiol 43:343–348PubMedCrossRef Bley TA, Baumann T, Saueressig U et al (2008) Comparison of radiologist and CAD performance in the detection of CT-confirmed subtle pulmonary nodules on digital chest radiographs. Invest Radiol 43:343–348PubMedCrossRef
4.
go back to reference White CS, Flukinger T, Jeudy J, Chen JJ (2009) Use of a computer-aided detection system to detect missed lung cancer at chest radiography. Radiology 252:273–281PubMedCrossRef White CS, Flukinger T, Jeudy J, Chen JJ (2009) Use of a computer-aided detection system to detect missed lung cancer at chest radiography. Radiology 252:273–281PubMedCrossRef
5.
go back to reference Sakai S, Soeda H, Takahashi N et al (2006) Computer-aided detection on digital chest radiography: validation test on consecutive T1 cases of resectable lung cancer. J Digit Imaging 19:376–382PubMedCrossRef Sakai S, Soeda H, Takahashi N et al (2006) Computer-aided detection on digital chest radiography: validation test on consecutive T1 cases of resectable lung cancer. J Digit Imaging 19:376–382PubMedCrossRef
6.
go back to reference Hardie RC, Rogers SK, Wilson T, Rogers A (2008) Performance analysis of a new computer aided detection system for identifying lung nodules on chest radiographs. Med Image Anal 12:240–258PubMedCrossRef Hardie RC, Rogers SK, Wilson T, Rogers A (2008) Performance analysis of a new computer aided detection system for identifying lung nodules on chest radiographs. Med Image Anal 12:240–258PubMedCrossRef
7.
go back to reference Xu Y, Ma D, He W (2011) Assessing the use of digital radiography and a real-time interactive pulmonary nodule analysis system for large population lung cancer screening. Eur J Radiol. doi:10.1016/j.ejrad.2011.04.031 Xu Y, Ma D, He W (2011) Assessing the use of digital radiography and a real-time interactive pulmonary nodule analysis system for large population lung cancer screening. Eur J Radiol. doi:10.​1016/​j.​ejrad.​2011.​04.​031
8.
go back to reference Song W, Fan L, Xie Y, Qian JZ, Jin Z (2005) A study of inter-observer variations of pulmonary nodule marking and characterizing on DR images. Proc SPIE 5749:272–280CrossRef Song W, Fan L, Xie Y, Qian JZ, Jin Z (2005) A study of inter-observer variations of pulmonary nodule marking and characterizing on DR images. Proc SPIE 5749:272–280CrossRef
9.
go back to reference de Hoop B, De Boo DW, Gietema HA et al (2010) Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology 257:532–540PubMedCrossRef de Hoop B, De Boo DW, Gietema HA et al (2010) Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology 257:532–540PubMedCrossRef
10.
go back to reference Taylor SA, Burling D, Roddie M et al (2008) Computer-aided detection for CT colonography: incremental benefit of observer training. Br J Radiol 81:180–186PubMedCrossRef Taylor SA, Burling D, Roddie M et al (2008) Computer-aided detection for CT colonography: incremental benefit of observer training. Br J Radiol 81:180–186PubMedCrossRef
11.
go back to reference Luo P, Qian W, Romilly P (2005) CAD-Aided mammogram training. Acad Rad 12:1039–1048CrossRef Luo P, Qian W, Romilly P (2005) CAD-Aided mammogram training. Acad Rad 12:1039–1048CrossRef
13.
go back to reference Giger ML, Chan HP, Boone J (2008) Anniversary paper: history and status of CAD and quantitative analysis—the image role of medical physics and AAPM. Med Phys 35:5799–5820PubMedCrossRef Giger ML, Chan HP, Boone J (2008) Anniversary paper: history and status of CAD and quantitative analysis—the image role of medical physics and AAPM. Med Phys 35:5799–5820PubMedCrossRef
14.
go back to reference Muhm JR, Miller WE, Fontana RS, Sanderson DR, Uhlenhoop MA (1983) Lung cancer detected during a screening program using 4-month chest radiographs. Radiology 148:609–615PubMed Muhm JR, Miller WE, Fontana RS, Sanderson DR, Uhlenhoop MA (1983) Lung cancer detected during a screening program using 4-month chest radiographs. Radiology 148:609–615PubMed
15.
go back to reference Heelan RT, Flehinger BJ, Melamed MR et al (1984) Non-small-cell lung cancer: results of the New York screening program. Radiology 151:289–293PubMed Heelan RT, Flehinger BJ, Melamed MR et al (1984) Non-small-cell lung cancer: results of the New York screening program. Radiology 151:289–293PubMed
16.
go back to reference Austin JH, Romney BM, Goldsmith LS (1992) Missed bronchogenic carcinoma: radiographic findings in 27 patients with potentially resectable lesion evident in retrospect. Radiology 182:115–122PubMed Austin JH, Romney BM, Goldsmith LS (1992) Missed bronchogenic carcinoma: radiographic findings in 27 patients with potentially resectable lesion evident in retrospect. Radiology 182:115–122PubMed
17.
go back to reference Quekel LG, Kessels AG, Goei R, Engelshoven JM (1999) Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 115:720–724PubMedCrossRef Quekel LG, Kessels AG, Goei R, Engelshoven JM (1999) Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 115:720–724PubMedCrossRef
18.
go back to reference Monnier-Cholley L, Arrivé L, Porcel A et al (2001) Characteristics of missed lung cancer on chest radiographs: a French experience. Eur Radiol 11:597–605PubMedCrossRef Monnier-Cholley L, Arrivé L, Porcel A et al (2001) Characteristics of missed lung cancer on chest radiographs: a French experience. Eur Radiol 11:597–605PubMedCrossRef
19.
go back to reference Shah PK, Austin JH, White CS et al (2003) Missed non-small cell lung cancers: Radiographic findings of potentially resectable lesions evident only in retrospect. Radiology 226:235–241PubMedCrossRef Shah PK, Austin JH, White CS et al (2003) Missed non-small cell lung cancers: Radiographic findings of potentially resectable lesions evident only in retrospect. Radiology 226:235–241PubMedCrossRef
20.
go back to reference Wu HM, Gotway MB, Lee TJ et al (2008) Features of non-small cell lung carcinomas overlooked at digital chest radiography. Clin Radiol 63:518–528PubMedCrossRef Wu HM, Gotway MB, Lee TJ et al (2008) Features of non-small cell lung carcinomas overlooked at digital chest radiography. Clin Radiol 63:518–528PubMedCrossRef
21.
go back to reference Samulski M, Hupse R, Boetes C, Mus RD, den Heeten GJ, Karssemeijer N (2010) Using computer-aided detection in mammography as a decision support. Eur Radiol 20:2323–2330PubMedCrossRef Samulski M, Hupse R, Boetes C, Mus RD, den Heeten GJ, Karssemeijer N (2010) Using computer-aided detection in mammography as a decision support. Eur Radiol 20:2323–2330PubMedCrossRef
Metadata
Title
Observer training for computer-aided detection of pulmonary nodules in chest radiography
Authors
Diederick W. De Boo
François van Hoorn
Joost van Schuppen
Laura Schijf
Maeke J. Scheerder
Nicole J. Freling
Onno Mets
Michael Weber
Cornelia M. Schaefer-Prokop
Publication date
01-08-2012
Publisher
Springer-Verlag
Published in
European Radiology / Issue 8/2012
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-012-2412-7

Other articles of this Issue 8/2012

European Radiology 8/2012 Go to the issue