Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ASCO Annual Meeting 2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

2024 ASCO Annual Meeting

Keep up to date with all the top stories and latest evidence with our hub page, including official congress videos and expert takeaways.
ADVANCED SEARCH
Log in
Top
European Radiology
Published in: European Radiology 2/2014

01-02-2014 | Breast

Experiences with a self-test for Dutch breast screening radiologists: lessons learnt

Authors: J. M. H. Timmers, A. L. M. Verbeek, R. M. Pijnappel, M. J. M. Broeders, G. J. den Heeten

Published in: European Radiology | Issue 2/2014

Login to get access

Abstract

Purpose

To evaluate a self-test for Dutch breast screening radiologists introduced as part of the national quality assurance programme.

Methods and materials

A total of 144 radiologists were invited to complete a test-set of 60 screening mammograms (20 malignancies). Participants assigned findings such as location, lesion type and BI-RADS. We determined areas under the receiver operating characteristics (ROC) curves (AUC), case and lesion sensitivity and specificity, agreement (kappa) and correlation between reader characteristics and case sensitivity (Spearman correlation coefficients).

Results

A total of 110 radiologists completed the test (76 %). Participants read a median number of 10,000 screening mammograms/year. Median AUC value was 0.93, case and lesion sensitivity was 91 % and case specificity 94 %. We found substantial agreement for recall (κ = 0.77) and laterality (κ = 0.80), moderate agreement for lesion type (κ = 0.57) and BI-RADS (κ = 0.45) and no correlation between case sensitivity and reader characteristics.

Conclusion

Areas under the ROC curve, case sensitivity and lesion sensitivity were satisfactory and recall agreement was substantial. However, agreement in lesion type and BI-RADS could be improved; further education might be aimed at reducing interobserver variation in interpretation and description of abnormalities. We offered individual feedback on interpretive performance and overall feedback at group level. Future research will determine whether performance has improved.

Key Points

• We introduced and evaluated a self-test for Dutch breast screening radiologists.
• ROC curves, case and lesion sensitivity and recall agreement were all satisfactory.
• Agreement in BI-RADS interpretation and description of abnormalities could be improved.
• These are areas that should be targeted with further education and training.
• We offered individual feedback on interpretative performance and overall group feedback.
Literature
1.
Holland R, Rijken HJ, Hendriks JH (2007) The Dutch population-based mammography screening: 30-years experience. Breast Care 2:12–18CrossRef
2.
National Evaluation Team for Breast Cancer Screening (2012) Preliminary results breast cancer screening programme in the Netherlands, 2011. NETCB, Rotterdam
3.
Australian Government Department of Health and Aging (2009) BreastScreen Australia Evaluation: evaluation final report. Department of Health and Aging, Canberra
4.
The Royal College of Radiologists (2007) Standards for self-assessment of performance. RCR, London
5.
6.
Soh BP, Lee W, Kench PL et al (2012) Assessing reader performance in radiology, an imperfect science: lessons from breast screening. Clin Radiol 67:623–628PubMedCrossRef
7.
Scott HJ, Gale A (2006) Breast screening: PERFORMS identifies key mammographic training needs. Br J Radiol 79:S127–S133PubMedCrossRef
8.
Cook AJ, Elmore JG, Zhu W et al (2012) Mammographic interpretation: radiologists' ability to accurately estimate their performance and compare it with that of their peers. AJR Am J Roentgenol 199:695–702PubMedCentralPubMedCrossRef
9.
Geller BM, Ichikawa L, Miglioretti DL, Eastman D (2012) Web-based mammography audit feedback. AJR Am J Roentgenol 198:562–567CrossRef
10.
Shapiro S, Coleman EA, Broeders M et al (1998) Breast cancer screening programmes in 22 countries: current policies, administration and guidelines. International Breast Cancer Screening Network (IBSN) and the European Network of Pilot Projects for Breast Cancer Screening. Int J Epidemiol 27:735–742PubMedCrossRef
11.
Yankaskas BC, Klabunde CN, Ancelle-Park R et al (2004) International comparison of performance measures for screening mammography: can it be done? J Med Screen 11:187–193PubMedCrossRef
12.
Timmers JM, van Doorne-Nagtegaal HJ, Zonderland HM et al (2012) The Breast Imaging Reporting and Data System (BI-RADS) in the Dutch breast cancer screening programme: its role as an assessment and stratification tool. Eur Radiol 22:5CrossRef
13.
14.
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174PubMedCrossRef
15.
Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46CrossRef
16.
Ciatto S, Ambrogetti D, Morrone D, Del Turco M (2006) Analysis of the results of a proficiency test in screening mammography at the CSPO of Florence: review of 705 tests. Radiol Med 111:797–803PubMedCrossRef
17.
Onega T, Smith M, Miglioretti DL et al (2012) Radiologist agreement for mammographic recall by case difficulty and finding type. J Am Coll Radiol 9:788–794PubMedCentralPubMedCrossRef
18.
Reed WM, Lee WB, Cawson JN, Brennan PC (2010) Malignancy detection in digital mammograms: important reader characteristics and required case numbers. Acad Radiol 17:1409–1413PubMedCrossRef
19.
Esserman L, Cowley H, Eberle C et al (2002) Improving the accuracy of mammography: volume and outcome relationships. J Natl Cancer Inst 94(5):369–375PubMedCrossRef
20.
Miglioretti DL, Gard CC, Carney PA et al (2009) When radiologists perform best: the learning curve in screening mammogram interpretation. Radiology 253:632–640PubMedCrossRef
21.
Pusic MV, Andrews JS, Kessler DO et al (2012) Prevalence of abnormal cases in an image bank affects the learning of radiograph interpretation. Med Educ 46:289–298PubMedCrossRef
22.
Carney PA, Bogart TA, Geller BM et al (2012) Association between time spent interpreting, level of confidence, and accuracy of screening mammography. Am J Roentgenol 198:970–978CrossRef
23.
Elmore JG, Jackson SL, Abraham L et al (2009) Variability in interpretive performance at screening mammography and radiologists' characteristics associated with accuracy. Radiology 253:641–651PubMedCrossRef
24.
Kerlikowske K, Grady D, Barclay J et al (1998) Variability and accuracy in mammographic interpretation using the American College of Radiology Breast Imaging Reporting and Data System. J Natl Cancer Inst 90:1801–1809PubMedCrossRef
25.
Lazarus E, Mainiero MB, Schepps B et al (2006) BI-RADS lexicon for US and mammography: interobserver variability and positive predictive value. Radiology 239:385–391PubMedCrossRef
26.
Pinto A, Acampora C, Pinto F et al (2011) Learning from diagnostic errors: a good way to improve education in radiology. Eur J Radiol 78:372–376PubMedCrossRef
27.
Gur D, Bandos AI, Cohen CS et al (2008) The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations. Radiology 249:47–53PubMedCrossRef
Metadata
Title
Experiences with a self-test for Dutch breast screening radiologists: lessons learnt
Authors
J. M. H. Timmers
A. L. M. Verbeek
R. M. Pijnappel
M. J. M. Broeders
G. J. den Heeten
Publication date
01-02-2014
Publisher
Springer Berlin Heidelberg
Published in
European Radiology / Issue 2/2014
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-013-3018-4

Other articles of this Issue 2/2014

European Radiology 2/2014 Go to the issue

Pediatric

Repetitive MR measurements of lung volume in fetuses with congenital diaphragmatic hernia: individual development of pulmonary hypoplasia during pregnancy and calculation of weekly lung growth rates

Gastrointestinal

Diffusion-weighted imaging with acquisition of three b-values for response evaluation of neuroendocrine liver metastases undergoing selective internal radiotherapy

Molecular Imaging

A low molecular weight zinc2+-dipicolylamine-based probe detects apoptosis during tumour treatment better than an annexin V-based probe

Cardiac

Automated Quantification of Epicardial Adipose Tissue Using CT Angiography: Evaluation of a Prototype Software

Computed Tomography

Radiation dose reduction in cerebral CT perfusion imaging using iterative reconstruction

Musculoskeletal

Dynamic contrast-enhanced magnetic resonance imaging can assess vascularity within fracture non-unions and predicts good outcome