Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
European Radiology
Published in: European Radiology 9/2019

Open Access 01-09-2019 | Breast Cancer | Breast

Can we reduce the workload of mammographic screening by automatic identification of normal exams with artificial intelligence? A feasibility study

Authors: Alejandro Rodriguez-Ruiz, Kristina Lång, Albert Gubern-Merida, Jonas Teuwen, Mireille Broeders, Gisella Gennaro, Paola Clauser, Thomas H. Helbich, Margarita Chevalier, Thomas Mertelmeier, Matthew G. Wallis, Ingvar Andersson, Sophia Zackrisson, Ioannis Sechopoulos, Ritse M. Mann

Published in: European Radiology | Issue 9/2019

Login to get access

Abstract

Purpose

To study the feasibility of automatically identifying normal digital mammography (DM) exams with artificial intelligence (AI) to reduce the breast cancer screening reading workload.

Methods and materials

A total of 2652 DM exams (653 cancer) and interpretations by 101 radiologists were gathered from nine previously performed multi-reader multi-case receiver operating characteristic (MRMC ROC) studies. An AI system was used to obtain a score between 1 and 10 for each exam, representing the likelihood of cancer present. Using all AI scores between 1 and 9 as possible thresholds, the exams were divided into groups of low- and high likelihood of cancer present. It was assumed that, under the pre-selection scenario, only the high-likelihood group would be read by radiologists, while all low-likelihood exams would be reported as normal. The area under the reader-averaged ROC curve (AUC) was calculated for the original evaluations and for the pre-selection scenarios and compared using a non-inferiority hypothesis.

Results

Setting the low/high-likelihood threshold at an AI score of 5 (high likelihood > 5) results in a trade-off of approximately halving (− 47%) the workload to be read by radiologists while excluding 7% of true-positive exams. Using an AI score of 2 as threshold yields a workload reduction of 17% while only excluding 1% of true-positive exams. Pre-selection did not change the average AUC of radiologists (inferior 95% CI > − 0.05) for any threshold except at the extreme AI score of 9.

Conclusion

It is possible to automatically pre-select exams using AI to significantly reduce the breast cancer screening reading workload.

Key Points

• There is potential to use artificial intelligence to automatically reduce the breast cancer screening reading workload by excluding exams with a low likelihood of cancer.
• The exclusion of exams with the lowest likelihood of cancer in screening might not change radiologists’ breast cancer detection performance.
• When excluding exams with the lowest likelihood of cancer, the decrease in true-positive recalls would be balanced by a simultaneous reduction in false-positive recalls.
Literature
1.
Smith RA, Cokkinides V, Brooks D, Saslow D, Brawley OW (2010) Cancer screening in the United States, 2010: a review of current American Cancer Society guidelines and issues in cancer screening. CA Cancer J Clin 60:99–119CrossRefPubMed
2.
Broeders M, Moss S, Nyström L et al (2012) The impact of mammographic screening on breast cancer mortality in Europe: a review of observational studies. J Med Screen 19:14–25CrossRefPubMed
3.
Independent UK Panel on Breast Cancer Screening (2012) The benefits and harms of breast cancer screening: an independent review. Lancet 380:1778–1786CrossRef
4.
Welch HG, Prorok PC, O’Malley AJ, Kramer BS (2016) Breast-cancer tumor size, overdiagnosis, and mammography screening effectiveness. N Engl J Med 375:1438–1447CrossRefPubMed
5.
6.
Brewer NT, Salz T, Lillie SE (2007) Systematic review: the long-term effects of false-positive mammograms. Ann Intern Med 146:502–510CrossRefPubMed
7.
Karssemeijer N, Bluekens AM, Beijerinck D et al (2009) Breast cancer screening results 5 years after introduction of digital mammography in a population-based screening program. Radiology 253:353–358CrossRefPubMed
8.
Evans KK, Birdwell RL, Wolfe JM (2013) If you don’t find it often, you often don’t find it: why some cancers are missed in breast cancer screening. PLoS One 8:e64366CrossRefPubMedPubMedCentral
9.
Huynh PT, Jarolimek AM, Daye S (1998) The false-negative mammogram. Radiographics 18:1137–1154 quiz 1243-1134CrossRefPubMed
10.
Fenton JJ, Taplin SH, Carney PA et al (2007) Influence of computer-aided detection on performance of screening mammography. N Engl J Med 356:1399–1409CrossRefPubMedPubMedCentral
11.
Lehman CD, Wellman RD, Buist DS et al (2015) Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Intern Med 175:1828–1837CrossRefPubMedPubMedCentral
12.
Litjens G, Kooi T, Bejnordi BE et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRef
13.
14.
15.
Rodríguez-Ruiz A, Krupinski E, Mordang J-J et al (2018) Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 181371
16.
Rimmer A (2017) Radiologist shortage leaves patient care at risk, warns royal college. BMJ 359
17.
18.
Wing P, Langelier MH (2009) Workforce shortages in breast imaging: impact on mammography utilization. Am J Roentgenol 192:370–378CrossRef
19.
Wallis MG, Moa E, Zanca F, Leifland K, Danielsson M (2012) Two-view and single-view tomosynthesis versus full-field digital mammography: high-resolution X-ray imaging observer study. Radiology 262:788–796CrossRefPubMed
20.
Visser R, Veldkamp WJ, Beijerinck D et al (2012) Increase in perceived case suspiciousness due to local contrast optimisation in digital screening mammography. Eur Radiol 22:908–914CrossRefPubMed
21.
Hupse R, Samulski M, Lobbes MB et al (2013) Computer-aided detection of masses at mammography: interactive decision support versus prompts. Radiology 266:123–129CrossRefPubMed
22.
Gennaro G, Hendrick RE, Ruppel P et al (2013) Performance comparison of single-view digital breast tomosynthesis plus single-view digital mammography with two-view digital mammography. Eur Radiol 23:664–672CrossRefPubMed
23.
24.
Garayoa J, Chevalier M, Castillo M et al (2018) Diagnostic value of the stand-alone synthetic image in digital breast tomosynthesis examinations. Eur Radiol 28:565–572CrossRefPubMed
25.
26.
Clauser P, Baltzer PA, Kapetas P et al (2019) Synthetic 2-dimensional mammography can replace digital mammography as an adjunct to wide-angle digital breast tomosynthesis. Invest Radiol 54:83–88CrossRefPubMed
27.
Blackwelder WC (1982) “Proving the null hypothesis” in clinical trials. Control Clin Trials 3:345–353CrossRefPubMed
28.
Chen W, Petrick NA, Sahiner B (2012) Hypothesis testing in noninferiority and equivalence MRMC ROC studies. Acad Radiol 19:1158–1165CrossRefPubMed
29.
Gallas BD, Bandos A, Samuelson FW, Wagner RF (2009) A framework for random-effects ROC analysis: biases with the bootstrap and other variance estimators. Commun Stat - Theory Methods 38:2586–2603CrossRef
30.
31.
Gennaro G (2018) The “perfect” reader study. Eur J Radiol In press
32.
Chen W, Gong Q, Gallas BD (2018) Efficiency gain of paired split-plot designs in MRMC ROC studies. Medical imaging 2018: image perception, observer performance, and technology assessment. International Society for Optics and Photonics, pp 105770F
33.
34.
35.
Dang PA, Freer PE, Humphrey KL, Halpern EF, Rafferty EA (2014) Addition of tomosynthesis to conventional digital mammography: effect on image interpretation time of screening examinations. Radiology 270:49–56CrossRefPubMed
Metadata
Title
Can we reduce the workload of mammographic screening by automatic identification of normal exams with artificial intelligence? A feasibility study
Authors
Alejandro Rodriguez-Ruiz
Kristina Lång
Albert Gubern-Merida
Jonas Teuwen
Mireille Broeders
Gisella Gennaro
Paola Clauser
Thomas H. Helbich
Margarita Chevalier
Thomas Mertelmeier
Matthew G. Wallis
Ingvar Andersson
Sophia Zackrisson
Ioannis Sechopoulos
Ritse M. Mann
Publication date
01-09-2019
Publisher
Springer Berlin Heidelberg
Published in
European Radiology / Issue 9/2019
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-019-06186-9

Other articles of this Issue 9/2019

European Radiology 9/2019 Go to the issue

Imaging Informatics and Artificial Intelligence

Differentiating kidney stones from phleboliths in unenhanced low-dose computed tomography using radiomics and machine learning

Nuclear Medicine

Does whole-body Patlak 18F-FDG PET imaging improve lesion detectability in clinical oncology?

Magnetic Resonance

Added value of apparent diffusion coefficient in distinguishing between serous and mucin-producing pancreatic cystic neoplasms

Editorial Comment

Is the future of breast imaging with AI?

Urogenital

Adnexal mass staging CT with a disease-specific structured report compared to simple structured report

Imaging Informatics and Artificial Intelligence

Machine learning classifiers can predict Gleason pattern 4 prostate cancer with greater accuracy than experienced radiologists