Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Insights into Imaging
Published in: Insights into Imaging 1/2020

01-12-2020 | Educational Review

Deep learning workflow in radiology: a primer

Authors: Emmanuel Montagnon, Milena Cerny, Alexandre Cadrin-Chênevert, Vincent Hamilton, Thomas Derennes, André Ilinca, Franck Vandenbroucke-Menu, Simon Turcotte, Samuel Kadoury, An Tang

Published in: Insights into Imaging | Issue 1/2020

Login to get access

Abstract

Interest for deep learning in radiology has increased tremendously in the past decade due to the high achievable performance for various computer vision tasks such as detection, segmentation, classification, monitoring, and prediction. This article provides step-by-step practical guidance for conducting a project that involves deep learning in radiology, from defining specifications, to deployment and scaling. Specifically, the objectives of this article are to provide an overview of clinical use cases of deep learning, describe the composition of multi-disciplinary team, and summarize current approaches to patient, data, model, and hardware selection. Key ideas will be illustrated by examples from a prototypical project on imaging of colorectal liver metastasis. This article illustrates the workflow for liver lesion detection, segmentation, classification, monitoring, and prediction of tumor recurrence and patient survival. Challenges are discussed, including ethical considerations, cohorting, data collection, anonymization, and availability of expert annotations. The practical guidance may be adapted to any project that requires automated medical image analysis.
Literature
1.
Chartrand G, Cheng PM, Vorontsov E et al (2017) Deep learning: a primer for radiologists. Radiographics 37:2113–2131CrossRef
2.
Miotto R, Wang F, Wang S, Jiang X, Dudley JT (2017) Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform 19:1236–1246CrossRef
3.
Litjens G, Kooi T, Bejnordi BE et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRef
4.
Luo W, Phung D, Tran T et al (2016) Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res 18:e323CrossRef
5.
Ben-Cohen A, Diamant I, Klang E, Amitai M, Greenspan H (2016) Fully convolutional network for liver segmentation and lesions detection. In: Carneiro G et al (Eds) Deep Learning and Data Labeling for Medical Applications. DLMIA 2016, LABELS 2016. Lecture Notes in Computer Science, vol 10008. Springer, Cham, pp 77-85CrossRef
6.
Roth HR, Lu L, Liu J et al (2016) Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 35:1170–1181CrossRef
7.
Yasaka K, Akai H, Abe O, Kiryu S (2017) Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286:887–896CrossRef
8.
9.
Vorontsov E, Cerny M, Régnier P et al (2019) Deep learning for automated segmentation of liver lesions at ct in patients with colorectal cancer liver metastases. Radiol Artif Intell 1:180014
10.
Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional neural networks: an overview and application in radiology. Insights Imaging:1–19
11.
Drozdzal M, Chartrand G, Vorontsov E et al (2018) Learning normalized inputs for iterative estimation in medical image segmentation. Med Image Anal 44:1–13CrossRef
12.
13.
Gillies RJ, Kinahan PE, Hricak H (2015) Radiomics: images are more than pictures, they are data. Radiology 278:563–577CrossRef
14.
Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446CrossRef
15.
Sun J, Li H, Xu Z (2016) Deep ADMM-Net for compressive sensing MRI. Advances In Neural Information Processing Systems, pp 10-18
16.
Yang Q, Yan P, Zhang Y et al (2018) Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Trans Med Imaging 37:1348–1357CrossRef
17.
Higaki T, Nakamura Y, Tatsugami F, Nakaura T, Awai K (2019) Improvement of image quality at CT and MRI using deep learning. Jpn J Radiology 37:73–80CrossRef
18.
19.
20.
Prasad SR, Jhaveri KS, Saini S, Hahn PF, Halpern EF, Sumner JE (2002) CT tumor measurement for therapeutic response assessment: comparison of unidimensional, bidimensional, and volumetric techniques initial observations. Radiology 225:416–419CrossRef
21.
Hayano K, Lee SH, Sahani DV (2015) Imaging for assessment of treatment response in hepatocellular carcinoma: current update. Indian J Radiol Imaging 25:121–128CrossRef
22.
Gotra A, Sivakumaran L, Chartrand G et al (2017) Liver segmentation: indications, techniques and future directions. Insights Imaging 8:377–392CrossRef
23.
Henze J, Maintz D, Persigehl T (2016) RECIST 1.1, irRECIST 1.1, and mRECIST: How to Do. Curr Radiol Rep 4:48
24.
25.
Nancarrow SA, Booth A, Ariss S, Smith T, Enderby P, Roots A (2013) Ten principles of good interdisciplinary team work. Hum Resour Health 11:19CrossRef
26.
Rubbia-Brandt L, Giostra E, Brezault C et al (2006) Importance of histological tumor response assessment in predicting the outcome in patients with colorectal liver metastases treated with neo-adjuvant chemotherapy followed by liver surgery. Ann Oncol 18:299–304CrossRef
27.
Whitney CW, Lind BK, Wahl PW (1998) Quality assurance and quality control in longitudinal studies. Epidemiol Rev 20:71–80CrossRef
28.
Knatterud GL, Rockhold FW, George SL et al (1998) Guidelines for quality assurance in multicenter trials: a position paper. Control Clin Trials 19:477–493CrossRef
29.
Nosowsky R, Giordano TJ (2006) The Health Insurance Portability and Accountability Act of 1996 (HIPAA) privacy rule: implications for clinical research. Annu Rev Med 57:575–590CrossRef
30.
Custers B, Dechesne F, Sears AM, Tani T, van der Hof S (2018) A comparison of data protection legislation and policies across the EU. Comput Law Security Rev 34:234–243CrossRef
31.
32.
Ballantyne A, Schaefer GO (2018) Consent and the ethical duty to participate in health data research. J Med Ethics 44:392–396CrossRef
33.
34.
35.
36.
37.
Pesapane F, Volonté C, Codari M, Sardanelli F (2018) Artificial intelligence as a medical device in radiology: ethical and regulatory issues in Europe and the United States. Insights Imaging 9:745–753CrossRef
38.
Murphy J, Scott J, Kaufman D, Geller G, LeRoy L, Hudson K (2009) Public perspectives on informed consent for biobanking. Am J Public Health 99:2128–2134CrossRef
39.
Nelson G (2015) Practical implications of sharing data: a primer on data privacy, anonymization, and de-identification. SAS Global Forum Proceedings
40.
Neubauer T, Heurix J (2011) A methodology for the pseudonymization of medical data. Int J Med Inform 80:190–204CrossRef
41.
42.
Tang A, Tam R, Cadrin-Chênevert A et al (2018) Canadian Association of Radiologists white paper on artificial intelligence in radiology. Can Assoc Radiol J
43.
Aryanto K, Oudkerk M, van Ooijen P (2015) Free DICOM de-identification tools in clinical research: functioning and safety of patient privacy. Eur Radiol 25:3685–3695CrossRef
44.
45.
46.
Chennubhotla C, Clarke L, Fedorov A et al (2017) An assessment of imaging informatics for precision medicine in cancer. Yearb Med Inform 26:110-119CrossRef
47.
48.
49.
Channin DS, Mongkolwat P, Kleper V, Rubin DL (2009) The Annotation and Image Mark-up Project. Radiology 253:590–592CrossRef
50.
Wolf I, Vetter M, Wegner I et al (2004) The medical imaging interaction toolkit (MITK): a toolkit facilitating the creation of interactive software by extending VTK and ITK. Proc. SPIE 5367, Medical Imaging 2004: Visualization, Image-Guided Procedures, and Display, pp 16-27
51.
Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning (synthesis lectures on artificial intelligence and machine learning). Morgan and Claypool Publishers 14
52.
Hinton GE, Sejnowski TJ, Poggio TA (1999) Unsupervised learning: foundations of neural computation. MIT press
53.
Bengio Y (2009) Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning 2:1–127CrossRef
54.
55.
Papandreou G, Chen L-C, Murphy KP, Yuille AL (2015) Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. Proceedings of the IEEE international conference on computer vision, pp 1742-1750
56.
57.
58.
Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 249-256
59.
Rodriguez JD, Perez A, Lozano JA (2009) Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Trans Pattern Anal Mach Intell 32:569–575CrossRef
60.
Goodfellow I, Bengio Y, Courville A (2016) Deep Learning, 1st edn. MIT Press, Cambridge
61.
Erickson BJ, Korfiatis P, Akkus Z, Kline T, Philbrick K (2017) Toolkits and libraries for deep learning. J Digit Imaging 30:400–405CrossRef
62.
Abadi M, Agarwal A, Barham P et al (2015) TensorFlow: large-scale machine learning on heterogeneous distributed systems. Preliminary White Paper, November 9, 2015
63.
Paszke A, Gross S, Chintala S, Chanan G (2017) Pytorch: tensors and dynamic neural networks in python with strong GPU acceleration. Pytorch: tensors and dynamic neural networks in python with strong gpu acceleration 6
64.
65.
66.
Jia Y, Shelhamer E, Donahue J et al (2014) Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM international conference on Multimedia. ACM, pp 675-678
67.
Soffer S, Ben-Cohen A, Shimon O, Amitai MM, Greenspan H, Klang E (2019) Convolutional neural networks for radiologic images: a radiologist’s guide. Radiology 290:590–606CrossRef
68.
Krizhevesky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (NIPS 2012)
69.
Summers RM (2016) Progress in fully automated abdominal CT interpretation. AJR Am J Roentgenol 207:67–79CrossRef
70.
71.
Thrun S, Pratt L (2012) Learning to learn. Springer Science & Business Media
72.
Walter SD (2005) The partial area under the summary ROC curve. Stat Med 24:2025–2040CrossRef
73.
Raina R, Madhavan A, Ng AY (2009) Large-scale deep unsupervised learning using graphics processors. Proceedings of the 26th Annual International Conference on Machine Learning. ACM, Montreal, Quebec, Canada, pp 873–880
74.
75.
Kaleeswari C, Maheswari P, Kuppusamy K, Jeyabalu M (2018) A brief review on cloud security scenarios. International Journal of Scientific Research in Science and Technology
76.
77.
Bernstein D (2014) Containers and cloud: from LXC to Docker to Kubernetes. IEEE Cloud Comput 1:81–84CrossRef
78.
79.
Hightower K, Burns B, Beda J (2017) Kubernetes: up and running dive into the future of infrastructure. O'Reilly Media, Inc.
80.
Spanou D (2013) Software as a Medical Device (SaMD): key definitions. IMDRF SaMD Working Group
81.
82.
IEC I (2006) 62304: 2006 Medical device software–software life cycle processes. International Electrotechnical Commission, Geneva
83.
Metadata
Title
Deep learning workflow in radiology: a primer
Authors
Emmanuel Montagnon
Milena Cerny
Alexandre Cadrin-Chênevert
Vincent Hamilton
Thomas Derennes
André Ilinca
Franck Vandenbroucke-Menu
Simon Turcotte
Samuel Kadoury
An Tang
Publication date
01-12-2020
Publisher
Springer Berlin Heidelberg
Published in
Insights into Imaging / Issue 1/2020
Electronic ISSN: 1869-4101
DOI
https://doi.org/10.1186/s13244-019-0832-5

Other articles of this Issue 1/2020

Insights into Imaging 1/2020 Go to the issue

Educational Review

Vascular lesions of the head and neck: an update on classification and imaging review

Educational Review

Sports-related lower limb muscle injuries: pattern recognition approach and MRI review

Statement

Radiology in the era of value-based healthcare: a multi-society expert statement from the ACR, CAR, ESR, IS3R, RANZCR, and RSNA

Educational Review

Imaging methods used in the assessment of environmental disease networks: a brief review for clinicians

Original Article

Trends of interventional radiology procedures during the COVID-19 pandemic: the first 27 weeks in the eye of the storm

Educational Review

Role of MRI as first-line modality in the detection of previously undiagnosed otosclerosis: a single tertiary institute experience