Skip to main content
Top
Published in: Pediatric Radiology 11/2022

23-08-2022 | Radiology | Artificial intelligence in pediatric radiology

The requirements for performing artificial-intelligence-related research and model development

Authors: Anuj Pareek, Matthew P. Lungren, Safwan S. Halabi

Published in: Pediatric Radiology | Issue 11/2022

Login to get access

Abstract

Artificial intelligence research in health care has undergone tremendous growth in the last several years thanks to the explosion of digital health care data and systems that can leverage large amounts of data to learn patterns that can be applied to clinical tasks. In addition, given broad acceleration in machine learning across industries like transportation, media and commerce, there has been a significant growth in demand for machine-learning practitioners such as engineers and data scientists, who have skill sets that can be applied to health care use cases but who simultaneously lack important health care domain expertise. The purpose of this paper is to discuss the requirements of building an artificial-intelligence research enterprise including the research team, technical software/hardware, and procurement and curation of health care data.
Literature
1.
go back to reference Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56CrossRef Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56CrossRef
6.
go back to reference Schuur F, Rezazade Mehrizi MH, Ranschaert E (2021) Training opportunities of artificial intelligence (AI) in radiology: a systematic review. Eur Radiol 31:6021–6029CrossRef Schuur F, Rezazade Mehrizi MH, Ranschaert E (2021) Training opportunities of artificial intelligence (AI) in radiology: a systematic review. Eur Radiol 31:6021–6029CrossRef
8.
go back to reference Willemink MJ, Koszek WA, Hardell C et al (2020) Preparing medical imaging data for machine learning. Radiology 295:4–15CrossRef Willemink MJ, Koszek WA, Hardell C et al (2020) Preparing medical imaging data for machine learning. Radiology 295:4–15CrossRef
9.
go back to reference Recht MP, Zbontar J, Sodickson DK et al (2020) Using deep learning to accelerate knee MRI at 3 T: results of an interchangeability study. AJR Am J Roentgenol 215:1421–1429CrossRef Recht MP, Zbontar J, Sodickson DK et al (2020) Using deep learning to accelerate knee MRI at 3 T: results of an interchangeability study. AJR Am J Roentgenol 215:1421–1429CrossRef
10.
go back to reference Chaudhari AS, Mittra E, Davidzon GA et al (2021) Low-count whole-body PET with deep learning in a multicenter and externally validated study. NPJ Digit Med 4:1–11 Chaudhari AS, Mittra E, Davidzon GA et al (2021) Low-count whole-body PET with deep learning in a multicenter and externally validated study. NPJ Digit Med 4:1–11
11.
go back to reference Irvin J, Rajpurkar P, Ko M et al (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 590–597 Irvin J, Rajpurkar P, Ko M et al (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 590–597
12.
go back to reference Davenport TH, Patil DJ (2012) Data scientist. Harv Bus Rev 90:70–76PubMed Davenport TH, Patil DJ (2012) Data scientist. Harv Bus Rev 90:70–76PubMed
13.
go back to reference Dai W, Berleant D (2019) Benchmarking contemporary deep learning hardware and frameworks: a survey of qualitative metrics. In: 2019 IEEE First International Conference on Cognitive Machine Intelligence (CogMI), pp 148–155 Dai W, Berleant D (2019) Benchmarking contemporary deep learning hardware and frameworks: a survey of qualitative metrics. In: 2019 IEEE First International Conference on Cognitive Machine Intelligence (CogMI), pp 148–155
14.
15.
go back to reference Jouppi NP, Young C, Patil N et al (2017) In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, ACM, New York, pp 1–12 Jouppi NP, Young C, Patil N et al (2017) In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, ACM, New York, pp 1–12
16.
go back to reference Lemley J, Bazrafkan S, Corcoran P (2017) Deep learning for consumer devices and services: pushing the limits for machine learning, artificial intelligence, and computer vision. IEEE Consum Electron Mag 6:48–56CrossRef Lemley J, Bazrafkan S, Corcoran P (2017) Deep learning for consumer devices and services: pushing the limits for machine learning, artificial intelligence, and computer vision. IEEE Consum Electron Mag 6:48–56CrossRef
17.
go back to reference Van Rossum G, Drake FL (2009) Python 3 reference manual. CreateSpace Independent Publishing Platform, North Charleston Van Rossum G, Drake FL (2009) Python 3 reference manual. CreateSpace Independent Publishing Platform, North Charleston
18.
go back to reference Abadi M, Barham P, Chen J et al (2016) TensorFlow: a system for large-scale machine learning. In: 12th USENIX Symposium on operating Systems Design and Implementation (OSDI 16), pp 265–283 Abadi M, Barham P, Chen J et al (2016) TensorFlow: a system for large-scale machine learning. In: 12th USENIX Symposium on operating Systems Design and Implementation (OSDI 16), pp 265–283
19.
go back to reference Paszke A, Gross S, Massa F et al (2019) PyTorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems 32. Curran Associates Inc., pp 8024–8035 Paszke A, Gross S, Massa F et al (2019) PyTorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems 32. Curran Associates Inc., pp 8024–8035
20.
go back to reference Chirodea MC, Novac OC, Novac CM et al (2021) Comparison of Tensorflow and PyTorch in convolutional neural network–based applications. In: 2021 13th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), pp 1–6 Chirodea MC, Novac OC, Novac CM et al (2021) Comparison of Tensorflow and PyTorch in convolutional neural network–based applications. In: 2021 13th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), pp 1–6
23.
go back to reference Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830 Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830
24.
go back to reference Esteva A, Robicquet A, Ramsundar B et al (2019) A guide to deep learning in healthcare. Nat Med 25:24–29CrossRef Esteva A, Robicquet A, Ramsundar B et al (2019) A guide to deep learning in healthcare. Nat Med 25:24–29CrossRef
25.
go back to reference McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94CrossRef McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94CrossRef
26.
go back to reference Hannun AY, Rajpurkar P, Haghpanahi M et al (2019) Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat Med 25:65–69CrossRef Hannun AY, Rajpurkar P, Haghpanahi M et al (2019) Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat Med 25:65–69CrossRef
27.
go back to reference Wang F, Casalino LP, Khullar D (2019) Deep learning in medicine — promise, progress, and challenges. JAMA Intern Med 179:293–294CrossRef Wang F, Casalino LP, Khullar D (2019) Deep learning in medicine — promise, progress, and challenges. JAMA Intern Med 179:293–294CrossRef
28.
go back to reference Voss SD, Reaman GH, Kaste SC, Slovis TL (2009) The ALARA concept in pediatric oncology. Pediatr Radiol 39:1142–1146CrossRef Voss SD, Reaman GH, Kaste SC, Slovis TL (2009) The ALARA concept in pediatric oncology. Pediatr Radiol 39:1142–1146CrossRef
29.
go back to reference Candemir S, Nguyen XV, Folio LR, Prevedello LM (2021) Training strategies for radiology deep learning models in data — limited scenarios. Radiol Artif Intell 3:e210014CrossRef Candemir S, Nguyen XV, Folio LR, Prevedello LM (2021) Training strategies for radiology deep learning models in data — limited scenarios. Radiol Artif Intell 3:e210014CrossRef
30.
go back to reference Azizi S, Mustafa B, Ryan F et al (2021) Big self-supervised models advance medical image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3478–3488 Azizi S, Mustafa B, Ryan F et al (2021) Big self-supervised models advance medical image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3478–3488
31.
go back to reference Kotia J, Kotwal A, Bharti R, Mangrulkar R (2021) Few shot learning for medical imaging. In: Das SK, Das SP, Dey N, Hassanien A-E (eds) Machine learning algorithms for industrial applications. Springer International Publishing, Cham, pp 107–132CrossRef Kotia J, Kotwal A, Bharti R, Mangrulkar R (2021) Few shot learning for medical imaging. In: Das SK, Das SP, Dey N, Hassanien A-E (eds) Machine learning algorithms for industrial applications. Springer International Publishing, Cham, pp 107–132CrossRef
32.
go back to reference Medela A, Picon A, Saratxaga CL et al (2019) Few shot learning in histopathological images: reducing the need of labeled data on biological datasets. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp 1860–1864 Medela A, Picon A, Saratxaga CL et al (2019) Few shot learning in histopathological images: reducing the need of labeled data on biological datasets. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp 1860–1864
33.
go back to reference Rieke N, Hancox J, Li W et al (2020) The future of digital health with federated learning. NPJ Digit Med 3:119CrossRef Rieke N, Hancox J, Li W et al (2020) The future of digital health with federated learning. NPJ Digit Med 3:119CrossRef
34.
go back to reference Huang S-C, Pareek A, Seyyedi S et al (2020) Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. NPJ Digit Med 3:136CrossRef Huang S-C, Pareek A, Seyyedi S et al (2020) Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. NPJ Digit Med 3:136CrossRef
35.
go back to reference Bluemke DA, Moy L, Bredella MA et al (2020) Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers — from the radiology editorial board. Radiology 294:487–489CrossRef Bluemke DA, Moy L, Bredella MA et al (2020) Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers — from the radiology editorial board. Radiology 294:487–489CrossRef
36.
go back to reference Eelbode T, Sinonquel P, Maes F, Bisschops R (2021) Pitfalls in training and validation of deep learning systems. Best Pract Res Clin Gastroenterol 52–53:101712CrossRef Eelbode T, Sinonquel P, Maes F, Bisschops R (2021) Pitfalls in training and validation of deep learning systems. Best Pract Res Clin Gastroenterol 52–53:101712CrossRef
37.
go back to reference Hestness J, Ardalani N, Diamos G (2019) Beyond human-level accuracy: computational challenges in deep learning. In: Proceedings of the 24th Symposium on Principles and Practice of Parallel Programming. ACM, New York, pp 1–14 Hestness J, Ardalani N, Diamos G (2019) Beyond human-level accuracy: computational challenges in deep learning. In: Proceedings of the 24th Symposium on Principles and Practice of Parallel Programming. ACM, New York, pp 1–14
38.
go back to reference Agarwal V, Podchiyska T, Banda JM et al (2016) Learning statistical models of phenotypes using noisy labeled training data. J Am Med Inform Assoc 23:1166–1173CrossRef Agarwal V, Podchiyska T, Banda JM et al (2016) Learning statistical models of phenotypes using noisy labeled training data. J Am Med Inform Assoc 23:1166–1173CrossRef
39.
go back to reference Karimi D, Dou H, Warfield SK, Gholipour A (2020) Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. Med Image Anal 65:101759CrossRef Karimi D, Dou H, Warfield SK, Gholipour A (2020) Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. Med Image Anal 65:101759CrossRef
40.
go back to reference Rolnick D, Veit A, Belongie S, Shavit N (2017) Deep learning is robust to massive label noise. arXiv:1705.10694 Rolnick D, Veit A, Belongie S, Shavit N (2017) Deep learning is robust to massive label noise. arXiv:1705.10694
41.
go back to reference Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2261–2269 Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2261–2269
42.
go back to reference Tan M, Le Q (2019) EfficientNet: rethinking model scaling for convolutional neural networks. Proc Mach Learn Res 97:6105–6114 Tan M, Le Q (2019) EfficientNet: rethinking model scaling for convolutional neural networks. Proc Mach Learn Res 97:6105–6114
43.
go back to reference Bischl B, Binder M, Lang M et al (2021) Hyperparameter optimization: foundations, algorithms, best practices and open challenges. arXiv:210705847 Bischl B, Binder M, Lang M et al (2021) Hyperparameter optimization: foundations, algorithms, best practices and open challenges. arXiv:210705847
44.
go back to reference Alibrahim H, Ludwig SA (2021) Hyperparameter optimization: comparing genetic algorithm against grid search and Bayesian optimization. In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp 1551–1559 Alibrahim H, Ludwig SA (2021) Hyperparameter optimization: comparing genetic algorithm against grid search and Bayesian optimization. In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp 1551–1559
45.
go back to reference Akiba T, Sano S, Yanase T et al (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, New York, pp 2623–2631 Akiba T, Sano S, Yanase T et al (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, New York, pp 2623–2631
46.
go back to reference Singh R, Bharti V, Purohit V et al (2021) MetaMed: few-shot medical image classification using gradient-based meta-learning. Pattern Recogn 120:108111CrossRef Singh R, Bharti V, Purohit V et al (2021) MetaMed: few-shot medical image classification using gradient-based meta-learning. Pattern Recogn 120:108111CrossRef
47.
go back to reference Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. Proc Mach Learn Res 119:1597–1607 Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. Proc Mach Learn Res 119:1597–1607
48.
go back to reference Sowrirajan H, Yang J, Ng AY, Rajpurkar P (2021) MoCo pretraining improves representation and transferability of chest X-ray models. Proc Mach Learn Res 143:728–744 Sowrirajan H, Yang J, Ng AY, Rajpurkar P (2021) MoCo pretraining improves representation and transferability of chest X-ray models. Proc Mach Learn Res 143:728–744
49.
go back to reference Couture HD, Williams LA, Geradts J et al (2018) Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer 4:30CrossRef Couture HD, Williams LA, Geradts J et al (2018) Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer 4:30CrossRef
50.
go back to reference Chaudhari AS, Sandino CM, Cole EK et al (2021) Prospective deployment of deep learning in MRI: a framework for important considerations, challenges, and recommendations for best practices. J Magn Reson Imaging 54:357–371CrossRef Chaudhari AS, Sandino CM, Cole EK et al (2021) Prospective deployment of deep learning in MRI: a framework for important considerations, challenges, and recommendations for best practices. J Magn Reson Imaging 54:357–371CrossRef
51.
go back to reference Abràmoff MD, Lavin PT, Birch M et al (2018) Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 1:39CrossRef Abràmoff MD, Lavin PT, Birch M et al (2018) Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 1:39CrossRef
52.
go back to reference Wu E, Wu K, Daneshjou R et al (2021) How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med 27:582–584CrossRef Wu E, Wu K, Daneshjou R et al (2021) How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med 27:582–584CrossRef
53.
go back to reference Benjamens S, Dhunnoo P, Meskó B (2020) The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit Med 3:118CrossRef Benjamens S, Dhunnoo P, Meskó B (2020) The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit Med 3:118CrossRef
Metadata
Title
The requirements for performing artificial-intelligence-related research and model development
Authors
Anuj Pareek
Matthew P. Lungren
Safwan S. Halabi
Publication date
23-08-2022
Publisher
Springer Berlin Heidelberg
Published in
Pediatric Radiology / Issue 11/2022
Print ISSN: 0301-0449
Electronic ISSN: 1432-1998
DOI
https://doi.org/10.1007/s00247-022-05483-8

Other articles of this Issue 11/2022

Pediatric Radiology 11/2022 Go to the issue