Top

European Radiology

Published in:

Open Access 01-10-2020 | Artificial Intelligence | Imaging Informatics and Artificial Intelligence

Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors

Authors: Lea Strohm, Charisma Hehakaya, Erik R. Ranschaert, Wouter P. C. Boon, Ellen H. M. Moors

Published in: European Radiology | Issue 10/2020

Abstract

Objective

The objective was to identify barriers and facilitators to the implementation of artificial intelligence (AI) applications in clinical radiology in The Netherlands.

Materials and methods

Using an embedded multiple case study, an exploratory, qualitative research design was followed. Data collection consisted of 24 semi-structured interviews from seven Dutch hospitals. The analysis of barriers and facilitators was guided by the recently published Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework for new medical technologies in healthcare organizations.

Results

Among the most important facilitating factors for implementation were the following: (i) pressure for cost containment in the Dutch healthcare system, (ii) high expectations of AI’s potential added value, (iii) presence of hospital-wide innovation strategies, and (iv) presence of a “local champion.” Among the most prominent hindering factors were the following: (i) inconsistent technical performance of AI applications, (ii) unstructured implementation processes, (iii) uncertain added value for clinical practice of AI applications, and (iv) large variance in acceptance and trust of direct (the radiologists) and indirect (the referring clinicians) adopters.

Conclusion

In order for AI applications to contribute to the improvement of the quality and efficiency of clinical radiology, implementation processes need to be carried out in a structured manner, thereby providing evidence on the clinical added value of AI applications.

Key Points

• Successful implementation of AI in radiology requires collaboration between radiologists and referring clinicians.

• Implementation of AI in radiology is facilitated by the presence of a local champion.

• Evidence on the clinical added value of AI in radiology is needed for successful implementation.

Available only for authorised users

BoneXpert is currently used in over 70 European hospitals, of which eight are located in The Netherlands [26]. From the eight hospitals, seven were included in the sample, due to non-response of the eighth hospital.

Obermeyer Z, Emanuel EJ (2016) Predicting the future — big data, machine learning, and clinical medicine. N Engl J Med 375(13):1216–1219. https://doi.org/10.1056/NEJMp1606181.PredictingCrossRefPubMedPubMedCentral

European Society of Radiology (ESR) (2019) Impact of artificial intelligence on radiology: a EuroAIM survey among members of the European Society of Radiology (ESR). Insights Imaging 10(1). https://doi.org/10.1186/s13244-019-0798-3

European Society of Radiology (ESR) (2019) What the radiologist should know about artificial intelligence – an ESR white paper. Insights Imaging 10(1):44. https://doi.org/10.1186/s13244-019-0738-2

Liew C (2018) The future of radiology augmented with Artificial Intelligence: a strategy for success. Eur J Radiol 102:152–156. https://doi.org/10.1016/j.ejrad.2018.03.019

Sogani J, Allen B Jr, Dreyer K, McGinty G (2019) Artificial intelligence in radiology: the ecosystem essential to improving patient care. Clin Imaging 59(July 2019):8–11. https://doi.org/10.1016/j.clinimag.2019.08.001

He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K (2019) The practical implementation of artificial intelligence technologies in medicine. Nat Med 25(1):30–36. https://doi.org/10.1038/s41591-018-0307-0CrossRefPubMedPubMedCentral

Jha S, Topol EJ (2016) Adapting to artificial intelligence: Radiologists and pathologists as information specialists. JAMA 316(22):2353–2354. https://doi.org/10.1001/jama.2016.17438CrossRefPubMed

Char DS, Shah NH, Magnus D (2018) Implementing machine learning in health care ’ addressing ethical challenges. N Engl J Med 378(11):981–983. https://doi.org/10.1056/NEJMp1714229CrossRefPubMedPubMedCentral

Chockley K, Emanuel E (2016) The end of radiology? Three threats to the future practice of radiology. J Am Coll Radiol 13(12):1415–1420. https://doi.org/10.1016/j.jacr.2016.07.010CrossRefPubMed

10.

Choy G, Samir AE, Brink JA (2018) Current applications and future impact of machine learning in radiology. Radiology 288(2):318–328. https://doi.org/10.1148/radiol.2018171820CrossRefPubMed

11.

Mazurowski MA (2019) Artificial intelligence may cause a significant disruption to the radiology workforce. J Am Coll Radiol 16(8):1077–1082. https://doi.org/10.1016/j.jacr.2019.01.026CrossRefPubMed

12.

Dreyer KJ, Geis JR (2017) When machines think: Radiology’s next frontier. Radiology. 285(3):713–718. https://doi.org/10.1148/radiol.2017171183CrossRefPubMed

13.

Yu KH, Kohane IS (2019) Framing the challenges of artificial intelligence in medicine. BMJ Qual Saf 28(3):238–241. https://doi.org/10.1136/bmjqs-2018-008551CrossRefPubMed

14.

van Ginneken B, Schaefer-Prokop CM, Prokop M (2011) Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology. 261(3):719–732. https://doi.org/10.1148/radiol.11091710CrossRefPubMed

15.

Kohli A, Jha S (2018) Why CAD failed in mammography. J Am Coll Radiol 15(3):535–537. https://doi.org/10.1016/j.jacr.2017.12.029CrossRefPubMed

16.

Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL (2018) Artificial intelligence in radiology. Nat Rev Cancer 18(8):500–510. https://doi.org/10.1038/s41568-018-0016-5CrossRefPubMedPubMedCentral

17.

Nishikawa RM, Bae KT (2018) Importance of better human-computer interaction in the era of deep learning: mammography computer-aided diagnosis as a use case. J Am Coll Radiol 15(1):49–52. https://doi.org/10.1016/j.jacr.2017.08.027CrossRefPubMed

18.

Pope C, Halford S, Turnbull J, Prichard J, Calestani M, May C (2013) Using computer decision support systems in NHS emergency and urgent care: Ethnographic study using normalisation process theory. BMC Health Serv Res 13(1)

19.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O (2004) Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 82(4):581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.xCrossRefPubMedPubMedCentral

20.

Greenhalgh T, Wherton J, Papoutsi C et al (2017) Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 19(11):e367. https://doi.org/10.2196/jmir.8775CrossRefPubMedPubMedCentral

21.

Recht MP, Dewey M, Dreyer K et al (2020) Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. Eur Radiol:1–9. https://doi.org/10.1007/s00330-020-06672-5

22.

AINED (2018) AI Voor Nederland. https://www.vno-ncw.nl/sites/default/files/aivnl_20181106_0.pdf. Accessed 2 Nov 2019

23.

WRR (2016) Big Data in Een Vrije En Veilige Samenleving. Vol 95. Den Haag. https://www.wrr.nl/publicaties/rapporten/2016/04/28/big-data-in-een-vrije-en-veilige-samenleving. Accessed 27 Oct 2019

24.

Nederland Digitaal (2019) Resultaten en opbrengsten van de Conferentie Nederland Digitaal 2019. In: Conferentie Nederland Digitaal. https://www.nederlanddigitaal.nl/documenten/publicaties/2019/03/21/opbrengsten-conferentie-nederland-digitaal-2019. Accessed 21 Nov 2019

25.

Lee H, Tajmir S, Lee J et al (2017) Fully automated deep learning system for bone age assessment. J Digit Imaging 30(4):427–441. https://doi.org/10.1007/s10278-017-9955-8CrossRefPubMedPubMedCentral

26.

Visiana (2018) Testimonials and references. https://www.bonexpert.com/documentation/testimonials-and-references. Accessed November 26, 2018.

27.

Ministerie van Volksgezondheid Welzijn en Sport (2018) Kamerbrief over Aanbieding Bestuurlijk Akkoord (Hoofdlijnenakkoord) Medisch-Specialistische Zorg 2019-2022. https://www.rijksoverheid.nl/documenten/kamerstukken/2018/06/04/kamerbrief-over-hoofdlijnenakkoord-medisch-specialistische-zorg-2019-2022. Accessed 21 Nov 2019

28.

Nederlandse Vereniging voor Radiologie (2016) De Rol van de Radioloog in 2020. Utrecht. https://www.radiologen.nl/system/files/bestanden/documenten/de_rol_van_de_radioloog_in_2020.pdf. Accessed 17 Nov 2019

29.

Nederlandse Vereniging van Ziekenhuizen (2018) Ziekenhuiszorg in Cijfers 2018. Utrecht. https://ziekenhuiszorgincijfers.nl/assets/uploads/NVZ-Brancherapport-2018.pdf

30.

Rubin DL (2019) Artificial intelligence in imaging: the radiologist’s role. J Am Coll Radiol 16(9):1309–1317. https://doi.org/10.1016/j.jacr.2019.05.036CrossRefPubMedPubMedCentral

31.

Pinto dos Santos D, Giese D, Brodehl S et al (2019) Medical students’ attitude towards artificial intelligence: a multicentre survey. Eur Radiol 29(4):1640–1646. https://doi.org/10.1007/s00330-018-5601-1CrossRefPubMed

32.

Parikh RB, Obermeyer Z, Navathe AS (2019) Regulation of predictive analytics in medicine. Science 363(6429):810–812. https://doi.org/10.1126/science.aaw0029CrossRefPubMedPubMedCentral

33.

Tsang L, Kracov DA, Mulryne J et al (2017) The impact of artificial intelligence on medical innovation in the European Union and United States. Intellect Prop Technol Law J 29(8):3–12

34.

Neri E, Coppola F, Miele V, Bibbolino C, Grassi R (2020) Artificial intelligence: who is responsible for the diagnosis? Radiol Med 0123456789. https://doi.org/10.1007/s11547-020-01135-9

35.

Waymel Q, Badr S, Demondion X, Cotten A, Jacques T (2019) Impact of the rise of artificial intelligence in radiology: what do radiologists think? Diagn Interv Imaging 100(6):327–336. https://doi.org/10.1016/j.diii.2019.03.015CrossRefPubMed

36.

Liberati EG, Ruggiero F, Galuppo L et al (2017) What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci 12(1):1–13. https://doi.org/10.1186/s13012-017-0644-2CrossRef

37.

Lugtenberg M, Weenink JW, Van Der Weijden T, Westert GP, Kool RB (2015) Implementation of multiple-domain covering computerized decision support systems in primary care: a focus group study on perceived barriers. BMC Med Inform Decis Mak 15(1):1–11. https://doi.org/10.1186/s12911-015-0205-zCrossRef

38.

Bezemer T, de Groot MC, Blasse E et al (2019) A Human(e) factor in clinical decision support systems. J Med Internet Res 21(3):e11732. https://doi.org/10.2196/11732CrossRefPubMedPubMedCentral

39.

Wade V, Eliott J (2012) The role of the champion in telehealth service development: a qualitative analysis. J Telemed Telecare 18(8):490–492. https://doi.org/10.1258/jtt.2012.GTH115CrossRefPubMed

40.

Marcial LH, Johnston DS, Shapiro MR, Jacobs SR, Blumenfeld B, Rojas SL (2019) A qualitative framework-based evaluation of radiology clinical decision support initiatives: eliciting key factors to physician adoption in implementation. JAMIA Open 2(1):187–196. https://doi.org/10.1093/jamiaopen/ooz002CrossRefPubMedPubMedCentral

41.

Turner S, D’Lima D, Hudson E et al (2017) Evidence use in decision-making on introducing innovations: a systematic scoping review with stakeholder feedback. Implement Sci 12(1):1–12. https://doi.org/10.1186/s13012-017-0669-6CrossRef

42.

Urquhart R, Kendell C, Geldenhuys L et al (2019) The role of scientific evidence in decisions to adopt complex innovations in cancer care settings: A multiple case study in Nova Scotia, Canada. Implement Sci 14(1):1–12. https://doi.org/10.1186/s13012-019-0859-5CrossRef

43.

Rai R, Kumar S, Batumalai V et al (2017) The integration of MRI in radiation therapy: collaboration of radiographers and radiation therapists. J Med Radiat Sci 64(1):61–68. https://doi.org/10.1002/jmrs.225CrossRefPubMedPubMedCentral

44.

Nawrocki T, Maldjian PD, Slasky SE, Contractor SG (2018) Artificial intelligence and radiology: have rumors of the radiologist’s demise been greatly exaggerated? Acad Radiol 25(8):967–972. https://doi.org/10.1016/j.acra.2017.12.027CrossRef

Title: Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors
Authors: Lea Strohm
Charisma Hehakaya
Erik R. Ranschaert
Wouter P. C. Boon
Ellen H. M. Moors
Publication date: 01-10-2020
Publisher: Springer Berlin Heidelberg
Keyword: Artificial Intelligence
Published in: European Radiology / Issue 10/2020
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI: https://doi.org/10.1007/s00330-020-06946-y

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors

Abstract

Objective

Materials and methods

Results

Conclusion

Key Points

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Objective

Materials and methods

Results

Conclusion

Key Points

Please log in to get access to this content

Other articles of this Issue 10/2020

The safety and efficacy of balloon-occluded transcatheter arterial chemoembolization for hepatocellular carcinoma refractory to conventional transcatheter arterial chemoembolization

Crohn’s disease: prevalence, MR features, and clinical significance of enteric and colonic sinus tracts

Patient preparation and image quality in female pelvic MRI: recommendations revisited

Development and validation of an 18F-FDG PET radiomic model for prognosis prediction in patients with nasal-type extranodal natural killer/T cell lymphoma

Functional tumour burden of peritoneal carcinomatosis derived from DWI could predict incomplete tumour debulking in advanced ovarian carcinoma

Scientific papers on musculoskeletal radiology presented at ECR 2020