Top

Published in:

Open Access 01-12-2023 | Artificial Intelligence | Brief Report

Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios

Authors: Marco Cascella, Jonathan Montomoli, Valentina Bellini, Elena Bignami

Published in: Journal of Medical Systems | Issue 1/2023

Abstract

This paper aims to highlight the potential applications and limits of a large language model (LLM) in healthcare. ChatGPT is a recently developed LLM that was trained on a massive dataset of text for dialogue with users. Although AI-based language models like ChatGPT have demonstrated impressive capabilities, it is uncertain how well they will perform in real-world scenarios, particularly in fields such as medicine where high-level and complex thinking is necessary. Furthermore, while the use of ChatGPT in writing scientific articles and other scientific outputs may have potential benefits, important ethical concerns must also be addressed. Consequently, we investigated the feasibility of ChatGPT in clinical and research scenarios: (1) support of the clinical practice, (2) scientific production, (3) misuse in medicine and research, and (4) reasoning about public health topics. Results indicated that it is important to recognize and promote education on the appropriate use and potential pitfalls of AI-based LLMs in medicine.

Floridi L, Chiriatti M (2020) GPT-3: Its Nature, Scope, Limits, and Consequences. Minds & Machines 30: 681–694. https://doi.org/10.1007/s11023-020-09548-1CrossRef

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin L (2017) Attention is All You Need. Advances in Neural Information Processing Systems 30:5998–6008.

Kung HT, Cheatham M, ChatGPT, Medenilla A, Sillos C, De Leon L, Elepaño C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, Tseng V (2022) Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models. medRxiv 2022.12.19.22283643; doi: https://doi.org/10.1101/2022.12.19.22283643

Liévin V, Egeberg Hother C, Winther O (2022). Can large language models reason about medical questions? arXiv. doi: https://doi.org/10.48550/ARXIV.2207.08143.

Hutson M (2022) Could AI help you to write your next paper? Nature 611(7934):192–193. doi: https://doi.org/10.1038/d41586-022-03479-w.CrossRefPubMed

Else H (2023) Abstracts written by ChatGPT fool scientists. Nature 613(7944):423. doi: https://doi.org/10.1038/d41586-023-00056-7.CrossRefPubMed

Andersen-Ranberg NC, Poulsen LM, Perner A, Wetterslev J, Estrup S, Hästbacka J, Morgan M, Citerio G, Caballero J, Lange T, Kjær MN, Ebdrup BH, Engstrøm J, Olsen MH, Oxenbøll Collet M, Mortensen CB, Weber SO, Andreasen AS, Bestle MH, Uslu B, Scharling Pedersen H, Gramstrup Nielsen L, Toft Boesen HC, Jensen JV, Nebrich L, La Cour K, Laigaard J, Haurum C, Olesen MW, Overgaard-Steensen C, Westergaard B, Brand B, Kingo Vesterlund G, Thornberg Kyhnauv P, Mikkelsen VS, Hyttel-Sørensen S, de Haas I, Aagaard SR, Nielsen LO, Eriksen AS, Rasmussen BS, Brix H, Hildebrandt T, Schønemann-Lund M, Fjeldsøe-Nielsen H, Kuivalainen AM, Mathiesen O; AID-ICU Trial Group (2022) Haloperidol for the Treatment of Delirium in ICU Patients. N Engl J Med 387(26):2425–2435. doi: https://doi.org/10.1056/NEJMoa2211868.CrossRefPubMed

Cheskes S, Verbeek PR, Drennan IR, McLeod SL, Turner L, Pinto R, Feldman M, Davis M, Vaillancourt C, Morrison LJ, Dorian P, Scales DC (2022) Defibrillation Strategies for Refractory Ventricular Fibrillation. N Engl J Med 387(21):1947–1956. doi: https://doi.org/10.1056/NEJMoa2207304.CrossRefPubMed

Devos D, Labreuche J, Rascol O, Corvol JC, Duhamel A, Guyon Delannoy P, Poewe W, Compta Y, Pavese N, Růžička E, Dušek P, Post B, Bloem BR, Berg D, Maetzler W, Otto M, Habert MO, Lehericy S, Ferreira J, Dodel R, Tranchant C, Eusebio A, Thobois S, Marques AR, Meissner WG, Ory-Magne F, Walter U, de Bie RMA, Gago M, Vilas D, Kulisevsky J, Januario C, Coelho MVS, Behnke S, Worth P, Seppi K, Ouk T, Potey C, Leclercq C, Viard R, Kuchcinski G, Lopes R, Pruvo JP, Pigny P, Garçon G, Simonin O, Carpentier J, Rolland AS, Nyholm D, Scherfler C, Mangin JF, Chupin M, Bordet R, Dexter DT, Fradette C, Spino M, Tricta F, Ayton S, Bush AI, Devedjian JC, Duce JA, Cabantchik I, Defebvre L, Deplanque D, Moreau C; FAIRPARK-II Study Group (2022) Trial of Deferiprone in Parkinson’s Disease. N Engl J Med 387(22):2045–2055. doi: https://doi.org/10.1056/NEJMoa2209254.

10.

Hugosson J, Månsson M, Wallström J, Axcrona U, Carlsson SV, Egevad L, Geterud K, Khatami A, Kohestani K, Pihl CG, Socratous A, Stranne J, Godtman RA, Hellström M; GÖTEBORG-2 Trial Investigators (2022) Prostate Cancer Screening with PSA and MRI Followed by Targeted Biopsy Only. N Engl J Med 387(23):2126–2137. doi: https://doi.org/10.1056/NEJMoa2209454.CrossRefPubMedPubMedCentral

11.

Furie RA, van Vollenhoven RF, Kalunian K, Navarra S, Romero-Diaz J, Werth VP, Huang X, Clark G, Carroll H, Meyers A, Musselli C, Barbey C, Franchimont N; LILAC Trial Investigators (2022) Trial of Anti-BDCA2 Antibody Litifilimab for Systemic Lupus Erythematosus. N Engl J Med 387(10):894–904. doi: https://doi.org/10.1056/NEJMoa2118025.CrossRefPubMed

12.

Stokel-Walker C (2023) ChatGPT listed as author on research papers: many scientists disapprove. Nature. 2023 Jan 18. doi: https://doi.org/10.1038/d41586-023-00107-z.

13.

Lee SB, Oh JH, Park JH, Choi SP, Wee JH (2018) Differences in youngest-old, middle-old, and oldest-old patients who visit the emergency department. Clin Exp Emerg Med 5(4):249–255. doi: https://doi.org/10.15441/ceem.17.261.CrossRefPubMedPubMedCentral

14.

Gordijn B, Have HT (2023) ChatGPT: evolution or revolution? Med Health Care Philos. 2023 Jan 19. doi: https://doi.org/10.1007/s11019-023-10136-0.

Title: Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios
Authors: Marco Cascella
Jonathan Montomoli
Valentina Bellini
Elena Bignami
Publication date: 01-12-2023
Publisher: Springer US
Keyword: Artificial Intelligence
Published in: Journal of Medical Systems / Issue 1/2023
Print ISSN: 0148-5598
Electronic ISSN: 1573-689X
DOI: https://doi.org/10.1007/s10916-023-01925-4

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios

Abstract

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2023

calf – Software for CEST Analysis with Lorentzian Fitting

Migrants face Barriers to Obtaining Electronic Identification: A population-based Study Among Older Russian-speakers in Finland

A Federated Learning Approach to Tumor Detection in Colon Histology Images

Multi-scale Bottleneck Residual Network for Retinal Vessel Segmentation

Case Volume Justification of 3D-Navigated Spinal Procedures: A Cost-Benefit Analysis

Digital Interventions for Social Participation in Adults with Long-term Physical Conditions: A Systematic Review