Top

European Archives of Oto-Rhino-Laryngology

Published in:

12-02-2024 | Artificial Intelligence | Short Communication

Exploring the role of ChatGPT in clinical decision-making in otorhinolaryngology: a ChatGPT designed study

Authors: Francisco Teixeira-Marques, Nuno Medeiros, Francisco Nazaré, Sandra Alves, Nuno Lima, Leandro Ribeiro, Rita Gama, Pedro Oliveira

Published in: European Archives of Oto-Rhino-Laryngology | Issue 4/2024

Abstract

Purpose

Since the beginning of 2023, ChatGPT emerged as a hot topic in healthcare research. The potential to be a valuable tool in clinical practice is compelling, particularly in improving clinical decision support by helping physicians to make clinical decisions based on the best medical knowledge available. We aim to investigate ChatGPT’s ability to identify, diagnose and manage patients with otorhinolaryngology-related symptoms.

Methods

A prospective, cross-sectional study was designed based on an idea suggested by ChatGPT to assess the level of agreement between ChatGPT and five otorhinolaryngologists (ENTs) in 20 reality-inspired clinical cases. The clinical cases were presented to the chatbot on two different occasions (ChatGPT-1 and ChatGPT-2) to assess its temporal stability.

Results

The mean score of ChatGPT-1 was 4.4 (SD 1.2; min 1, max 5) and of ChatGPT-2 was 4.15 (SD 1.3; min 1, max 5), while the ENTs mean score was 4.91 (SD 0.3; min 3, max 5). The Mann–Whitney U test revealed a statistically significant difference (p < 0.001) between both ChatGPT’s and the ENTs’s score. ChatGPT-1 and ChatGPT-2 gave different answers in five occasions.

Conclusions

Artificial intelligence will be an important instrument in clinical decision-making in the near future and ChatGPT is the most promising chatbot so far. Despite needing further development to be used with safety, there is room for improvement and potential to aid otorhinolaryngology residents and specialists in making the most correct decision for the patient.

National Center for Biotechnology Information (NCBI) (1988) Bethesda (MD): National Library of Medicine (US), National Center for Biotechnology Information. Available from: https://pubmed.ncbi.nlm.nih.gov/?term=chatgpt&timeline=expanded&timeline=expanded

Chiesa-Estomba CM, Lechien JR, Vaira LA et al (2023) Exploring the potential of Chat-GPT as a supportive tool for sialoendoscopy clinical decision making a patient information support. Eur Arch Oto-Rhino-L. https://doi.org/10.1007/s00405-023-08104-8CrossRef

Densen P (2011) Challenges and opportunities facing medical education. Trans Am Coin Climate Assoc 122:48–58

Grunebaum A, Chervenak J, Pollet SL, Katz A, Chervenak FA (2023) The exciting potential for ChatGPT in obstetrics and gynecology. Am K Obstet Gynecol. https://doi.org/10.1016/j.ajog.2023.03.009CrossRef

Liu J, Wang C, Liu S (2023) Utility of ChatGPT in clinical practice. J Med Internet Res. https://doi.org/10.2196/48568CrossRefPubMedPubMedCentral

ChatGPT: Optimizing Language Models for Dialogue (2023) OpenAI. Published November 30, 2022. https://openai.com/blog/chatgpt/. Accessed 1st Aug 2023.

Morthorst T, Roy N (2023) Decoding ChatGPT—Why ChatGPT’s potential is yet to be realised. EY. https://www.ey.com/en_ie/consulting/why-chatgpts-potential-is-yet-to-be-realised. Accessed 22 Sep 2023.

Patel SB, Lam K (2023) ChatGPT: the future of discharge summaries? Lancet Digital Health. https://doi.org/10.1016/S2589-7500(23)00021-3CrossRefPubMed

Rao A, Pang M, Kim J et al (2023) Assessing the utility of ChatGPT throughout the entire clinical workflow: development and usability study. J Med Internet Res. https://doi.org/10.2196/48659CrossRefPubMedPubMedCentral

10.

Liu S, Wright AP, Patterson BL et al (2023) Using AI-generated suggestions from ChatGPT to optimize clinical decision support. J Am Med Inform Assoc. https://doi.org/10.1093/jamia/ocad072CrossRefPubMedPubMedCentral

11.

Editorial (2023) ChatGPT: friend or foe? Lancet Digital Health. https://doi.org/10.1016/S2589-7500(23)00023-7CrossRef

12.

Koch CC, Wollenberg B, Luers JC et al (2023) ChatGPT’s quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions. Eur Arch Oto-Rhino-L. https://doi.org/10.1007/s00405-023-08051-4CrossRef

13.

Vaira LA, Lechien JR, Abbate V et al (2023) Accuracy of ChatGPT-Generated information on Head and Neck and Oromaxillofacial surgery: a multicenter collaborative analysis. Otolaryngol Head Neck Surg. https://doi.org/10.1002/ohn.489CrossRefPubMed

14.

Qu RW, Qureshi U, Petersen G et al (2023) Diagnostic and management applications of ChatGPT in structured otolaryngology clinical scenarios. OTO Open. https://doi.org/10.1002/oto2.67CrossRefPubMedPubMedCentral

15.

Zalzal HG, Cheng J, Shah RK (2023) Evaluating the current ability of ChatGPT to assist in professional otolaryngology evaluation. OTO Open. https://doi.org/10.1002/oto2.94CrossRefPubMedPubMedCentral

16.

Buhr CR, Smith H, Huppertz T et al (2023) ChatGPT versus consultants: blinded evaluation on answering otorhinolaryngology case-based questions. JMIR Med Educ. https://doi.org/10.2196/49183CrossRefPubMedPubMedCentral

17.

Yeo YH, Samaan JS, Ng WH et al (2023) Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin Mol Hepatol. https://doi.org/10.3350/cmh.2023.0089CrossRefPubMedPubMedCentral

18.

Potapenko I, Boberg-Ans LC, Hansen MS, Klefter ON, van Dijk EHC, Subhi Y (2023) Artificial intelligence-based chatbot patient information on common retinal diseases using ChatGPT. Acta Ophtalmol. https://doi.org/10.1111/aos.15661CrossRef

19.

Shen Y, Heacock L, Elias J et al (2023) ChatGPT and other large language models are double-edged swords. Radiology. https://doi.org/10.1148/radiol.230163CrossRefPubMedPubMedCentral

Title: Exploring the role of ChatGPT in clinical decision-making in otorhinolaryngology: a ChatGPT designed study
Authors: Francisco Teixeira-Marques
Nuno Medeiros
Francisco Nazaré
Sandra Alves
Nuno Lima
Leandro Ribeiro
Rita Gama
Pedro Oliveira
Publication date: 12-02-2024
Publisher: Springer Berlin Heidelberg
Keyword: Artificial Intelligence
Published in: European Archives of Oto-Rhino-Laryngology / Issue 4/2024
Print ISSN: 0937-4477
Electronic ISSN: 1434-4726
DOI: https://doi.org/10.1007/s00405-024-08498-z

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Exploring the role of ChatGPT in clinical decision-making in otorhinolaryngology: a ChatGPT designed study

Abstract

Purpose

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 4/2024

Anterolateral advancement pharyngoplasty versus barbed reposition pharyngoplasty in patients with obstructive sleep apnea

Accuracy of ChatGPT-3.5 and -4 in providing scientific references in otolaryngology–head and neck surgery

A deep learning pipeline for automated classification of vocal fold polyps in flexible laryngoscopy

Endoscopic transorbital decompression for traumatic superior orbital fissure syndrome: from cadaveric study to clinical application

Comment on: Tranexamic acid in bleeding reduction and operative time of nasal surgeries: systematic review and meta‐analysis

Causal analysis between gastroesophageal reflux disease and chronic rhinosinusitis