Skip to main content
Top
Published in:

03-01-2024 | Artificial Intelligence | Research

Progression of an Artificial Intelligence Chatbot (ChatGPT) for Pediatric Cardiology Educational Knowledge Assessment

Authors: Michael N. Gritti, Hussain AlTurki, Pedrom Farid, Conall T. Morgan

Published in: Pediatric Cardiology | Issue 2/2024

Login to get access

Abstract

Artificial intelligence chatbots, like ChatGPT, have become powerful tools that are disrupting how humans interact with technology. The potential uses within medicine are vast. In medical education, these chatbots have shown improvements, in a short time span, in generalized medical examinations. We evaluated the overall performance and improvement between ChatGPT 3.5 and 4.0 in a test of pediatric cardiology knowledge. ChatGPT 3.5 and ChatGPT 4.0 were used to answer text-based multiple-choice questions derived from a Pediatric Cardiology Board Review textbook. Each chatbot was given an 88 question test, subcategorized into 11 topics. We excluded questions with modalities other than text (sound clips or images). Statistical analysis was done using an unpaired two-tailed t-test. Of the same 88 questions, ChatGPT 4.0 answered 66% of the questions correctly (n = 58/88) which was significantly greater (p < 0.0001) than ChatGPT 3.5, which only answered 38% (33/88). The ChatGPT 4.0 version also did better on each subspeciality topic as compared to ChatGPT 3.5. While acknowledging that ChatGPT does not yet offer subspecialty level knowledge in pediatric cardiology, the performance in pediatric cardiology educational assessments showed a considerable improvement in a short period of time between ChatGPT 3.5 and 4.0.
Literature
1.
2.
go back to reference Bart NK, Pepe S, Gregory AT, Denniss AR (2023) Emerging roles of Artificial Intelligence (AI) in cardiology: benefits and barriers in a “Brave New World.” Heart Lung Circ 32(8):883–888CrossRefPubMed Bart NK, Pepe S, Gregory AT, Denniss AR (2023) Emerging roles of Artificial Intelligence (AI) in cardiology: benefits and barriers in a “Brave New World.” Heart Lung Circ 32(8):883–888CrossRefPubMed
3.
go back to reference Dave T, Athaluri SA, Singh S (2023) ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell 6:1169595CrossRefPubMedPubMedCentral Dave T, Athaluri SA, Singh S (2023) ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell 6:1169595CrossRefPubMedPubMedCentral
4.
go back to reference Sardana D, Fagan TR, Wright JT (2023) ChatGPT: a disruptive innovation or disrupting innovation in academia? J Am Dent Assoc 154(5):361–364CrossRefPubMed Sardana D, Fagan TR, Wright JT (2023) ChatGPT: a disruptive innovation or disrupting innovation in academia? J Am Dent Assoc 154(5):361–364CrossRefPubMed
5.
go back to reference Gao CA, Howard FM, Markov NS et al (2023) Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. NPJ Digit Med 6(1):75CrossRefPubMedPubMedCentral Gao CA, Howard FM, Markov NS et al (2023) Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. NPJ Digit Med 6(1):75CrossRefPubMedPubMedCentral
6.
go back to reference Ruksakulpiwat S, Kumar A, Ajibade A (2023) Using ChatGPT in medical research: current status and future directions. J Multidiscip Healthc 16:1513–1520CrossRefPubMedPubMedCentral Ruksakulpiwat S, Kumar A, Ajibade A (2023) Using ChatGPT in medical research: current status and future directions. J Multidiscip Healthc 16:1513–1520CrossRefPubMedPubMedCentral
7.
go back to reference Teixeira da Silva JA (2023) Can ChatGPT rescue or assist with language barriers in healthcare communication? Patient Educ Couns 115:107940CrossRefPubMed Teixeira da Silva JA (2023) Can ChatGPT rescue or assist with language barriers in healthcare communication? Patient Educ Couns 115:107940CrossRefPubMed
8.
go back to reference Mihalache A, Popovic MM, Muni RH (2023) Performance of an Artificial Intelligence Chatbot in ophthalmic knowledge assessment. JAMA Ophthalmol 141(6):589–597CrossRefPubMed Mihalache A, Popovic MM, Muni RH (2023) Performance of an Artificial Intelligence Chatbot in ophthalmic knowledge assessment. JAMA Ophthalmol 141(6):589–597CrossRefPubMed
10.
go back to reference Patel SB, Lam K (2023) ChatGPT: the future of discharge summaries? Lancet Digit Health 5(3):e107–e108CrossRefPubMed Patel SB, Lam K (2023) ChatGPT: the future of discharge summaries? Lancet Digit Health 5(3):e107–e108CrossRefPubMed
11.
go back to reference Singh S, Djalilian A, Ali MJ (2023) ChatGPT and ophthalmology: exploring its potential with discharge summaries and operative notes. Semin Ophthalmol 38(5):503–507CrossRefPubMed Singh S, Djalilian A, Ali MJ (2023) ChatGPT and ophthalmology: exploring its potential with discharge summaries and operative notes. Semin Ophthalmol 38(5):503–507CrossRefPubMed
12.
go back to reference Skalidis I, Cagnina A, Luangphiphat W et al (2023) ChatGPT takes on the European exam in core cardiology: an artificial intelligence success story? Eur Heart J Digit Health 4(3):279–281CrossRefPubMedPubMedCentral Skalidis I, Cagnina A, Luangphiphat W et al (2023) ChatGPT takes on the European exam in core cardiology: an artificial intelligence success story? Eur Heart J Digit Health 4(3):279–281CrossRefPubMedPubMedCentral
13.
go back to reference Gibson A (2022) How does ChatGPT perform on the medical licensing exams? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ 9:e45312 Gibson A (2022) How does ChatGPT perform on the medical licensing exams? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ 9:e45312
14.
go back to reference Kung TH, Cheatham M, Medenilla A et al (2023) Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLoS Digit Health 2(2):e0000198CrossRefPubMedPubMedCentral Kung TH, Cheatham M, Medenilla A et al (2023) Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLoS Digit Health 2(2):e0000198CrossRefPubMedPubMedCentral
15.
go back to reference Mihalache A, Huang RS, Popovic MM, Muni RH (2023) Performance of an upgraded Artificial Intelligence Chatbot for ophthalmic knowledge assessment. JAMA Ophthalmol 141(8):798–800CrossRefPubMed Mihalache A, Huang RS, Popovic MM, Muni RH (2023) Performance of an upgraded Artificial Intelligence Chatbot for ophthalmic knowledge assessment. JAMA Ophthalmol 141(8):798–800CrossRefPubMed
16.
go back to reference Waisberg E, Ong J, Masalkhi M et al (2023) GPT-4: a new era of artificial intelligence in medicine. Ir J Med Sci. 192:3197–3200CrossRefPubMed Waisberg E, Ong J, Masalkhi M et al (2023) GPT-4: a new era of artificial intelligence in medicine. Ir J Med Sci. 192:3197–3200CrossRefPubMed
17.
go back to reference Eidem B (2023) Pediatric cardiology board review, 2nd edn. Wolters Kluwer, Philadelphia, PA Eidem B (2023) Pediatric cardiology board review, 2nd edn. Wolters Kluwer, Philadelphia, PA
18.
go back to reference Sallam M (2023) ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel) 11(6):887CrossRefPubMed Sallam M (2023) ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel) 11(6):887CrossRefPubMed
19.
go back to reference Vaid A, Jiang J, Sawant A et al (2023) A foundational vision transformer improves diagnostic performance for electrocardiograms. NPJ Digit Med 6(1):108CrossRefPubMedPubMedCentral Vaid A, Jiang J, Sawant A et al (2023) A foundational vision transformer improves diagnostic performance for electrocardiograms. NPJ Digit Med 6(1):108CrossRefPubMedPubMedCentral
20.
21.
go back to reference The Daily: ‘Hard Fork—An Interview with Sam Altman’ [Internet]. The New York Times, 2023 November 24, 2023, Podcast The Daily: ‘Hard Fork—An Interview with Sam Altman’ [Internet]. The New York Times, 2023 November 24, 2023, Podcast
Metadata
Title
Progression of an Artificial Intelligence Chatbot (ChatGPT) for Pediatric Cardiology Educational Knowledge Assessment
Authors
Michael N. Gritti
Hussain AlTurki
Pedrom Farid
Conall T. Morgan
Publication date
03-01-2024
Publisher
Springer US
Published in
Pediatric Cardiology / Issue 2/2024
Print ISSN: 0172-0643
Electronic ISSN: 1432-1971
DOI
https://doi.org/10.1007/s00246-023-03385-6

Keynote series | Spotlight on menopause

Menopause can have a significant impact on the body, with effects ranging beyond the endocrine and reproductive systems. Learn about the systemic effects of menopause, so you can help patients in your clinics through the transition.   

Prof. Martha Hickey
Dr. Claudia Barth
Dr. Samar El Khoudary
Developed by: Springer Medicine
Watch now

A quick guide to ECGs

Improve your ECG interpretation skills with this comprehensive, rapid, interactive course. Expert advice provides detailed feedback as you work through 50 ECGs covering the most common cardiac presentations to ensure your practice stays up to date. 

PD Dr. Carsten W. Israel
Developed by: Springer Medizin
Start the cases

Keynote webinar | Spotlight on medication adherence

  • Webinar | 27-06-2024 | 18:00 (CEST)

Medication non-adherence is a major barrier to effective healthcare delivery; half of all patients do not follow their doctor’s recommendations or treatment plan.

Our experts explain the fundamentals with a practical discussion of how to manage non-adherence in two common scenarios: hypertension and asthma control.

Prof. Kevin Dolgin
Prof. Florian Limbourg
Prof. Anoop Chauhan
Developed by: Springer Medicine
Watch now