Top

Published in:

01-02-2018 | Original Scientific Report

Measuring Decision-Making During Thyroidectomy: Validity Evidence for a Web-Based Assessment Tool

Authors: Amin Madani, Jordan Gornitsky, Yusuke Watanabe, Cassandre Benay, Maria S. Altieri, Philip H. Pucher, Roger Tabah, Elliot J. Mitmaker

Published in: World Journal of Surgery | Issue 2/2018

Abstract

Background

Errors in judgment during thyroidectomy can lead to recurrent laryngeal nerve injury and other complications. Despite the strong link between patient outcomes and intraoperative decision-making, methods to evaluate these complex skills are lacking. The purpose of this study was to develop objective metrics to evaluate advanced cognitive skills during thyroidectomy and to obtain validity evidence for them.

Methods

An interactive online learning platform was developed (www.thinklikeasurgeon.com). Trainees and surgeons from four institutions completed a 33-item assessment, developed based on a cognitive task analysis and expert Delphi consensus. Sixteen items required subjects to make annotations on still frames of thyroidectomy videos, and accuracy scores were calculated based on an algorithm derived from experts’ responses (“visual concordance test,” VCT). Seven items were short answer (SA), requiring users to type their answers, and scores were automatically calculated based on their similarity to a pre-populated repertoire of correct responses. Test–retest reliability, internal consistency, and correlation of scores with self-reported experience and training level (novice, intermediate, expert) were calculated.

Results

Twenty-eight subjects (10 endocrine surgeons and otolaryngologists, 18 trainees) participated. There was high test–retest reliability (intraclass correlation coefficient = 0.96; n = 10) and internal consistency (Cronbach’s α = 0.93). The assessment demonstrated significant differences between novices, intermediates, and experts in total score (p < 0.01), VCT score (p < 0.01) and SA score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.95, p < 0.01), between total case number and VCT score (ρ = 0.93, p < 0.01), and between total case number and SA score (ρ = 0.83, p < 0.01).

Conclusion

This study describes the development of novel metrics and provides validity evidence for an interactive Web-based platform to objectively assess decision-making during thyroidectomy.

Chan WF, Lo CY (2006) Pitfalls of intraoperative neuromonitoring for predicting postoperative recurrent laryngeal nerve function during thyroidectomy. World J Surg 30:806–812. https://doi.org/10.1007/s00268-005-0355-8 CrossRefPubMed

Jeannon JP, Orabi AA, Bruch GA et al (2009) Diagnosis of recurrent laryngeal nerve palsy after thyroidectomy: a systematic review. Int J Clin Pract 63:624–629CrossRefPubMed

Lo CY, Kwok KF, Yuen PW (2000) A prospective evaluation of recurrent laryngeal nerve paralysis during thyroidectomy. Arch Surg 135:204–207CrossRefPubMed

Serpell JW, Yeung MJ, Grodski S (2009) The motor fibers of the recurrent laryngeal nerve are located in the anterior extralaryngeal branch. Ann Surg 249:648–652CrossRefPubMed

Steurer M, Passler C, Denk DM et al (2002) Advantages of recurrent laryngeal nerve identification in thyroidectomy and parathyroidectomy and the importance of preoperative and postoperative laryngoscopic examination in more than 1000 nerves at risk. Laryngoscope 112:124–133CrossRefPubMed

Bononi M, de Cesare A, Atella F et al (2000) Surgical treatment of multinodular goiter: incidence of lesions of the recurrent nerves after total thyroidectomy. Int Surg 85:190–193PubMed

Erbil Y, Barbaros U, Issever H et al (2007) Predictive factors for recurrent laryngeal nerve palsy and hypoparathyroidism after thyroid surgery. Clin Otolaryngol 32:32–37CrossRefPubMed

Abadin SS, Kaplan EL, Angelos P (2010) Malpractice litigation after thyroid surgery: the role of recurrent laryngeal nerve injuries, 1989–2009. Surgery 148:718–722CrossRefPubMed

Kern KA (1993) Medicolegal analysis of errors in diagnosis and treatment of surgical endocrine disease. Surgery 114:1167–1173PubMed

10.

Strasberg SM, Sanabria JR, Clavien PA (1992) Complications of laparoscopic cholecystectomy. Can J Surg 35:275–280PubMed

11.

Strasberg SM, Hertl M, Soper NJ (1995) An analysis of the problem of biliary injury during laparoscopic cholecystectomy. J Am Coll Surg 180:101–125PubMed

12.

Way LW, Stewart L, Gantert W et al (2003) Causes and prevention of laparoscopic bile duct injuries: analysis of 252 cases from a human factors and cognitive psychology perspective. Ann Surg 237:460–469PubMedPubMedCentral

13.

Strasberg SM, Eagon CJ, Drebin JA (2000) The “hidden cystic duct” syndrome and the infundibular technique of laparoscopic cholecystectomy–the danger of the false infundibulum. J Am Coll Surg 191:661–667CrossRefPubMed

14.

Strasberg SM, Gouma DJ (2012) ‘Extreme’ vasculobiliary injuries: association with fundus-down cholecystectomy in severely inflamed gallbladders. HPB 14:1–8CrossRefPubMedPubMedCentral

15.

Chiang FY, Lu IC, Kuo WR et al (2008) The mechanism of recurrent laryngeal nerve injury during thyroid surgery–the application of intraoperative neuromonitoring. Surgery 143:743–749CrossRefPubMed

16.

Rulli F, Ambrogi V, Dionigi G et al (2014) Meta-analysis of recurrent laryngeal nerve injury in thyroid surgery with or without intraoperative nerve monitoring. Acta Otorhinolaryngol Ital 34:223–229PubMedPubMedCentral

17.

Wheeler MH (1999) Thyroid surgery and the recurrent laryngeal nerve. Br J Surg 86:291–292CrossRefPubMed

18.

Rogers SO Jr, Gawande AA, Kwaan M et al (2006) Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery 140:25–33CrossRefPubMed

19.

Guru V, Tu JV, Etchells E et al (2008) Relationship between preventability of death after coronary artery bypass graft surgery and all-cause risk-adjusted mortality rates. Circulation 117:2969–2976CrossRefPubMed

20.

Madani A, Watanabe Y, Vassiliou M et al (2016) Defining competencies for safe thyroidectomy: an international Delphi consensus. Surgery 159(86–94):96–101

21.

Madani A, Watanabe Y, Bilgic E et al (2017) Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool. Surg Endosc 31:1203–1212CrossRefPubMed

22.

Maertens H, Madani A, Landry T et al (2016) Systematic review of e-learning for surgical training. Br J Surg 103:1428–1437CrossRefPubMed

23.

Charlin B, Roy L, Brailovsky C et al (2000) The script concordance test: a tool to assess the reflective clinician. Teach Learn Med 12:189–195CrossRefPubMed

24.

Meterissian SH (2006) A novel method of assessing clinical reasoning in surgical residents. Surg Innov 13:115–119CrossRefPubMed

25.

Nouh T, Boutros M, Gagnon R et al (2012) The script concordance test as a measure of clinical reasoning: a national validation study. Am J Surg 203:530–534CrossRefPubMed

26.

Lubarsky S, Dory V, Duggan P et al (2013) Script concordance testing: from theory to practice: AMEE guide no. 75. Med Teach 35:184–193CrossRefPubMed

27.

Downing SM (2003) Validity: on meaningful interpretation of assessment data. Med Educ 37:830–837CrossRefPubMed

28.

Adam MA, Thomas S, Youngwirth L et al (2017) Is there a minimum number of thyroidectomies a surgeon should perform to optimize patient outcomes? Ann Surg 265:402–407CrossRefPubMed

29.

Flin R, Youngson G, Yule S (2007) How do surgeons make intraoperative decisions? Qual Saf Health Care 16:235–239CrossRefPubMedPubMedCentral

30.

Frank JR, Snell LS, Cate OT et al (2010) Competency-based medical education: theory to practice. Med Teach 32:638–645CrossRefPubMed

31.

Singh P, Aggarwal R, Tahir M et al (2015) A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills. Ann Surg 261:862–869CrossRefPubMed

32.

Ericsson KA (2014) Necessity is the mother of invention: video recording firsthand perspectives of critical medical procedures to make simulated training more effective. Acad Med 89:17–20CrossRefPubMed

33.

Andersen DK (2012) How can educators use simulation applications to teach and assess surgical judgment? Acad Med 87:934–941CrossRefPubMed

34.

Al-Qurayshi Z, Robins R, Hauch A et al (2016) Association of surgeon volume with outcomes and cost savings following thyroidectomy: a national forecast. JAMA Otolaryngol Head Neck Surg 142:32–39CrossRefPubMed

Title: Measuring Decision-Making During Thyroidectomy: Validity Evidence for a Web-Based Assessment Tool
Authors: Amin Madani
Jordan Gornitsky
Yusuke Watanabe
Cassandre Benay
Maria S. Altieri
Philip H. Pucher
Roger Tabah
Elliot J. Mitmaker
Publication date: 01-02-2018
Publisher: Springer International Publishing
Published in: World Journal of Surgery / Issue 2/2018
Print ISSN: 0364-2313
Electronic ISSN: 1432-2323
DOI: https://doi.org/10.1007/s00268-017-4322-y

At a glance: The STEP trials

Springer Medicine

Measuring Decision-Making During Thyroidectomy: Validity Evidence for a Web-Based Assessment Tool

Abstract

Background

Methods

Results

Conclusion

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 2/2018

Trauma Leagues—A Novel Option to Attract Medical Students to a Surgical Career

Minimally Invasive Adrenalectomy for Adrenocortical Carcinoma: Five-Year Trends and Predictors of Conversion

An Evolving Understanding of the Clinical Implications of NIFTP

The Hair in the Sinus: Sharp-Ended Rootless Head Hair Fragments can be Found in Large Amounts in Pilonidal Sinus Nests

Refining the Definitions of Biochemical and Clinical Cure for Primary Aldosteronism Using the Primary Aldosteronism Surgical Outcome (PASO) Classification System

Modification of the Surgical Strategy for the Dissection of the Recurrent Laryngeal Nerve Using Continuous Intraoperative Nerve Monitoring