Top

Surgical Endoscopy

Published in:

01-12-2017

Psychometric properties of the Fundamentals of Endoscopic Surgery (FES) skills examination

Authors: Matthew Lineberry, E. Matthew Ritter

Published in: Surgical Endoscopy | Issue 12/2017

Abstract

Background

The Fundamentals of Endoscopic Surgery (FES) manual skills examination is a simulation-based assessment of five foundational skills in endoscopic surgery. With the FES skills exam becoming part of the board certification process in general surgery, continual investigation is needed to determine the validity with which the exam is supporting inferences and decision-making about examinees, as well as how it might be improved.

Methods

The present study retrospectively analyzed performance and demographic details for the initial 344 examinees completing the FES skills exam.

Results

The five tasks showed distinct degrees of difficulty, with Loop Reduction being especially difficult for examinees. Tasks related to one another positively but moderately, suggesting that the exam assesses both general and task-specific skills. The number of lower-endoscopic cases completed by an examinee strongly predicted performance, while upper endoscopy experience and career level (e.g., resident vs. fellow vs. practicing) did not. Hand dominance and the type of simulator used were not found to be related to scores. However, three demographic variables that related to one another—gender, glove size, and height—were also related to performance and pass/fail status.

Conclusions

This study’s results generally support the validity argument for the FES skills exam while pointing to additional investigations to be undertaken as the exam is applied more broadly.

Hazey JW, Marks JM, Mellinger JD, Trus TL, Chand B, Delaney CP, Dunkin BJ, Fanelli RD, Fried GM, Martinez JM, Pearl JP, Poulose BK, Sillin LF, Vassiliou MC, Melvin WS (2014) Why fundamentals of endoscopic surgery (FES)? Surg Endosc 28:701–703CrossRefPubMed

Vassiliou MC, Dunkin BJ, Fried GM, Mellinger JD, Trus T, Kaneva P, Lyons C, Korndorffer JR Jr, Ujiki M, Velanovich V, Kochman ML, Tsuda S, Martinez J, Scott DJ, Korus G, Park A, Marks JM (2014) Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28:704–711CrossRefPubMed

Poulose BK, Vassiliou MC, Dunkin BJ, Mellinger JD, Fanelli RD, Martinez JM, Hazey JW, Sillin LF, Delaney CP, Velanovich V, Fried GM, Korndorffer JR Jr, Marks JM (2014) Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence. Surg Endosc 28:631–638CrossRefPubMed

Mueller CL, Kaneva P, Fried GM, Feldman LS, Vassiliou MC (2014) Colonoscopy performance correlates with scores on the FES™ manual skills test. Surg Endosc 28(11):3081–3085CrossRefPubMed

American Board of Surgery (2017) Flexible Endoscopy Curriculum for General Surgery Residents. Available at https://www.absurgery.org/default.jsp?certgsqe_fec Accessed 22 March 2017

American Board of Surgery (2017) ABS Establishes New Requirement for Endoscopic Training and Assessment. Available at https://www.absurgery.org/default.jsp?news_fec Accessed 22 March 2017

American Educational Research Association, American Psychological Association, National Council on Measurement in Education (2014) Standards for Educational and Psychological Testing. Washington, DC: AERA

APA Publications and Communications Board Working Group on Journal Article Reporting Standards (2008) Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist 63(9):839–851

Cheng A, Kessler D, Mackinnon R, Chang TP, Nadkarni VM, Hunt EA, Duval-Arnold J, Yiqun L, Cook DA, Pusic M, Hui J, Moher D, Egger M, Auerbach M (2016) Reporting guidelines for health care simulation research: extensions to the CONSORT and STROBE statements. Adv Simul 1:25CrossRef

10.

Mueller CL, Kaneva P, Fried GM, Mellinger JD, Marks JM, Dunkin BJ, van Sickle K, Vassiliou MC (2016) Validity evidence for a new portable, lower-cost platform for the fundamentals of endoscopic surgery skills test. Surg Endosc 30:1107–1112CrossRefPubMed

11.

Edwards JR, Bagozzi RP (2000) On the nature and direction of relationships between constructs and measures. Psychol Methods 5(2):155–174CrossRefPubMed

12.

Cortina JM (1993) What is coefficient alpha? An examination of theory and applications. J Appl Psychol 78:98–104CrossRef

13.

Cronbach LJ, Shavelson RJ (2004) My current thoughts on coefficient alpha and successor procedures. Educ Psychol Measur 64(3):391–418CrossRef

14.

Green SB, Yang Y (2009) Commentary on coefficient alpha: a cautionary tale. Psychometrika 74:121–135CrossRef

15.

Schmitt N (1996) Uses and abuses of coefficient alpha. Psychol Assess 8:350–353CrossRef

16.

Sijtsma K (2009) On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika 74(1):107CrossRefPubMed

17.

Yudkowsky R, Park YS, Lineberry M, Knox A, Ritter EM (2015) Setting mastery learning standards. Acad Med 90(11):1495–1500CrossRefPubMed

18.

Cook DA (2014) Much ado about differences: why expert-novice comparisons add little to the validity argument. Adv Health Sci Educ 27:1–6CrossRef

19.

Society for Industrial & Organizational Psychology (2003) Principles for the validation and use of personnel selection procedures, 4th edn. SIOP, Bowling Green

20.

Campbell SM, Collaer ML (2009) Stereotype threat and gender differences in performance on a novel visuospatial task. Psychol Women Q 33(4):437–444CrossRef

21.

Kass SJ, Ahlers RH, Dugger M (1998) Eliminating gender differences through practice in an applied visual spatial task. Human Perform 11(4):337–349CrossRef

22.

Ali A, Subhi Y, Ringsted C, Konge L (2015) Gender differences in the acquisition of surgical skills: a systematic review. Surg Endosc 29(11):3065–3073CrossRefPubMed

23.

Adams DM, Fenton SJ, Schirmer BD, Mahvi DM, Horvath K, Nichol P (2008) One size does not fit all: current disposable laparoscopic devices do not fit the needs of female laparoscopic surgeons. Surg Endosc 22(10):2310–2313CrossRefPubMed

24.

Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R (2014) What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Health Sci Educ 19(2):233–250CrossRef

25.

Cook DA, Lineberry M (2016) Consequences validity evidence: evaluating the impact of educational assessments. Acad Med 91(6):785–795CrossRefPubMed

26.

Tolsgaard MG, Ringsted C (2014) Using equivalence designs to improve methodological rigor in medical education trials. Med Educ 48(2):220–221CrossRefPubMed

27.

Cook DA, Hatala R (2016) Validation of educational assessments: a primer for simulation and beyond. Adv Simul 1(1):31CrossRef

Title: Psychometric properties of the Fundamentals of Endoscopic Surgery (FES) skills examination
Authors: Matthew Lineberry
E. Matthew Ritter
Publication date: 01-12-2017
Publisher: Springer US
Published in: Surgical Endoscopy / Issue 12/2017
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-017-5590-1

At a glance: The STEP trials

Springer Medicine

Psychometric properties of the Fundamentals of Endoscopic Surgery (FES) skills examination

Abstract

Background

Methods

Results

Conclusions

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 12/2017

Safety and efficacy of endoscopic submucosal dissection of large (≥3 cm) subepithelial tumors located in the cardia

Short-term outcomes of laparoscopy-assisted hybrid living donor hepatectomy: a comparison with the conventional open procedure

C-reactive protein, fibrinogen, and procalcitonin levels as early markers of staple line leak after laparoscopic sleeve gastrectomy in morbidly obese patients within an Enhanced Recovery After Surgery (ERAS) program

The effect of an online referral system on referrals to bariatric surgery

Comparison of endoscopic ultrasonography and magnifying endoscopy for assessment of the invasion depth of shallow gastrointestinal neoplasms: a systematic review and meta-analysis

Comparison of endoscopic sphincterotomy techniques after Billroth II gastrectomy using a novel mechanical simulator