Top

Published in:

01-11-2015 | New Technology

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

Authors: Fernando Pérez Escamirosa, Ricardo Manuel Ordorica Flores, Ignacio Oropesa García, Cristian Rubén Zalles Vidal, Arturo Minor Martínez

Published in: Surgical Endoscopy | Issue 11/2015

Abstract

Background

The aim of this study is to present face, content, and constructs validity of the endoscopic orthogonal video system (EndoViS) training system and determines its efficiency as a training and objective assessment tool of the surgeons’ psychomotor skills.

Methods

Thirty-five surgeons and medical students participated in this study: 11 medical students, 19 residents, and 5 experts. All participants performed four basic skill tasks using conventional laparoscopic instruments and EndoViS training system. Subsequently, participants filled out a questionnaire regarding the design, realism, overall functionality, and its capabilities to train hand–eye coordination and depth perception, rated on a 5-point Likert scale. Motion data of the instruments were obtained by means of two webcams built into a laparoscopic physical trainer. To identify the surgical instruments in the images, colored markers were placed in each instrument. Thirteen motion-related metrics were used to assess laparoscopic performance of the participants. Statistical analysis of performance was made between novice, intermediate, and expert groups. Internal consistency of all metrics was analyzed with Cronbach’s α test.

Results

Overall scores about features of the EndoViS system were positives. Participants agreed with the usefulness of tasks and the training capacities of EndoViS system (score >4). Results presented significant differences in the execution of three skill tasks performed by participants. Seven metrics showed construct validity for assessment of performance with high consistency levels.

Conclusions

EndoViS training system has been successfully validated. Results showed that EndoViS was able to differentiate between participants of varying laparoscopic experience. This simulator is a useful and effective tool to objectively assess laparoscopic psychomotor skills of the surgeons.

Aziz O, Constantinides V, Tekkis PP, Athanasiou T, Purkayastha S, Paraskeva P, Darzi AW, Heriot AG (2006) Laparoscopic versus open surgery for rectal cancer: a meta-analysis. Ann Surg Oncol 13(3):413–424CrossRefPubMed

Staudacher C, Vignali A (2010) Laparoscopic surgery for rectal cancer: the state of the art. World J Gastrointest Surg 2(9):275–282. doi:10.4240/wjgs.v2.i9.275 PubMedCentralCrossRefPubMed

Klarenbeek BR, Bergamaschi R, Veenhof AA, van der Peet DL, van den Broek WT, de Lange ES, Bemelman WA, Heres P, Lacy AM, Cuesta MA (2011) Laparoscopic versus open sigmoid resection for diverticular disease: follow-up assessment of the randomized control Sigma trial. Surg Endosc 25(4):1121–1126. doi:10.1007/s00464-010-1327-0 CrossRefPubMed

Figert PL, Park AE, Witzke DB, Schwartz RW (2001) Transfer of training in acquiring laparoscopic skills. J Am Coll Surg 193(5):533–537CrossRefPubMed

Livingston EH, Rege RV (2004) A nationwide study of conversion from laparoscopic to open cholecystectomy. Am J Surg 188(3):205–211CrossRefPubMed

Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165(3):358–361CrossRefPubMed

Bridges M, Diamond DL (1999) The financial impact of teaching surgical residents in the operating room. Am J Surg 177(1):28–32CrossRefPubMed

Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240(3):518–525PubMedCentralCrossRefPubMed

Cosman PH, Hugh TJ, Shearer CJ, Merrett ND, Biankin AV, Cartmill JA (2007) Skills acquired on virtual reality laparoscopic simulators transfer into the operating room in a blinded, randomised, controlled trial. Stud Health Technol Inform 125:76–81PubMed

10.

Samia H, Khan S, Lawrence J, Delaney CP (2013) Simulation and its role in training. Clin Colon Rectal Surg 26(1):47–55. doi:10.1055/s-0033-1333661 PubMedCentralCrossRefPubMed

11.

Khan MW, Lin D, Marlow N, Altree M, Babidge W, Field J, Hewett P, Maddern G (2014) Laparoscopic skills maintenance: a randomized trial of virtual reality and box trainer simulators. J Surg Educ 71(1):79–84. doi:10.1016/j.jsurg.2013.05.009 CrossRefPubMed

12.

Laski D, Stefaniak TJ, Makarewicz W, Proczko M, Gruca Z, Sledziński Z (2011) Structuralized box-trainer laparoscopic training significantly improves performance in complex virtual reality laparoscopic tasks. Wideochir Inne Tech Malo Inwazyjne 7(1):27–32. doi:10.5114/wiitm.2011.25666 PubMedCentralPubMed

13.

Martinez AM, Espinoza DL (2007) Novel laparoscopic home trainer. Surg Laparosc Endosc Percutan Tech 17(4):300–302CrossRefPubMed

14.

Martinez AM, Kalach AC, Espinoza DL (2008) Millimetric laparoscopic surgery training on a physical trainer using rats. Surg Endosc 22:246–249CrossRefPubMed

15.

Hinata N, Iwamoto H, Morizane S, Hikita K, Yao A, Muraoka K, Honda M, Isoyama T, Sejima T, Takenaka A (2013) Dry box training with three-dimensional vision for the assistant surgeon in robot-assisted urological surgery. Int J Urol 20(10):1037–1041. doi:10.1111/iju.12101 PubMed

16.

Verdaasdonk EG, Stassen LP, Monteny LJ, Dankelman J (2006) Validation of a new basic virtual reality simulator for training of basic endoscopic skills: the SIMENDO. Surg Endosc 20(3):511–518CrossRefPubMed

17.

Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L, Gauger PG, Minter RM (2006) Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 191(1):28–32CrossRefPubMed

18.

Ayodeji ID, Schijven M, Jakimowicz J, Greve JW (2007) Face validation of the Simbionix LAP Mentor virtual reality training module and its applicability in the surgical curriculum. Surg Endosc 21(9):1641–1649CrossRefPubMed

19.

Iwata N, Fujiwara M, Kodera Y, Tanaka C, Ohashi N, Nakayama G, Koike M, Nakao A (2011) Construct validity of the LapVR virtual-reality surgical simulator. Surg Endosc 25(2):423–428. doi:10.1007/s00464-010-1184-x CrossRefPubMed

20.

Stylopoulos N, Cotin S, Maithel SK, Ottensmeye M, Jackson PG, Bardsley RS, Neumann PF, Rattner DW, Dawson SL (2004) Computer-enhanced laparoscopic training system (CELTS): bridging the gap. Surg Endosc 18(5):782–789CrossRefPubMed

21.

Soyinka AS, Schollmeyer T, Meinhold-Heerlein I, Gopalghare DV, Hasson H, Mettler L (2008) Enhancing laparoscopic performance with the LTS3E: a computerized hybrid physical reality simulator. Fertil Steril 90(5):1988–1994. doi:10.1016/j.fertnstert.2007.08.077 CrossRefPubMed

22.

Pellen MG, Horgan LF, Barton JR, Attwood SE (2009) Construct validity of the ProMIS laparoscopic simulator. Surg Endosc 23(1):130–139. doi:10.1007/s00464-008-0066-y CrossRefPubMed

23.

Botden SM, Jakimowicz JJ (2009) What is going on in augmented reality simulation in laparoscopic surgery? Surg Endosc 23(8):1693–1700. doi:10.1007/s00464-008-0144-1 PubMedCentralCrossRefPubMed

24.

Chmarra MK, Grimbergen CA, Dankelman J (2007) Systems for tracking minimally invasive surgical instruments. Minim Invasive Ther Allied Technol 16:328–340CrossRefPubMed

25.

Oropesa I, Sánchez-González P, Chmarra MK, Lamata P, Fernández A, Sánchez-Margallo JA, Jansen FW, Dankelman J, Sánchez-Margallo FM, Gómez EJ (2013) EVA: laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment. Surg Endosc 27(3):1029–1039. doi:10.1007/s00464-012-2513-z CrossRefPubMed

26.

Van Empel PJ, Commandeur JP, van Rijssen LB, Verdam MG, Huirne JA, Scheele F, Bonjer HJ, Jeroen Meijerink W (2013) Learning curve on the TrEndo laparoscopic simulator compared to an expert level. Surg Endosc 27(8):2934–2939. doi:10.1007/s00464-013-2859-x CrossRefPubMed

27.

Kovac E, Azhar RA, Quirouet A, Delisle J, Anidjar M (2012) Construct validity of the LapSim virtual reality laparoscopic simulator within a urology residency program. Can Urol Assoc J 6(4):253–259. doi:10.5489/cuaj.12047 PubMedCentralCrossRefPubMed

28.

Ritter EM, Kindelan TW, Michael C, Pimentel EA, Bowyer MW (2007) Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS). Surg Endosc 21(8):1441–1445CrossRefPubMed

29.

Maithel S, Sierra R, Korndorffer J, Neumann P, Dawson S, Callery M, Jones D, Scott D (2006) Construct and face validity of MIST-VR, Endotower, and CELTS: are we ready for skills assessment using simulators? Surg Endosc 20(1):104–112CrossRefPubMed

30.

Pérez F, Sossa H, Martínez R, Lorias D, Minor A (2013) Video-based tracking of laparoscopic instruments using an orthogonal webcams system. World Acad Sci Eng Technol Int J Med Health Pharm Biomed Eng 7(8):184–187

31.

Cotin S, Stylopoulos N, Ottensmeyer MP, Neumann P, Rattner DW, Dawson SL (2002) Metrics for laparoscopic skills trainers: the weakest link! In: MICCAI 2002. LNCS, 2488, pp. 35–43. Springer, Heidelberg

32.

Chmarra MK, Kolkman W, Jansen FW, Grimbergen CA, Dankelman J (2007) The influence of experience and camera holding on laparoscopic instrument movements with the TrEndo tracking system. Surg Endosc 21:2069–2075CrossRefPubMed

33.

Hofstad EF, Våpenstad C, Chmarra MK, Langø T, Kuhry E, Mårvik R (2013) A study of psychomotor skills in minimally invasive surgery: what differentiates expert and nonexpert performance. Surg Endosc 27(3):854–863. doi:10.1007/s00464-012-2524-9 CrossRefPubMed

34.

Oropesa I, Chmarra MK, Sánchez-González P, Lamata P, Rodrigues SP, Enciso S, Sánchez-Margallo FM, Jansen FW, Dankelman J, Gómez EJ (2013) Relevance of motion-related assessment metrics in laparoscopic surgery. Surg Innov 20(3):299–312. doi:10.1177/1553350612459808 CrossRefPubMed

35.

Vassiliou MC, Ghitulescu GA, Feldman LS, Stanbridge D, Leffondré K, Sigman HH, Fried GM (2006) The MISTELS program to measure technical skill in laparoscopic surgery: evidence for reliability. Surg Endosc 20(5):744–747CrossRefPubMed

36.

Rivard JD, Vergis AS, Unger BJ, Hardy KM, Andrew CG, Gillman LM, Park J (2014) Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator. Surg Endosc 28(6):1921–1928. doi:10.1007/s00464-013-3414-5 CrossRefPubMed

37.

Arikatla VS, Sankaranarayanan G, Ahn W, Chellali A, De S, Caroline GL, Hwabejire J, DeMoya M, Schwaitzberg S, Jones DB (2013) Face and construct validation of a virtual peg transfer simulator. Surg Endosc 27(5):1721–1729. doi:10.1007/s00464-012-2664-y PubMedCentralCrossRefPubMed

38.

Chellali A, Zhang L, Sankaranarayanan G, Arikatla VS, Ahn W, Derevianko A, Schwaitzberg SD, Jones DB, Demoya M, Cao CG (2014) Validation of the VBLaST peg transfer task: a first step toward an alternate training standard. Surg Endosc. doi:10.1007/s00464-014-3538-2

39.

Chmarra MK, de Klein S, Winter JC, Jansen FW, Dankelman J (2011) Objective classification of residents based on their psychomotor laparoscopic skills. Surg Endosc 24(5):1031–1039. doi:10.1007/s00464-009-0721-y CrossRef

40.

Van Sickle KR, McClusky DA 3rd, Gallagher AG, Smith CD (2005) Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc 19(9):1227–1231CrossRefPubMed

41.

Yamaguchi S, Yoshida D, Kenmotsu H, Yasunaga T, Konishi K, Ieiri S, Nakashima H, Tanoue K, Hashizume M (2011) Objective assessment of laparoscopic suturing skills using a motion-tracking system. Surg Endosc 25(3):771–775. doi:10.1007/s00464-010-1251-3 CrossRefPubMed

Title: Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons
Authors: Fernando Pérez Escamirosa
Ricardo Manuel Ordorica Flores
Ignacio Oropesa García
Cristian Rubén Zalles Vidal
Arturo Minor Martínez
Publication date: 01-11-2015
Publisher: Springer US
Published in: Surgical Endoscopy / Issue 11/2015
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-014-4032-6

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 11/2015

Computer-assessed performance of psychomotor skills in endoscopic otolaryngology surgery: construct validity of the Dundee Endoscopic Psychomotor Otolaryngology Surgery Trainer (DEPOST)

Endoluminal full-thickness suture repair of gastrotomy: a survival study

A feasibility study of chemically assisted endoscopic submucosal mechanical dissection using mesna for superficial esophageal squamous cell carcinomas

A prospective non-randomised single-center study comparing laparoscopic and robotic distal pancreatectomy

Surgical outcomes of intracorporeal circular-stapled esophagojejunostomy using modified over-and-over suture technique in laparoscopic total gastrectomy

Endoscopic full-thickness resection with defect closure using an over-the-scope clip for gastric subepithelial tumors originating from the muscularis propria