Top

Surgical Endoscopy

Published in:

01-10-2018

Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure

Authors: Lisette Hvid Hovgaard, Steven Arild Wuyts Andersen, Lars Konge, Torur Dalsgaard, Christian Rifbjerg Larsen

Published in: Surgical Endoscopy | Issue 10/2018

Abstract

Background

The use of robotic surgery for minimally invasive procedures has increased considerably over the last decade. Robotic surgery has potential advantages compared to laparoscopic surgery but also requires new skills. Using virtual reality (VR) simulation to facilitate the acquisition of these new skills could potentially benefit training of robotic surgical skills and also be a crucial step in developing a robotic surgical training curriculum. The study's objective was to establish validity evidence for a simulation-based test for procedural competency for the vaginal cuff closure procedure that can be used in a future simulation-based, mastery learning training curriculum.

Methods

Eleven novice gynaecological surgeons without prior robotic experience and 11 experienced gynaecological robotic surgeons (> 30 robotic procedures) were recruited. After familiarization with the VR simulator, participants completed the module ‘Guided Vaginal Cuff Closure’ six times. Validity evidence was investigated for 18 preselected simulator metrics. The internal consistency was assessed using Cronbach’s alpha and a composite score was calculated based on metrics with significant discriminative ability between the two groups. Finally, a pass/fail standard was established using the contrasting groups’ method.

Results

The experienced surgeons significantly outperformed the novice surgeons on 6 of the 18 metrics. The internal consistency was 0.58 (Cronbach’s alpha). The experienced surgeons’ mean composite score for all six repetitions were significantly better than the novice surgeons’ (76.1 vs. 63.0, respectively, p < 0.001). A pass/fail standard of 75/100 was established. Four novice surgeons passed this standard (false positives) and three experienced surgeons failed (false negatives).

Conclusion

Our study has gathered validity evidence for a simulation-based test for procedural robotic surgical competency in the vaginal cuff closure procedure and established a credible pass/fail standard for future proficiency-based training.

Available only for authorised users

Schreuder HW, Verheijen RH (2009) Robotic surgery. BJOG 116(2):198–213. https://doi.org/10.1111/J.1471-0528.2008.02038.X CrossRefPubMed

Weinberg L, Rao S, Escobar P (2011) Robotic surgery in gynecology: an updated systematic review. Obstet Gynecol Int 2011:852061. https://doi.org/10.1155/2011/852061 CrossRefPubMedPubMedCentral

Yates DR, Vaessen C, Roupret M (2011) From Leonardo to da Vinci: the history of robot-assisted surgery in urology. BJU Int 108(11):1708–1713. https://doi.org/10.1111/J.1464-410x.2011.10576.X (discussion 1714)CrossRefPubMed

Maan ZN, Gibbins N, Al-Jabri T, D’souza AR (2012) The use of robotics in otolaryngology-head and neck surgery: a systematic review. Am J Otolaryngol 33(1):137–146. https://doi.org/10.1016/J.Amjoto.2011.04.003 CrossRefPubMed

Veronesi G (2013) Robotic surgery for the treatment of early-stage lung cancer. Curr Opin Oncol 25(2):107–114. https://doi.org/10.1097/Cco.0b013e32835daf4f CrossRefPubMed

Baek SK, Carmichael JC, Pigazzi A (2013) Robotic surgery: colon and rectum. Cancer J 19(2):140–146. https://doi.org/10.1097/Ppo.0b013e31828ba0fd CrossRefPubMed

Liu Z, Li X, Tian S, Zhu T, Yao Y, Tao Y (2017) Superiority of robotic surgery for cervical cancer in comparison with traditional approaches: a systematic review and meta-analysis. Int J Surg 40:145–154. https://doi.org/10.1016/J.Ijsu.2017.02.062 CrossRefPubMed

Xie W, Cao D, Yang J, Shen K, Zhao L (2016) Robot-assisted surgery versus conventional laparoscopic surgery for endometrial cancer: a systematic review and meta-analysis. J Cancer Res Clin Oncol 142(10):2173–2183. https://doi.org/10.1007/S00432-016-2180-X CrossRefPubMed

Foote JR, Valea FA (2016) Robotic surgical training: where are we? Gynecol Oncol 143(1):179–183. https://doi.org/10.1016/J.Ygyno.2016.05.031 CrossRefPubMed

10.

Bric J, Connolly M, Kastenmeier A, Goldblatt M, Gould J (2014) Proficiency training on a virtual reality robotic surgical skills curriculum. Surg Endosc 28(12):3343–3348. https://doi.org/10.1007/S00464-014-3624-5 CrossRefPubMed

11.

Sandadi S, Gadzinski JA, Lee S, Chi DS, Sonoda Y, Jewell EL, Brown CL, Gardner GJ, Barakat RR, Leitao MM (2014) Fellowship learning curve associated with completing a robotic assisted total laparoscopic hysterectomy. Gynecol Oncol 132(1):102–106. https://doi.org/10.1016/J.Ygyno.2013.11.017 CrossRefPubMed

12.

Seamon LG, Fowler JM, Richardson DL, Carlson MJ, Valmadre S, Phillips GS, Cohn DE (2009) A detailed analysis of the learning curve: robotic hysterectomy and pelvic-aortic lymphadenectomy for endometrial cancer. Gynecol Oncol 114(2):162–167. https://doi.org/10.1016/J.Ygyno.2009.04.017 CrossRefPubMed

13.

Balasundaram I, Aggarwal R, Darzi A (2008) Short-phase training on a virtual reality simulator improves technical performance in tele-robotic surgery. Int J Med Robot 4(2):139–145. https://doi.org/10.1002/Rcs.181 CrossRefPubMed

14.

Schreuder HW, Wolswijk R, Zweemer RP, Schijven MP, Verheijen RH (2012) Training and learning robotic surgery, time for a more structured approach: a systematic review. BJOG 119(2):137–149. https://doi.org/10.1111/J.1471-0528.2011.03139.X CrossRefPubMed

15.

Andersen SA, Konge L, Cayé-Thomasen P, Sørensen MS (2015) Learning curves of virtual mastoidectomy in distributed and massed practice. JAMA Otolaryngol 141(10):913–918. https://doi.org/10.1001/Jamaoto.2015.1563 CrossRef

16.

Lendvay TS, Brand TC, White L, Kowalewski T, Jonnadula S, Mercer LD, Khorsand D, Andros J, Hannaford B, Satava R (2013) Virtual reality robotic surgery warm-up improves task performance in a dry laboratory environment: a prospective randomized controlled study. J Am Coll Surg 216(6):1181–1192. https://doi.org/10.1016/J.Jamcollsurg.2013.02.012 CrossRefPubMedPubMedCentral

17.

Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:B1802. https://doi.org/10.1136/Bmj.B1802 CrossRefPubMedPubMedCentral

18.

Larsen CR, Oestergaard J, Ottosen C, Soerensen JL, (2012) The efficacy of virtual reality simulation training in laparoscopy: a systematic review of randomized trials. Acta Obstet Gynecol Scand 91(9):1015–1028. https://doi.org/10.1111/J.1600-0412.2012.01482.X CrossRefPubMed

19.

Culligan P, Gurshumov E, Lewis C, Priestley J, Komar J, Salamon C (2014) Predictive validity of a training protocol using a robotic surgery simulator. Female Pelvic Med Reconstr Surg 20(1):48–51. https://doi.org/10.1097/Spv.0000000000000045 CrossRefPubMed

20.

Kiely DJ, Gotlieb WH, Lau S, Zeng X, Samouelian V, Ramanakumar AV, Zakrzewski H, Brin S, Fraser SA, Korsieporn P, Drudi L (2015) Virtual reality robotic surgery simulation curriculum to teach robotic suturing: a randomized controlled trial. J Robot Surg 9(3):179–186. https://doi.org/10.1007/S11701-015-0513-4 CrossRefPubMed

21.

Hung AJ, Jayaratna IS, Teruya K, Desai MM, Gill IS, Goh AC (2013) Comparative assessment of three standardized robotic surgery training methods. BJU Int 112(6):864–871. https://doi.org/10.1111/Bju.12045 CrossRefPubMed

22.

Kenney PA, Wszolek MF, Gould JJ, Libertino JA, Moinzadeh A (2009) Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology 73(6):1288–1292. https://doi.org/10.1016/J.Urology.2008.12.044 CrossRefPubMed

23.

Raza SJ, Froghi S, Chowriappa A, Ahmed K, Field E, Stegemann AP, Rehman S, Sharif M, Shi Y, Wilding GE, Kesavadas T (2014) Construct validation of the key components of fundamental skills of robotic surgery (FSRS) curriculum—a multi-institution prospective study. J Surg Educ 71(3):316–324. https://doi.org/10.1016/J.Jsurg.2013.10.006 CrossRefPubMed

24.

Whittaker G, Aydin A, Raison N, Kum F, Challacombe B, Khan MS, Dasgupta P, Ahmed K (2016) Validation of the robotix mentor robotic surgery simulator. J Endourol 30(3):338–346. https://doi.org/10.1089/End.2015.0620 CrossRefPubMed

25.

Simbionix (2017) http://simbionix.com/simulators/robotix-mentor/robotix-library-of-modules/. Accessed 21 Feb 2017

26.

Fisher RA, Dasgupta P, Mottrie A, Volpe A, Khan MS, Challacombe B, Ahmed K (2015) An over-view of robot assisted surgery curricula and the status of their validation. Int J Surg 13:115–123. https://doi.org/10.1016/J.Ijsu.2014.11.033 CrossRefPubMed

27.

Bric JD, Lumbard DC, Frelich MJ, Gould JC (2016) current state of virtual reality simulation in robotic surgery training: a review. Surg Endosc 30(6):2169–2178. https://doi.org/10.1007/S00464-015-4517-Y CrossRefPubMed

28.

Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R (2013) Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis. Acad Med 88(8):1178–1186. https://doi.org/10.1097/Acm.0b013e31829a365d CrossRefPubMed

29.

Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R (2014) What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Health Sci Educ Theory Pract 19(2):233–250. https://doi.org/10.1007/S10459-013-9458-4 CrossRefPubMed

30.

Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I, Farrell T (2015) Technical skills assessment toolbox: a review using the unitary framework of validity. Ann Surg 261(2):251–262. https://doi.org/10.1097/Sla.0000000000000520 CrossRefPubMed

31.

Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sorensen JL, Dalsgaard T, Ottesen B (2006) objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 20(9):1460–1466. https://doi.org/10.1007/S00464-005-0745-X CrossRefPubMed

32.

Tulandi T, Einarsson J (2014) The use of barbed suture for laparoscopic hysterectomy and myomectomy: a systematic review and meta-analysis. J Minim Invasive Gynecol 21(2):210–216. https://doi.org/10.1016/J.Jmig.2013.09.014 CrossRefPubMed

33.

American Educational Research Association APA, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing (2014) Standards for educational and psychological testing. American Educational Research Association, Washington, DC

34.

Bloch R, Norman G (2012) Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68. Med Teach 34(11):960–992. https://doi.org/10.3109/0142159x.2012.703791 CrossRefPubMed

35.

Downing SM, Yudkowsky R (2009) Assessment in Health Professions Education. Routledge, New York

36.

Dyre L, Norgaard LN, Tabor A, Madsen ME, Sorensen JL, Ringsted C, Tolsgaard M (2016) Collecting validity evidence for the assessment of mastery learning in simulation-based ultrasound training. Ultraschall Med 37(4):386–392. https://doi.org/10.1055/S-0041-107976 CrossRefPubMed

37.

Madsen ME, Konge L, Norgaard LN, Tabor A, Ringsted C, Klemmensen AK, Ottesen B, Tolsgaard M (2014) Assessment of performance measures and learning curves for use of a virtual-reality ultrasound simulator in transvaginal ultrasound examination. Ultrasound Obstet Gynecol 44(6):693–699. https://doi.org/10.1002/Uog.13400 CrossRefPubMed

38.

Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241(2):364–372CrossRefPubMedPubMedCentral

39.

Connolly M, Seligman J, Kastenmeier A, Goldblatt M, Gould J (2014) Validation of a virtual reality-based robotic surgical skills curriculum. Surg Endosc 28(5):1691–1694. https://doi.org/10.1007/S00464-013-3373-X CrossRefPubMed

40.

Lee JY, Mucksavage P, Kerbl DC, Huynh VB, Etafy M, McDougall EM (2012) Validation study of a virtual reality robotic simulator—role as an assessment tool? J Urol 187(3):998–1002. https://doi.org/10.1016/J.Juro.2011.10.160 CrossRefPubMed

41.

Sethi AS, Peine WJ, Mohammadi Y, Sundaram CP (2009) Validation of a novel virtual reality robotic simulator. J Endourol 23(3):503–508. https://doi.org/10.1089/End.2008.0250 CrossRefPubMed

42.

Yule S, Flin R, Paterson-Brown S, Maran N (2006) Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 139(2):140–149. https://doi.org/10.1016/J.Surg.2005.06.017 CrossRefPubMed

43.

Lyons C, Goldfarb D, Jones SL, Badhiwala N, Miles B, Link R, Dunkin B (2013) Which skills really matter? Proving face, content, and construct validity for a commercial robotic simulator. Surg Endosc 27(6):2020–2030. https://doi.org/10.1007/S00464-012-2704-7 CrossRefPubMed

Title: Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure
Authors: Lisette Hvid Hovgaard
Steven Arild Wuyts Andersen
Lars Konge
Torur Dalsgaard
Christian Rifbjerg Larsen
Publication date: 01-10-2018
Publisher: Springer US
Published in: Surgical Endoscopy / Issue 10/2018
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-018-6165-5

At a glance: The STEP trials

Springer Medicine

Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure

Abstract

Background

Methods

Results

Conclusion

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 10/2018

Towards spill-free in-bag morcellation: a health failure mode and effects analysis

Safety of orogastric tubes in foregut and bariatric surgery

Examining the impact of surgical coaching on trainee physiologic response and basic skill acquisition

Laparoscopic pancreaticoduodenectomy with major venous resection and reconstruction: anterior superior mesenteric artery first approach

Transferability of laparoscopic skills using the virtual reality simulator

Learning curve for performing choledochotomy bile duct exploration with primary closure after laparoscopic cholecystectomy