Top

Surgical Endoscopy

Published in:

01-10-2021 | New Technology

Evaluation of a novel universal robotic surgery virtual reality simulation proficiency index that will allow comparisons of users across any virtual reality simulation curriculum

Authors: Christopher Simmonds, Mark Brentnall, John Lenihan

Published in: Surgical Endoscopy | Issue 10/2021

Abstract

Study objective

Evaluate a universal proficiency metric for Robotic Surgery Virtual Reality (VR) simulation that will allow comparison of all users across any VR curriculum.

Design

Retrospective analysis of VR Simulation metrics.

Setting

Two training institutions.

Patients or participants

Residents, fellows and practicing surgeons.

Interventions

Analysis of the Mimic robotic Virtual Reality (VR)-Simulation database of over 600,000 sessions was utilized to calculate Mean scores for each exercise. Those Mean scores were then normalized to 100. Subject's scores were also averaged and normalized to 100. We called this Index score the MScore Proficiency Index (MPI^©). Scores above 100 were better than average; Less than 100 were worse than average.

Measurements and main results

Seventeen thousand six hundred and forty eight sessions were analyzed (2017–2020) comparing 77 students (residents to practicing surgeons) working in 7 different curriculums. On average, each student spent 8 h and 24 min on simulation, attempted 26.5 different exercises, and became proficient in 20.6 exercises per user. The MPI^© mean score for all participants in all curricula was an MPI^© of 104.9 (SD: 15.5).

Thirteen students were 1 standard deviation below the norm with an average MPI^© of 80.15. This group averaged 9 h 27 min each on the simulator attempting 23.46 exercises but becoming proficient in only 10.38 (47%) of them in 224 sessions. Twelve students were 1 standard deviation above the norm with an average MPI^© of 127.05. This group averaged 6 h 31 min each on the simulator attempting 29.08 exercises but becoming proficient in 27.5 (95%) of them in 196 sessions.

Conclusion

A universal skill-based performance index (MPI^©) was calculated and found to be a reliable tool that could be used to identify relative proficiency among students in different robotic surgery VR Simulation curriculums. An individual user’s proficiency can be utilized to identify a student’s progress in a given curriculum. Future studies of MPI^© will determine if machine learning can provide timely personalized feedback to the user.

Thomsen SS, Bach-Holm D, Kjaerbo H, Hojgaard-Olsen K, Subhi Y, Saleh G, SooPark Y, Konge L (2017) Operating room performance improves after proficiency-based virtual reality cataract surgery training. Ophthamology 24(14):524–531. https://doi.org/10.1016/j.ophtha.2016.11.015CrossRef

Satava RM, Gallagher AG, Pellegrini CA (2003) Surgical competence and surgical proficiency: definitions, taxonomy, and metrics. J Am Coll Surg 196(6):933–937. https://doi.org/10.1016/S1072-7515(03)00237-CrossRefPubMed

Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241(2):364–372. https://doi.org/10.1097/01.sla.0000151982.85062.80CrossRefPubMedPubMedCentral

Zheng A, Hur H-C, Johnson S, Swanstrom L (2010) Validity of using fundamentals of laparoscopic surgery (FLS) program to assess laparoscopic competence for gynecologists. Surg Endosc 21(1):152–160. https://doi.org/10.1007/s00464-009-0539-7CrossRef

Sroka G, Feldman L, Pepa V, Kaneva A, Fayez R, Fried G (2010) Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room—a randomized controlled trial. Am JourSurg 199(1):115–120. https://doi.org/10.1016/j.amjsurg.2009.07.035CrossRef

Mowat A, Maher C, Ballard E (2016) Surgical outcomes for low-volume vs. high-volume surgeons in gynecology surgery: a systematic review and meta-analysis. Am J Obstet Gynecol 215(1):21–33. https://doi.org/10.1016/j.ajog.2016.02.048CrossRefPubMed

Hung A, Chen J, Liu Z, Nguyen J, Oh P, Stewart D, Remulla D, Chu T, Lee R, Aron K, Ghodoussipour S, Purushotham S, Gil I (2019) Deep learning model to predict time to urinary continence recovery after robot-assisted radical prostatectomy using automated performance metrics and clinical data. J Urol. https://doi.org/10.1097/01.JU.0000555971.26833.flCrossRefPubMed

Lendvay TS, White L, Kowalewski T (2015) Crowdsourcing to Assess Surgical Skill. JAMA Surg 150(11):1086–1087. https://doi.org/10.1001/jamasurg.2015.2405CrossRefPubMed

Niitsu H, Hirabayashi N, Yoshimitsu M, Mimura T, Taomoto J, Sugiyama Y, Murakami S, Saeki S, Mukaidaq H, Takiyama W (2013) Using the objective structured assessment of technical skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg Today 43(3):271–275. https://doi.org/10.1007/s00595-012-0313-7CrossRefPubMed

10.

Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012) Global evaluative assessment of robotic skills (GEARS): validation of a clinical assessment tool to measure robotic surgical skills. JUrol 187(1):247–252. https://doi.org/10.1016/j.juro.2011.09.032CrossRef

11.

Khalid S, Goldenberg M, Grantcharov T, Taati B, Rudzicz F (2020) Evaluation of deep learning models for identifying surgical actions and measuring performance. JAMA Newt Open 3(30):e201664. https://doi.org/10.1001/jamanetworkopen.2020.1664CrossRef

12.

Hashimoto DA, Rosman G, Volkov M, Rus D, Meireles O (2017) Artificial intelligence for intraoperative video analysis: machine learning’s role in surgical education. JACS. https://doi.org/10.1016/j.amcollsurg.2017.07.387CrossRef

13.

Satava R, Stefanidis D, Levy J, Smith R, Martin J, Monfared S, Timsina L, Darzi A, Moglia A, Brand T, Dorin R, Dumon K, Francone T, Georgiou E, Goh A, Marcet J, Martino M, Sudan R, Vale J, Gallagher A (2019) Proving the effectiveness of the fundamentals of robotic surgery (FRS) skills curriculum: a single-blinded, multispecialty multi-institutional randomized control trial. Ann Surg. https://doi.org/10.1097/SLA.0000000000003220CrossRef

14.

Lara J, Julian D, Smith R, Magnuson JS. Optimizing Simulation-Based Skill Acquisition for Novice Robotic Surgeons. Presented at ACS-AEI Annual Virtual Meeting, Oct 28, 2020. www.facs.org/education/accreditation/aei/surgical-simulation-summit.

15.

Training Centers: One was a post graduate robotic surgery training center and the other was an academic training program.

16.

Liss MA, Abdelshehid C, Quach S, Lusch A, Gravferson J, Landman J, McGougall E (2012) Validation, correlation, and comparison of the da Vinci trainer (dV-Trainer) and the da Vinci surgical skills simulator (dVSSS) using Mimic Software for urologic robotic surgical education. J Endo Urol. https://doi.org/10.1089/end.2012.0328CrossRef

Title: Evaluation of a novel universal robotic surgery virtual reality simulation proficiency index that will allow comparisons of users across any virtual reality simulation curriculum
Authors: Christopher Simmonds
Mark Brentnall
John Lenihan
Publication date: 01-10-2021
Publisher: Springer US
Published in: Surgical Endoscopy / Issue 10/2021
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-021-08609-5

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Evaluation of a novel universal robotic surgery virtual reality simulation proficiency index that will allow comparisons of users across any virtual reality simulation curriculum

Abstract

Study objective

Design

Setting

Patients or participants

Interventions

Measurements and main results

Conclusion

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Study objective

Design

Setting

Patients or participants

Interventions

Measurements and main results

Conclusion

Please log in to get access to this content

Other articles of this Issue 10/2021

Impact of laparoscopic surgical experience on the learning curve of robotic rectal cancer surgery

Endoscopic onlay repair for ventral hernia and rectus abdominis diastasis repair: Why so many different names for the same procedure? A qualitative systematic review

The use of tamsulosin to prevent postoperative urinary retention in laparoscopic inguinal hernia repair: a randomized double-blind placebo-controlled study

POEM: clinical outcomes beyond 5 years

Endoscopic submucosal dissection using a detachable assistant robot: a comparative in vivo feasibility study (with video)

Measurement of outflow resistance imposed by magnetic sphincter augmentation: defining normal values and clinical implication