Top

Published in:

01-06-2017

Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy

Authors: Katrine Jensen, Flemming Bjerrum, Henrik Jessen Hansen, René Horsleben Petersen, Jesper Holst Pedersen, Lars Konge

Published in: Surgical Endoscopy | Issue 6/2017

Abstract

Background

The societies of thoracic surgery are working to incorporate simulation and competency-based assessment into specialty training. One challenge is the development of a simulation-based test, which can be used as an assessment tool. The study objective was to establish validity evidence for a virtual reality simulator test of a video-assisted thoracoscopic surgery (VATS) lobectomy of a right upper lobe.

Methods

Participants with varying experience in VATS lobectomy were included. They were familiarized with a virtual reality simulator (LapSim^®) and introduced to the steps of the procedure for a VATS right upper lobe lobectomy. The participants performed two VATS lobectomies on the simulator with a 5-min break between attempts. Nineteen pre-defined simulator metrics were recorded.

Results

Fifty-three participants from nine different countries were included. High internal consistency was found for the metrics with Cronbach’s alpha coefficient for standardized items of 0.91. Significant test–retest reliability was found for 15 of the metrics (p-values <0.05). Significant correlations between the metrics and the participants VATS lobectomy experience were identified for seven metrics (p-values <0.001), and 10 metrics showed significant differences between novices (0 VATS lobectomies performed) and experienced surgeons (>50 VATS lobectomies performed). A pass/fail level defined as approximately one standard deviation from the mean metric scores for experienced surgeons passed none of the novices (0 % false positives) and failed four of the experienced surgeons (29 % false negatives).

Conclusion

This study is the first to establish validity evidence for a VATS right upper lobe lobectomy virtual reality simulator test. Several simulator metrics demonstrated significant differences between novices and experienced surgeons and pass/fail criteria for the test were set with acceptable consequences. This test can be used as a first step in assessing thoracic surgery trainees’ VATS lobectomy competency.

McGaghie WC (2008) Research opportunities in simulation-based medical education using deliberate practice. Acad Emerg Med 15:995–1001. doi:10.1111/j.1553-2712.2008.00246.x CrossRefPubMed

Price J, Naik V, Boodhwani M, Brandys T, Hendry P, Lam BBK (2011) A randomized evaluation of simulation training on performance of vascular anastomosis on a high-fidelity in vivo model: the role of deliberate practice. J Thorac Cardiovasc Surg 142:496–503. doi:10.1016/j.jtcvs.2011.05.015 CrossRefPubMed

Marshall MB (2012) Simulation for technical skills. J Thorac Cardiovasc Surg 144:S43–S47. doi:10.1016/j.jtcvs.2012.06.004 CrossRefPubMed

Crochet P, Aggarwal R, Dubb SS, Ziprin P, Rajaretnam N, Grantcharov T, Ericsson KA, Darzi A (2011) Deliberate practice on a virtual reality laparoscopic simulator enhances the quality of surgical technical skills. Ann Surg 253:1216–1222. doi:10.1097/SLA.0b013e3182197016 CrossRefPubMed

Seymour NE (2008) VR to OR: a review of the evidence that virtual reality simulation improves operating room performance. World J Surg 32:182–188. doi:10.1007/s00268-007-9307-9 CrossRefPubMed

Schijven MP, Jakimowicz JJ, Broeders IAMJ, Tseng LNL (2005) The Eindhoven laparoscopic cholecystectomy training course—improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc Other Interv Tech 19:1220–1226. doi:10.1007/s00464-004-2240-1 CrossRef

Jensen K, Bjerrum F, Hansen HJ, Petersen RH, Pedersen JH, Konge L (2015) A new possibility in thoracoscopic virtual reality simulation training: development and testing of a novel virtual reality simulator for video-assisted thoracoscopic surgery lobectomy. Interact Cardiovasc Thorac Surg 21:420–426. doi:10.1093/icvts/ivv183 CrossRefPubMed

Seymour NE, Røtnes JS (2006) Challenges to the development of complex virtual reality surgical simulations. Surg Endosc Other Interv Tech 20:1774–1777. doi:10.1007/s00464-006-0107-3 CrossRef

Jensen K, Ringsted C, Hansen HJ, Petersen RH, Konge L (2014) Simulation-based training for thoracoscopic lobectomy: a randomized controlled trial: virtual-reality versus black-box simulation. Surg Endosc Other Interv Tech 28:1821–1829. doi:10.1007/s00464-013-3392-7 CrossRef

10.

Bjerrum F, Sorensen JL, Konge L, Rosthøj S, Lindschou J, Ottesen B, Strandbygaard J (2016) Randomized clinical trial to examine procedure-to-procedure transfer in laparoscopic simulator training. Br J Surg 103:44–50. doi:10.1002/bjs.9966 CrossRefPubMed

11.

Carpenter AJ, Yang SC, Uhlig PN, Colson YL (2008) Envisioning simulation in the future of thoracic surgical education. J Thorac Cardiovasc Surg 135:477–484. doi:10.1016/j.jtcvs.2007.12.005 CrossRefPubMed

12.

Verrier ED (2011) Joint Council on Thoracic Surgical Education: an investment in our future. J Thorac Cardiovasc Surg 141:318–321. doi:10.1016/j.jtcvs.2010.10.013 CrossRefPubMed

13.

Moon MR (2014) Technical skills assessment in thoracic surgery education: we won’t get fooled again. J Thorac Cardiovasc Surg 148:2497–2498. doi:10.1016/j.jtcvs.2014.09.057 CrossRefPubMed

14.

Lodge D, Grantcharov T (2011) Training and assessment of technical skills and competency in cardiac surgery. Eur J Cardiothorac Surg 39:287–293. doi:10.1016/j.ejcts.2010.06.035 CrossRefPubMed

15.

Aggarwal R, Darzi A (2011) Simulation to enhance patient safety: Why aren’t we there yet? Chest 140:854–858. doi:10.1378/chest.11-0728 CrossRefPubMed

16.

Stefanidis D, Sevdalis N, Paige J, Zevin B, Aggarwal R, Grantcharov T, Jones DB (2015) Simulation in surgery: What’s needed next? Ann Surg 261:846–853. doi:10.1097/SLA.0000000000000826 CrossRefPubMed

17.

Baumgartner WA (2003) Cardiothoracic surgery: a specialty in transition—Good to great? Ann Thorac Surg 75:1685–1692. doi:10.1016/S0003-4975(03)00538-1 CrossRefPubMed

18.

Chitwood WR, Spray TL, Feins RH, Mack MJ (2008) Mission critical: thoracic surgery education reform. Ann Thorac Surg 86:1061–1062. doi:10.1016/j.athoracsur.2008.08.047 CrossRefPubMed

19.

Crawford FA (2003) Presidential address: thoracic surgery education—responding to a changing environment. J Thorac Cardiovasc Surg 126:1235–1242. doi:10.1016/S0022-5223(03)00814-6 CrossRefPubMed

20.

Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus DM, Jeyarajah DR, Thompson WM, Jones DB (2000) Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg 191:272–283. doi:10.1016/S1072-7515(00)00339-2 CrossRefPubMed

21.

Torkington J, Smith SGT, Rees BI, Darzi A (2001) Skill transfer from virtual reality to a real laparoscopic task. Surg Endosc 15:1076–1079. doi:10.1007/s004640000233 CrossRefPubMed

22.

Paisley AM, Baldwin PJ, Paterson-Brown S (2001) Validity of surgical simulation for the assessment of operative skill. Br J Surg 88:1525–1532. doi:10.1046/j.0007-1323.2001.01880.x CrossRefPubMed

23.

Reznick RK, MacRae H (2006) Teaching surgical skills—changes in the wind. N Engl J Med 355:2664–2669. doi:10.1016/S0084-392X(08)70199-0 CrossRefPubMed

24.

Lee R, Enter D, Lou X, Feins RH, Hicks GL, Gasparri M, Takayama H, Young JN, Calhoon JH, Crawford FA, Mokadam NA, Fann JI (2013) The joint council on thoracic surgery education coronary artery assessment tool has high interrater reliability. Ann Thorac Surg 95:2064–2070. doi:10.1016/j.athoracsur.2012.10.090 CrossRefPubMed

25.

Van Bruwaene S, Schijven MP, Miserez M (2014) Assessment of procedural skills using virtual simulation remains a challenge. J Surg Educ 71:654–661. doi:10.1016/j.jsurg.2014.01.005 CrossRefPubMed

26.

Seagull FJ, Rooney DM (2014) Filling a void: developing a standard subjective assessment tool for surgical simulation through focused review of current practices. Surgery 156:718–722. doi:10.1016/j.surg.2014.04.048 CrossRefPubMed

27.

Van Hove PD, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987. doi:10.1002/bjs.7115 CrossRefPubMed

28.

Hansen HJ, Petersen RH, Christensen M (2011) Video-assisted thoracoscopic surgery (VATS) lobectomy using a standardized anterior approach. Surg Endosc Other Interv Tech 25:1263–1269. doi:10.1007/s00464-010-1355-9 CrossRef

29.

Konge L, Ringsted C, Bjerrum F, Tolsgaard MG, Bitsch M, Sørensen JL, Schroeder TV (2015) The simulation centre at Rigshospitalet, Copenhagen, Denmark. J Surg Educ 72:362–365. doi:10.1016/j.jsurg.2014.11.012 CrossRefPubMed

30.

Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sørensen JL, Dalsgaard T, Ottesen B (2006) Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 20:1460–1466. doi:10.1007/s00464-005-0745-x CrossRefPubMed

31.

Van Dongen KW, Tournoij E, Van Der Zee DC, Schijven MP, Broeders IAMJ (2007) Construct validity of the LapSim: Can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc Other Interv Tech 21:1413–1417. doi:10.1007/s00464-006-9188-2 CrossRef

32.

Eriksen JR, Grantcharov T (2005) Objective assessment of laparoscopic skills using a virtual reality stimulator. Surg Endosc 19:1216–1219. doi:10.1007/s00464-004-2154-y CrossRefPubMed

33.

Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I, Farrell TM (2015) Technical skills assessment toolbox: a review using the unitary framework of validity. Ann Surg 261:251–262. doi:10.1097/SLA.0000000000000520 CrossRefPubMed

34.

Vassiliou MC, Ghitulescu GA, Feldman LS, Stanbridge D, Leffondré K, Sigman HH, Fried GM (2006) The MISTELS program to measure technical skill in laparoscopic surgery: evidence for reliability. Surg Endosc Other Interv Tech 20:744–747. doi:10.1007/s00464-005-3008-y CrossRef

35.

Downing SM (2003) Validity: on the meaningful interpretation of assessment data. Med Educ 37:830–837. doi:10.1046/j.1365-2923.2003.01594.x CrossRefPubMed

36.

Rivard JD, Vergis AS, Unger BJ, Hardy KM, Andrew CG, Gillman LM, Park J (2014) Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator. Surg Endosc Other Interv Tech 28:1921–1928. doi:10.1007/s00464-013-3414-5 CrossRef

37.

Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L, Gauger PG, Minter RM (2006) Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 191:28–32. doi:10.1016/j.amjsurg.2005.10.018 CrossRefPubMed

38.

Fried GM, Feldman LS (2008) Objective assessment of technical performance. World J Surg 32:156–160. doi:10.1007/s00268-007-9143-y CrossRefPubMed

39.

MacRae HM (2014) Technical skills assessment: time to take it seriously. Dis Colon Rectum 57:141–142. doi:10.1097/DCR.0000000000000032 CrossRefPubMed

40.

Hogle NJ, Briggs WM, Fowler DL (2007) Documenting a learning curve and test–retest reliability of two tasks on a virtual reality training simulator in laparoscopic surgery. J Surg Educ 64:424–430. doi:10.1016/j.jsurg.2007.08.007 CrossRefPubMed

41.

Loukas C, Nikiteas N, Schizas D, Lahanas V, Georgiou E (2012) A head-to-head comparison between virtual reality and physical reality simulation training for basic skills acquisition. Surg Endosc 26:2550–2558. doi:10.1007/s00464-012-2230-7 CrossRefPubMed

42.

Zendejas B, Ruparel RK, Cook DA (2015) Validity evidence for the fundamentals of laparoscopic surgery (FLS) program as an assessment tool: a systematic review. Surg Endosc Other Interv Tech 30:512–520. doi:10.1007/s00464-015-4233-7 CrossRef

43.

Våpenstad C, Buzink SN (2013) Procedural virtual reality simulation in minimally invasive surgery. Surg Endosc 27:364–377. doi:10.1007/s00464-012-2503-1 CrossRefPubMed

44.

Rocco G, Internullo E, Cassivi SD, Van Raemdonck D, Ferguson MK (2008) The variability of practice in minimally invasive thoracic surgery for pulmonary resections. Thorac Surg Clin 18:235–247. doi:10.1016/j.thorsurg.2008.06.002 CrossRefPubMed

45.

Petersen RH, Hansen HJ (2012) Learning curve associated with VATS lobectomy. Ann Cardiothorac Surg 1:47–50. doi:10.3978/478 PubMedPubMedCentral

46.

Li X, Wang J, Ferguson MK (2014) Competence versus mastery: the time course for developing proficiency in video-assisted thoracoscopic lobectomy. J Thorac Cardiovasc Surg 147:1150–1154. doi:10.1016/j.jtcvs.2013.11.036 CrossRefPubMed

47.

Ferguson J, Walker W (2006) Developing a VATS lobectomy programme—Can VATS lobectomy be taught? Eur J Cardiothorac Surg 29:806–809. doi:10.1016/j.ejcts.2006.02.012 CrossRefPubMed

48.

McKenna RJ (2008) Complications and learning curves for video-assisted thoracic surgery lobectomy. Thorac Surg Clin 18:275–280. doi:10.1016/j.thorsurg.2008.04.004 CrossRefPubMed

49.

Gjeraa K, Spanager L, Konge L, Petersen RH, Ostergaard D (2016) Non-technical skills in minimally invasive surgery teams: a systematic review. Surg Endosc. doi:10.1007/s00464-016-4890-1 PubMed

50.

Savran MM, Hansen HJ, Petersen RH, Walker W, Schmid T, Bojsen SR, Konge L (2015) Development and validation of a theoretical test of proficiency for video-assisted thoracoscopic surgery (VATS) lobectomy. Surg Endosc 29:2598–2604. doi:10.1007/s00464-014-3975-y CrossRefPubMed

51.

Vilmann P, Frost Clementsen P, Colella S, Siemsen M, De Leyn P, Dumonceau JM, Herth FJ, Larghi A, Vazquez-Sequeiros E, Hassan C, Crombag L, Korevaar DA, Konge L, Annema JT (2015) Combined endobronchial and esophageal endosonography for the diagnosis and staging of lung cancer: European Society of gastrointestinal endoscopy (ESGE) guideline, in cooperation with the European Respiratory Society (ERS) and the European Society of thor. Eur J Cardiothorac Surg 48:1–15. doi:10.1093/ejcts/ezv194 CrossRefPubMed

52.

Aggarwal R, Mytton OT, Derbrew M, Hananel D, Heydenburg M, Issenberg B, MacAulay C, Mancini ME, Morimoto T, Soper N, Ziv A, Reznick R (2010) Training and simulation for patient safety. Qual Saf Health Care 19(Suppl 2):i34–i43. doi:10.1136/qshc.2009.038562 CrossRefPubMed

53.

Barsuk JH, Cohen ER, Wayne DB, Siddall VJ, McGaghie WC (2016) Developing a simulation-based mastery learning curriculum. Simul Healthc J Soc Simul Healthc 11:52–59. doi:10.1097/SIH.0000000000000120 CrossRef

54.

Mcgaghie WC (2015) Mastery learning : it is time for medical education to join the 21st century. Acad Med 90:1–4. doi:10.1097/ACM.0000000000000911 CrossRef

55.

Konge L, Vilmann P, Clementsen P, Annema JT, Ringsted C (2012) Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer. Endoscopy 44:928–933. doi:10.1055/s-0032-1309892 CrossRefPubMed

56.

Tong BC, Gustafson MR, Balderson SS, D’Amico TA, Meyerson SL (2012) Validation of a thoracoscopic lobectomy simulator. Eur J Cardiothorac Surg 42:364–369. doi:10.1093/ejcts/ezs012 CrossRefPubMed

57.

Meyerson SL, Tong BC, Balderson SS, D’Amico TA, Phillips JD, Decamp MM, Darosa DA (2012) Needs assessment for an errors-based curriculum on thoracoscopic lobectomy. Ann Thorac Surg 94:368–373. doi:10.1016/j.athoracsur.2012.04.023 CrossRefPubMedPubMedCentral

Title: Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy
Authors: Katrine Jensen
Flemming Bjerrum
Henrik Jessen Hansen
René Horsleben Petersen
Jesper Holst Pedersen
Lars Konge
Publication date: 01-06-2017
Publisher: Springer US
Published in: Surgical Endoscopy / Issue 6/2017
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-016-5254-6

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy

Abstract

Background

Methods

Results

Conclusion

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 6/2017

The surgical outcomes of robot-assisted laparoscopic pancreaticoduodenectomy versus laparoscopic pancreaticoduodenectomy for periampullary neoplasms: a comparative study of a single center

Fluorescent incisionless cholangiography as a teaching tool for identification of Calot’s triangle

Erratum to: Laparoscopic gastrectomy for gastric cancer in the elderly patients

Identification of risk factors in minimally invasive surgery: a prospective multicenter study

Self-assessment in laparoscopic surgical skills training: Is it reliable?

Laparoscopic versus robotic colectomy: a national surgical quality improvement project analysis