Skip to main content
Top
Published in: Surgical Endoscopy 12/2005

01-12-2005

Consensus guidelines for validation of virtual reality surgical simulators

Authors: F. J. Carter, M. P. Schijven, R. Aggarwal, T. Grantcharov, N. K. Francis, G. B. Hanna, J. J. Jakimowicz

Published in: Surgical Endoscopy | Issue 12/2005

Login to get access

Abstract.

The Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes is a newly formed sub-group of the European Association of Endoscopic Surgeons (EAES). This work group undertook a review of validation evidence for surgical simulators and the resulting consensus is presented in this article. Using clinical guidelines criteria, the evidence for validation for six different simulators was rated and subsequently translated to a level of recommendation for each system. The simulators could be divided into two basic types; systems for laparoscopic general surgery and flexible gastrointestinal endoscopy. Selection of simulators for inclusion in this consensus was based on their availability and relatively widespread usage as of July 2004. Whilst level 2 recommendations were achieved for a few systems, it was clear that there was an overall lack of published validation studies with rigorous experimental methodology. Since the consensus meeting, there have been a number of new articles, system upgrades and new devices available. The work group intends to update these consensus guidelines on a regular basis, with the resulting article available on the EAES website (http://​www.​eaes-eur.​org ).
Literature
2.
go back to reference Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91: 146–150CrossRefPubMed Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91: 146–150CrossRefPubMed
3.
go back to reference Seymour NE, Gallagher AG, Roman SA, O’Brian MK, Bansal VK (2002) Virtual reality improves operating room performance: results of a randomized, double-blind study. Ann Surg 236: 458–463CrossRefPubMed Seymour NE, Gallagher AG, Roman SA, O’Brian MK, Bansal VK (2002) Virtual reality improves operating room performance: results of a randomized, double-blind study. Ann Surg 236: 458–463CrossRefPubMed
4.
go back to reference Hyltander A, Liljegren E, Rhodin PH, Lonroth H (2002) The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc 16: 1324–1328CrossRefPubMed Hyltander A, Liljegren E, Rhodin PH, Lonroth H (2002) The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc 16: 1324–1328CrossRefPubMed
5.
go back to reference Hamilton EC, Scott DJ, Fleming RV, Rege RV, Laycock R (2002) Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skill. Surg Endosc 16: 406–411CrossRefPubMed Hamilton EC, Scott DJ, Fleming RV, Rege RV, Laycock R (2002) Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skill. Surg Endosc 16: 406–411CrossRefPubMed
6.
go back to reference Ahlberg G, Heikkinen T, Iselius L, Leijonmarck CE, Rutqvist J (2002) Does training in a virtual reality simulator improve surgical performance? Surg Endosc 16: 126–129CrossRefPubMed Ahlberg G, Heikkinen T, Iselius L, Leijonmarck CE, Rutqvist J (2002) Does training in a virtual reality simulator improve surgical performance? Surg Endosc 16: 126–129CrossRefPubMed
7.
go back to reference Fry H, Ketteridge S, Marshall S (1999) Understanding student learning. In: Fry H, Ketteridge S, Marshall S (eds) A handbook for teaching and learning in higher education. Kogan Page, London Fry H, Ketteridge S, Marshall S (1999) Understanding student learning. In: Fry H, Ketteridge S, Marshall S (eds) A handbook for teaching and learning in higher education. Kogan Page, London
8.
go back to reference Kolb DA (1984) Experiential learning. Prentice-Hall, Englewood Cliffs, New Jersey Kolb DA (1984) Experiential learning. Prentice-Hall, Englewood Cliffs, New Jersey
9.
go back to reference Murphy KR, Davidshofer CO (1998) Psychological testing: principle and applications. 4th ed. Prentice-Hall, Upper Saddle River, New Jersey Murphy KR, Davidshofer CO (1998) Psychological testing: principle and applications. 4th ed. Prentice-Hall, Upper Saddle River, New Jersey
10.
go back to reference Ayodeji ID, Schijven MP, Jakimowicz JJ, Bonjer HJ (2004) Face validation of the LapMentor laparoscopy trainer (Abstract). Fall meeting of the Dutch Society of Surgery, Ede, The Netherlands Ayodeji ID, Schijven MP, Jakimowicz JJ, Bonjer HJ (2004) Face validation of the LapMentor laparoscopy trainer (Abstract). Fall meeting of the Dutch Society of Surgery, Ede, The Netherlands
11.
go back to reference Prystowski JB, Regehr G, Rogers DA, Loan JP, Hiemenz LL, Smith KM (1999) A virtual reality module for intravenous catheter placement. Am J Surg 177: 171–175 Prystowski JB, Regehr G, Rogers DA, Loan JP, Hiemenz LL, Smith KM (1999) A virtual reality module for intravenous catheter placement. Am J Surg 177: 171–175
12.
go back to reference Schijven M, Jakimowicz J (2003) Construct validity: experts and novices performing on the Xitact LS500 laparoscopy simulator. Surg Endosc 17: 803–810PubMed Schijven M, Jakimowicz J (2003) Construct validity: experts and novices performing on the Xitact LS500 laparoscopy simulator. Surg Endosc 17: 803–810PubMed
13.
go back to reference Schijven M (2005) Virtual reality simulation for laparoscopic cholecystectomy: the process of validation and implementation in the surgical curriculum outlined. Thesis, ISBN 90-9019048-1. University of Leiden, Leiden, the Netherlands Schijven M (2005) Virtual reality simulation for laparoscopic cholecystectomy: the process of validation and implementation in the surgical curriculum outlined. Thesis, ISBN 90-9019048-1. University of Leiden, Leiden, the Netherlands
15.
go back to reference Neugebauer E, Sauerland S (2000) Recommendations for evidence-based endoscopic surgery. Springer-Verlag, Paris, France Neugebauer E, Sauerland S (2000) Recommendations for evidence-based endoscopic surgery. Springer-Verlag, Paris, France
16.
go back to reference Fink A, Kosecoff J, Chassin M, Brook RH (1984) Consensus methods: characteristics and guidelines for use. Am J Public Health 74: 979–983PubMedCrossRef Fink A, Kosecoff J, Chassin M, Brook RH (1984) Consensus methods: characteristics and guidelines for use. Am J Public Health 74: 979–983PubMedCrossRef
17.
go back to reference (NABON), N.B.O.N. and N.B.C.C.t. Netherlands (1992) Richtlijn behandeling van het mammacarcinoom (Guidelines for treatment of breast cancer). van Zuiden communications, Alphen aan de Rijn. (NABON), N.B.O.N. and N.B.C.C.t. Netherlands (1992) Richtlijn behandeling van het mammacarcinoom (Guidelines for treatment of breast cancer). van Zuiden communications, Alphen aan de Rijn.
18.
go back to reference Datta VK, Mandalia M, Mackay SD, Darzi AW (2001) Evaluation and validation of a virtual reality–based flexible sigmoidoscopy trainer (Abstract). Gut 48((Suppl 1)): A97–A98 Datta VK, Mandalia M, Mackay SD, Darzi AW (2001) Evaluation and validation of a virtual reality–based flexible sigmoidoscopy trainer (Abstract). Gut 48((Suppl 1)): A97–A98
19.
go back to reference Garuda S, Keshavarzian A, Losurdo J, Brown MD (2002) Efficacy of a computer-assisted endoscopic simulator in training residents in flexible endoscopy (Poster). Proceedings of ACG 2002, Seattle, WA Garuda S, Keshavarzian A, Losurdo J, Brown MD (2002) Efficacy of a computer-assisted endoscopic simulator in training residents in flexible endoscopy (Poster). Proceedings of ACG 2002, Seattle, WA
20.
go back to reference Carter FJ, Steele RJC, Kennovin MF, Cuschieri A (2003) Validation of a virtual reality colonoscopy simulator using subjects of differing experience (Abstract). Proc. 1st European Endoscopic Surgical Week and 11th EAES Congress, 15th–18th June 2003, Glasgow, UK Carter FJ, Steele RJC, Kennovin MF, Cuschieri A (2003) Validation of a virtual reality colonoscopy simulator using subjects of differing experience (Abstract). Proc. 1st European Endoscopic Surgical Week and 11th EAES Congress, 15th–18th June 2003, Glasgow, UK
21.
go back to reference Sedlack RE, Kolars JC (2003) Validation of a computer-based colonoscopy simulator. Gastrointest Endosc 57: 214–218CrossRefPubMed Sedlack RE, Kolars JC (2003) Validation of a computer-based colonoscopy simulator. Gastrointest Endosc 57: 214–218CrossRefPubMed
22.
go back to reference Mahmood T, Darzi A (2003) A study to validate the colonoscopy simulator: it is usefully discriminatory for more than one measurable outcome. Surg Endosc 17: 1583–1589CrossRefPubMed Mahmood T, Darzi A (2003) A study to validate the colonoscopy simulator: it is usefully discriminatory for more than one measurable outcome. Surg Endosc 17: 1583–1589CrossRefPubMed
23.
go back to reference Thomas-Gibson S, Vance ME, Saunders BP (2003) Can a colonoscopy computer simulator differentiate between a novice and expert? (Abstract) Gut 52(Suppl 1): A73 Thomas-Gibson S, Vance ME, Saunders BP (2003) Can a colonoscopy computer simulator differentiate between a novice and expert? (Abstract) Gut 52(Suppl 1): A73
24.
go back to reference Sedlack RE, Kolars JC (2004) Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study. Am J Gastroenterol 99: 33–37CrossRefPubMed Sedlack RE, Kolars JC (2004) Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study. Am J Gastroenterol 99: 33–37CrossRefPubMed
25.
go back to reference Sedlack RE, Kolars JC (2003) Direct comparison of ERCP teaching models. Gastrointest Endosc 57: 886–890CrossRefPubMed Sedlack RE, Kolars JC (2003) Direct comparison of ERCP teaching models. Gastrointest Endosc 57: 886–890CrossRefPubMed
26.
go back to reference Aalbakken L, Adamsen A, Kruse A (2000) Performance of a colonoscopy simulator: experience from a hands-on endoscopy course. Endoscopy 32: 898–900 Aalbakken L, Adamsen A, Kruse A (2000) Performance of a colonoscopy simulator: experience from a hands-on endoscopy course. Endoscopy 32: 898–900
27.
go back to reference Grantcharov TP, Everbusch MD, Funch-Jensen P (2004) Teaching and testing surgical skills on a VR endoscopy simulator: learning curves and impact of psychomotor training on performance in simulated colonoscopy (Poster). SAGES meeting Denver, Colorado, March–April, 2004, P179 Grantcharov TP, Everbusch MD, Funch-Jensen P (2004) Teaching and testing surgical skills on a VR endoscopy simulator: learning curves and impact of psychomotor training on performance in simulated colonoscopy (Poster). SAGES meeting Denver, Colorado, March–April, 2004, P179
28.
go back to reference Yousfi MM, Darius Sorbi MD, Baron T, Fleischer DE (2002) Flexible sigmoidoscopy: assessing endoscopic skills using a computer-based simulator (Abstract). ACG meeting, Seattle, Washington, 2002 Yousfi MM, Darius Sorbi MD, Baron T, Fleischer DE (2002) Flexible sigmoidoscopy: assessing endoscopic skills using a computer-based simulator (Abstract). ACG meeting, Seattle, Washington, 2002
29.
go back to reference Ritter EM, McClusky DA III, Lederman AB, Gallagher AG, Smith CD (2003) Objective psychomotor skills assessment of experienced and novice flexible endoscopists with a virtual reality simulator. J Gastrointest Surg 7: 871–878CrossRefPubMed Ritter EM, McClusky DA III, Lederman AB, Gallagher AG, Smith CD (2003) Objective psychomotor skills assessment of experienced and novice flexible endoscopists with a virtual reality simulator. J Gastrointest Surg 7: 871–878CrossRefPubMed
30.
go back to reference Adamsen S, Funch-Jensen PM, Drewes AM, Rosenberg J, Grantcharov TP (2005) A comparative study of skills in virtual laparoscopy and endoscopy. Surg Endosc 19: 229–234CrossRefPubMed Adamsen S, Funch-Jensen PM, Drewes AM, Rosenberg J, Grantcharov TP (2005) A comparative study of skills in virtual laparoscopy and endoscopy. Surg Endosc 19: 229–234CrossRefPubMed
31.
go back to reference Ferlitsch A, Glauninger P, Gupper A, Schillinger M, Haefner M, Gangi A, Schoefl R (2002) Evaluation of a virtual endoscopy simulator for training in gastrointestinal endoscopy. Endoscopy 34: 698–702PubMed Ferlitsch A, Glauninger P, Gupper A, Schillinger M, Haefner M, Gangi A, Schoefl R (2002) Evaluation of a virtual endoscopy simulator for training in gastrointestinal endoscopy. Endoscopy 34: 698–702PubMed
32.
go back to reference Moorthy K, Munz Y, Jiwanji M, Bahn S, Chang A, Darzi A (2004) Validity and reliability of a virtual reality upper gastrointestinal simulator and cross validation using structured assessment of individual performance with video playback. Surg. Endosc 28: 328–333 Moorthy K, Munz Y, Jiwanji M, Bahn S, Chang A, Darzi A (2004) Validity and reliability of a virtual reality upper gastrointestinal simulator and cross validation using structured assessment of individual performance with video playback. Surg. Endosc 28: 328–333
33.
go back to reference Enochson L, Isaksson B, Tour R, Kjellin A, Hedman L, Wredmark T, Tsai-Fellander L (2004) Visuospatial skills and computer game experience influence the performance of virtual endoscopy. J Gastrointest Surg 8: 874–880 Enochson L, Isaksson B, Tour R, Kjellin A, Hedman L, Wredmark T, Tsai-Fellander L (2004) Visuospatial skills and computer game experience influence the performance of virtual endoscopy. J Gastrointest Surg 8: 874–880
34.
go back to reference Fanelli RD, Mainella MT, Justin RC, Gersin KS (2003) Initial experience using an endoscopic simulator to train residents in flexible endoscopy in a community medical center–based residency program. Flexible diagnostic and therapeutic endoscopy (Poster). Proceedings of the SAGES meeting, 12–15 March 2003, Los Angeles, California P233 Fanelli RD, Mainella MT, Justin RC, Gersin KS (2003) Initial experience using an endoscopic simulator to train residents in flexible endoscopy in a community medical center–based residency program. Flexible diagnostic and therapeutic endoscopy (Poster). Proceedings of the SAGES meeting, 12–15 March 2003, Los Angeles, California P233
35.
go back to reference Taffinder N, Russell RCG, McManus IC, Jansen J, Darzi A (1998) An objective assessment of surgeons’ psychomotor skills: validation of the MIST-VR laparoscopic simulator. Br J Surg 85(Supp 1): 75 Taffinder N, Russell RCG, McManus IC, Jansen J, Darzi A (1998) An objective assessment of surgeons’ psychomotor skills: validation of the MIST-VR laparoscopic simulator. Br J Surg 85(Supp 1): 75
36.
go back to reference Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P (2001) Virtual reality computer simulation: an objective method for the evaluation of laparoscopic surgical skills. Surg Endosc 15: 242–244CrossRefPubMed Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P (2001) Virtual reality computer simulation: an objective method for the evaluation of laparoscopic surgical skills. Surg Endosc 15: 242–244CrossRefPubMed
37.
go back to reference McNatt SS, Daniel Smith C (2001) A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons. Surg Endosc 15: 1085–1089CrossRefPubMed McNatt SS, Daniel Smith C (2001) A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons. Surg Endosc 15: 1085–1089CrossRefPubMed
38.
go back to reference Gallagher HJ, Allan JD, Tolley DA (2001) Spatial awareness in urologists: are they different?’ BJU Int 88: 666–670CrossRefPubMed Gallagher HJ, Allan JD, Tolley DA (2001) Spatial awareness in urologists: are they different?’ BJU Int 88: 666–670CrossRefPubMed
39.
go back to reference Gallagher AG, Richie K, McClure N, McGuigan J (2001) Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality. World J Surg 25: 1478–1483PubMed Gallagher AG, Richie K, McClure N, McGuigan J (2001) Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality. World J Surg 25: 1478–1483PubMed
40.
go back to reference Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills: learning curves and reliability measures. Surg Endosc 16: 1746–1752CrossRefPubMed Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills: learning curves and reliability measures. Surg Endosc 16: 1746–1752CrossRefPubMed
41.
go back to reference Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003) Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am J Surg 185: 146–149CrossRefPubMed Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003) Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am J Surg 185: 146–149CrossRefPubMed
42.
go back to reference Gallagher AG, Lederman AB, McGlade K, Satava RM, Smith CD (2004) Discriminative validity of the Minimally Invasive Surgical Trainer in Virtual Reality (MIST-VR) using criteria levels based on expert performance. Surg Endosc 18: 660–665CrossRefPubMed Gallagher AG, Lederman AB, McGlade K, Satava RM, Smith CD (2004) Discriminative validity of the Minimally Invasive Surgical Trainer in Virtual Reality (MIST-VR) using criteria levels based on expert performance. Surg Endosc 18: 660–665CrossRefPubMed
43.
go back to reference Carter FJ, Francis NK, Tang B, Martindale JP, Cuschieri A (2003) Validation of a virtual reality simulator with error analysis of a simulated surgical procedure (Abstract). Surg. Endosc 17(Suppl 1): S61 Carter FJ, Francis NK, Tang B, Martindale JP, Cuschieri A (2003) Validation of a virtual reality simulator with error analysis of a simulated surgical procedure (Abstract). Surg. Endosc 17(Suppl 1): S61
44.
go back to reference Lonroth H, Olsson JH, Johnsson E, Pazooki D (2003) Can virtual reality training tools differentiate between different levels of expertise and experience? (Abstract) Surg. Endosc 17(Suppl 1): S63 Lonroth H, Olsson JH, Johnsson E, Pazooki D (2003) Can virtual reality training tools differentiate between different levels of expertise and experience? (Abstract) Surg. Endosc 17(Suppl 1): S63
45.
go back to reference Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos PJ, Fowler DL (2003). Construct validity for the lapsim laparoscopic surgical simulator (Abstract). Surg. Endosc 17(Suppl 1): S230 Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos PJ, Fowler DL (2003). Construct validity for the lapsim laparoscopic surgical simulator (Abstract). Surg. Endosc 17(Suppl 1): S230
46.
go back to reference Broe D, Ridgway PF, Johnson S, Tierney C, Conlon KC (2004) Validation of a novel hybrid surgical simulator. Proc. 12th EAES Congress 9th–12th June 2004, Barcelona, Spain Broe D, Ridgway PF, Johnson S, Tierney C, Conlon KC (2004) Validation of a novel hybrid surgical simulator. Proc. 12th EAES Congress 9th–12th June 2004, Barcelona, Spain
47.
go back to reference Hance J, Aggarwal R, Undre S, Patel H, Darzi A (2004) Evaluation of a laparoscopic video trainer with built in measures of performance. Proc. 13th SLS meeting and EndoExpo 29th September–2nd October 2004, New York Hance J, Aggarwal R, Undre S, Patel H, Darzi A (2004) Evaluation of a laparoscopic video trainer with built in measures of performance. Proc. 13th SLS meeting and EndoExpo 29th September–2nd October 2004, New York
48.
go back to reference McClusky DAM, Van Sickle K, Gallagher AG (2004) Relationship between motion analysis, time, accuracy, and errors during performance of a laparoscopic suturing task on an augmented reality simulator’. Proc. 12th EAES Congress 9th–12th June 2004, Barcelona, Spain McClusky DAM, Van Sickle K, Gallagher AG (2004) Relationship between motion analysis, time, accuracy, and errors during performance of a laparoscopic suturing task on an augmented reality simulator’. Proc. 12th EAES Congress 9th–12th June 2004, Barcelona, Spain
Metadata
Title
Consensus guidelines for validation of virtual reality surgical simulators
Authors
F. J. Carter
M. P. Schijven
R. Aggarwal
T. Grantcharov
N. K. Francis
G. B. Hanna
J. J. Jakimowicz
Publication date
01-12-2005
Publisher
Springer-Verlag
Published in
Surgical Endoscopy / Issue 12/2005
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-005-0384-2

Other articles of this Issue 12/2005

Surgical Endoscopy 12/2005 Go to the issue