Top

Surgical Endoscopy

Published in:

Open Access 01-03-2010 | Review

Validation and implementation of surgical simulators: a critical review of present, past, and future

Authors: B. M. A. Schout, A. J. M. Hendrikx, F. Scheele, B. L. H. Bemelmans, A. J. J. A. Scherpbier

Published in: Surgical Endoscopy | Issue 3/2010

Abstract

Background

In the past 20 years the surgical simulator market has seen substantial growth. Simulators are useful for teaching surgical skills effectively and with minimal harm and discomfort to patients. Before a simulator can be integrated into an educational program, it is recommended that its validity be determined. This study aims to provide a critical review of the literature and the main experiences and efforts relating to the validation of simulators during the last two decades.

Methods

Subjective and objective validity studies between 1980 and 2008 were identified by searches in Pubmed, Cochrane, and Web of Science.

Results

Although several papers have described definitions of various subjective types of validity, the literature does not offer any general guidelines concerning methods, settings, and data interpretation. Objective validation studies on endourological simulators were mainly characterized by a large variety of methods and parameters used to assess validity and in the definition and identification of expert and novice levels of performance.

Conclusion

Validity research is hampered by a paucity of widely accepted definitions and measurement methods of validity. It would be helpful to those considering the use of simulators in training programs if there were consensus on guidelines for validating surgical simulators and the development of training programs. Before undertaking a study to validate a simulator, researchers would be well advised to conduct a training needs analysis (TNA) to evaluate the existing need for training and to determine program requirements in a training program design (TPD), methods that are also used by designers of military simulation programs. Development and validation of training models should be based on a multidisciplinary approach involving specialists (teachers), residents (learners), educationalists (teaching the teachers), and industrial designers (providers of teaching facilities). In addition to technical skills, attention should be paid to contextual, interpersonal, and task-related factors.

Satava RM (2008) Historical review of surgical simulation–a personal perspective. World J Surg 32(2):141–148CrossRefPubMed

Dent JA, Harden RM (2005) A practical guide for medical teachers, 2nd edn. Elsevier, Dundee

Aggarwal R, Undre S, Moorthy K, Vincent C, Darzi A (2004) The simulated operating theatre: comprehensive training for surgical teams. Qual Saf Health Care 13(Suppl 1):i27–i32CrossRefPubMed

Blumenthal D (1994) Making medical errors into “medical treasures”. JAMA 272(23):1867–1868CrossRefPubMed

Bradley P, Bligh J (2005) Clinical skills centres: where are we going? Med Educ 39(7):649–650CrossRefPubMed

Bradley P (2006) The history of simulation in medical education and possible future directions. Med Educ 40(3):254–262CrossRefPubMed

Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G et al (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241(2):364–372CrossRefPubMed

Kneebone RL, Scott W, Darzi A, Horrocks M (2004) Simulation and clinical practice: strengthening the relationship. Med Educ 38(10):1095–1102CrossRefPubMed

Kneebone RL, Nestel D, Vincent C, Darzi A (2007) Complexity, risk and simulation in learning procedural skills. Med Educ 41(8):808–814CrossRefPubMed

10.

Reznick RK, MacRae H (2006) Teaching surgical skills—changes in the wind. N Engl J Med 355(25):2664–2669CrossRefPubMed

11.

Ziv A, Wolpe PR, Small SD, Glick S (2003) Simulation-based medical education: an ethical imperative. Acad Med 78(8):783–788CrossRefPubMed

12.

Seymour NE (2008) VR to OR: a review of the evidence that virtual reality simulation improves operating room performance. World J Surg 32(2):182–188CrossRefPubMed

13.

Issenberg SB, McGaghie WC, Hart IR, Mayer JW, Felner JM, Petrusa ER et al (1999) Simulation technology for health care professional skills training and assessment. JAMA 282(9):861–866CrossRefPubMed

14.

Kneebone R (2003) Simulation in surgical training: educational issues and practical implications. Med Educ 37(3):267–277CrossRefPubMed

15.

McDougall EM (2007) Validation of surgical simulators. J Endourol 21(3):244–247CrossRefPubMed

16.

Bajka M, Tuchschmid S, Streich M, Fink D, Szekely G, Harders M (2008) Evaluation of a new virtual-reality training simulator for hysteroscopy. Surg Endosc. doi: 10.1007/s00464-008-9927-7, Apr 24, 2008

17.

Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB et al (2005) Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 19(12):1523–1532CrossRefPubMed

18.

Gallagher AG, Ritter EM, Satava RM (2003) Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17(10):1525–1529CrossRefPubMed

19.

Ayodeji ID, Schijven M, Jakimowicz J, Greve JW (2007) Face validation of the Simbionix LAP Mentor virtual reality training module and its applicability in the surgical curriculum. Surg Endosc 21(9):1641–1649CrossRefPubMed

20.

Hruby GW, Sprenkle PC, Abdelshehid C, Clayman RV, McDougall EM, Landman J (2008) The EZ trainer: validation of a portable and inexpensive simulator for training basic laparoscopic skills. J Urol 179(2):662–666CrossRefPubMed

21.

Koch AD, Buzink SN, Heemskerk J, Botden SM, Veenendaal R, Jakimowicz JJ et al (2008) Expert and construct validity of the Simbionix GI Mentor II endoscopy simulator for colonoscopy. Surg Endosc 22(1):158–162CrossRefPubMed

22.

Schijven M, Jakimowicz J (2002) Face-, expert, and referent validity of the Xitact LS500 laparoscopy simulator. Surg Endosc 16(12):1764–1770CrossRefPubMed

23.

Sweet R, Kowalewski T, Oppenheimer P, Weghorst S, Satava R (2004) Face, content and construct validity of the University of Washington virtual reality transurethral prostate resection trainer. J Urol 172(5 Pt 1):1953–1957CrossRefPubMed

24.

Verdaasdonk EG, Stassen LP, Monteny LJ, Dankelman J (2006) Validation of a new basic virtual reality simulator for training of basic endoscopic skills: the SIMENDO. Surg Endosc 20(3):511–518CrossRefPubMed

25.

Schout BM, Hendrikx AJ, Scherpbier AJ, Bemelmans BL (2008) Update on training models in endourology: a qualitative systematic review of the literature between January 1980 and April 2008. Eur Urol 54(6):1247–1261CrossRefPubMed

26.

Wignall GR, Denstedt JD, Preminger GM, Cadeddu JA, Pearle MS, Sweet RM et al (2008) Surgical simulation: a urological perspective. J Urol 179(5):1690–1699CrossRefPubMed

27.

Schijven MP, Jakimowicz J (2004) The learning curve on the Xitact LS 500 laparoscopy simulator: profiles of performance. Surg Endosc 18(1):121–127CrossRefPubMed

28.

Thijssen AS, Schijven MP (2008) Validation of performance metrics in virtual reality laparoscopy trainers: a systematic review of the literature. In 16th international conference of the EAES, Stockholm, Abstract no. Oo11

29.

Brehmer M, Swartz R (2005) Training on bench models improves dexterity in ureteroscopy. Eur Urol 48(3):458–463PubMed

30.

Knoll T, Trojan L, Haecker A, Alken P, Michel MS (2005) Validation of computer-based training in ureterorenoscopy. BJU Int 95(9):1276–1279CrossRefPubMed

31.

Ogan K, Jacomides L, Shulman MJ, Roehrborn CG, Cadeddu JA, Pearle MS (2004) Virtual ureteroscopy predicts ureteroscopic proficiency of medical students on a cadaver. J Urol 172(2):667–671CrossRefPubMed

32.

Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DAIII et al (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193(6):797–804CrossRefPubMed

33.

Ganai S, Donroe JA, St Louis MR, Lewis GM, Seymour NE (2007) Virtual-reality training improves angled telescope skills in novice laparoscopists. Am J Surg 193(2):260–265CrossRefPubMed

34.

Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91(2):146–150CrossRefPubMed

35.

Naik VN, Matsumoto ED, Houston PL, Hamstra SJ, Yeung RY, Mallon JS et al (2001) Fiberoptic orotracheal intubation on anesthetized patients: do manipulation skills learned on a simple model transfer into the operating room? Anesthesiology 95(2):343–348CrossRefPubMed

36.

Park J, MacRae H, Musselman LJ, Rossos P, Hamstra SJ, Wolman S et al (2007) Randomized controlled trial of virtual reality simulator training: transfer to live patients. Am J Surg 194(2):205–211CrossRefPubMed

37.

Schijven MP, Jakimowicz JJ, Broeders IA, Tseng LN (2005) The Eindhoven laparoscopic cholecystectomy training course—improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc 19(9):1220–1226CrossRefPubMed

38.

Gettman MT, Le CQ, Rangel LJ, Slezak JM, Bergstralh EJ, Krambeck AE (2008) Analysis of a computer based simulator as an educational tool for cystoscopy: subjective and objective results. J Urol 179(1):267–271CrossRefPubMed

39.

Shah J, Darzi A (2002) Virtual reality flexible cystoscopy: a validation study. BJU Int 90(9):828–832CrossRefPubMed

40.

Davis DA, Mazmanian PE, Fordis M, Van HR, Thorpe KE, Perrier L (2006) Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 296(9):1094–1102CrossRefPubMed

41.

Farmer et al (1999) Handbook of simulator-based training. Ashgate, England

42.

Peyton JWR (1998) Teaching and learning in medical practice. Manticore Europe Ltd, UK

43.

Verdaasdonk EG, Stassen LP, Schijven MP, Dankelman J (2007) Construct validity and assessment of the learning curve for the SIMENDO endoscopic simulator. Surg Endosc 21(8):1406–1412CrossRefPubMed

44.

Satava RM (2001) Surgical education and surgical simulation. World J Surg 25(11):1484–1489CrossRefPubMed

45.

Spencer FC (1978) Teaching and measuring surgical techniques—the technical evaluation of competence. Am Coll Surg 63:9–12

46.

Dankelman J, Chmarra MK, Verdaasdonk EG, Stassen LP, Grimbergen CA (2005) Fundamental aspects of learning minimally invasive surgical skills. Minim Invasive Ther Allied Technol 14(4):247–256CrossRefPubMed

47.

Enochsson L, Isaksson B, Tour R, Kjellin A, Hedman L, Wredmark T et al (2004) Visuospatial skills and computer game experience influence the performance of virtual endoscopy. J Gastrointest Surg 8(7):876–882CrossRefPubMed

48.

Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003) Impact of hand dominance, gender, and experience with computer games on performance in virtual reality laparoscopy. Surg Endosc 17(7):1082–1085CrossRefPubMed

49.

Hutchinson L (2003) Educational environment. Br Med J 326(7393):810–812CrossRef

50.

Lacorte MA, Risucci DA (1993) Personality, clinical performance and knowledge in paediatric residents. Med Educ 27(2):165–169CrossRefPubMed

51.

Mitchell M, Srinivasan M, West DC, Franks P, Keenan C, Henderson M et al (2005) Factors affecting resident performance: development of a theoretical model and a focused literature review. Acad Med 80(4):376–389CrossRefPubMed

52.

Reich DL, Uysal S, Bodian CA, Gabriele S, Hibbard M, Gordon W et al (1999) The relationship of cognitive, personality, and academic measures to anesthesiology resident clinical performance. Anesth Analg 88(5):1092–1100CrossRefPubMed

53.

Ringsted C, Skaarup AM, Henriksen AH, Davis D (2006) Person-task-context: a model for designing curriculum and in-training assessment in postgraduate education. Med Teach 28(1):70–76CrossRefPubMed

54.

Schwind CJ, Boehler ML, Rogers DA, Williams RG, Dunnington G, Folse R et al (2004) Variables influencing medical student learning in the operating room. Am J Surg 187(2):198–200CrossRefPubMed

55.

Williams S, Dale J, Glucksman E, Wellesley A (1997) Senior house officers’ work related stressors, psychological distress, and confidence in performing clinical tasks in accident and emergency: a questionnaire study. Br Med J 314(7082):713–718

56.

Kneebone R (2005) Evaluating clinical simulations for learning procedural skills: a theory-based approach. Acad Med 80(6):549–553CrossRefPubMed

57.

Teunissen PW, Stapel DA, Scheele F, Scherpbier AJ, Boor K, van Diemen-Steenvoorde JA et al. (2007) The influence of context on residents’ evaluations: effects of priming on clinical judgment and affect. Adv Health Sci Educ Theory Pract. doi: 10.1007/s10459-007-9082-2, Oct 17, 2007

58.

Vincent C (2003) Understanding and responding to adverse events. N Engl J Med 348(11):1051–1056CrossRefPubMed

59.

Kneebone R, Nestel D, Wetzel C, Black S, Jacklin R, Aggarwal R et al (2006) The human face of simulation: patient-focused simulation training. Acad Med 81(10):919–924CrossRefPubMed

60.

Kneebone R, Kidd J, Nestel D, Asvall S, Paraskeva P, Darzi A (2002) An innovative model for teaching and learning clinical procedures. Med Educ 36(7):628–634CrossRefPubMed

61.

Satava RM, Cuschieri A, Hamdorf J (2003) Metrics for objective assessment. Surg Endosc 17(2):220–226CrossRefPubMed

62.

Brigden D, Dangerfield P (2008) The role of simulation in medical education. Clin Teach 5:167–170CrossRef

63.

Chmarra MK, Grimbergen CA, Dankelman J (2007) Systems for tracking minimally invasive surgical instruments. Minim Invasive Ther Allied Technol 16(6):328–340CrossRefPubMed

64.

Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Andersen DK, Satava RM (2004) Analysis of errors in laparoscopic surgical procedures. Surg Endosc 18(4):592–595CrossRefPubMed

65.

Van Sickle KR, Baghai M, Huang IP, Goldenberg A, Smith CD, Ritter EM (2008) Construct validity of an objective assessment method for laparoscopic intracorporeal suturing and knot tying. Am J Surg 196(1):74–80CrossRefPubMed

66.

Eubanks TR, Clements RH, Pohl D, Williams N, Schaad DC, Horgan S et al (1999) An objective scoring system for laparoscopic cholecystectomy. J Am Coll Surg 189(6):566–574CrossRefPubMed

67.

Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C et al (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84(2):273–278CrossRefPubMed

68.

Miller GE (1990) The assessment of clinical skills/competence/performance. Acad Med 65(9 Suppl):S63–S67PubMedCrossRef

69.

Shah J, Montgomery B, Langley S, Darzi A (2002) Validation of a flexible cystoscopy course. BJU Int 90(9):833–835CrossRefPubMed

70.

Reich O, Noll M, Gratzke C, Bachmann A, Waidelich R, Seitz M et al (2006) High-level virtual reality simulator for endourologic procedures of lower urinary tract. Urology 67(6):1144–1148CrossRefPubMed

71.

Watterson JD, Beiko DT, Kuan JK, Denstedt JD (2002) Randomized prospective blinded study validating acquistion of ureteroscopy skills using computer based virtual reality endourological simulator. J Urol 168(5):1928–1932CrossRefPubMed

72.

Jacomides L, Ogan K, Cadeddu JA, Pearle MS (2004) Use of a virtual reality simulator for ureteroscopy training. J Urol 171(1):320–323CrossRefPubMed

73.

Wilhelm DM, Ogan K, Roehrborn CG, Cadeddu JA, Pearle MS (2002) Assessment of basic endoscopic performance using a virtual reality simulator. J Am Coll Surg 195(5):675–681CrossRefPubMed

74.

Matsumoto ED, Pace KT, D’A Honey RJ (2006) Virtual reality ureteroscopy simulator as a valid tool for assessing endourological skills. Int J Urol 13(7):896–901CrossRefPubMed

75.

Matsumoto ED, Hamstra SJ, Radomski SB, Cusimano MD (2001) A novel approach to endourological training: training at the Surgical Skills Center. J Urol 166(4):1261–1266CrossRefPubMed

76.

Rashid HH, Kowalewski T, Oppenheimer P, Ooms A, Krieger JN, Sweet RM (2007) The virtual reality transurethral prostatic resection trainer: evaluation of discriminate validity. J Urol 177(6):2283–2286CrossRefPubMed

77.

Kallstrom R, Hjertberg H, Kjolhede H, Svanvik J (2005) Use of a virtual reality, real-time, simulation model for the training of urologists in transurethral resection of the prostate. Scand J Urol Nephrol 39(4):313–320CrossRefPubMed

78.

Knudsen BE, Matsumoto ED, Chew BH, Johnson B, Margulis V, Cadeddu JA, Pearle MS, Pautler SE, Denstedt JD (2006) A randomized, controlled, prospective study validating the acquisition of percutaneous renal collecting system access skills using a computer based hybrid virtual reality surgical simulator: phase I. J Urol 176(5):2173–2178CrossRefPubMed

Title: Validation and implementation of surgical simulators: a critical review of present, past, and future
Authors: B. M. A. Schout
A. J. M. Hendrikx
F. Scheele
B. L. H. Bemelmans
A. J. J. A. Scherpbier
Publication date: 01-03-2010
Publisher: Springer-Verlag
Published in: Surgical Endoscopy / Issue 3/2010
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-009-0634-9

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Validation and implementation of surgical simulators: a critical review of present, past, and future

Abstract

Background

Methods

Results

Conclusion

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 3/2010

Hospital volume as a surrogate for laparoscopically assisted colectomy

Is laparoscopic fiberoptic choledochoscopy for common bile duct stones a fine option or a mandatory step?

Percutaneous transhepatic insertion of self-expanding short metal stents for biliary obstruction before resection of pancreatic or duodenal malignancy proves to be safe and effective

News and Notices

Robotic gastrectomy for gastric cancer: surgical techniques and clinical merits

Routine management of stricture after gastric bypass and predictors of subsequent weight loss