Top

Surgical Endoscopy

Published in:

01-03-2012

Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level

Authors: Danilo Miskovic, Melody Ni, Susannah M. Wyles, Amjad Parvaiz, George B. Hanna

Published in: Surgical Endoscopy | Issue 3/2012

Abstract

Background

There are no valid and reliable tools to assess competency in advanced laparoscopic surgery at a specialist level. The observational clinical human reliability analysis (OCHRA) may have the required characteristics of such a tool. The aim of this study was to evaluate construct and concurrent validity of OCHRA for competency assessment at a specialist level.

Methods

Thirty-two video-recorded laparoscopic colorectal resections, performed by experts and delegates of the National Training Program in England, were evaluated. Each video was analysed using OCHRA by identifying errors enacted during surgery. The number of tissue-handling, instrument-misuse, and consequential errors was recorded using video-rating software. Times spent on dissecting (D) and on exposing (E) tissues were also measured (D/E ratio). In addition, two independent expert surgeons globally assessed each video regarding competency (pass vs. fail). Logistic regression was used to predict outcomes.

Results

A total of 399 errors were identified. There was a significant difference when comparing the expert, pass, and fail groups for total errors (median counts for experts = 4, pass = 10, fail = 17; P < 0.001). When comparing the pass and fail groups excluding experts, differences could be found for tissue-handling errors (7 vs. 12; P = 0.005), but not for consequential errors (4 vs. 7; P = 0.059) and instrument-handling errors (4 vs. 5; P = 0.320). The D/E ratio was significantly lower for delegates than for experts (0.6 vs. 1.0; P = 0.001). When all four independent variables were used to predict delegates who passed or failed, the area under the receiver operating characteristic curve was 0.867, sensitivity was 71.4%, and specificity was 90.9%.

Conclusion

OCHRA is a valid tool for assessing competency at a specialist level in advanced laparoscopic surgery. It has the potential to be used for recertification and revalidation of specialists.

Available only for authorised users

Tekkis PP, Senagore AJ, Delaney CP, Fazio VW (2005) Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right-sided and left-sided resections. Ann Surg 242:83–91PubMedCrossRef

Dincler S, Koller MT, Steurer J, Bachmann LM, Christen D, Buchmann P (2003) Multidimensional analysis of learning curves in laparoscopic sigmoid resection: eight-year results. Dis Colon Rectum 46:1371–1378 discussion 1378–1379PubMedCrossRef

Miskovic D, Wyles SM, Ni M, Darzi AW, Hanna GB (2010) Systematic review on mentoring and simulation in laparoscopic colorectal surgery. Ann Surg 252:943–951PubMedCrossRef

Maeda T, Tan KY, Konishi F, Tsujinaka S, Mizokami K, Sasaki J, Kawamura YJ (2010) Accelerated learning curve for colorectal resection, open versus laparoscopic approach, can be attained with expert supervision. Surg Endosc 24:2850–2854PubMedCrossRef

Coleman MG, Hanna GB, Kennedy R (2011) National Training Programme Lapco. The National Training Programme for Laparoscopic Colorectal Surgery in England: a new training paradigm. Colorectal Dis 13(6):614–616PubMedCrossRef

Miskovic D, Wyles SM, Carter F, Coleman MG, Hanna GB (2011) Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program. Surg Endosc 25:1136–1142PubMedCrossRef

Okrainec A, Soper NJ, Swanstrom LL, Fried GM (2011) Trends and results of the first 5 years of Fundamentals of Laparoscopic Surgery (FLS) certification testing. Surg Endosc 25:1192–1198PubMedCrossRef

Moorthy K, Munz Y, Dosis A, Bello F, Darzi A (2003) Motion analysis in the training and assessment of minimally invasive surgery. Minim Invasive Ther Allied Technol 12:137–142PubMed

Hanna GB, Drew T, Clinch P, Hunter B, Cuschieri A (1998) Computer-controlled endoscopic performance assessment system. Surg Endosc 12:997–1000PubMedCrossRef

10.

Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278PubMedCrossRef

11.

Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113PubMedCrossRef

12.

Aggarwal R, Grantcharov T, Moorthy K, Milland T, Darzi A (2008) Toward feasible, valid, and reliable video-based assessments of technical surgical skills in the operating room. Ann Surg 247:372–379PubMedCrossRef

13.

Datta V, Bann S, Aggarwal R, Mandalia M, Hance J, Darzi A (2006) Technical skills examination for general surgical trainees. Br J Surg 93:1139–1146PubMedCrossRef

14.

Francis NK, Hanna GB, Cuschieri A (2002) The performance of master surgeons on the Advanced Dundee Endoscopic Psychomotor Tester: contrast validity study. Arch Surg 137:841–844PubMedCrossRef

15.

Munz Y, Moorthy K, Bann S, Shah J, Ivanova S, Darzi SA (2004) Ceiling effect in technical skills of surgical residents. Am J Surg 188:294–300PubMedCrossRef

16.

Joice P, Hanna GB, Cuschieri A (1998) Errors enacted during endoscopic surgery—a human reliability analysis. Appl Ergon 29:409–414PubMedCrossRef

17.

Tang B, Hanna GB, Joice P, Cuschieri A (2004) Identification and categorization of technical errors by Observational Clinical Human Reliability Assessment (OCHRA) during laparoscopic cholecystectomy. Arch Surg 139:1215–1220PubMedCrossRef

18.

Tang B, Hanna GB, Cuschieri A (2005) Analysis of errors enacted by surgical trainees during skills training courses. Surgery 138:14–20PubMedCrossRef

19.

van der Vleuten CP, Schuwirth LW (2005) Assessing professional competence: from methods to programmes. Med Educ 39:309–317PubMedCrossRef

20.

Kirwan B (1992) Human error identification in human reliability assessment. Part 1: overview of approaches. Appl Ergon 23:299–318PubMedCrossRef

21.

Miskovic D, Wyles SM, Francis N, Rockall T, Kennedy R, Hanna GB (2010) Laparoscopic Colorectal Competency Assessment Tool (LCAT) for the National Training Program. Annual Meeting of the American Society of Colon and Rectal Surgeons, Minneapolis, MN, May 15–19, 2010

22.

Peduzzi P, Concato J, Feinstein AR, Holford TR (1995) Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 48:1503–1510PubMedCrossRef

23.

Scheer A, Martel G, Moloo H, Sabri E, Poulin EC, Mamazza J, Boushey RP (2009) Laparoscopic colon surgery: does operative time matter? Dis Colon Rectum 52:1746–1752PubMedCrossRef

24.

Tang B, Hanna GB, Bax NM, Cuschieri A (2004) Analysis of technical surgical errors during initial experience of laparoscopic pyloromyotomy by a group of Dutch pediatric surgeons. Surg Endosc 18:1716–1720PubMedCrossRef

25.

van Hove PD, Tuijthof GJ, Verdaasdonk EG, Stassen LP, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987PubMedCrossRef

26.

Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB (2011) Observational assessment of procedural skills—a systematic review. Am J Surg 202(4):469–480.e6PubMedCrossRef

27.

Fraser SA, Klassen DR, Feldman LS, Ghitulescu GA, Stanbridge D, Fried GM (2003) Evaluating laparoscopic skills: setting the pass/fail score for the MISTELS system. Surg Endosc 17:964–967PubMedCrossRef

28.

Vassiliou MC, Ghitulescu GA, Feldman LS, Stanbridge D, Leffondre K, Sigman HH, Fried GM (2006) The MISTELS program to measure technical skill in laparoscopic surgery: evidence for reliability. Surg Endosc 20:744–747PubMedCrossRef

29.

Aggarwal R, Grantcharov T, Moorthy K, Hance J, Darzi A (2006) A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg 191:128–133PubMedCrossRef

30.

Gallagher AG, Lederman AB, McGlade K, Satava RM, Smith CD (2004) Discriminative validity of the Minimally Invasive Surgical Trainer in Virtual Reality (MIST-VR) using criteria levels based on expert performance. Surg Endosc 18:660–665PubMedCrossRef

31.

Ahmed AR, Miskovic D, Vijayaseelan T, O’Malley W, Hanna GB (2011) Root cause analysis of internal hernia and Roux limb compression following laparoscopic Roux-Y Gastric Bypass using Observational Clinical Human Reliability Analysis (OCHRA). Surg Obes Relat Dis. doi:10.1016/j.soard.2010.12.009

Title: Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level
Authors: Danilo Miskovic
Melody Ni
Susannah M. Wyles
Amjad Parvaiz
George B. Hanna
Publication date: 01-03-2012
Publisher: Springer-Verlag
Published in: Surgical Endoscopy / Issue 3/2012
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-011-1955-z

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level

Abstract

Background

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 3/2012

Mentorship for participants in a laparoscopic colectomy course

Does a preoperative medically supervised weight loss program improve bariatric surgery outcomes? A pilot randomized study

Transgastric versus laparoendoscopic single-site peritoneoscopy in a rat model: effects on motility, inflammation, and nociception

Long-term outcomes after transoral incisionless fundoplication in patients with GERD and LPR symptoms

Short esophagus: selection of patients for surgery and long-term results

Miniature surgical robot for laparoendoscopic single-incision colectomy