Top

Published in:

01-12-2017

Evaluation of crowd-sourced assessment of the critical view of safety in laparoscopic cholecystectomy

Authors: Shanley B. Deal, Dimitrios Stefanidis, Dana Telem, Robert D. Fanelli, Marian McDonald, Michael Ujiki, L. Michael Brunt, Adnan A. Alseidi

Published in: Surgical Endoscopy | Issue 12/2017

Abstract

Background

Faculty experts (FE) and crowd workers (CW) can assess technical skill, but assessment of operative technique has not been explored. We sought to evaluate if CW could be taught to assess completion of the critical view of safety (CVS) in laparoscopic cholecystectomy.

Methods

We prepared 160 blinded, surgical videos of laparoscopic cholecystectomy from public domain websites. Videos were edited to ≤60 s, ending when a structure was cut/clipped. CW analyzed videos using Global Objective Assessment of Laparoscopic Skills (GOALS) and CVS criteria assessment tools after watching an instructional tutorial. Ten videos were randomly selected from each performance quartile based on GOALS. Five FE rated the 40 videos using GOALS and CVS. Linear mixed effects models derived average CW and FE ratings for GOALS and CVS for each video. Spearman correlation coefficients (SCC) were used to assess the degree of correlation between performance measures. Satisfactory completion of the CVS was defined as scoring an average CVS ≥ 5. Videos with an average GOALS ≥ 15 were considered top technical performers.

Results

A high degree of correlation was seen between all performance measures: CVS ratings between CW and FE, SCC 0.89 (p < 0.001); GOALS and CVS ratings SCC 0.77 (p < 0.001) for CW, and SCC 0.71 (p < 0.001) for FE. Sixteen videos were assigned top technical performer ratings by both CW and FE but the average CVS was inadequate (3.8 and 3.6, respectively), and the percentage of satisfactory CVS ≥ 5 was 12.5%.

Conclusions

A high degree of correlation was found between CW and FE in assessment of the CVS. However, in this video analysis, high technical performers did not achieve a complete CVS in most cases. Educating CW to assess operative technique for the identification of low or average performers is feasible and may broaden the application of this assessment and feedback tool.

Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR et al (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369(15):1434–1442CrossRefPubMed

Chan MM, Hamza N, Ammori BJ (2013) Duration of surgery independently influences risk of venous thromboembolism after laparoscopic bariatric surgery. Surg Obes Relat Dis 9(1):88–93CrossRefPubMed

Procter LD, Davenport DL, Bernard AC, Zwischenberger JB (2010) General surgical operative duration is associated with increased risk-adjusted infectious complication rates and length of hospital stay. J Am Coll Surg 210(1):60–65 e1–2

Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A et al (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245(6):992–999CrossRefPubMedPubMedCentral

Downing SM (2003) Validity: on meaningful interpretation of assessment data. Med Educ 37(9):830–837CrossRefPubMed

Chen C, White L, Kowalewski T, Aggarwal R, Lintott C, Comstock B et al (2014) Crowd-sourced assessment of technical skills: a novel method to evaluate surgical performance. J Surg Res 187(1):65–71CrossRefPubMed

Holst D, Kowalewski TM, White LW, Brand TC, Harper JD, Sorensen MD et al (2015) Crowd-sourced assessment of technical skills: differentiating animate surgical skill through the wisdom of crowds. J Endourol 29(10):1183–1188CrossRefPubMed

Holst D, Kowalewski TM, White LW, Brand TC, Harper JD, Sorenson MD et al (2015) Crowd-sourced assessment of technical skills: an adjunct to urology resident surgical simulation training. J Endourol 29(5):604–609CrossRefPubMed

Lendvay TS, White L, Kowalewski T (2015) Crowdsourcing to assess surgical skill. JAMA Surg 150(11):1086–1087CrossRefPubMed

10.

White LW, Kowalewski TM, Dockter RL, Comstock B, Hannaford B, Lendvay TS (2015) Crowd-sourced assessment of technical skill: a valid method for discriminating basic robotic surgery skills. J Endourol 29(11):1295–1301CrossRefPubMed

11.

Deal SB, Lendvay TS, Haque MI, Brand T, Comstock B, Warren J et al (2016) Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills. Am J Surg 211(2):398–404CrossRefPubMed

12.

Buddingh KT, Nieuwenhuijs VB, van Buuren L, Hulscher JB, de Jong JS, van Dam GM (2011) Intraoperative assessment of biliary anatomy for prevention of bile duct injury: a review of current and future patient safety interventions. Surg Endosc 25(8):2449–2461CrossRefPubMedPubMedCentral

13.

Buddingh KT, Weersma RK, Savenije RA, van Dam GM, Nieuwenhuijs VB (2011) Lower rate of major bile duct injury and increased intraoperative management of common bile duct stones after implementation of routine intraoperative cholangiography. J Am Coll Surg 213(2):267–274CrossRefPubMed

14.

Worth PJ, Kaur T, Diggs BS, Sheppard BC, Hunter JG, Dolan JP (2016) Major bile duct injury requiring operative reconstruction after laparoscopic cholecystectomy: a follow-on study. Surg Endosc 30(5):1839–1846CrossRefPubMed

15.

Savader SJ, Lillemoe KD, Prescott CA, Winick AB, Venbrux AC, Lund GB et al (1997) Laparoscopic cholecystectomy-related bile duct injuries: a health and financial disaster. Ann Surg 225(3):268–273CrossRefPubMedPubMedCentral

16.

Nijssen MA, Schreinemakers JM, Meyer Z, van der Schelling GP, Crolla RM, Rijken AM (2015) complications after laparoscopic cholecystectomy: a video evaluation study of whether the critical view of safety was reached. World J Surg 39(7):1798–1803CrossRefPubMed

17.

Buddingh KT, Morks AN, ten Cate Hoedemaker HO, Blaauw CB, van Dam GM, Ploeg RJ et al (2012) Documenting correct assessment of biliary anatomy during laparoscopic cholecystectomy. Surg Endosc 26(1):79–85CrossRefPubMed

18.

Sanford DE, Strasberg SM (2014) A simple effective method for generation of a permanent record of the critical view of safety during laparoscopic cholecystectomy by intraoperative “doublet” photography. J Am Coll Surg 218(2):170–178CrossRefPubMed

19.

Chen CB, Palazzo F, Doane SM, Winter JM, Lavu H, Chojnacki KA et al (2016) Increasing resident utilization and recognition of the critical view of safety during laparoscopic cholecystectomy: a pilot study from an academic medical center. Surg Endosc 31(4):1627–1635CrossRefPubMed

20.

Emous M, Westerterp M, Wind J, Eerenberg JP, van Geloven AA (2010) Registering the critical view of safety: photo or video? Surg Endosc 24(10):2527–2530CrossRefPubMed

21.

Lam T, Usatoff V, Chan ST (2014) Are we getting the critical view? A prospective study of photographic documentation during laparoscopic cholecystectomy. HPB 16(9):859–863CrossRefPubMedPubMedCentral

22.

Strasberg SM, Brunt LM (2017) The critical view of safety: why it is not the only method of ductal identification within the standard of care in laparoscopic cholecystectomy. Ann Surg 265(3):464–465CrossRefPubMed

23.

Strasberg SM, Pucci MJ, Brunt LM, Deziel DJ (2016) Subtotal cholecystectomy-”fenestrating” vs “reconstituting” subtypes and the prevention of bile duct injury: definition of the optimal procedure in difficult operative conditions. J Am Coll Surg 222(1):89–96CrossRefPubMed

24.

C-SATS Inc. (2017). http://www.csats.com/

25.

Deal SS, Brunt SLM, Alseidi A (2017) Development of a multimedia tutorial to educate how to assess the critical view of safety in laparoscopic cholecystectomy using expert review and crowd-sourcing. Am J Surg. doi:10.1016/j.amjsurg.2017.03.023 PubMed

26.

Deal SS, Brunt LMS, Alseidi A (2017) Laparoscopic cholecystectomy educational tutorial: explaining the critical view of safety [Video]. Vimeo. https://vimeo.com/190486987

27.

Kramp KH, van Det MJ, Hoff C, Lamme B, Veeger NJ, Pierie JP (2015) Validity and reliability of global operative assessment of laparoscopic skills (GOALS) in novice trainees performing a laparoscopic cholecystectomy. J Surg Educ 72(2):351–358CrossRefPubMed

28.

Krippendorff K (2013) Content analysis: an introduction to its methodology, 3rd edn. Sage, Thousand Oaks, CA

29.

Cronbach LJ, Shavelson RJ (2004) My current thoughts on coefficient alpha and successor procedures. Educ Psychol Meas 3:391–418CrossRef

30.

Laird NM, James HW (1983) Random effects models for longitudinal studies. Biometrics 38:963–974CrossRef

31.

Cook DA, Beckman TJ (2006) Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 119(2):166 e7–16CrossRefPubMed

Title: Evaluation of crowd-sourced assessment of the critical view of safety in laparoscopic cholecystectomy
Authors: Shanley B. Deal
Dimitrios Stefanidis
Dana Telem
Robert D. Fanelli
Marian McDonald
Michael Ujiki
L. Michael Brunt
Adnan A. Alseidi
Publication date: 01-12-2017
Publisher: Springer US
Published in: Surgical Endoscopy / Issue 12/2017
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-017-5574-1

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Evaluation of crowd-sourced assessment of the critical view of safety in laparoscopic cholecystectomy

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 12/2017

Transumbilical laparoscopic-assisted appendectomy in children: is it worth it?

Remnant gastropathy due to bile reflux after Roux-en-Y gastric bypass: a unique cause of abdominal pain and successful treatment with ursodiol

Diagnostic yield of small histological cores obtained with a new EUS-guided fine needle biopsy system

Laparoscopy may decrease morbidity and length of stay after elective colon cancer resection, especially in frail patients: results from an observational real-life study

Comparison of the multichannel intraluminal impedance pH and conventional pH for measuring esophageal acid exposure: a propensity score-matched analysis

Impaired laparoscopic performance of novice surgeons due to phone call distraction: a single-centre, prospective study