Top

Surgical Endoscopy

Published in:

Open Access 01-03-2022

RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings

Authors: Julia Schreyer, Amelie Koch, Annika Herlemann, Armin Becker, Boris Schlenker, Ken Catchpole, Matthias Weigl

Published in: Surgical Endoscopy | Issue 3/2022

Abstract

Background

Non-technical skills (NTS) are essential for safe surgical practice as they impact workflow and patient outcomes. Observational tools to measure operating room (OR) teams’ NTS have been introduced. However, there are none that account for the specific teamwork challenges introduced by robotic-assisted surgery (RAS). We set out to develop and content-validate a tool to assess multidisciplinary NTS in RAS.

Methodology

Stepwise, multi-method procedure. Observations in different surgical departments and a scoping literature review were first used to compile a set of RAS-specific teamwork behaviours. This list was refined and expert validated using a Delphi consensus approach consisting of qualitative interviews and a quantitative survey. Then, RAS-specific behaviours were merged with a well-established assessment tool on OR teamwork (NOTECHS II). Finally, the new tool—RAS-NOTECHS—was applied in standardized observations of real-world procedures to test its reliability (inter-rater agreement via intra-class correlations).

Results

Our scoping review revealed 5242 articles, of which 21 were included based on pre-established inclusion criteria. We elicited 16 RAS-specific behaviours from the literature base. These were synthesized with further 18 behavioural markers (obtained from 12 OR-observations) into a list of 26 behavioural markers. This list was reviewed by seven RAS experts and condensed to 15 expert-validated RAS-specific behavioural markers which were then merged into NOTECHS II. For five observations of urologic RAS procedures (duration: 13 h and 41 min), inter-rater agreement for identification of behavioural markers was strong. Agreement of RAS-NOTECHS scores indicated moderate to strong agreement.

Conclusions

RAS-NOTECHS is the first observational tool for multidisciplinary NTS in RAS. In preliminary application, it has been shown to be reliable. Since RAS is rapidly increasing and challenges for effective and safe teamwork remain at the forefront of quality and safety of surgical care, RAS-NOTECHS may contribute to training and improvement efforts in technology-facilitated surgeries.

Available only for authorised users

Pfandler M, Stefan P, Mehren C, Lazarovici M, Weigl M (2019) Technical and nontechnical skills in surgery: a simulated operating room environment study. Spine (Phila Pa 1976) 44(23):E1396–E400. https://doi.org/10.1097/brs.0000000000003154CrossRef

Hull L, Arora S, Aggarwal R, Darzi A, Vincent C, Sevdalis N (2012) The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 214(2):214–230. https://doi.org/10.1016/j.jamcollsurg.2011.10.016CrossRefPubMed

Schmutz JB, Meier LL, Manser T (2019) How effective is teamwork really? The relationship between teamwork and performance in healthcare teams: a systematic review and meta-analysis. BMJ Open 9(9):e028280. https://doi.org/10.1136/bmjopen-2018-028280CrossRefPubMedPubMedCentral

Doumouras AG, Hamidi M, Lung K, Tarola CL, Tsao MW, Scott JW et al (2017) Non-technical skills of surgeons and anaesthetists in simulated operating theatre crises. Br J Surg 104(8):1028–1036. https://doi.org/10.1002/bjs.10526CrossRefPubMed

Fecso AB, Kuzulugil SS, Babaoglu C, Bener AB, Grantcharov TP (2018) Relationship between intraoperative non-technical performance and technical events in bariatric surgery. Br J Surg 105(8):1044–1050. https://doi.org/10.1002/bjs.10811CrossRefPubMed

Gjeraa K, Spanager L, Konge L, Petersen RH, Østergaard D (2016) Non-technical skills in minimally invasive surgery teams: a systematic review. Surg Endosc 30(12):5185–5199. https://doi.org/10.1007/s00464-016-4890-1CrossRefPubMed

Flin R, O’Connor P, Crichton M (2008) Safety at the sharp end: a guide to non-technical Skills. Ashgate, Aldershot

Gillespie BM, Harbeck E, Kang E, Steel C, Fairweather N, Chaboyer W (2017) Correlates of non-technical skills in surgery: a prospective study. BMJ Open 7(1):e014480. https://doi.org/10.1136/bmjopen-2016-014480CrossRefPubMedPubMedCentral

Al-Hakim L (2011) The impact of preventable disruption on the operative time for minimally invasive surgery. Surg Endosc 25(10):3385–3392. https://doi.org/10.1007/s00464-011-1735-9CrossRefPubMed

10.

Mushtaq F, O’Driscoll C, Smith F, Wilkins D, Kapur N, Lawton R (2018) Contributory factors in surgical incidents as delineated by a confidential reporting system. Ann R Coll Surg Engl 100(5):401–405. https://doi.org/10.1308/rcsann.2018.0025CrossRefPubMedPubMedCentral

11.

Siu J, Maran N, Paterson-Brown S (2016) Observation of behavioural markers of non-technical skills in the operating room and their relationship to intra-operative incidents. Surgeon 14(3):119–128. https://doi.org/10.1016/j.surge.2014.06.005CrossRefPubMed

12.

Li N, Marshall D, Sykes M, McCulloch P, Shalhoub J, Maruthappu M (2018) Systematic review of methods for quantifying teamwork in the operating theatre. BJS Open 2(2):42–51. https://doi.org/10.1002/bjs5.40CrossRefPubMedPubMedCentral

13.

McMullan RD, Urwin R, Sunderland N, Westbrook J (2020) Observational tools that quantify nontechnical skills in the operating room: a systematic review. J Surg Res 247:306–322. https://doi.org/10.1016/j.jss.2019.10.012CrossRefPubMed

14.

Mishra A, Catchpole K, McCulloch P (2009) The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care 18(2):104–108. https://doi.org/10.1136/qshc.2007.024760CrossRefPubMed

15.

Catchpole K, Bisantz A, Hallbeck MS, Weigl M, Randell R, Kossack M et al (2019) Human factors in robotic assisted surgery: lessons from studies “in the Wild.” Appl Ergon 78:270–276. https://doi.org/10.1016/j.apergo.2018.02.011CrossRefPubMed

16.

Gillespie BM, Gillespie J, Boorman RJ, Granqvist K, Stranne J, Erichsen-Andersson A (2020) The impact of robotic-assisted surgery on team performance: a systematic mixed studies review. Hum Factors. https://doi.org/10.1177/0018720820928624CrossRefPubMed

17.

Tiferes J, Hussein AA, Bisantz A, Kozlowski JD, Sharif MA, Winder NM et al (2016) the loud surgeon behind the console: understanding team activities during robot-assisted surgery. J Surg Educ 73(3):504–512. https://doi.org/10.1016/j.jsurg.2015.12.009CrossRefPubMed

18.

Raison N, Wood T, Brunckhorst O, Abe T, Ross T, Challacombe B et al (2017) Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system. Surg Endosc 31(12):5403–5410. https://doi.org/10.1007/s00464-017-5622-xCrossRefPubMed

19.

Robertson ER, Hadi M, Morgan LJ, Pickering SP, Collins G, New S et al (2014) Oxford NOTECHS II: a modified theatre team non-technical skills scoring system. PLoS ONE 9(3):e90320. https://doi.org/10.1371/journal.pone.0090320CrossRefPubMedPubMedCentral

20.

Peters MDJ GC, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping reviews (2020 version). In: Aromataris E, Munn Z (eds) JBI manual for evidence synthesis. JBI. 2020. http://doi.org/https://doi.org/10.46658/JBIMES-20-12. https://reviewersmanual.joannabriggs.org/

21.

Flin R, Martin L, Goeters KM, Hoermann H, Amalberti R, Valot C et al (2003) Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Hum Factors Aerosp Saf 3:97–119

22.

Myklebust MV, Storheim H, Hartvik M, Dysvik E (2020) Anesthesia professionals’ perspectives of teamwork during robotic-assisted surgery. Aorn J 111(1):87–96. https://doi.org/10.1002/aorn.12897CrossRefPubMed

23.

Iqbal S, Pipon-Young L (2009) The Delphi method. A step-by-step guide. Psychologist 22:598–601

24.

Hull L, Arora S, Kassab E, Kneebone R, Sevdalis N (2011) Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg 212(2):234-43.e1–5. https://doi.org/10.1016/j.jamcollsurg.2010.11.001CrossRefPubMed

25.

Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163. https://doi.org/10.1016/j.jcm.2016.02.012CrossRefPubMedPubMedCentral

26.

Gwet K (2002) Kappa statistic is not satisfactory for assessing the extent of agreement between raters. Stat Methods Inter-rater Reliab Assess 1:1–6

27.

Pelikan HR (2018) “What’s going on there?” Negotiating common ground in robotic vs. open surgery: a comparison of surgeon-initiated requests for action in open and robotic surgery. University of Twente, Enschede

28.

Hsu RL, Kaye AD, Urman RD (2013) Anesthetic challenges in robotic-assisted urologic surgery. Rev Urol 15(4):178–184. https://doi.org/10.3909/riu0589CrossRefPubMedPubMedCentral

29.

Cao CG, Taylor H (2004) Effects of new technology on the operating room team. Department of Mechanical Engineering, Tufts University, Medford, pp 309–12

30.

Higuchi T, Gettman M (2011) Robotic instrumentation, personnel and operating room setup. In: Su LM (ed) Atlas of robotic urologic surgery. Humana Press, New York, pp 15–30. https://doi.org/10.1007/978-1-60761-026-7_2CrossRef

31.

Lai F, Entin E (2005) Robotic surgery and the operating room team. Proc Hum Factors Ergon Soc Annu Meet 49(11):1070–1073. https://doi.org/10.1177/154193120504901115CrossRef

32.

Wastler KE (2015) Robotic surgical and anesthesia communication tool. J Robot Surg 9(1):97–98. https://doi.org/10.1007/s11701-014-0494-8CrossRefPubMed

33.

Payne TN, Pitter MC (2011) Robotic-assisted surgery for the community gynecologist: can it be adopted? Clin Obstet Gynecol 54(3):391–411. https://doi.org/10.1097/GRF.0b013e31822b4998CrossRefPubMed

34.

Satchidanand A, Higginbotham J, Bisantz A, Aldhaam NA, Elsayed AS, Kannappan R et al (2019) Vague swirling ends in no communication: referencing strategies during robot assisted surgery. J Urol 201(4):E288–E289

35.

Raheem S, Ahmed YE, Hussein AA, Johnson A, Cavuoto L, May P et al (2018) Variability and interpretation of communication taxonomy during robot-assisted surgery: do we all speak the same language? BJU Int 122(1):99–105. https://doi.org/10.1111/bju.14150CrossRefPubMed

36.

Randell R, Greenhalgh J, Hindmarsh J, Honey S, Pearman A, Alvarado N et al (2019) How do team experience and relationships shape new divisions of labour in robot-assisted surgery? A realist investigation. Health. https://doi.org/10.1177/1363459319874115CrossRefPubMed

37.

Randell R, Honey S, Hindmarsh J, Alvarado N, Greenhalgh J, Pearman A et al (2017) A realist process evaluation of robot-assisted surgery: integration into routine practice and impacts on communication, collaboration and decision-making. Health Serv Deliv Res. https://doi.org/10.3310/hsdr05200CrossRef

38.

Alvarado N, Honey S, Greenhalgh J, Pearman A, Dowding D, Cope A et al (2017) Eliciting context-mechanism-outcome configurations: experiences from a realist evaluation investigating the impact of robotic surgery on teamwork in the operating theatre. Evaluation 23(4):444–462. https://doi.org/10.1177/1356389017732172CrossRef

39.

Gill A, Randell R (2016) Robotic surgery and its impact on teamwork in the operating theatre. J Perioper Pract 26(3):42–45. https://doi.org/10.1177/175045891602600303CrossRefPubMed

40.

El-Hamamsy D, Walton TJ, Griffiths TRL, Anderson ES, Tincello DG (2020) Surgeon-team separation in robotic theaters: a qualitative observational and interview study. Female Pelvic Med Reconstr Surg 26(2):86–91. https://doi.org/10.1097/spv.0000000000000829CrossRefPubMed

41.

Nyssen A-S, Blavier A (2009) Verbal communication as a sign of adaptation in socio-technical systems: the case of robotic surgery. In: 9th Bi-annual international conference on naturalistic decision making (NDM9), vol 9. pp 267–72. http://doi.org/https://doi.org/10.14236/ewic/NDM2009.39

42.

Almeras C (2019) Operating room communication in robotic surgery: place, modalities and evolution of a safe system of interaction. J Visc Surg 156(5):397–403. https://doi.org/10.1016/j.jviscsurg.2019.02.004CrossRefPubMed

43.

Sergeeva A, Huysman M, Faraj S (2015) Transforming work practices of operating room teams: the case of the Da Vinci robot. In: Transforming surgery practice with the Da Vinci robot thirty sixth international conference on information systems. Fort Worth.

44.

Viswanathan M, Berkman N (2011) Development of the RTI Item Bank on risk of bias and precision of observational studies [Internet] Appendix A, AC1 Statistic. Agency for Healthcare Research and Quality (US), Rockville. https://www.ncbi.nlm.nih.gov/books/NBK82266/.

45.

Fleiss J (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76:378–382. https://doi.org/10.1037/h0031619CrossRef

46.

Peters BS, Armijo PR, Krause C, Choudhury SA, Oleynikov D (2018) Review of emerging surgical robotic technology. Surg Endosc 32(4):1636–1655. https://doi.org/10.1007/s00464-018-6079-2CrossRefPubMed

47.

Perez RE, Schwaitzberg SD (2019) Robotic surgery: finding value in 2019 and beyond. Ann Laparosc Endosc Surg. https://doi.org/10.21037/ales.2019.05.02CrossRef

48.

Sheetz KH, Claflin J, Dimick JB (2020) Trends in the adoption of robotic surgery for common surgical procedures. JAMA Netw Open 3(1):e1918911. https://doi.org/10.1001/jamanetworkopen.2019.18911CrossRefPubMedPubMedCentral

49.

Undre S, Sevdalis N, Healey AN, Darzi A, Vincent CA (2007) Observational teamwork assessment for surgery (OTAS): refinement and application in urological surgery. World J Surg 31(7):1373–1381. https://doi.org/10.1007/s00268-007-9053-zCrossRefPubMed

50.

Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S (2008) Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 32(4):548–556. https://doi.org/10.1007/s00268-007-9320-zCrossRefPubMed

51.

Robertson JM, Dias RD, Yule S, Smink DS (2017) Operating room team training with simulation: a systematic review. J Laparoendosc Adv Surg Tech A 27(5):475–480. https://doi.org/10.1089/lap.2017.0043CrossRefPubMed

52.

Yule S, Parker SH, Wilkinson J, McKinley A, MacDonald J, Neill A et al (2015) Coaching non-technical skills improves surgical residents’ performance in a simulated operating room. J Surg Educ 72(6):1124–1130. https://doi.org/10.1016/j.jsurg.2015.06.012CrossRefPubMed

53.

Ali MR, Loggins JP, Fuller WD, Miller BE, Hasser CJ, Yellowlees P et al (2008) 3-D telestration: a teaching tool for robotic surgery. J Laparoendosc Adv Surg Tech A 18(1):107–112. https://doi.org/10.1089/lap.2007.0051CrossRefPubMed

54.

Russ S, Hull L, Rout S, Vincent C, Darzi A, Sevdalis N (2012) Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training. Ann Surg 255(4):804–809. https://doi.org/10.1097/SLA.0b013e31824a9a02CrossRefPubMed

55.

Manuguerra A, Mazeaud C, Hubert N, Eschwège P, Roumiguié M, Salleron J et al (2020) Non-technical skills in robotic surgery and impact on near-miss events: a multi-center study. Surg Endosc. https://doi.org/10.1007/s00464-020-07988-5CrossRefPubMed

56.

Manser T (2009) Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand 53(2):143–151. https://doi.org/10.1111/j.1399-6576.2008.01717.xCrossRefPubMed

57.

Leonard M, Graham S, Bonacum D (2004) The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care 13(Suppl 1):i85-90. https://doi.org/10.1136/qhc.13.suppl_1.i85CrossRefPubMedPubMedCentral

58.

Rao PP (2018) Robotic surgery: new robots and finally some real competition! World J Urol 36(4):537–41. https://doi.org/10.1007/s00345-018-2213-yCrossRefPubMed

59.

Brodie A, Vasdev N (2018) The future of robotic surgery. Ann R Coll Surg Engl 100(Suppl 7):4–13. https://doi.org/10.1308/rcsann.supp2.4CrossRefPubMedPubMedCentral

60.

Paradis E, Sutkin G (2017) Beyond a good story: from Hawthorne effect to reactivity in health professions education research. Med Educ 51(1):31–39. https://doi.org/10.1111/medu.13122CrossRefPubMed

61.

Lynch A (2020) Simulation-based acquisition of non-technical skills to improve patient safety. Semin Pediatr Surg 29(2):150906. https://doi.org/10.1016/j.sempedsurg.2020.150906CrossRefPubMed

62.

Nicksa GA, Anderson C, Fidler R, Stewart L (2015) Innovative approach using interprofessional simulation to educate surgical residents in technical and nontechnical skills in high-risk clinical scenarios. JAMA Surg 150(3):201–207. https://doi.org/10.1001/jamasurg.2014.2235CrossRefPubMed

63.

Weaver S, Dy S, Rosen M (2014) Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf 23(5):359–372. https://doi.org/10.1136/bmjqs-2013-001848CrossRefPubMedPubMedCentral

64.

Koch A, Burns J, Catchpole K, Weigl M (2020) Associations of workflow disruptions in the operating room with surgical outcomes: a systematic review and narrative synthesis. BMJ Qual Saf. https://doi.org/10.1136/bmjqs-2019-010639CrossRefPubMed

65.

Catchpole KR, Dale TJ, Hirst DG, Smith JP, Giddings TA (2010) A multicenter trial of aviation-style training for surgical teams. J Patient Saf 6(3):180–186. https://doi.org/10.1097/PTS.0b013e3181f100eaCrossRefPubMed

66.

Stewart-Parker E, Galloway R, Vig S (2017) S-TEAMS: a truly multiprofessional course focusing on nontechnical skills to improve patient safety in the operating theater. J Surg Educ 74(1):137–144. https://doi.org/10.1016/j.jsurg.2016.06.020CrossRefPubMed

Title: RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings
Authors: Julia Schreyer
Amelie Koch
Annika Herlemann
Armin Becker
Boris Schlenker
Ken Catchpole
Matthias Weigl
Publication date: 01-03-2022
Publisher: Springer US
Published in: Surgical Endoscopy / Issue 3/2022
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI: https://doi.org/10.1007/s00464-021-08474-2

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings

Abstract

Background

Methodology

Results

Conclusions

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methodology

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 3/2022

Short-term outcome in robotic vs laparoscopic and open rectal tumor surgery within an ERAS protocol: a retrospective cohort study from the Swedish ERAS database

Safety comparison of minimally invasive abdomen-only esophagectomy versus minimally invasive Ivor Lewis esophagectomy: a retrospective cohort study

Assessment of indocyanine green fluorescence lymphography on lymphadenectomy during minimally invasive gastric cancer surgery: a systematic review and meta-analysis

Correction to: Improved submucosal tunneling endoscopic resection with slant tunnel for submucosal tumors in proximal esophagus

Laparoscopic total pelvic peritonectomy for colorectal cancer pelvic carcinomatosis: a retrospective case series and photographic/videographic step-by-step guide

Correction to: Trends and risk factors for 30‑day readmissions in patients with acute cholangitis: analysis from the national readmission database