Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

05-10-2022 | Expert Opinion | Original Article

“Stand-up straight!”: human pose estimation to evaluate postural skills during orthopedic surgery simulations

Authors: Tiphaine Casy, Alexandre Tronchot, Hervé Thomazeau, Xavier Morandi, Pierre Jannin, Arnaud Huaulmé

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 2/2023

Abstract

Purpose

Surgery simulators can be used to learn technical and non-technical skills and, to analyse posture. Ergonomic skill can be automatically detected with a Human Pose Estimation algorithm to help improve the surgeon’s work quality. The objective of this study was to analyse the postural behaviour of surgeons and identify expertise-dependent movements. Our hypothesis was that hesitation and the occurrence of surgical instruments interfering with movement (defined as interfering movements) decrease with expertise.

Material and methods

Sixty surgeons with three expertise levels (novice, intermediate, and expert) were recruited. During a training session using an arthroscopic simulator, each participant’s movements were video-recorded with an RGB camera. A modified OpenPose algorithm was used to detect the surgeon's joints. The detection frequency of each joint in a specific area was visualized with a heatmap-like approach and used to calculate a mobility score.

Results

This analysis allowed quantifying surgical movements. Overall, the mean mobility score was 0.823, 0.816, and 0.820 for novice, intermediate and expert surgeons, respectively. The mobility score alone was not enough to identify postural behaviour differences. A visual analysis of each participants’ movements highlighted expertise-dependent interfering movements.

Conclusion

Video-recording and analysis of surgeon’s movements are a non-invasive approach to obtain quantitative and qualitative ergonomic information in order to provide feedback during training. Our findings suggest that the interfering movements do not decrease with expertise but differ in function of the surgeon’s level.

Available only for authorised users

https://www.github.com/quanhua92/human-pose-estimation-opencv.

Satava RM (2008) Historical review of surgical simulation—A personal perspective. World J Surg 32:141–148. https://doi.org/10.1007/s00268-007-9374-yCrossRef

Badash I, Burtt K, Solorzano CA, Carey JN (2016) Innovations in surgery simulation: a review of past, current and future techniques. Ann Transl Med. https://doi.org/10.21037/atm.2016.12.24CrossRef

Sutherland LM, Middleton PF, Anthony A et al (2006) Surgical simulation. Ann Surg 243:291–300. https://doi.org/10.1097/01.sla.0000200839.93965.26CrossRef

Satava RM (2001) Surgical education and surgical simulation. World J Surg 25:1484–1489. https://doi.org/10.1007/s00268-001-0134-0CrossRef

Brewin J, Tang J, Dasgupta P et al (2015) Full immersion simulation: validation of a distributed simulation environment for technical and non-technical skills training in Urology. BJU Int 116:156–162. https://doi.org/10.1111/bju.12875CrossRef

Narazaki K, Oleynikov D, Stergiou N (2006) Robotic surgery training and performance. Surg Endosc Interv Tech 20:96–103. https://doi.org/10.1007/s00464-005-3011-3CrossRef

Smith CD, Farrell TM, McNatt SS, Metreveli RE (2001) Assessing laparoscopic manipulative skills. Am J Surg 181:547–550. https://doi.org/10.1016/s0002-9610(01)00639-0CrossRef

Agha RA, Fowler AJ, Sevdalis N (2015) The role of non-technical skills in surgery. Ann Med Surg 4:422–427. https://doi.org/10.1016/j.amsu.2015.10.006CrossRef

Sharma B, Mishra A, Aggarwal R, Grantcharov TP (2011) Non-technical skills assessment in surgery. Surg Oncol 20:169–177. https://doi.org/10.1016/j.suronc.2010.10.001CrossRef

10.

Fecso AB, Kuzulugil SS, Babaoglu C et al (2018) Relationship between intraoperative non-technical performance and technical events in bariatric surgery. Br J Surg 105:1044–1050. https://doi.org/10.1002/bjs.10811CrossRef

11.

Azari DP, Miller BL, Le BV et al (2020) Quantifying surgeon maneuevers across experience levels through marker-less hand motion kinematics of simulated surgical tasks. Appl Ergon 87:103136. https://doi.org/10.1016/j.apergo.2020.103136CrossRef

12.

Meltzer AJ, Hallbeck MS, Morrow MM et al (2020) Measuring ergonomic risk in operating surgeons by using wearable technology. JAMA Surg 155:444–446. https://doi.org/10.1001/jamasurg.2019.6384CrossRef

13.

Farella E, Pieracci A, Benini L, Acquaviva A (2006) A wireless body area sensor network for posture detection. In: 11th IEEE symposium on computers and communications (ISCC’06), pp 454–459

14.

Xu Y, Chen J, Yang Q, Guo (2019) Human posture recognition and fall detection using kinect V2 camera. In: IEEE conference publication, IEEE Xplore

15.

Nisky I, Hsieh MH, Okamura AM (2013) A framework for analysis of surgeon arm posture variability in robot-assisted surgery. In: IEEE conference publication, IEEE Xplore

16.

Dalager T, Jensen PT, Eriksen JR et al (2020) Surgeons’ posture and muscle strain during laparoscopic and robotic surgery. Br J Surg 107:756–766. https://doi.org/10.1002/bjs.11394CrossRef

17.

Athanasiadis DI, Monfared S, Asadi H et al (2021) An analysis of the ergonomic risk of surgical trainees and experienced surgeons during laparoscopic procedures. Surgery 169:496–501. https://doi.org/10.1016/j.surg.2020.10.027CrossRef

18.

Lobo D, Anuarbe P, López-Higuera JM et al (2019) Estimation of surgeons’ ergonomic dynamics with a structured light system during endoscopic surgery. Int Forum Allergy Rhinol 9:857–864. https://doi.org/10.1002/alr.22353CrossRef

19.

Berguer R, Rab GT, Abu-Ghaida H et al (1997) A comparison of surgeons’ posture during laparoscopic and open surgical procedures. Surg Endosc 11:139–142. https://doi.org/10.1007/s004649900316CrossRef

20.

Sarafianos N, Boteanu B, Ionescu B, Kakadiaris IA (2016) 3D Human pose estimation: a review of the literature and analysis of covariates. Comput Vis Image Underst 152:1–20. https://doi.org/10.1016/j.cviu.2016.09.002CrossRef

21.

Aggarwal JK, Cai Q (1997) Human motion analysis: a review. In: Proceedings IEEE nonrigid and articulated motion workshop, pp 90–102

22.

Gavrila DM (1999) The visual analysis of human movement: a survey. Comput Vis Image Underst 73:82–98. https://doi.org/10.1006/cviu.1998.0716CrossRef

23.

Fischler MA, Elschlager RA (1973) The representation and matching of pictorial structures. IEEE Trans Comput C 22:67–92. https://doi.org/10.1109/T-C.1973.223602CrossRef

24.

Khan NU, Wan W (2018) A review of human pose estimation from single image. In: 2018 international conference on audio, language and image processing (ICALIP), pp 230–236

25.

Liu Y, Xu Y, Li S (2018) 2-D human pose estimation from images based on deep learning: a review. In: 2018 2nd IEEE advanced information management, communicates, electronic and automation control conference (IMCEC), pp 462–465

26.

Toshev A, Szegedy C (2014) DeepPose: human pose estimation via deep neural networks. In: 2014 IEEE conference on computer vision and pattern recognition. IEEE, Columbus, OH, USA, pp 1653–1660

27.

Cao Z, Simon T, Wei S-E, Sheikh Y (2017) Realtime multi-person 2D pose estimation using part affinity fields. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Honolulu, HI, pp 1302–1310

28.

Cao Z, Hidalgo G, Simon T et al (2021) OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans Pattern Anal Mach Intell 43:172–186. https://doi.org/10.1109/TPAMI.2019.2929257CrossRef

29.

Chen Y, Tian Y, He M (2020) Monocular human pose estimation: a survey of deep learning-based methods. Comput Vis Image Underst 192:102897. https://doi.org/10.1016/j.cviu.2019.102897CrossRef

30.

Dang Q, Yin J, Wang B, Zheng W (2019) Deep learning based 2D human pose estimation: a survey. Tsinghua Sci Technol 24:663–676. https://doi.org/10.26599/TST.2018.9010100CrossRef

31.

Tanugraha P (2019) Understanding OpenPose (with code reference)—Part 1. In: Anal. Vidhya. https://medium.com/analytics-vidhya/understanding-openpose-with-code-reference-part-1-b515ba0bbc73. Accessed 8 Oct 2021

32.

Lichtman A (2020) An overview of human pose estimation with deep learning. In: BeyondMinds. https://beyondminds.ai/blog/an-overview-of-human-pose-estimation-with-deep-learning/. Accessed 8 Oct 2021

33.

Viswakumar A, Rajagopalan V, Ray T, Parimi C (2019) Human gait analysis using OpenPose. In: 2019 Fifth international conference on image information processing (ICIIP), pp 310–314

34.

Tronchot A, Berthelemy J, Thomazeau H et al (2021) Validation of virtual reality arthroscopy simulator relevance in characterising experienced surgeons. Orthop Traumatol Surg Res. https://doi.org/10.1016/j.otsr.2021.103079CrossRef

35.

Anetzberger H, Reppenhagen S, Eickhoff H et al (2021) Ten hours of simulator training in arthroscopy are insufficient to reach the target level based on the Diagnostic Arthroscopic Skill Score. Knee Surg Sports Traumatol Arthrosc Off J ESSKA. https://doi.org/10.1007/s00167-021-06648-yCrossRef

36.

Sadeghi M, Behnia F (2018). Optimum window length of Savitzky-Golay filters with arbitrary order. https://doi.org/10.48550/arXiv.1808.10489CrossRef

37.

Carbonaro N, Mascherini G, Bartolini I et al (2021) A wearable sensor-based platform for surgeon posture monitoring: a tool to prevent musculoskeletal disorders. Int J Environ Res Public Health 18:3734. https://doi.org/10.3390/ijerph18073734CrossRef

38.

Dias R, Yule S, Kennedy-Metz L, Zenati M (2019) Psychophysiological data and computer vision to assess cognitive load and team dynamics in cardiac surgery. https://www.semanticscholar.org/paper/Psychophysiological-Data-and-Computer-Vision-to-and-Dias-Yule/43d6d81bc5b78f5a222e93c719f9348e5f2ce564. Accessed 5 Jul 2022

39.

Kishi S, Suzuki N, Tsuyuki S et al (2021) Convolutional neural network based on temporal pose features for surgical procedure recognition. Proc ISCIE Int Symp Stoch Syst Theory Appl 2021:60–64. https://doi.org/10.5687/sss.2021.60CrossRef

40.

Morineau T, Morandi X, Le Moëllic N et al (2009) Decision making during preoperative surgical planning. Hum Factors 51:67–77. https://doi.org/10.1177/0018720809332847CrossRef

41.

Kapralos B, Moussa F, Dubrowski A (2014) An overview of virtual simulation and serious gaming for surgical education and training. In: Brooks AL, Brahnam S, Jain LC (eds) Technologies of inclusive well-being: serious games, alternative realities, and play therapy. Springer, Berlin, Heidelberg, pp 289–306CrossRef

Title: “Stand-up straight!”: human pose estimation to evaluate postural skills during orthopedic surgery simulations
Authors: Tiphaine Casy
Alexandre Tronchot
Hervé Thomazeau
Xavier Morandi
Pierre Jannin
Arnaud Huaulmé
Publication date: 05-10-2022
Publisher: Springer International Publishing
Keywords: Expert Opinion
Expert Opinion
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 2/2023
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-022-02762-5

At a glance: The STEP trials

Springer Medicine

“Stand-up straight!”: human pose estimation to evaluate postural skills during orthopedic surgery simulations

Abstract

Purpose

Material and methods

Results

Conclusion

At a glance: The STEP trials

Springer Medicine

Abstract

Purpose

Material and methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 2/2023

Enhancement of instrumented ultrasonic tracking images using deep learning

Multi-scale feature pyramid fusion network for medical image segmentation

“Image to patient” equal-resolution surface registration supported by a surface scanner: analysis of algorithm efficiency for computer-aided surgery

Efficient contour-based annotation by iterative deep learning for organ segmentation from volumetric medical images

Accurate and robust registration method for computer-assisted high tibial osteotomy surgery

Effects of different virtual monoenergetic CT image data on chest wall post-processing “unfolded ribs” and proposal of an algorithm improvement