Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

Open Access 01-03-2020 | Original Article

Augmentation of haptic feedback for teleoperated robotic surgery

Authors: Philipp Schleer, Philipp Kaiser, Sergey Drobinsky, Klaus Radermacher

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 3/2020

Abstract

Purpose

A frequently mentioned lack of teleoperated surgical robots is the lack of haptic feedback. Haptics are not only able to mirror force information from the situs, but also to provide spatial guidance according to a surgical plan. However, superposition of the two haptic information can lead to overlapping and masking of the feedback and guidance forces. This study investigates different approaches toward a combination of both information and investigates effects on system usability.

Methods

Preliminary studies are conducted to define parameters for two main experiments. The two main experiments constitute simulated surgical interventions where haptic guidance as well as haptic feedback provide information for the surgeon. The first main experiment considers drilling for pedicle screw placements, while the second main experiment refers to three-dimensional milling tasks such as during partial knee replacements or craniectomies. For both experiments, different guidance modes in combination with haptic feedback are evaluated regarding effectiveness (e.g., distance to target depth), efficiency and user satisfaction (e.g., detectability of discrepancies in case of technical guidance error).

Results

Regarding pedicle screw placements a combination of a peripheral visual signal and a vibration constitutes a good compromise regarding distance to target depth and detectability of discrepancies. For milling tasks, trajectory guidance is able to improve efficiency and user satisfaction (e.g., perceived workload), while boundary constraints improve effectiveness. If, assistance cannot be offered in all degrees of freedom (e.g., craniectomies), a visual substitution of the haptic force feedback shows the best results, though participants prefer using haptic force feedback.

Conclusion

Our results suggest that in case haptic feedback and haptic assistance are combined appropriately, benefits of both haptic modalities can be exploited. Thereby, capabilities of the human–machine system are improved compared to usage of exclusively one of the haptic information.

Troccaz J, Peshkin M, Davies B (1998) Synergistic mechanical devices: a new generation of medical robots. In: Robotics research. Springer, pp 317–323

Schleer P, Drobinsky S, de la Fuente M, Radermacher K (2019) Toward versatile cooperative surgical robotics: a review and future challenges. Int J Comput Assist Radiol Surg 14(10):1673–1686CrossRef

Herron DM, Marohn M (2008) A consensus document on robotic surgery. Surg Endosc 22:313–325CrossRef

Okamura AM (2004) Methods for haptic feedback in teleoperated robot-assisted surgery. Ind Robot Int J 31:499–508CrossRef

Bholat OS, Haluck RS, Murray WB, Gorman PJ, Krummel TM (1999) Tactile feedback is present during minimally invasive surgery. J Am Coll Surg 189:349–355CrossRef

Wagner CR, Stylopoulos N, Howe RD (2002) The role of force feedback in surgery: analysis of blunt dissection. In: Proceedings 10th symposium on haptic interfaces for virtual environment and teleoperator systems. HAPTICS 2002. Citeseer, pp 68–74

Kazi A (2001) Operator performance in surgical telemanipulation. Presence Teleoper Virtual Environ 10:495–510CrossRef

Westebring-van der Putten EP, Goossens RHM, Jakimowicz JJ, Dankelman J (2008) Haptics in minimally invasive surgery—a review. Minim Invasive Ther Allied Technol 17:3–16. https://doi.org/10.1080/13645700701820242 CrossRefPubMed

Bowyer SA, Davies BL, y Baena FR (2014) Active constraints/virtual fixtures: a survey. IEEE Trans Robot 30:138–157CrossRef

10.

Park S, Howe RD, Torchiana DF (2001) Virtual fixtures for robotic cardiac surgery. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 1419–1420

11.

Rosenberg LB (1993) Virtual fixtures: perceptual tools for telerobotic manipulation. Proc IEEE Virtual Real Annu Int Symp. https://doi.org/10.1109/VRAIS.1993.380795 CrossRef

12.

Schleer P, Drobinsky S, Radermacher K (2019) Evaluation of different modes of haptic guidance for robotic surgery. IFAC-PapersOnLine 51:97–103. https://doi.org/10.1016/j.ifacol.2019.01.035 CrossRef

13.

Olivieri E, Barresi G, Caldwell DG, Mattos LS (2017) Haptic feedback for control and active constraints in contactless laser surgery: concept, implementation, and evaluation. IEEE Trans Haptics 11:241–254CrossRef

14.

Smisek J, van Paassen MM, Schiele A (2015) Haptic guidance in bilateral teleoperation: effects of guidance inaccuracy. In: World haptics conference (WHC), 2015 IEEE. IEEE, pp 500–505

15.

van Oosterhout J, Wildenbeest JGW, Boessenkool H, Heemskerk CJM, de Baar MR, van der Helm FCT, Abbink DA (2015) Haptic shared control in tele-manipulation: effects of inaccuracies in guidance on task execution. IEEE Trans Haptics 8:164–175CrossRef

16.

Makous JC, Friedman RM, Vierck CJ (1995) A critical band filter in touch. J Neurosci 15:2808–2818CrossRef

17.

Strasburger H, Rentschler I, Jüttner M (2011) Peripheral vision and pattern recognition: a review. J Vis 11:13CrossRef

18.

Burke JL, Prewett MS, Gray AA, Yang L, Stilson FRB, Coovert MD, Elliot LR, Redden E (2006) Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis. In: Proceedings of the 8th international conference on multimodal interfaces. ACM, pp 108–117

19.

Borys B-B (2015) Interaktionsuntersuchungen für haptisches feedback elektronischer Eingabegeräte (InterHapt): Schlussbericht: Projektlaufzeit: 01.05. 2014 bis 31.10. 2015. Universität Kassel, Fachgebiet Mensch-Maschine-Systemtechnik

20.

Harada N, Griffin MJ (1991) Factors influencing vibration sense thresholds used to assess occupational exposures to hand transmitted vibration. Occup Environ Med 48:185–192CrossRef

21.

Hölscher U, Laurig W, Müller-Arnecke HW (2008) Prinziplösungen zur ergonomischen Gestaltung von Medizingeräten: Erkenntnisse, Empfehlungen und Prinziplösungen zur ergonomischen Produktgestaltung am Beispiel der Gebrauchstauglichkeit von Medizingeräten; Forschung Projekt F 1902;[Abschlussbericht]. Bundesanst. für Arbeitsschutz und Arbeitsmedizin

22.

Peon AR, Prattichizzo D (2013) Reaction times to constraint violation in haptics: comparing vibration, visual and audio stimuli. In: World haptics conference (WHC), 2013. IEEE, pp 657–661

23.

Vaccaro AR, Rizzolo SJ, Balderston RA, Allardyce TJ, Garfin SR, Dolinskas C, An HS (1995) Placement of pedicle screws in the thoracic spine. Part II: an anatomical and radiographic assessment. JBJS 77:1200–1206CrossRef

24.

Brix C (2008) Platzierung von Pedikelschrauben mit einer mechanischen Zielhilfe im Vergleich zur konventionellen Methode-erste klinische Ergebnisse

25.

Puvanesarajah V, Liauw JA, Lo S, Lina IA, Witham TF (2014) Techniques and accuracy of thoracolumbar pedicle screw placement. World J Orthop 5:112CrossRef

26.

Hu Y, Jin H, Zhang L, Zhang P, Zhang J (2013) State recognition of pedicle drilling with force sensing in a robotic spinal surgical system. IEEE/ASME Trans Mechatron 19:357–365CrossRef

27.

Preusche C, Hirzinger G (2007) Haptics in telerobotics. Vis Comput 23:273–284CrossRef

28.

Banerjee S, Cherian JJ, Elmallah RK, Jauregui JJ, Pierce TP, Mont MA (2015) Robotic-assisted knee arthroplasty. Expert Rev Med Devices 12:727–735CrossRef

29.

Bast P (2010) Sicherheitsstrategie für ein Parallelrobotersystem beim Einsatz in der cranialen Neurochirurgie: ein Beitrag aus dem Lehrstuhl für Medizintechnik der RWTH Aachen. Shaker

30.

Cunha-Cruz V, Follmann A, Popovic A, Bast P, Wu T, Heger S, Engelhardt M, Schmieder K, Radermacher K (2010) Robot-and computer-assisted craniotomy (CRANIO): from active systems to synergistic man—machine interaction. Proc Inst Mech Eng Part H J Eng Med 224:441–452CrossRef

31.

Wiles AD, Thompson DG, Frantz DD (2004) Accuracy assessment and interpretation for optical tracking systems. In: Medical imaging 2004: visualization, image-guided procedures, and display. International Society for Optics and Photonics, pp 421–432

32.

Bast P, Popovic A, Wu T, Heger S, Engelhardt M, Lauer W, Radermacher K, Schmieder K (2006) Robot-and computer-assisted craniotomy: resection planning, implant modelling and robot safety. Int J Med Robot Comput Assist Surg 2:168–178CrossRef

33.

Hollensteiner M, Fürst D, Esterer B, Hunger S, Malek M, Augat P, Schrödl F, Stephan D, Schrempf A (2016) Development of parietal bone surrogates for parietal graft lift training. Curr Dir Biomed Eng 2:637–641

34.

Hart SG (2006) NASA-task load index (NASA-TLX); 20 years later. In: Proceedings of the human factors and ergonomics society annual meeting. Sage Publications Sage CA, Los Angeles, CA, pp 904–908CrossRef

35.

Brooke J (1996) SUS—a quick and dirty usability scale. Usabil Eval Ind 189:4–7

36.

Black D, Lilge S, Fellmann C, Reinschluessel AV, Kreuer L, Nabavi A, Hahn HK, Kikinis R, Burgner-Kahrs J (2019) Auditory display for telerobotic transnasal surgery using a continuum robot. J Med Robot Res 4:1950004CrossRef

Title: Augmentation of haptic feedback for teleoperated robotic surgery
Authors: Philipp Schleer
Philipp Kaiser
Sergey Drobinsky
Klaus Radermacher
Publication date: 01-03-2020
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 3/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-020-02118-x

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Augmentation of haptic feedback for teleoperated robotic surgery

Abstract

Purpose

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 3/2020

3D printing method for next-day acetabular fracture surgery using a surface filtering pipeline: feasibility and 1-year clinical results

Automated measurement of bone scan index from a whole-body bone scintigram

Toward automatic quantification of knee osteoarthritis severity using improved Faster R-CNN

Touchless scanner control to support MRI-guided interventions

Ultrasound needle segmentation and trajectory prediction using excitation network

Transrectal ultrasound image-based real-time augmented reality guidance in robot-assisted laparoscopic rectal surgery: a proof-of-concept study