Top

Current Reviews in Musculoskeletal Medicine

Published in:

12-04-2024

Feasibility and Usability of Augmented Reality Technology in the Orthopaedic Operating Room

Authors: Stephen P. Canton, Confidence Njoku Austin, Fritz Steuer, Srujan Dadi, Nikhil Sharma, Nicolás M. Kass, David Fogg, Elizabeth Clayton, Onaje Cunningham, Devon Scott, Dukens LaBaze, Edward G. Andrews, Jacob T. Biehl, MaCalus V. Hogan

Published in: Current Reviews in Musculoskeletal Medicine | Issue 5/2024

Abstract

Purpose of Review

Augmented reality (AR) has gained popularity in various sectors, including gaming, entertainment, and healthcare. The desire for improved surgical navigation within orthopaedic surgery has led to the evaluation of the feasibility and usability of AR in the operating room (OR). However, the safe and effective use of AR technology in the OR necessitates a proper understanding of its capabilities and limitations. This review aims to describe the fundamental elements of AR, highlight limitations for use within the field of orthopaedic surgery, and discuss potential areas for development.

Recent Findings

To date, studies have demonstrated evidence that AR technology can be used to enhance navigation and performance in orthopaedic procedures. General hardware and software limitations of the technology include the registration process, ergonomics, and battery life. Other limitations are related to the human response factors such as inattentional blindness, which may lead to the inability to see complications within the surgical field. Furthermore, the prolonged use of AR can cause eye strain and headache due to phenomena such as the vergence-convergence conflict.

Summary

AR technology may prove to be a better alternative to current orthopaedic surgery navigation systems. However, the current limitations should be mitigated to further improve the feasibility and usability of AR in the OR setting. It is important for both non-clinicians and clinicians to work in conjunction to guide the development of future iterations of AR technology and its implementation into the OR workflow.

Cherian JJ, Kapadia BH, Banerjee S, Jauregui JJ, Issa K, Mont MA. Mechanical, anatomical, and kinematic axis in TKA: concepts and practical applications. Curr Rev Musculoskelet Med. 2014;7:89–95. https://doi.org/10.1007/s12178-014-9218-y.CrossRefPubMedPubMedCentral

Liebmann F, Roner S, von Atzigen M, Scaramuzza D, Sutter R, Snedeker J, Farshad M, Furnstahl P. Pedicle screw navigation using surface digitization on the Microsoft HoloLens. Int J Comput Assist Radiol Surg. 2019;14:1157–65. https://doi.org/10.1007/s11548-019-01973-7.CrossRefPubMed

Casari FA, Navab N, Hruby LA, Kriechling P, Nakamura R, Tori R, Nunes F de LDS, Queiroz MC, Furnstahl P, Farshad M. Augmented reality in orthopedic surgery is emerging from proof of concept towards clinical studies: a literature review explaining the technology and current state of the art. Curr Rev Musculoskelet Med. 2021;14:192–203. https://doi.org/10.1007/s12178-021-09699-3.

Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans Information Systems 1994;E77-D, no. 12:1321–1329.

Cipresso P, Giglioli IAC, Raya MA, Riva G. The past, present, and future of virtual and augmented reality research: a network and cluster analysis of the literature. Front Psychol. 2018;9:2086. https://doi.org/10.3389/fpsyg.2018.02086.CrossRefPubMedPubMedCentral

Alexander C, Loeb AE, Fotouhi J, Navab N, Armand M, Khanuja HS. Augmented reality for acetabular component placement in direct anterior total hip arthroplasty. J Arthroplasty. 2020;35:1636-1641.e3. https://doi.org/10.1016/J.ARTH.2020.01.025.CrossRefPubMed

Cho HS, Park YK, Gupta S, Yoon C, Han I, Kim HS, Choi H, Hong J. Augmented reality in bone tumour resection: an experimental study. Bone Joint Res. 2017;6:137–43. https://doi.org/10.1302/2046-3758.63.BJR-2016-0289.R1.CrossRefPubMedPubMedCentral

Fotouhi J, Alexander CP, Unberath M, Taylor G, Lee SC, Fuerst B, Johnson A, Osgood G, Taylor RH, Khanuja H, Armand M, Navab N. Plan in 2-D, execute in 3-D: an augmented reality solution for cup placement in total hip arthroplasty. J Med Imaging (Bellingham). 2018;5:21205–21205. https://doi.org/10.1117/1.Jmi.5.2.021205.CrossRef

Kriechling P, Roner S, Liebmann F, Casari F, Furnstahl P, Wieser K. Augmented reality for base plate component placement in reverse total shoulder arthroplasty: a feasibility study. Arch Orthop Trauma Surg. 2020;141:1447–53. https://doi.org/10.1007/s00402-020-03542-z.CrossRefPubMedPubMedCentral

10.

Molina CA, Phillips FM, Colman MW, Ray WZ, Khan M, Orru E, Poelstra K, Khoo L. A cadaveric precision and accuracy analysis of augmented reality-mediated percutaneous pedicle implant insertion. J Neurosurg Spine. 2020;1–9. https://doi.org/10.3171/2020.6.SPINE20370.

11.

Ogawa H, Kurosaka K, Sato A, Hirasawa N, Matsubara M, Tsukada S. Does an augmented reality-based portable navigation system improve the accuracy of acetabular component orientation during THA? A randomized controlled trial. Clin Orthop Relat Res. 2020;478:935–43. https://doi.org/10.1097/CORR.0000000000001083.CrossRefPubMed

12.

Tu P, Gao Y, Lungu AJ, Li D, Wang H, Chen X. Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2. Comput Biol Med. 2021;133:104402–104402. https://doi.org/10.1016/j.compbiomed.2021.104402.CrossRefPubMed

13.

Vávra P, Roman J, Zonča P, Ihnát P, Němec M, Kumar J, Habib N, El-Gendi A. Recent development of augmented reality in surgery: a review. J Healthc Eng. 2017;2017:4574172. https://doi.org/10.1155/2017/4574172.CrossRefPubMedPubMedCentral

14.

Roth T, Carrillo F, Wieczorek M, Ceschi G, Esfandiari H, Sutter R, Vlachopoulos L, Wein W, Fucentese SF, Fürnstahl P. Three-dimensional preoperative planning in the weight-bearing state: validation and clinical evaluation. Insights Imaging. 2021;12:44–44. https://doi.org/10.1186/s13244-021-00994-8.CrossRefPubMedPubMedCentral

15.

Fida B, Cutolo F, di Franco G, Ferrari M, Ferrari V. Augmented reality in open surgery. Updates Surg. 2018;70:389–400. https://doi.org/10.1007/S13304-018-0567-8.CrossRefPubMed

16.

Fischer M, Fuerst B, Lee SC, Fotouhi J, Habert S, Weidert S, Euler E, Osgood G, Navab N. Preclinical usability study of multiple augmented reality concepts for K-wire placement. Int J Comput Assist Radiol Surg. 2016;11:1007–14. https://doi.org/10.1007/s11548-016-1363-x.CrossRefPubMed

17.

Keating TC, Jacobs JJ. Augmented reality in orthopedic practice and education. Orthop Clin North Am. 2021;52:15–26. https://doi.org/10.1016/j.ocl.2020.08.002.CrossRefPubMed

18.

Lungu AJ, Swinkels W, Claesen L, Tu P, Egger J, Chen X. A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert Rev Med Devices. 2021;18:47–62. https://doi.org/10.1080/17434440.2021.1860750.CrossRefPubMed

19.

Park BJ, Hunt SJ, Martin C 3rd, Nadolski GJ, Wood BJ, Gade TP. Augmented and mixed reality: technologies for enhancing the future of IR. J Vasc Interv Radiol. 2020;31:1074–82. https://doi.org/10.1016/j.jvir.2019.09.020.CrossRefPubMedPubMedCentral

20.

Silva ACDH, Gaber M, Dolenc M, Silva ACDH, Gaber M, Dolenc M. Using augmented reality in different BIM workflows. In: Augmented reality and its application. IntechOpen; 2021.

21.

Elmi-Terander A, Burstrom G, Nachabe R, Fagerlund M, Stahl F, Charalampidis A, Edstrom E, Gerdhem P. Augmented reality navigation with intraoperative 3D imaging vs fluoroscopy-assisted free-hand surgery for spine fixation surgery: a matched-control study comparing accuracy. Sci Rep. 2020;10:707–707. https://doi.org/10.1038/s41598-020-57693-5.CrossRefPubMedPubMedCentral

22.

Guo Q, Li X, Tang Y, Huang Y, Luo L. Augmented reality and three-dimensional plate library-assisted posterior minimally invasive surgery for scapula fracture. Int Orthop. 2022;46:875–82. https://doi.org/10.1007/s00264-022-05303-6.CrossRefPubMed

23.

Hiranaka T, Fujishiro T, Hida Y, Shibata Y, Tsubosaka M, Nakanishi Y, Okimura K, Uemoto H. Augmented reality: the use of the PicoLinker smart glasses improves wire insertion under fluoroscopy. World J Orthop. 2017;8:891–4. https://doi.org/10.5312/wjo.v8.i12.891.CrossRefPubMedPubMedCentral

24.

Liu H, Auvinet E, Giles J, Rodriguez YBF. Augmented reality based navigation for computer assisted hip resurfacing: a proof of concept study. Ann Biomed Eng. 2018;46:1595–605. https://doi.org/10.1007/s10439-018-2055-1.CrossRefPubMedPubMedCentral

25.

Silva R, Oliveira JC, Giraldi GA. Introduction to augmented reality. National laboratory for Scientific Computation 2003; 11:1–11.

26.

Tsukada S, Ogawa H, Nishino M, Kurosaka K, Hirasawa N. Augmented reality-assisted femoral bone resection in total knee arthroplasty. JB JS Open Access 2021;6. https://doi.org/10.2106/JBJS.OA.21.00001.

27.

Aydındoğan G, Kavaklı K, Şahin A, Artal P, Ürey H. Applications of augmented reality in ophthalmology [Invited]. Biomed Opt Express. 2021;12:511–38. https://doi.org/10.1364/boe.405026.CrossRefPubMed

28.

Palumbo A. Microsoft HoloLens 2 in medical and healthcare context: state of the art and future prospects. Sensors (Basel). 2022;22:7709. https://doi.org/10.3390/s22207709.CrossRefPubMed

29.

Furman AA, Hsu WK. Augmented reality (AR) in orthopedics: current applications and future directions. Curr Rev Musculoskelet Med. 2021;14:397–405. https://doi.org/10.1007/s12178-021-09728-1.CrossRefPubMedPubMedCentral

30.

Iacono V, Farinelli L, Natali S, Piovan G, Screpis D, Gigante A, Zorzi C. The use of augmented reality for limb and component alignment in total knee arthroplasty: systematic review of the literature and clinical pilot study. J Exp Orthop. 2021;8:52–52. https://doi.org/10.1186/s40634-021-00374-7.CrossRefPubMedPubMedCentral

31.

Lee SC, Fuerst B, Tateno K, Johnson A, Fotouhi J, Osgood G, Tombari F, Navab N. Multi-modal imaging, model-based tracking, and mixed reality visualisation for orthopaedic surgery. Healthc Technol Lett. 2017;4:168–73. https://doi.org/10.1049/htl.2017.0066.CrossRefPubMedPubMedCentral

32.

Tanzer M, Laverdiere C, Barimani B, Hart A. Augmented reality in arthroplasty: an overview of clinical applications, benefits, and limitations. J Am Acad Orthop Surg. 2022;30:e760–8. https://doi.org/10.5435/JAAOS-D-21-00964.CrossRefPubMed

33.

Cutolo F, Mamone V, Carbonaro N, Ferrari V, Tognetti A. Ambiguity-free optical-inertial tracking for augmented reality headsets. Sensors (Basel) 2020;20. https://doi.org/10.3390/s20051444.

34.

Hoff WA, Vincent TL. Analysis of head pose accuracy in augmented reality. IEEE Trans Vis Comput Graph. 2000;6:319–34.CrossRef

35.

Jud L, Fotouhi J, Andronic O, Aichmair A, Osgood G, Navab N, Farshad M. Applicability of augmented reality in orthopedic surgery - a systematic review. BMC Musculoskelet Disord. 2020; 21. https://doi.org/10.1186/S12891-020-3110-2.

36.

Sutherland J, Belec J, Sheikh A, Chepelev L, Althobaity W, Chow BJW, Mitsouras D, Christensen A, Rybicki FJ, Russa DJL. Applying modern virtual and augmented reality technologies to medical images and models. J Digit Imaging. 2019;32:38–53. https://doi.org/10.1007/s10278-018-0122-7.CrossRefPubMed

37.

Barcali E, Iadanza E, Manetti L, Francia P, Nardi C, Bocchi L. Augmented reality in surgery: a scoping review. Appl Sci. 2022;12. https://doi.org/10.3390/app12146890.

38.

Fucentese SF, Koch PP. A novel augmented reality-based surgical guidance system for total knee arthroplasty. Arch Orthop Trauma Surg. 2021;141:2227–33. https://doi.org/10.1007/S00402-021-04204-4.CrossRefPubMedPubMedCentral

39.

Butler AJ, Colman MW, Lynch J, Phillips FM. Augmented reality in minimally invasive spine surgery: early efficiency and complications of percutaneous pedicle screw instrumentation. Spine J. 2022. https://doi.org/10.1016/j.spinee.2022.09.008.CrossRefPubMed

40.

Yoon JW, Chen RE, Han PK, Si P, Freeman WD, Pirris SM. Technical feasibility and safety of an intraoperative head-up display device during spine instrumentation. Int J Med Robot 2017;13. https://doi.org/10.1002/rcs.1770.

41.

Muller F, Roner S, Liebmann F, Spirig JM, Furnstahl P, Farshad M. Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging. Spine J. 2020;20:621–8. https://doi.org/10.1016/j.spinee.2019.10.012.CrossRefPubMed

42.

Logishetty K, Western L, Morgan R, Iranpour F, Cobb JP, Auvinet E. Can an augmented reality headset improve accuracy of acetabular cup orientation in simulated THA? A randomized trial. Clin Orthop Relat Res. 2019;477:1190–9. https://doi.org/10.1097/CORR.0000000000000542.CrossRefPubMed

43.

Farshad M, Furnstahl P, Spirig JM. First in man in-situ augmented reality pedicle screw navigation. N Am Spine Soc J. 2021;6:100065–100065. https://doi.org/10.1016/j.xnsj.2021.100065.CrossRefPubMedPubMedCentral

44.

Kriechling P, Loucas R, Loucas M, Casari F, Fürnstahl P, Wieser K. Augmented reality through head-mounted display for navigation of baseplate component placement in reverse total shoulder arthroplasty: a cadaveric study. Arch Orthop Trauma Surg. 2021. https://doi.org/10.1007/s00402-021-04025-5.CrossRefPubMedPubMedCentral

45.

Schlueter-Brust K, Henckel J, Katinakis F, Buken C, Opt-Eynde J, Pofahl T, Rodriguez YBF, Tatti F. Augmented-reality-assisted K-wire placement for glenoid component positioning in reversed shoulder arthroplasty: a proof-of-concept study. J Pers Med. 2021;11. https://doi.org/10.3390/jpm11080777.

46.

Weidert S, Wang L, Landes J, Sandner P, Suero EM, Navab N, Kammerlander C, Euler E, Heide A von D. Video-augmented fluoroscopy for distal interlocking of intramedullary nails decreased radiation exposure and surgical time in a bovine cadaveric setting. Int J Med Robot. 2019;15:e1995–e1995. https://doi.org/10.1002/rcs.1995.

47.

Gregory T, Hurst SA, Moslemi A. Mixed reality assisted percutaneous scaphoid fixation: a proposed new surgical technique. Tech Hand Up Extrem Surg. 2021;26:32–6. https://doi.org/10.1097/bth.0000000000000353.CrossRefPubMed

48.

Chen F, Cui X, Han B, Liu J, Zhang X, Liao H. Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy. Comput Methods Programs Biomed. 2021;201:105952–105952. https://doi.org/10.1016/j.cmpb.2021.105952.CrossRefPubMed

49.

Chan A, Parent E, Narvacan K, San C, Lou E. Intraoperative image guidance compared with free-hand methods in adolescent idiopathic scoliosis posterior spinal surgery: a systematic review on screw-related complications and breach rates. Spine J. 2017;17:1215–29. https://doi.org/10.1016/j.spinee.2017.04.001.CrossRefPubMed

50.

van Dijk JD, van den Ende RPJ, Stramigioli S, Köchling M, Höss N. Clinical pedicle screw accuracy and deviation from planning in robot-guided spine surgery: robot-guided pedicle screw accuracy. Spine (Phila Pa 1976) 2015;40:E986-991. https://doi.org/10.1097/BRS.0000000000000960.

51.

Laudato PA, Pierzchala K, Schizas C. Pedicle screw insertion accuracy using O-arm, robotic guidance, or freehand technique: a comparative study. Spine (Phila Pa 1976) 2018;43:E373–E378. https://doi.org/10.1097/BRS.0000000000002449.

52.

Mason A, Paulsen R, Babuska JM, Rajpal S, Burneikiene S, Nelson EL, Villavicencio AT. The accuracy of pedicle screw placement using intraoperative image guidance systems. J Neurosurg Spine. 2014;20:196–203. https://doi.org/10.3171/2013.11.SPINE13413.CrossRefPubMed

53.

Nevzati E, Marbacher S, Soleman J, Perrig WN, Diepers M, Khamis A, Fandino J. Accuracy of pedicle screw placement in the thoracic and lumbosacral spine using a conventional intraoperative fluoroscopy-guided technique: a national neurosurgical education and training center analysis of 1236 consecutive screws. World Neurosurg. 2014;82(866–871):e1-2. https://doi.org/10.1016/j.wneu.2014.06.023.CrossRef

54.

Carucci LR. Imaging obese patients: problems and solutions. Abdom Imaging. 2013;38:630–46. https://doi.org/10.1007/s00261-012-9959-2.CrossRefPubMed

55.

James CR, Peterson BE, Crim JR, Cook JL, Crist BD. The use of fluoroscopy during direct anterior hip arthroplasty: powerful or misleading? J Arthroplasty. 2018;33:1775–9. https://doi.org/10.1016/j.arth.2018.01.040.CrossRefPubMed

56.

Laverdière C, Corban J, Khoury J, Ge SM, Schupbach J, Harvey EJ, Reindl R, Martineau PA. Augmented reality in orthopaedics: a systematic review and a window on future possibilities. Bone Joint J. 2019;101-b:1479–1488. https://doi.org/10.1302/0301-620x.101b12.Bjj-2019-0315.R1.

57.

Cho HS, Park MS, Gupta S, Han I, Kim H-S, Choi H, Hong J. Can augmented reality be helpful in pelvic bone cancer surgery? An in vitro study. Clin Orthop Relat Res. 2018;476:1719–25. https://doi.org/10.1007/s11999.0000000000000233.CrossRefPubMedPubMedCentral

58.

Gavaghan K, Oliveira-Santos T, Peterhans M, Reyes M, Kim H, Anderegg S, Weber S. Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies. Int J Comput Assist Radiol Surg. 2012;7:547–56. https://doi.org/10.1007/S11548-011-0660-7.CrossRefPubMed

59.

Rochlen LR, Levine R, Tait AR. First-person point-of-view-augmented reality for central line insertion training: a usability and feasibility study. Simul Healthc. 2017;12:57–62. https://doi.org/10.1097/SIH.0000000000000185.CrossRefPubMedPubMedCentral

60.

Parsons D, Maccallum K. Current perspectives on augmented reality in medical education: applications, affordances and limitations. Adv Med Educ Pract. 2021;12:77–77. https://doi.org/10.2147/AMEP.S249891.CrossRefPubMedPubMedCentral

61.

Li N, Wakim J, Koethe Y, Huber T, Schenning R, Gade TP, Hunt SJ, Park BJ. Multicenter assessment of augmented reality registration methods for image-guided interventions. Radiol Med. 2022;127:857–65. https://doi.org/10.1007/s11547-022-01515-3.CrossRefPubMed

62.

Condino S, Turini G, Parchi PD, Viglialoro RM, Piolanti N, Gesi M, Ferrari M, Ferrari V. How to build a patient-specific hybrid simulator for orthopaedic open surgery: benefits and limits of mixed-reality using the Microsoft HoloLens. J Healthc Eng. 2018;2018:5435097–5435097. https://doi.org/10.1155/2018/5435097.CrossRefPubMedPubMedCentral

63.

Lin CJ, Canny S. Effects of virtual target size, position, and parallax on vergence-accommodation conflict as estimated by actual gaze. Sci Rep. 2022;12:20100–20100. https://doi.org/10.1038/s41598-022-24450-9.CrossRefPubMedPubMedCentral

64.

Lin Y-H, Huang T-W, Huang H-H, Wang Y-J. Liquid crystal lens set in augmented reality systems and virtual reality systems for rapidly varifocal images and vision correction. Optics Express. 2022;30:22768–78. https://doi.org/10.1364/OE.461378.CrossRefPubMed

65.

Matthews JH, Shields JS. The clinical application of augmented reality in orthopaedics: where do we stand? Curr Rev Musculoskelet Med. 2021;14:316–9. https://doi.org/10.1007/s12178-021-09713-8.CrossRefPubMedPubMedCentral

66.

Ha J, Parekh P, Gamble D, Masters J, Jun P, Hester T, Daniels T, Halai M. Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: a narrative review. J Clin Orthop Trauma. 2021;18:209–15. https://doi.org/10.1016/j.jcot.2021.04.031.CrossRefPubMedPubMedCentral

67.

Babichenko D, Andrews EG, Canton SP, Littleton EB, Patel R, Labaze D, Mills A. Evaluating effect of Microsoft HoloLens on extraneous cognitive load during simulated cervical lateral mass screw placement. In: Yang X-S, Sherratt S, Dey N, Joshi A, editors. Proceedings of Seventh International Congress on Information and Communication Technology. Singapore: Springer Nature; 2023. pp. 191–201.

68.

Dixon BJ, Daly MJ, Chan HH, Vescan A, Witterick IJ, Irish JC. Inattentional blindness increased with augmented reality surgical navigation. Am J Rhinol Allergy. 2014;28:433–7. https://doi.org/10.2500/ajra.2014.28.4067.CrossRefPubMed

69.

Dennler C, Bauer DE, Scheibler AG, Spirig J, Gotschi T, Furnstahl P, Farshad M. Augmented reality in the operating room: a clinical feasibility study. BMC Musculoskelet Disord. 2021;22:451–451. https://doi.org/10.1186/s12891-021-04339-w.CrossRefPubMedPubMedCentral

70.

Khan T, Biehl JT, Andrews EG, Babichenko D. A systematic comparison of the accuracy of monocular RGB tracking and LiDAR for neuronavigation. Healthc Technol Lett. 2022;9:91–101. https://doi.org/10.1049/htl2.12036.CrossRefPubMedPubMedCentral

Title: Feasibility and Usability of Augmented Reality Technology in the Orthopaedic Operating Room
Authors: Stephen P. Canton
Confidence Njoku Austin
Fritz Steuer
Srujan Dadi
Nikhil Sharma
Nicolás M. Kass
David Fogg
Elizabeth Clayton
Onaje Cunningham
Devon Scott
Dukens LaBaze
Edward G. Andrews
Jacob T. Biehl
MaCalus V. Hogan
Publication date: 12-04-2024
Publisher: Springer US
Published in: Current Reviews in Musculoskeletal Medicine / Issue 5/2024
Electronic ISSN: 1935-9748
DOI: https://doi.org/10.1007/s12178-024-09888-w

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Feasibility and Usability of Augmented Reality Technology in the Orthopaedic Operating Room

Abstract

Purpose of Review

Recent Findings

Summary

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose of Review

Recent Findings

Summary

Please log in to get access to this content