Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-07-2013 | Original Article

Mobile augmented reality for computer-assisted percutaneous nephrolithotomy

Authors: Michael Müller, Marie-Claire Rassweiler, Jan Klein, Alexander Seitel, Matthias Gondan, Matthias Baumhauer, Dogu Teber, Jens J. Rassweiler, Hans-Peter Meinzer, Lena Maier-Hein

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 4/2013

Abstract

Purpose

Percutaneous nephrolithotomy (PCNL) plays an integral role in treatment of renal stones. Creating percutaneous renal access is the most important and challenging step in the procedure. To facilitate this step, we evaluated our novel mobile augmented reality (AR) system for its feasibility of use for PCNL.

Methods

A tablet computer, such as an iPad\(^{\circledR }\), is positioned above the patient with its camera pointing toward the field of intervention. The images of the tablet camera are registered with the CT image by means of fiducial markers. Structures of interest can be superimposed semi-transparently on the video images. We present a systematic evaluation by means of a phantom study. An urological trainee and two experts conducted 53 punctures on kidney phantoms.

Results

The trainee performed best with the proposed AR system in terms of puncturing time (mean: 99 s), whereas the experts performed best with fluoroscopy (mean: 59 s). iPad assistance lowered radiation exposure by a factor of 3 for the inexperienced physician and by a factor of 1.8 for the experts in comparison with fluoroscopy usage. We achieve a mean visualization accuracy of 2.5 mm.

Conclusions

The proposed tablet computer-based AR system has proven helpful in assisting percutaneous interventions such as PCNL and shows benefits compared to other state-of-the-art assistance systems. A drawback of the system in its current state is the lack of depth information. Despite that, the simple integration into the clinical workflow highlights the potential impact of this approach to such interventions.

Available only for authorised users

iKlip stand adapter, IK Multimedia Production, http://www.ikmultimedia.com/products/iklip/.

Braun Melsungen AG, http://www.bbraun.de/.

Rebeck patient markers, Fobeck GbR, http://fobeck.com/cms2/en.

For a detailed specification, please refer to the manufacturer‘s website http://www.siemens.com.

Romero V, Akpinar H, Assimos DG (2010) Kidney stones: a global picture of prevalence, incidence, and associated risk factors. Rev Urol 12(2–3):86–96

Skolarikos A, Alivizatos G, de la Rosette JJ (2005) Percutaneous nephrolithotomy and its legacy. Eur Urol 47(1):22–28PubMedCrossRef

Michel MS, Trojan L, Rassweiler JJ (2007) Complications in percutaneous nephrolithotomy. Eur Urol 51(4):899–906

Andonian S, Scoffone C, Louie MK, Gross AJ, Grabe M, Daels F, Shah HN, De La Rosette J (2012) Does imaging modality used for percutaneous renal access make a difference? a matched case analysis. J Endourol 27(1):24–28

Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3): 189–201

Maier-Hein L, Tekbas A, Seitel A, Pianka F, Müller SA, Satzl S, Schawo S, Radeleff B, Tetzlaff R, Franz AM, Müller-Stich BP, Wolf I, Kauczor H-U, Schmied BM, Meinzer H-P (2008) In vivo accuracy assessment of a needle-based navigation system for CT-guided radiofrequency ablation of the liver. Med Phys 35(12):5385–5396PubMedCrossRef

Fichtinger G, Deguet A, Fischer G, Iordachita I, Balogh E, Masamune K, Taylor RH, Fayad LM, de Oliveira M, Zinreich SJ (2005) Image overlay for CT-guided needle insertions. Comput Aided Surg 10(4):241–255PubMed

Song DY, Burdette EC, Fiene J, Armour E, Kronreif G, Deguet A, Zhang Z, Iordachita I, Fichtinger G, Kazanzides P (2011) Robotic needle guide for prostate brachytherapy: clinical testing of feasibility and performance. Brachytherapy 10(1):57–63PubMedCrossRef

Wein W, Brunke S, Khamene A, Callstrom MR, Navab N (2008) Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention. Med Image Anal 12(5):577–585PubMedCrossRef

10.

Wood BJ, Kruecker J, Abi-Jaoudeh N, Locklin JK, Levy E, Xu S, Solbiati L, Kapoor A, Amalou H, Venkatesan AM (2010) Navigation systems for ablation. J Vasc Interv Radiol 21(8 Suppl):257–263CrossRef

11.

Lazarus J, Williams J (2011) The locator: novel percutaneous nephrolithotomy apparatus to aid collecting system puncture-a preliminary report. J Endourol 25(5):747–750PubMedCrossRef

12.

Huber J, Wegner I, Meinzer HP, Hallscheidt P, Hadaschik B, Pahernik S, Hohenfellner M (2011) Navigated renal access using electromagnetic tracking: an initial experience. Surg Endosc 25(4):1307–1312PubMedCrossRef

13.

Ritter M, Rassweiler MC, Hacker A, Michel MS (2012) Laser-guided percutaneous kidney access with the Uro Dyna-CT: first experience of three-dimensional puncture planning with an ex vivo model. World J Urol. 30:1–5

14.

Baumhauer M, Simpfendörfer T, Stich BM, Teber D, Gutt C, Rassweiler J, Meinzer H-P, Wolf I (2008) Soft tissue navigation for laparoscopic partial nephrectomy. Int J Comput Assist Radiol Surg 3:307–314CrossRef

15.

Maier-Hein L, Franz AM, Fangerau M, Schmidt M, Seitel A, Mersmann S, Kilgus T, Groch A, Yung K, dos Santos TR, Meinzer H-P (2011) Towards mobile augmented reality for on-patient visualization of medical images. In: Handels H, Ehrhardt J, Deserno TM, Meinzer H-P, Tolxdorff T (eds) Bildverarbeitung für die Medizin. Springer, Berlin, pp 389–393

16.

Rassweiler JJ, Müller M, Fangerau M, Klein J, Goezen AS, Pereira P, Meinzer HP, Teber D (2012) iPad-assisted percutaneous access to the kidney using marker-based navigation: initial clinical experience. Eur Urol 61(3):628–631PubMedCrossRef

17.

Wolf I, Vetter M, Wegner I, Böttger T, Nolden M, Schöbinger M, Hastenteufel M, Kunert T, Meinzer H-P (2005) The medical imaging interaction toolkit. Med Image Anal 9(6):594–604

18.

Fangerau M (2011) Medical imaging interaction toolkit for mobile devices. Master’s thesis, Hochschule Mannheim, University of Applied Sciences

19.

Müller M, Groch A, Baumhauer M, Maier-Hein L, Teber D, Rassweiler J, Meinzer H-P, Wegner I (2012) Robust and efficient fiducial tracking for augmented reality in HD-laparoscopic video streams. In: DRH III, Wong KH, (eds) SPIE medical imaging 2012: visualization, image-guided procedures, and modeling, vol 8316, No 1. SPIE, p 83161M

20.

Zhang Z (2000) A flexible new technique for camera calibration. IEEE T Pattern Anal 22:1330–1334CrossRef

21.

Lowe DG (1991) Fitting parameterized three-dimensional models to images. IEEE Trans Pattern Anal Mach Intell 13:441–450CrossRef

22.

DeMenthon DF, Davis LS (1995) Model-based object pose in 25 lines of code. Int J Comput Vis 15:123–141CrossRef

23.

Lu C-P, Hager GD, Mjolsness E (2000) Fast and globally convergent pose estimation from video images. IEEE Trans Pattern Anal Mach Intell 22:610–622CrossRef

24.

Lepetit V, Moreno-Noguer F, Fua P (2008) EP\(n\)P: an accurate o(\(n\)) solution to the P\(n\)P problem. Int J Comput Vis 81(2):155–166CrossRef

25.

Sarkis M, Diepold K (2012) Camera-pose estimation via projective Newton optimization on the manifold. IEEE Trans Image Process 21(4):1729–1741PubMedCrossRef

26.

Li S, Xu C, Xie M (2012) A robust o(n) solution to the perspective-n-point problem. IEEE Trans Pattern Anal Mach Intell. 34(7):1444–1450CrossRef

27.

Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395CrossRef

28.

Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. ser. Springer series in operations research and financial engineering. Springer, New York

29.

Holloway RL (1997) Registration error analysis for augmented reality. Presence Teleoper Virtual Environ 6(4):413–432

30.

Baumhauer M (2008) Real-time compensation of organ motion for augmented reality in laparoscopic surgery. Ph.D. dissertation, Ruprecht-Karls University Heidelberg

31.

Simpfendörfer T, Baumhauer M, Müller M, Gutt CN, Meinzer HP, Rassweiler JJ, Guven S, Teber D (2011) Augmented reality visualization during laparoscopic radical prostatectomy. J Endourol 25:1841–1845

32.

Laganière R (2011) OpenCV 2 computer vision application programming cookbook. Packt Publishing, Birmingham

33.

Miller NL, Lingeman JE (2007) Management of kidney stones. BMJ 334(7591):468–472PubMedCrossRef

34.

Ritter M, Siegel F, Krombach P, Martinschek A, Weiss C, Hacker A, Pelzer AE (2012) Influence of surgeon’s experience on fluoroscopy time during endourological interventions. World J Urol 31(1): 183–187

35.

Seitel A, Maier-Hein L, Schawo S, Radeleff B, Müller SA, Pianka F, Schmied BM, Wolf I, Meinzer H-P (2007) In-vitro evaluation of different visualization approaches for computer assisted targeting in soft tissue. In: Lemke H, Inamura K, Doi K, Vannier M, Farman A (eds) International journal of Computer Assisted Radiology and Surgery. Berlin (Germany), pp 188–190, June 2007

36.

Karami H, Rezaei A, Mohammadhosseini M, Javanmard B, Mazloomfard M, Lotfi B (2010) Ultrasonography-guided percutaneous nephrolithotomy in the flank position versus fluoroscopy-guided percutaneous nephrolithotomy in the prone position: a comparative study. J Endourol 24(8):1357–1361

37.

Agarwal M, Agrawal MS, Jaiswal A, Kumar D, Yadav H, Lavania P (2011) Safety and efficacy of ultrasonography as an adjunct to fluoroscopy for renal access in percutaneous nephrolithotomy (PCNL). BJU Int 108(8):1346–1349PubMedCrossRef

38.

Maier-Hein L, Walsh CJ, Seitel A, Hanumara NC, Shepard J-A, Franz AM, Pianka F, Müller SA, Schmied B, Slocum AH, Gupta R, Meinzer H-P (2009) Human vs. robot operator error in a needle-based navigation system for percutaneous liver interventions. In: SPIE medical imaging 2009: visualization, image-guided procedures, and modelling, vol 7261, p 72610Y (12 p)

39.

Franz A, März K, Hummel J, Birkfellner W, Bendl R, Delorme S, Schlemmer H-P, Meinzer H-P, Maier-Hein L (2012) Electromagnetic tracking for us-guided interventions: standardized assessment of a new compact field generator. Int J Comp Assist Radiol Surg 7:1–6

40.

Yaniva Z, Wilson E, Lindisch D, Cleary K (2009) Electromagnetic tracking in the clinical environment. Med Phys 36(3):876–892CrossRef

41.

Lee T, Hollerer T (2009) Multithreaded hybrid feature tracking for markerless augmented reality. IEEE Trans Vis Comput Graph 15(3):355–368PubMedCrossRef

42.

Grimm R, Bauer S, Sukkau J, Hornegger J, Greiner G (2012) Markerless estimation of patient orientation, posture and pose using range and pressure imaging. Int J Comput Assist Radiol Surg 1:1–9

43.

Newcombe RA, Davison AJ, Izadi S, Kohli P, Hilliges O, Shotton J, Molyneaux D, Hodges S, Kim D, Fitzgibbon A (2011) Kinectfusion: real-time dense surface mapping and tracking. In: Mixed and augmented reality (ISMAR), 2011 10th IEEE international symposium on, pp. 127–136, Oct 2011

44.

Mersmann S, Gergel I, Seitel A, Gaa J, Wegner I, Meinzer H-P, Maier-Hein L (2011) Microsoft kinect controller as intra-operative imaging modality. Int J Comput Assist Radiol Surg 6(Suppl 1): 251–252

45.

Seitel A, Engel M, Sommer CM, Radeleff BA, Essert-Villard C, Baegert C, Fangerau M, Fritzsche KH, Yung K, Meinzer H-P, Maier-Hein L (2011) Computer-assisted trajectory planning for percutaneous needle insertions. Med Phys 38(6):3246–3259PubMedCrossRef

Title: Mobile augmented reality for computer-assisted percutaneous nephrolithotomy
Authors: Michael Müller
Marie-Claire Rassweiler
Jan Klein
Alexander Seitel
Matthias Gondan
Matthias Baumhauer
Dogu Teber
Jens J. Rassweiler
Hans-Peter Meinzer
Lena Maier-Hein
Publication date: 01-07-2013
Publisher: Springer Berlin Heidelberg
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 4/2013
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-013-0828-4

At a glance: The STEP trials

Springer Medicine

Mobile augmented reality for computer-assisted percutaneous nephrolithotomy

Abstract

Purpose

Methods

Results

Conclusions

At a glance: The STEP trials

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 4/2013

Fast rigid registration of pre-operative magnetic resonance images to intra-operative ultrasound for neurosurgery based on high confidence gradient orientations

Multimodality GPU-based computer-assisted diagnosis of breast cancer using ultrasound and digital mammography images

A robust method to track colonoscopy videos with non-informative images

Discriminative generalized Hough transform for object localization in medical images

Three-dimensional histological specimen preparation for accurate imaging and spatial reconstruction of the middle and inner ear

Phantom-based comparison of conventional versus phase-contrast mammography for LCD soft-copy diagnosis