Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-09-2016 | Original Article

Automated visibility map of the internal colon surface from colonoscopy video

Authors: Mohammad Ali Armin, Girija Chetty, Hans De Visser, Cedric Dumas, Florian Grimpen, Olivier Salvado

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 9/2016

Abstract

Purpose

Optical colonoscopy is a prominent procedure by which clinicians examine the surface of the colon for cancerous polyps using a flexible colonoscope. One of the main concerns regarding the quality of the colonoscopy is to ensure that the whole colonic surface has been inspected for abnormalities. In this paper, we aim at estimating areas that have not been covered thoroughly by providing a map from the internal colon surface.

Methods

Camera parameters were estimated using optical flow between consecutive colonoscopy frames. A cylinder model was fitted to the colon structure using 3D pseudo stereo vision and projected into each frame. A circumferential band from the cylinder was extracted to unroll the internal colon surface (band image). By registering these band images, drift in estimating camera motion could be reduced, and a visibility map of the colon surface could be generated, revealing uncovered areas by the colonoscope. Hidden areas behind haustral folds were ignored in this study. The method was validated on simulated and actual colonoscopy videos. The realistic simulated videos were generated using a colonoscopy simulator with known ground truth, and the actual colonoscopy videos were manually assessed by a clinical expert.

Results

The proposed method obtained a sensitivity and precision of 98 and 96 % for detecting the number of uncovered areas on simulated data, whereas validation on real videos showed a sensitivity and precision of 96 and 78 %, respectively. Error in camera motion drift could be reduced by almost 50 % using results from band image registration.

Conclusion

Using a simple cylindrical model for the colon and reducing drift by registering band images allows for the generation of visibility maps. The current results also suggest that the provided feedback through the visibility map could enhance clinicians’ awareness of uncovered areas, which in return could reduce the probability of missing polyps.

Australian Institute of Health and Welfare. http://www.aihw.gov.au/. Accessed 16 Oct 2013

World Health Organization (WHO) Fact sheet # 297: Cancer. In: WHO. http://www.who.int/mediacentre/factsheets/fs297/en/. Accessed 21 Oct 2015

de Groen PC (2010) Advanced systems to assess colonoscopy. Gastrointest Endosc Clin N Am 20:699–716. doi:10.1016/j.giec.2010.07.012 CrossRefPubMed

Edakkanambeth Varayil J, Enders F, Tavanapong W, Oh J, Wong J, de Groen PC (2011) Colonoscopy: what endoscopists inspect under optimal conditions. Gastroenterology 140:S-718. doi:10.1016/S0016-5085(11)62982-X

Cotton PB, Williams CB (eds) (2008) Colonoscopy and flexible sigmoidoscopy. In: Practical gastrointestinal endoscopy: the fundamentals, 5th edn. Blackwell Publishing Ltd, Oxford, pp 81–175. doi:10.1002/9780470987032.ch6

Zauber AG, Winawer SJ, O’Brien MJ, Lansdorp-Vogelaar I, van Ballegooijen M, Hankey BF, Shi W, Bond JH, Schapiro M, Panish JF, Stewart ET, Waye JD (2012) Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. N Engl J Med 366:687–696. doi:10.1056/NEJMoa1100370 CrossRefPubMedPubMedCentral

Barclay RL, Vicari JJ, Doughty AS, Johanson JF, Greenlaw RL (2006) Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N Engl J Med 355:2533–2541. doi:10.1056/NEJMoa055498 CrossRefPubMed

Oh JH, Hwang S, Cao Y, Tavanapong W, Liu D, Wong J, de Groen PC (2009) Measuring objective quality of colonoscopy. IEEE Trans Biomed Eng 56:2190–2196. doi:10.1109/TBME.2008.2006035 CrossRefPubMed

Filip D (2012) Colometer: a real-time quality feedback system for screening colonoscopy. World J Gastroenterol 18:4270. doi:10.3748/wjg.v18.i32.4270 CrossRefPubMedPubMedCentral

10.

Hong D, Tavanapong W, Wong J, Oh J, de Groen PC (2013) 3D reconstruction of virtual colon structures from colonoscopy images. Comput Med Imaging Graph. doi:10.1016/j.compmedimag.2013.10.005 PubMed

11.

Bergen T, Wittenberg T (2015) Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform 20:304–321. doi:10.1109/JBHI.2014.2384134 CrossRef

12.

Armin MA, De Visser H, Chetty G, Dumas C, Conlan D, Grimpen F, Salvado O (2015) Visibility map: a new method in evaluation quality of optical colonoscopy. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. Springer, Cham, pp 396–404CrossRef

13.

Kaufman A, Wang J (2008) 3D surface reconstruction from endoscopic videos. In: Linsen L, Hagen H, Hamann B (eds) Visualization in medicine and life sciences. Springer, Berlin, pp 61–74CrossRef

14.

Koppel D, Chen C-I,Wang Y-F, Lee H, Gu J, Poirson A,Wolters R (2007) Toward automated model building from video in computer-assisted diagnoses in colonoscopy. In: Cleary KR, Miga MI (eds) Medical imaging 2007: visualization and image-guided procedures, vol 6509. San Diego, CA. doi:10.1117/12.709595

15.

Bao G, Pahlavan K, Mi L (2015) Hybrid localization of microrobotic endoscopic capsule inside small intestine by data fusion of vision and RF sensors. IEEE Sens J 15:2669–2678. doi:10.1109/JSEN.2014.2367495 CrossRef

16.

Drozdzal M, Seguí S, Vitrià J, Malagelada C, Azpiroz F, Radeva P (2013) Adaptable image cuts for motility inspection using WCE. Comput Med Imaging Graph 37:72–80. doi:10.1016/j.compmedimag.2012.09.002 CrossRefPubMed

17.

Bao G, Mi L, Pahlavan K (2013) A video aided RF localization technique for the wireless capsule endoscope (WCE) inside small intestine. ACM, BrusselsCrossRef

18.

Mountney P, Stoyanov D, Davison A, Yang G-Z (2006) Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery. In: Larsen R, Nielsen M, Sporring J (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2006. Springer, Berlin, pp 347–354CrossRef

19.

Grasa OG, Civera J, Montiel JMM (2011) EKF monocular SLAM with relocalization for laparoscopic sequences. IEEE, ShanghaiCrossRef

20.

Puerto-Souza GA, Staranowicz AN, Bell CS, Valdastri P, Mariottini G-L (2014) A comparative study of ego-motion estimation algorithms for teleoperated robotic endoscopes. In: Luo X, Reichl T, Mirota D, Soper T (eds) Computer-assisted and robotic endoscopy. Springer, Cham, pp 64–76

21.

Liu J, Subramanian KR, Yoo TS (2013) A robust method to track colonoscopy videos with non-informative images. Int J Comput Assist Radiol Surg 8:575–592. doi:10.1007/s11548-013-0814-x CrossRefPubMed

22.

Mori K, Deguchi D, Sugiyama J, Suenaga Y, Toriwaki J, Maurer CR, Takabatake H, Natori H (2002) Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images. A preliminary version of this paper was presented at the Medical Image Computing and Computer-Assisted Intervention (MICCAI) conference, Utrecht, The Netherlands (2001). Med Image Anal 6:321–336. doi:10.1016/S1361-8415(02)00089-0

23.

Rai L, Helferty JP, Higgins WE (2008) Combined video tracking and image-video registration for continuous bronchoscopic guidance. Int J Comput Assist Radiol Surg 3:315–329. doi:10.1007/s11548-008-0241-6 CrossRef

24.

Luó X, Feuerstein M, Deguchi D, Kitasaka T, Takabatake H, Mori K (2012) Development and comparison of new hybrid motion tracking for bronchoscopic navigation. Med Image Anal 16:577–596. doi:10.1016/j.media.2010.11.001 CrossRefPubMed

25.

Valdastri P, Ciuti G, Verbeni A, Menciassi A, Dario P, Arezzo A, Morino M (2012) Magnetic air capsule robotic system: proof of concept of a novel approach for painless colonoscopy. Surg Endosc 26:1238–1246. doi:10.1007/s00464-011-2054-x CrossRefPubMed

26.

Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge

27.

Wang H, Mirota D, Ishii M, Hager GD (2008) Robust motion estimation and structure recovery from endoscopic image sequences with an Adaptive Scale Kernel Consensus estimator. IEEE, pp 1–7

28.

Scaramuzza D, Martinelli A, Siegwart R (2006) A flexible technique for accurate omnidirectional camera calibration and structure from motion. IEEE, p 45

29.

Shi J, Tomasi C (1994) Good features to track. In: IEEE computer vision and pattern recognition, Society Press, pp 593–600

30.

Armin MA, Chetty G, Jurgen F, Visser HD, Dumas C, Fazlollahi A, Grimpen F, Salvado O (2015) Uninformative frame detection in colonoscopy through motion, edge and color features. In: International workshop on computer-assisted and robotic. doi:10.1007/978-3-319-29965-5_15

31.

Torr PHS, Zisserman A (2000) MLESAC: a new robust estimator with application to estimating image geometry. Comput Vis Image Underst 78:138–156. doi:10.1006/cviu.1999.0832 CrossRef

32.

Nister D (2004) An efficient solution to the five-point relative pose problem. IEEE Trans Pattern Anal Mach Intell 26:756–770. doi:10.1109/TPAMI.2004.17 CrossRefPubMed

33.

More J (1978) The Levenberg–Marquardt algorithm: implementation and theory. Springer, Berlin

34.

Scaramuzza D, Fraundorfer F (2011) Visual odometry: Part I—The first 30 years and fundamentals. IEEE Robot Autom Mag 18:80–92. doi:10.1109/MRA.2011.943233 CrossRef

35.

Welch G, Bishop G (1995) An introduction to the Kalman filter. Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill

36.

Nagao J, Mori K, Enjouji T, Deguchi D, Kitasaka T, Suenaga Y, Hasegawa J, Toriwaki J, Takabatake H, Natori H (2004) Fast and accurate bronchoscope tracking using image registration and motion prediction. In: Barillot C, Haynor DR, Hellier P (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2004. Springer, Berlin, pp 551–558CrossRef

37.

Rabbani T, Dijkman S, van den Heuvel F, Vosselman G (2007) An integrated approach for modelling and global registration of point clouds. ISPRS J Photogramm Remote Sens 61:355–370. doi:10.1016/j.isprsjprs.2006.09.006 CrossRef

38.

Zhen Z, Jinwu Q, Yanan Z, Linyong S (2006) An intelligent endoscopic navigation system. IEEE, pp 1653–1657

39.

Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110:346–359. doi:10.1016/j.cviu.2007.09.014 CrossRef

40.

De Visser H, Passenger J, Conlan D, Russ C, Hellier D, Cheng M, Acosta O, Ourselin S, Salvado O (2010) Developing a next generation colonoscopy simulator. Int J Image Graph 10:203–217. doi:10.1142/S0219467810003731 CrossRef

41.

OpenGL—the industry standard for high performance graphics. https://www.opengl.org/. Accessed 1 Feb 2016

42.

Han J (2006) Data mining: concepts and techniques, 2nd edn. Elsevier; Morgan Kaufmann, Amsterdam; Boston

43.

Konda V, Chauhan SS, Abu Dayyeh BK, Hwang JH, Komanduri S, Manfredi MA, Maple JT, Murad FM, Siddiqui UD, Banerjee S (2015) Endoscopes and devices to improve colon polyp detection. Gastrointest Endosc 81:1122–1129. doi:10.1016/j.gie.2014.10.006 CrossRefPubMed

44.

Zhou Jin, Das A, Li Feng, Li Baoxin (2008) Circular generalized cylinder fitting for 3D reconstruction in endoscopic imaging based on MRF. In: IEEE computer vision and pattern recognition workshop, pp 1–8

Title: Automated visibility map of the internal colon surface from colonoscopy video
Authors: Mohammad Ali Armin
Girija Chetty
Hans De Visser
Cedric Dumas
Florian Grimpen
Olivier Salvado
Publication date: 01-09-2016
Publisher: Springer Berlin Heidelberg
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 9/2016
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-016-1462-8

At a glance: The STEP trials

Springer Medicine

Automated visibility map of the internal colon surface from colonoscopy video

Abstract

Purpose

Methods

Results

Conclusion

At a glance: The STEP trials

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 9/2016

Edge map analysis in chest X-rays for automatic pulmonary abnormality screening

A versatile intensity-based 3D/2D rigid registration compatible with mobile C-arm for endovascular treatment of abdominal aortic aneurysm

Automated video-based assessment of surgical skills for training and evaluation in medical schools

IJCARS MICCAI 2015 Special Issue

Fully automatic reconstruction of personalized 3D volumes of the proximal femur from 2D X-ray images

Random walks with shape prior for cochlea segmentation in ex vivo