Top

Journal of Imaging Informatics in Medicine

Published in:

01-10-2019

Image Annotation by Eye Tracking: Accuracy and Precision of Centerlines of Obstructed Small-Bowel Segments Placed Using Eye Trackers

Authors: Alfredo Lucas, Kang Wang, Cynthia Santillan, Albert Hsiao, Claude B. Sirlin, Paul M. Murphy

Published in: Journal of Imaging Informatics in Medicine | Issue 5/2019

Abstract

Small-bowel obstruction (SBO) is a common and important disease, for which machine learning tools have yet to be developed. Image annotation is a critical first step for development of such tools. This study assesses whether image annotation by eye tracking is sufficiently accurate and precise to serve as a first step in the development of machine learning tools for detection of SBO on CT. Seven subjects diagnosed with SBO by CT were included in the study. For each subject, an obstructed segment of bowel was chosen. Three observers annotated the centerline of the segment by manual fiducial placement and by visual fiducial placement using a Tobii 4c eye tracker. Each annotation was repeated three times. The distance between centerlines was calculated after alignment using dynamic time warping (DTW) and statistically compared to clinical thresholds for diagnosis of SBO. Intra-observer DTW distance between manual and visual centerlines was calculated as a measure of accuracy. These distances were 1.1 ± 0.2, 1.3 ± 0.4, and 1.8 ± 0.2 cm for the three observers and were less than 1.5 cm for two of three observers (P < 0.01). Intra- and inter-observer DTW distances between centerlines placed with each method were calculated as measures of precision. These distances were 0.6 ± 0.1 and 0.8 ± 0.2 cm for manual centerlines, 1.1 ± 0.4 and 1.9 ± 0.6 cm for visual centerlines, and were less than 3.0 cm in all cases (P < 0.01). Results suggest that eye tracking–based annotation is sufficiently accurate and precise for small-bowel centerline annotation for use in machine learning–based applications.

Alajlan N: HopDSW: An approximate dynamic space warping algorithm for fast shape matching and retrieval. Journal of King Saud University - Computer and Information Sciences 23(1): 7–14, 2011. https://doi.org/10.1016/j.jksuci.2010.01.001. http://www.sciencedirect.com/science/article/pii/S1319157810000030 CrossRef

Allen BC, Leyendecker JR: MR enterography for assessment and management of small bowel Crohn disease. Radiol Clin N Am 52(4): 799–810, 2014. https://doi.org/10.1016/j.rcl.2014.02.001 CrossRefPubMed

Aquina CT, Becerra AZ, Probst CP, Xu Z, Hensley BJ, Iannuzzi JC, Noyes K, Monson JRT, Fleming FJ: Patients with adhesive small bowel obstruction should be primarily managed by a surgical team. Ann Surg 264(3): 437–447, 2016. https://doi.org/10.1097/SLA.0000000000001861 CrossRefPubMed

Berndt DJ, Clifford J: Using dynamic time warping to find patterns in time series.. In: Proceedings of the 3rd international conference on knowledge discovery and data mining, AAAIWS’94. AAAI Press, Seattle, 1994, pp 359–370. http://dl.acm.org/citation.cfm?id=3000850.3000887

Bruining DH, Zimmermann EM, Loftus EV, Sandborn WJ, Sauer CG, Strong SA: Society of Abdominal Radiology Crohn’s Disease-Focused Panel: Consensus Recommendations for Evaluation, Interpretation, and Utilization of Computed Tomography and Magnetic Resonance Enterography in Patients With Small Bowel Crohn’s Disease. Radiology 286(3): 776–799, 2018. https://doi.org/10.1148/radiol.2018171737 CrossRefPubMed

Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, Kadoury S, Tang A: Deep learning: A primer for radiologists. RadioGraphics 37(7): 2113–2131, 2017. https://doi.org/10.1148/rg.2017170077. http://pubs.rsna.org/doi/10.1148/rg.2017170077 CrossRefPubMed

Dale AM, Fischl B, Sereno MI: Cortical surface-based analysis. I. Segmentation and surface reconstruction. NeuroImage 9(2): 179–194, 1999. https://doi.org/10.1006/nimg.1998.0395 CrossRefPubMed

Ebner L, Tall M, Choudhury KR, Ly DL, Roos JE, Napel S, Rubin GD: Variations in the functional visual field for detection of lung nodules on chest computed tomography: Impact of nodule size, distance, and local lung complexity. Med Phys 44(7): 3483–3490, 2017. https://doi.org/10.1002/mp.12277 CrossRefPubMed

Feuerstein JD, Cheifetz AS: Crohn disease: Epidemiology, diagnosis, and management. Mayo Clin Proc 92(7): 1088–1103, 2017. https://doi.org/10.1016/j.mayocp.2017.04.010 CrossRefPubMed

10.

Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, van der Kouwe A, Killiany R, Kennedy D, Klaveness S, Montillo A, Makris N, Rosen B, Dale AM: Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron 33(3): 341–355, 2002CrossRefPubMed

11.

Frager D, Medwid SW, Baer JW, Mollinelli B, Friedman M: CT of small-bowel obstruction: value in establishing the diagnosis and determining the degree and cause. AJR Am J Roentgenol 162(1): 37–41, 1994. https://doi.org/10.2214/ajr.162.1.8273686 CrossRefPubMed

12.

Frangi AF, Niessen WJ, Vincken KL, Viergever MA: Multiscale vessel enhancement filtering.. In: Medical Image Computing and Computer-Assisted Intervention — MICCAI’98, pp 130–137, 1998

13.

Frimmel H, Näppi J, Yoshida H: Fast and robust computation of colon centerline in CT colonography. Med Phys 31(11): 3046–3056, 2004. https://doi.org/10.1118/1.1790111 CrossRefPubMed

14.

Frimmel H, Näppi J, Yoshida H: Centerline-based colon segmentation for CT colonography. Med Phys 32(8): 2665–2672, 2005. https://doi.org/10.1118/1.1990288 CrossRefPubMed

15.

Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Advances In Neural Information Processing Systems, 1–9

16.

Lecun Y, Bottou L, Bengio Y, Haffner P: Gradient-based learning applied to document recognition. Proc IEEE 86(11): 2278–2324, 1998. https://doi.org/10.1109/5.726791 CrossRef

17.

Li Q, Sone S, Doi K: Selective enhancement filters for nodules, vessels, and airway walls in two- and three-dimensional CT scans. Med Phys 30(8): 2040–2051, 2003. https://doi.org/10.1118/1.1581411 CrossRefPubMed

18.

Litjens G, Toth R, van de Ven W, Hoeks C, Kerkstra S, van Ginneken B, Vincent G, Guillard G, Birbeck N, Zhang J, Strand R, Malmberg F, Ou Y, Davatzikos C, Kirschner M, Jung F, Yuan J, Qiu W, Gao Q, Edwards PE, Maan B, van der Heijden F, Ghose S, Mitra J, Dowling J, Barratt D, Huisman H, Madabhushi A: Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge. Med Image Anal 18(2): 359–373, 2014. https://doi.org/10.1016/j.media.2013.12.002 CrossRefPubMed

19.

Lu L, Zhang D, Li L, Zhao J: Fully automated colon segmentation for the computation of complete colon centerline in virtual colonoscopy. IEEE Trans Biomed Eng 59(4): 996–1004, 2012. https://doi.org/10.1109/TBME.2011.2182051 CrossRefPubMed

20.

Magdy N, Sakr MA, Mostafa T, El-Bahnasy K (2015) Review on trajectory similarity measures. In: 2015 IEEE 7th international conference on intelligent computing and information systems (ICICIS), pp 613–619. https://doi.org/10.1109/IntelCIS.2015.7397286

21.

Maiora J, Ayerdi B, Graña M: Random forest active learning for AAA thrombus segmentation in computed tomography angiography images. Neurocomputing 126: 71–77, 2014. https://doi.org/10.1016/j.neucom.2013.01.051. http://www.sciencedirect.com/science/article/pii/S0925231213006978 CrossRef

22.

Maiora J, Graña M (2012) Abdominal CTA image analisys through active learning and decision random forests: Aplication to AAA segmentation. In: The 2012 international joint conference on neural networks (IJCNN), pp 1–7. https://doi.org/10.1109/IJCNN.2012.6252801

23.

Metz CT, Schaap M, Weustink AC, Mollet NR, Van Walsum T, Niessen WJ: Coronary centerline extraction from CT coronary angiography images using a minimum cost path approach. Med Phys 36(12): 5568–5579, 2009. https://doi.org/10.1118/1.3254077 CrossRefPubMed

24.

Milletari F, Navab N, Ahmadi SA (2016) V-Net: Fully convolutional neural networks for volumetric medical image segmentation, 1–11. arXiv:1606.04797

25.

Mittal D, Kumar V, Saxena SC, Khandelwal N, Kalra N: Neural network based focal liver lesion diagnosis using ultrasound images. Comput Med Imaging Graph 35(4): 315–323, 2011. https://doi.org/10.1016/j.compmedimag.2011.01.007. http://www.sciencedirect.com/science/article/pii/S0895611111000188 CrossRefPubMed

26.

Papadopoulos DP, Clarke ADF, Keller F, Ferrari V: Training object class detectors from eye tracking data.. In: Computer Vision – ECCV 2014, Lecture Notes in Computer Science. Springer, Cham, 2014, pp 361–376, https://doi.org/10.1007/978-3-319-10602-1_24. https://link.springer.com/chapter/10.1007/978-3-319-10602-1_24

27.

Paulson EK, Thompson WM: Review of small-bowel obstruction: the diagnosis and when to worry. Radiology 275(2): 332–342, 2015. https://doi.org/10.1148/radiol.15131519 CrossRefPubMed

28.

Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation, 1–8. arXiv:1505.04597

29.

Rubin GD, Roos JE, Tall M, Harrawood B, Bag S, Ly DL, Seaman DM, Hurwitz LM, Napel S, Roy Choudhury K: Characterizing search, recognition, and decision in the detection of lung nodules on CT scans: Elucidation with eye tracking. Radiology 274(1): 276–286, 2015. https://doi.org/10.1148/radiol.14132918. http://pubs.rsna.org/doi/10.1148/radiol.14132918 CrossRefPubMed

30.

Santillan CS: MR imaging techniques of the bowel. Magn Reson Imaging Clin N Am 22(1): 1–11, 2014. https://doi.org/10.1016/j.mric.2013.07.004 CrossRefPubMed

31.

Scrima A, Lubner MG, King S, Pankratz J, Kennedy G, Pickhardt PJ: Value of MDCT and clinical and laboratory data for predicting the need for surgical intervention in suspected small-bowel obstruction. AJR Am J Roentgenol 208(4): 785–793, 2017. https://doi.org/10.2214/AJR.16.16946 CrossRefPubMed

32.

Silva AC, Pimenta M, Guimarães LS: Small bowel obstruction: what to look for Radiographics: A review publication of the radiological. Society of North America, Inc 29(2): 423–439, 2009. https://doi.org/10.1148/rg.292085514 CrossRef

33.

Spuhler C (2006) Interactive centerline finding in complex tubular structures (16697)

34.

Spuhler C, Harders M, Szekely G: Fast and robust extraction of centerlines in 3d tubular structures using a scattered-snakelet approach. SPIE Medical Imaging 6144: 614442–614442–8, 2006. https://doi.org/10.1117/12.653169. http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=1277145 CrossRef

35.

Tall M, Choudhury KR, Napel S, Roos JE, Rubin GD: Accuracy of a remote eye tracker for radiologic observer studies: Effects of calibration and recording environment. Acad Radiol 19(2): 196–202, 2012. https://doi.org/10.1016/j.acra.2011.10.011. http://www.sciencedirect.com/science/article/pii/S1076633211004946 CrossRefPubMed

36.

Vaughan N, Gabrys B: Comparing and combining time series trajectories using dynamic time warping. Proc Comput Sci 96: 465–474, 2016. https://doi.org/10.1016/j.procs.2016.08.106. http://www.sciencedirect.com/science/article/pii/S187705091631907X CrossRef

37.

Vincent G, Guillard G, Bowes M (2012) Fully automatic segmentation of the prostate using active appearance models, Medical image computing and computer-assisted intervention – MICCAI 2012. http://promise12.grand-challenge.org/Results/displayFile?resultId=20120629193617_302_Imorphics_Results&;type=Public&;file=Imorphics.pdf

38.

Wang S, Summers RM: Machine learning and radiology. Med Image Anal 16(5): 933–951, 2012. https://doi.org/10.1016/j.media.2012.02.005 CrossRefPubMedPubMedCentral

39.

Yasaka K, Akai H, Abe O, Kiryu S: Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: A preliminary study. Radiology 286(3): 887–896, 2017. https://doi.org/10.1148/radiol.2017170706. https://pubs.rsna.org/doi/abs/10.1148/radiol.2017170706 CrossRefPubMed

40.

Yu L, Yang X, Chen H, Qin J, Heng PA (2017) Volumetric convnets with mixed residual connections for automated prostate segmentation from 3d mr images, 31st AAAI conference on artificial intelligence, pp 66–72

41.

Zar JH (2014) Biostatistical analysis, 5. ed., pearson new internat. ed edn. Always learning. Pearson Education Limited, Harlow. OCLC: 862984228

42.

Zhang W, Liu J, Yao J, Louie A, Nguyen TB, Wank S, Nowinski WL, Summers RM: Mesenteric vasculature-guided small bowel segmentation on 3d CT. IEEE Trans Med Imaging 32(11): 2006–2021, 2013. https://doi.org/10.1109/TMI.2013.2271487. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4224016/ CrossRefPubMedPubMedCentral

Title: Image Annotation by Eye Tracking: Accuracy and Precision of Centerlines of Obstructed Small-Bowel Segments Placed Using Eye Trackers
Authors: Alfredo Lucas
Kang Wang
Cynthia Santillan
Albert Hsiao
Claude B. Sirlin
Paul M. Murphy
Publication date: 01-10-2019
Publisher: Springer International Publishing
Published in: Journal of Imaging Informatics in Medicine / Issue 5/2019
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI: https://doi.org/10.1007/s10278-018-0169-5

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Image Annotation by Eye Tracking: Accuracy and Precision of Centerlines of Obstructed Small-Bowel Segments Placed Using Eye Trackers

Abstract

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 5/2019

CT-Based 3D Printing of the Glenoid Prior to Shoulder Arthroplasty: Bony Morphology and Model Evaluation

Hybrid Airway Segmentation Using Multi-Scale Tubular Structure Filters and Texture Analysis on 3D Chest CT Scans

Transcending Dimensions: a Comparative Analysis of Cloaca Imaging in Advancing the Surgeon’s Understanding of Complex Anatomy

Centralized Clinical Trial Imaging Data Management: Practical Guidance from a Comprehensive Cancer Center’s Experience

Impact of Patient Photos on Detection Accuracy, Decision Confidence and Eye-Tracking Parameters in Chest and Abdomen Images with Tubes and Lines

Generalizable Inter-Institutional Classification of Abnormal Chest Radiographs Using Efficient Convolutional Neural Networks