Top

Journal of Imaging Informatics in Medicine

Published in:

01-04-2011

Content-Based Image Retrieval in Radiology: Current Status and Future Directions

Authors: Ceyhun Burak Akgül, Daniel L. Rubin, Sandy Napel, Christopher F. Beaulieu, Hayit Greenspan, Burak Acar

Published in: Journal of Imaging Informatics in Medicine | Issue 2/2011

Abstract

Diagnostic radiology requires accurate interpretation of complex signals in medical images. Content-based image retrieval (CBIR) techniques could be valuable to radiologists in assessing medical images by identifying similar images in large archives that could assist with decision support. Many advances have occurred in CBIR, and a variety of systems have appeared in nonmedical domains; however, permeation of these methods into radiology has been limited. Our goal in this review is to survey CBIR methods and systems from the perspective of application to radiology and to identify approaches developed in nonmedical applications that could be translated to radiology. Radiology images pose specific challenges compared with images in the consumer domain; they contain varied, rich, and often subtle features that need to be recognized in assessing image similarity. Radiology images also provide rich opportunities for CBIR: rich metadata about image semantics are provided by radiologists, and this information is not yet being used to its fullest advantage in CBIR systems. By integrating pixel-based and metadata-based image feature analysis, substantial advances of CBIR in medicine could ensue, with CBIR systems becoming an important tool in radiology practice.

Rubin GD: Data explosion: the challenge of multidetector-row CT. Eur J Radiol 36(2):74–80, 2000PubMedCrossRef

Siegle RL, et al: Rates of disagreement in imaging interpretation in a group of community hospitals. Acad Radiol 5(3):148–154, 1998PubMedCrossRef

Barlow WE, et al: Accuracy of screening mammography interpretation by characteristics of radiologists. J Natl Cancer Inst 96(24):1840–1850, 2004PubMedCrossRef

McDonald CJ: Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med 124:56–62, 1996PubMed

Doi K: Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph 31(4–5):198–211, 2007PubMedCrossRef

Burnside ES, et al: Bayesian network to predict breast cancer risk of mammographic microcalcifications and reduce number of benign biopsy results: initial experience. Radiology 240(3):666–673, 2006PubMedCrossRef

Rubin DL, Burnside ES, Shachter R: A Bayesian Network to assist mammography interpretation. In: M.L. Brandeau, F.S. F, and W.P. Pierskalla, Eds. Operations Research and Health Care. Kluwer Academic Publishers: Boston, 2004, pp 695–720

Kahn CE: Artificial intelligence in radiology: decision support systems. Radiographics 14:849–861, 1994PubMed

Datta R, et al: Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2), 2008

10.

Muller H, et al: A review of content-based image retrieval systems in medical applications—clinical benefits and future directions. Int J Med Informatics 73(1):1–23, 2004CrossRef

11.

Smeulders AWM, et al: Content-based image retrieval at the end of the early years. IEEE Trans Patt Anal Mach Intell 22(12):1349–1380, 2000CrossRef

12.

Swain MJ, Ballard DH: Color indexing. Int J Comput Vis 7(1):11–32, 1991CrossRef

13.

Comaniciu D, Meer P, Foran DJ: Image-guided decision support system for pathology. Mach Vis Appl 11(4):213–224, 1999CrossRef

14.

Kwak DM, et al: Content-based ultrasound image retrieval using a coarse to fine approach. Ann N Y Acad Sci 980:212–224, 2002PubMedCrossRef

15.

Lim J, Chevallet J-P: Vismed: A visual vocabulary approach for medical image indexing and retrieval. in Second Asia Information Retrieval Symposium. 2005. Jeju Island, Korea

16.

Shyu CR, et al: ASSERT: a physician-in-the-loop content-based image retrieval system for HRCT image databases. Comput Vis Image Underst 75(1/2):111–132, 1999CrossRef

17.

Cauvin JM, et al: Computer-assisted diagnosis system in digestive endoscopy. IEEE Trans Inf Technol Biomed 7(4):256–262, 2003PubMedCrossRef

18.

Güld MO, et al: Content-Based Retrieval of Medical Images by Combining Global Features. Accessing Multilingual Information Repositories. in Accessing Multilingual Information Repositories. 2005: Springer LNCS 4022

19.

Lubbers K, et al: A Probabilistic Approach to Medical Image Retrieval, in Multilingual Information Access for Text, Speech and Images. Springer Berlin, 2005, pp 761–772

20.

Doyle S, et al: Using manifold learning for content-based image retrieval of prostate histopathology. in MICCAI 2007 Workshop on Content-Based Image Retrieval for Biomedical Image Archives. Brisbane, Australia, 2007

21.

Gletsos M, et al: A computer-aided diagnostic system to characterize CT focal liver lesions: design and optimization of a neural network classifier. IEEE Trans Inf Technol Biomed 7(3):153–162, 2003PubMedCrossRef

22.

Zhu H, et al: A new local multiscale Fourier analysis for medical imaging. Med Phys 30:1134–1141, 2003PubMedCrossRef

23.

Oliveira MC, Cirne W, Marques PDA: Towards applying content-based image retrieval in the clinical routine. Future Gener Comput Syst 23(3):466–474, 2007CrossRef

24.

Rahman M, Bhattacharya P, Desai BC: A framework for medical image retrieval using machine learning and statistical similarity matching techniques with relevance feedback. IEEE Trans Inf Technol Biomed 11(1):58–69, 2007PubMedCrossRef

25.

Mao J, Jain AK: Texture classification and segmentation using multiresolution simultaneous autoregressive models. Pattern Recogn 25(2):173–188, 1992CrossRef

26.

Iqbal Q, Aggarwal JK: Combining structure, color and texture for image retrieval: a performance evaluation. in International Conference on Pattern Recognition (ICPR). Quebec City, Canada, 2002

27.

Belongie S, Malik J, Puzicha J: Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Machine Intelligence 24(4):509–522, 2002CrossRef

28.

Akgul CB, et al: 3D model retrieval using probability density-based shape descriptors. IEEE Trans Pattern Anal Machine Intelligence 31(6):1117–1133, 2009CrossRef

29.

Gokturk SB, et al: A statistical 3-D pattern processing method for computer-aided detection of polyps in CT colonography. IEEE Trans Med Imag 20(12):1251–1260, 2001CrossRef

30.

Yoshida H, Nappi J: Three-dimensional computer-aided diagnosis scheme for detection of colonic polyps. IEEE Trans Med Imag 20(12):1261–1274, 2001CrossRef

31.

Rubin GD, et al: Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 234(1):274–283, 2005PubMedCrossRef

32.

Alto H, Rangayyan RM, Desautels JEL: Content-based retrieval and analysis of mammographic masses. Journal of Electronic Imaging 14(2):1–17, 2005CrossRef

33.

Qian XN, Tagare HD: Optimal embedding for shape indexing in medical image databases. in Medical Image Computing and Computer-Assisted Intervention (MICCAI). 2005

34.

Antani S, et al: Evaluation of shape similarity measurement methods for spine X-ray images. J Vis Commun Image Represent 15(3):285–302, 2004CrossRef

35.

Balmachnova E, et al: Content-based image retrieval by means of scale-space top-points and differential invariants. in MICCAI 2007 Workshop on Content-based Image Retrieval for Biomedical Image Archives. Brisbane, Australia, 2007

36.

Golland P, et al: Detection and analysis of statistical differences in anatomical shape. Med Image Anal 9(1):69–86, 2005PubMedCrossRef

37.

Bansal R, et al: Statistical analyses of brain surfaces using Gaussian random fields on 2-D manifolds. IEEE Trans Med Imaging 26(1):46–57, 2007PubMedCrossRef

38.

Toews M, Arbel T: A statistical parts-based model of anatomical variability. IEEE Trans Med Imag 26(4):497–508, 2007CrossRef

39.

Wang JZ: Pathfinder: multiresolution region-based searching of pathology images using IRM. in AMIA Symp. 2000

40.

Pokrajac D, et al: Applying spatial distribution analysis techniques to classification of 3D medical images. Artif Intell Med 33(3):261–280, 2005PubMedCrossRef

41.

Nielsen J, Nelson M, Liu L: Image-matching as a medical diagnostic support tool (DST) for brain diseases in children. Comput Med Imaging Graph 29(2/3):195–202, 2005PubMedCrossRef

42.

Sasso G, et al: A visual query-by-example image database for chest CT images: potential role as a decision and educational support tool for radiologists. J Digit Imaging 18(1):78–84, 2005PubMedCrossRef

43.

Petrakis EGM, Faloutsos C, Lin KI: ImageMap: an image indexing method based on spatial similarity. IEEE Trans Knowl Data Eng 14(5):979–987, 2002CrossRef

44.

Shantanu HJ, Washington M: Statistical shape analysis: clustering, learning, and testing. IEEE Trans Pattern Anal Mach Intell 27(4):590–602, 2005CrossRef

45.

Tong L, Hongbin Z: Riemannian manifold learning. IEEE Trans Pattern Anal Mach Intell 30(5):796–809, 2008CrossRef

46.

Zhang J, et al: Object representation and recognition in shape spaces. Pattern Recogn 36(5):1143–1154, 2003CrossRef

47.

Rubner Y, Tomasi C, Guibas LJ: The earth mover's distance as a metric for image retrieval. Int J Comput Vis 40(2):99–121, 2000CrossRef

48.

Bunke H, Irniger C, Neuhaus M: Graph Matching—Challenges and Potential Solutions, in Image Analysis and Processing–ICIAP 2005, 2005, pp 1–10

49.

Glaunès J, et al: Large deformation diffeomorphic metric curve mapping. Int J Comput Vis 80(3):317–336, 2008PubMedCrossRef

50.

Veltkamp RC: Shape matching: similarity measures and algorithms. in Shape Modeling and Applications, SMI 2001 International Conference on, 2001

51.

Akgül CB, et al: Similarity learning for 3D object retrieval using relevance feedback and risk minimization. To appear in Int. Journal of Computer Vision, 2010

52.

Rahmani R, et al: Localized content-based image retrieval. IEEE Trans Pattern Anal Mach Intell 30(11):1902–1912, 2008PubMedCrossRef

53.

Rui Y, et al: Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans Circuits Syst Video Technol 8(5):644–655, 1998CrossRef

54.

Tao D, et al: Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Trans Pattern Anal Mach Intell 28(7):1088–1099, 2006PubMedCrossRef

55.

Keysers D, et al: Statistical framework for model-based image retrieval in medical applications. J Electron Imaging 12(1):59–68, 2003CrossRef

56.

Müller H, et al: The Use of MedGIFT and EasyIR for ImageCLEF 2005. in Accessing Multilingual Information Repositories. 2005: Springer LNCS 4022

57.

Mohammad-Reza S, et al: Content-based image database system for epilepsy. Comput Methods Programs Biomed 79(3):209–226, 2005CrossRef

58.

El-Naqa I, et al: A similarity learning approach to content-based image retrieval: application to digital mammography. IEEETransactions On Medical Imaging 23(10):1233–1244, 2004CrossRef

59.

Aschkenasy VS, et al: Unsupervised image classification of medical ultrasound data by multiresolution elastic registration. Ultrasound Med Biol 32(7):1047–1054, 2006PubMedCrossRef

60.

Kim J, et al: A new way for multidimensional medical data management: Volume of interest (VOI)-based retrieval of medical images with visual and functional features. IEEE Trans Inf Technol Biomed 10(3):598–607, 2006PubMedCrossRef

61.

Amores J, Radeva P: Registration and retrieval of highly elastic bodies using contextual information. Pattern Recognit Lett 26(11):1720–1731, 2005CrossRef

62.

Chin Y, et al: An automatic liver segmentation initialization information retrieval strategy for a CBIR followed by a new liver volume segmentation method for CT and MRI image datasets. in MICCAI 2007 Workshop on Content-based Image Retrieval for Biomedical Image Archives. 2007. Brisbane, Australia

63.

Miller MI, Younes L: Group actions, homeomorphisms, and matching: a general framework. Int J Comput Vis 41(1):61–84, 2001CrossRef

64.

Carson C, et al: Blobworld: image segmentation using expectation-maximization and its application to image querying. IEEE Trans Pattern Anal Mach Intell 24(8):1026–1038, 2002CrossRef

65.

Korn P, et al: Fast and effective retrieval of medical tumor shapes. IEEE Trans Knowl Data Eng 10(6):889–904, 1998CrossRef

66.

Greenspan H, Pinhas AT: Medical image categorization and retrieval for PACS using the GMM-KL framework. IEEE Trans Inf Technol Biomed 11(2):190–202, 2007PubMedCrossRef

67.

Avni U, et al., X_ray image categorization and retrieval using patch-based visual words representation, in IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI). 2009

68.

Brown R, et al: The use of magnetic resonance imaging to noninvasively detect genetic signatures in oligodendroglioma. Clin Cancer Res 14(8):2357–2362, 2008PubMedCrossRef

69.

Badr Y, Chbeir R: Automatic Image Description Based on Textual Data, in Journal on Data Semantics VII. 2006. pp. 196–218

70.

Besançon R, et al: Cross-Media Feedback Strategies: Merging Text and Image Information to Improve Image Retrieval, in Multilingual Information Access for Text, Speech and Images. 2005. pp. 709–717

71.

Barb AS, Chi-Ren S, Sethi YP: Knowledge representation and sharing using visual semantic modeling for diagnostic medical image databases. IEEE Trans Inf Technol Biomed 9(4):538–553, 2005PubMedCrossRef

72.

Langlotz CP: RadLex: a new method for indexing online educational materials. RadioGraphics 26(6):1595–1597, 2006PubMedCrossRef

73.

Rubin DL, et al: Medical Imaging on the Semantic Web: Annotation and Image Markup. in AAAI Spring Symposium Series, Semantic Scientific Knowledge Integration. 2008. Stanford University

74.

Syeda-Mahmood T, et al: AALIM: Multimodal Mining for Cardiac Decision Support. Comput Cardiol 1 and 2:209–212, 2007CrossRef

75.

Syeda-Mahmood T, Beymer D, Wang F: Shape-based matching of ECG recordings. 2007 Annual International Conference of the Ieee Engineering in Medicine and Biology Society, Vols 1–16:2012–2018, 2007

76.

Syeda-Mahmood T, et al: Characterizing spatio-temporal patterns for disease discrimination in cardiac echo videos. Medical Image Computing and Computer-Assisted Intervention–MICCAI, Pt 1. Proceedings 4791:261–269, 2007

77.

Syeda-Mahmood T, Beymer D, Amir A: Disease-specific extraction of text from cardiac echo videos for decision support, in Intl. Conf on Document Analysis and Recognition (ICDAR). 2009

Title: Content-Based Image Retrieval in Radiology: Current Status and Future Directions
Authors: Ceyhun Burak Akgül
Daniel L. Rubin
Sandy Napel
Christopher F. Beaulieu
Hayit Greenspan
Burak Acar
Publication date: 01-04-2011
Publisher: Springer-Verlag
Published in: Journal of Imaging Informatics in Medicine / Issue 2/2011
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI: https://doi.org/10.1007/s10278-010-9290-9

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Content-Based Image Retrieval in Radiology: Current Status and Future Directions

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 2/2011

Assessment of Performance and Reliability of Computer-Aided Detection Scheme Using Content-Based Image Retrieval Approach and Limited Reference Database

TCP/IP Optimization over Wide Area Networks: Implications for Teleradiology

Freehand Liver Volumetry by Using an Electromagnetic Pen Tablet: Accuracy, Precision, and Rapidity

Will the Next Generation of PACS Be Sitting on a Cloud?

Design of a Web-Tool for Diagnostic Clinical Trials Handling Medical Imaging Research

Improving Healthcare Delivery Through Patient Informatics and Quality Centric Data