Skip to main content
Top
Published in: European Radiology 12/2012

01-12-2012 | Computer Applications

Intelligent image retrieval based on radiology reports

Authors: Axel Gerstmair, Philipp Daumke, Kai Simon, Mathias Langer, Elmar Kotter

Published in: European Radiology | Issue 12/2012

Login to get access

Abstract

Objectives

To create an advanced image retrieval and data-mining system based on in-house radiology reports.

Methods

Radiology reports are semantically analysed using natural language processing (NLP) techniques and stored in a state-of-the-art search engine. Images referenced by sequence and image number in the reports are retrieved from the picture archiving and communication system (PACS) and stored for later viewing. A web-based front end is used as an interface to query for images and show the results with the retrieved images and report text. Using a comprehensive radiological lexicon for the underlying terminology, the search algorithm also finds results for synonyms, abbreviations and related topics.

Results

The test set was 108 manually annotated reports analysed by different system configurations. Best results were achieved using full syntactic and semantic analysis with a precision of 0.929 and recall of 0.952. Operating successfully since October 2010, 258,824 reports have been indexed and a total of 405,146 preview images are stored in the database.

Conclusions

Data-mining and NLP techniques provide quick access to a vast repository of images and radiology reports with both high precision and recall values. Consequently, the system has become a valuable tool in daily clinical routine, education and research.

Key Points

Radiology reports can now be analysed using sophisticated natural language-processing techniques.
Semantic text analysis is backed by terminology of a radiological lexicon.
The search engine includes results for synonyms, abbreviations and compositions.
Key images are automatically extracted from radiology reports and fetched from PACS.
Such systems help to find diagnoses, improve report quality and save time.
Literature
2.
go back to reference Ramaswamy MR, Patterson DS, Yin L, Goodacre BW (1996) MoSearch: a radiologist-friendly tool for finding-based diagnostic report and image retrieval. Radiographics 16:923–933PubMed Ramaswamy MR, Patterson DS, Yin L, Goodacre BW (1996) MoSearch: a radiologist-friendly tool for finding-based diagnostic report and image retrieval. Radiographics 16:923–933PubMed
3.
go back to reference Dreyer KJ, Kalra MK, Maher MM et al (2005) Application of recently developed computer algorithm for automatic classification of unstructured radiology reports: validation study. Radiology 234:323–329PubMedCrossRef Dreyer KJ, Kalra MK, Maher MM et al (2005) Application of recently developed computer algorithm for automatic classification of unstructured radiology reports: validation study. Radiology 234:323–329PubMedCrossRef
4.
go back to reference Friedman C, Alderson PO, Austin JHM, Cimino JJ, Johnson SB (1994) A general natural-language text processor for clinical radiology. J Am Med Inform Assoc 1:161–174PubMedCrossRef Friedman C, Alderson PO, Austin JHM, Cimino JJ, Johnson SB (1994) A general natural-language text processor for clinical radiology. J Am Med Inform Assoc 1:161–174PubMedCrossRef
5.
go back to reference Hripcsak G, Friedman C, Alderson PO et al (1995) Unlocking clinical data from narrative reports: a study of natural language processing. Ann Intern Med 122:681–688PubMed Hripcsak G, Friedman C, Alderson PO et al (1995) Unlocking clinical data from narrative reports: a study of natural language processing. Ann Intern Med 122:681–688PubMed
6.
go back to reference Do BH, Wu A, Biswal S, Kamaya A, Rubin DL (2010) Informatics in radiology: RADTF: a semantic search-enabled, natural language processor-generated radiology teaching file. Radiographics 30:2039–2048PubMedCrossRef Do BH, Wu A, Biswal S, Kamaya A, Rubin DL (2010) Informatics in radiology: RADTF: a semantic search-enabled, natural language processor-generated radiology teaching file. Radiographics 30:2039–2048PubMedCrossRef
7.
go back to reference Mendonça EA, Haas J, Shagina L, Larson E, Friedman C (2005) Extracting information on pneumonia in infants using natural language processing of radiology reports. J Biomed Inform 38:314–321PubMedCrossRef Mendonça EA, Haas J, Shagina L, Larson E, Friedman C (2005) Extracting information on pneumonia in infants using natural language processing of radiology reports. J Biomed Inform 38:314–321PubMedCrossRef
8.
go back to reference Schulz S, Daumke P, Fischer P, Müller M, Müller ML (2008) Evaluation of a document search engine in a clinical department system. AMIA Annu Symp Proc 647–651 Schulz S, Daumke P, Fischer P, Müller M, Müller ML (2008) Evaluation of a document search engine in a clinical department system. AMIA Annu Symp Proc 647–651
9.
go back to reference Dang PA, Kalra MK, Schultz TJ, Graham SA, Dreyer KJ (2009) Informatics in radiology: Render: an online searchable radiology study repository. Radiographics 29:1233–1246PubMedCrossRef Dang PA, Kalra MK, Schultz TJ, Graham SA, Dreyer KJ (2009) Informatics in radiology: Render: an online searchable radiology study repository. Radiographics 29:1233–1246PubMedCrossRef
10.
go back to reference Erinjeri JP, Picus D, Prior FW, Rubin DA, Koppel P (2008) Development of a Google-based search engine for data mining radiology reports. J Digit Imaging 22:348–356PubMedCrossRef Erinjeri JP, Picus D, Prior FW, Rubin DA, Koppel P (2008) Development of a Google-based search engine for data mining radiology reports. J Digit Imaging 22:348–356PubMedCrossRef
13.
go back to reference Wermter J, Hahn U (2004) An Annotated German-Language Medical Text Corpus as Language Resource, Presented at the International Conference on Language Resources and Evaluation Wermter J, Hahn U (2004) An Annotated German-Language Medical Text Corpus as Language Resource, Presented at the International Conference on Language Resources and Evaluation
14.
15.
go back to reference Markó K, Schulz S, Medelyan O, Hahn U (2005) Bootstrapping Dictionaries for Cross-Language Information Retrieval, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 528–535, Salvador, Brazil Markó K, Schulz S, Medelyan O, Hahn U (2005) Bootstrapping Dictionaries for Cross-Language Information Retrieval, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 528–535, Salvador, Brazil
16.
go back to reference Markó K, Schulz S, Hahn U (2005) MorphoSaurus—design and evaluation of an interlingua-based, cross-language document retrieval engine for the medical domain. Methods Inf Med 44:537–545PubMed Markó K, Schulz S, Hahn U (2005) MorphoSaurus—design and evaluation of an interlingua-based, cross-language document retrieval engine for the medical domain. Methods Inf Med 44:537–545PubMed
17.
go back to reference Chapman WW, Bridewell W, Hanbury P, Cooper GF, Buchanan BG (2001) A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform 34:301–310PubMedCrossRef Chapman WW, Bridewell W, Hanbury P, Cooper GF, Buchanan BG (2001) A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform 34:301–310PubMedCrossRef
18.
go back to reference Huang Y, Lowe HJ (2005) A grammar-based classification of negations in clinical radiology reports. AMIA Annu Symp Proc 2005:988–988 Huang Y, Lowe HJ (2005) A grammar-based classification of negations in clinical radiology reports. AMIA Annu Symp Proc 2005:988–988
19.
go back to reference Huang Y, Lowe HJ (2007) A novel hybrid approach to automated negation detection in clinical radiology reports. J Am Med Inform Assoc 14:304–311PubMedCrossRef Huang Y, Lowe HJ (2007) A novel hybrid approach to automated negation detection in clinical radiology reports. J Am Med Inform Assoc 14:304–311PubMedCrossRef
20.
go back to reference Wu AS, Do BH, Kim J, Rubin DL (2009) Evaluation of negation and uncertainty detection and its impact on precision and recall in search. J Digit Imaging 24:234–242PubMedCrossRef Wu AS, Do BH, Kim J, Rubin DL (2009) Evaluation of negation and uncertainty detection and its impact on precision and recall in search. J Digit Imaging 24:234–242PubMedCrossRef
24.
go back to reference Tanenblatt M, Coden A, Sominsky I (2010) The ConceptMapper Approach to Named Entity Recognition, Presented at the International Conference on Language Resources and Evaluation Tanenblatt M, Coden A, Sominsky I (2010) The ConceptMapper Approach to Named Entity Recognition, Presented at the International Conference on Language Resources and Evaluation
25.
go back to reference Rector AL (1999) Clinical terminology: why is it so hard? Methods Inf Med 38:239–252PubMed Rector AL (1999) Clinical terminology: why is it so hard? Methods Inf Med 38:239–252PubMed
30.
go back to reference Lim CCT, Yang GL, Nowinski WL, Hui F (2003) Medical image resource center—making electronic teaching files from PACS. J Digit Imaging 16:331–336PubMedCrossRef Lim CCT, Yang GL, Nowinski WL, Hui F (2003) Medical image resource center—making electronic teaching files from PACS. J Digit Imaging 16:331–336PubMedCrossRef
31.
go back to reference Kahn CE, Thao C (2007) GoldMiner: a radiology image search engine. Am J Roentgenol 188:1475–1478CrossRef Kahn CE, Thao C (2007) GoldMiner: a radiology image search engine. Am J Roentgenol 188:1475–1478CrossRef
32.
go back to reference Ekins J (2007) What is STATdx. S Afr J Radiol 11:110–111 Ekins J (2007) What is STATdx. S Afr J Radiol 11:110–111
33.
go back to reference Savova GK, Masanz JJ, Ogren PV et al (2010) Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc 17:507–513PubMedCrossRef Savova GK, Masanz JJ, Ogren PV et al (2010) Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc 17:507–513PubMedCrossRef
34.
go back to reference Eisenhauer EA, Therasse P, Bogaerts J et al (2009) New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 45:228–247PubMedCrossRef Eisenhauer EA, Therasse P, Bogaerts J et al (2009) New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 45:228–247PubMedCrossRef
35.
go back to reference Rubin DL, Desser TS (2008) A data warehouse for integrating radiologic and pathologic data. J Am Coll Radiol 5:210–217PubMedCrossRef Rubin DL, Desser TS (2008) A data warehouse for integrating radiologic and pathologic data. J Am Coll Radiol 5:210–217PubMedCrossRef
36.
go back to reference Wong STC, Hoo KS Jr, Cao X et al (2004) A neuroinformatics database system for disease-oriented neuroimaging research. Acad Radiol 11:345–358PubMedCrossRef Wong STC, Hoo KS Jr, Cao X et al (2004) A neuroinformatics database system for disease-oriented neuroimaging research. Acad Radiol 11:345–358PubMedCrossRef
Metadata
Title
Intelligent image retrieval based on radiology reports
Authors
Axel Gerstmair
Philipp Daumke
Kai Simon
Mathias Langer
Elmar Kotter
Publication date
01-12-2012
Publisher
Springer-Verlag
Published in
European Radiology / Issue 12/2012
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-012-2608-x

Other articles of this Issue 12/2012

European Radiology 12/2012 Go to the issue