Top

Journal of Imaging Informatics in Medicine

Published in:

01-04-2020 | Original Paper

Automatic Staging of Cancer Tumors Using AIM Image Annotations and Ontologies

Authors: E. F. Luque, N. Miranda, D. L. Rubin, D. A. Moreira

Published in: Journal of Imaging Informatics in Medicine | Issue 2/2020

Abstract

A second opinion about cancer stage is crucial when clinicians assess patient treatment progress. Staging is a process that takes into account description, location, characteristics, and possible metastasis of tumors in a patient. It should follow standards, such as the TNM Classification of Malignant Tumors. However, in clinical practice, the implementation of this process can be tedious and error prone. In order to alleviate these problems, we intend to assist radiologists by providing a second opinion in the evaluation of cancer stage. For doing this, we developed a TNM classifier based on semantic annotations, made by radiologists, using the ePAD tool. It transforms the annotations (stored using the AIM format), using axioms and rules, into AIM4-O ontology instances. From then, it automatically calculates the liver TNM cancer stage. The AIM4-O ontology was developed, as part of this work, to represent annotations in the Web Ontology Language (OWL). A dataset of 51 liver radiology reports with staging data, from NCI’s Genomic Data Commons (GDC), were used to evaluate our classifier. When compared with the stages attributed by physicians, the classifier stages had a precision of 85.7% and recall of 81.0%. In addition, 3 radiologists from 2 different institutions manually reviewed a random sample of 4 of the 51 records and agreed with the tool staging. AIM4-O was also evaluated with good results. Our classifier can be integrated into AIM aware imaging tools, such as ePAD, to offer a second opinion about staging as part of the cancer treatment workflow.

https://wiki.nci.nih.gov/display/AIM/Extending+the+AIM+Model

http://groovy-lang.org/

https://wiki.nci.nih.gov/display/AIM/Extending+the+AIM+Model#ExtendingtheAIMModel-AIMUMLModeling

http://bioportal.bioontology.org/ontologies/RADLEX/classes

https://bioportal.bioontology.org/ontologies/ONLIRA

http://lucene.apache.org/

https://protege.stanford.edu/

https://lucene.apache.org/core/6_4_2/core/org/apache/lucene/analysis/standard/StandardAnalyzer.html

https://gdc.cancer.gov/

https://public.cancerimagingarchive.net/ncia/login.jsf

https://portal.gdc.cancer.gov/cases/52292ffc-0902-4d97-b461-20723987a177

Levy M, O’Connor MJ, Rubin DL: Semantic reasoning with image annotations for tumor assessment. AMIA Ann Symp Proc 2009: 359–63, 2009

Wennerberg P, Schulz K, Buitelaar P: Ontology modularization to improve semantic medical image annotation. J Biomed Inform 44: 155–162, 2011. https://doi.org/10.1016/j.jbi.2010.12.005 CrossRefPubMed

Bretschneider C, Zillner S, Hammon M (2013) Grammar-based lexicon enhancement for aligning German radiology text and images. In: Proceedings of the Recent Advances in Natural Language Processing (RANLP 2013), Hissar, Bulgaria, pp 105– 112

Zillner S: Reasoning-based Patient Classification for Enhanced Medical Image Annotation.. In: Extended Semantic Web Conference. Springer, Berlin, 2010, pp 243–257

Rubin DL, Willrett D, O’Connor MJ, Hage C, Kurtz C, Moreira DA: Automated Tracking of Quantitative Assessments of Tumor Burden in Clinical Trials. Transl Oncol 7: 23–35, 2014. https://doi.org/10.1593/tlo.13796 CrossRefPubMedPubMedCentral

Channin DS, Mongkolwat P, Kleper V, Rubin DL: The Annotation and Image Mark-up Project. Radiology 253: 590–592, 2009. https://doi.org/10.1148/radiol.2533090135. pMID: 19952021CrossRefPubMed

Channin DS, Mongkolwat P, Kleper V, Sepukar K, Rubin DL: The caBIG^TM Annotation and Image Markup Project. J Digit Imaging 23: 217–225, 2010. https://doi.org/10.1007/s10278-009-9193-9 CrossRefPubMed

Rubin DL, Rodriguez C, Shah P, Beaulieu C (2008) Ipad: semantic annotation and markup of radiological images. In: AMIA Annual Symposium Proceedings, pp 626–630

Kundu S, Itkin M, Gervais D. a., Krishnamurthy VN, Wallace MJ, Cardella JF, Rubin DL, Langlotz CP: The IR Radlex Project: An Interventional Radiology Lexicon-A Collaborative Project of the Radiological Society of North America and the Society of Interventional Radiology. J Vasc Interv Radiol 20: 433–435, 2009. https://doi.org/10.1016/j.jvir.2008.10.022 CrossRefPubMed

10.

Faria SC, Szklaruk J, Kaseb AO, Hassabo HM, Elsayes KM: TNM/Okuda/Barcelona/UNOS/CLIP International Multidisciplinary Classification of Hepatocellular Carcinoma: concepts, perspectives, and radiologic implications. Abdom Imaging 39: 1070–1087, 2014. https://doi.org/10.1007/s00261-014-0130-0 CrossRefPubMed

11.

Dameron O, Roques É, Rubin D, Marquet G, Burgun A (2006) Grading lung tumors using OWL-DL based reasoning, in: 9th International Protégé Conference-Presentation Abstracts. Stanford, USA: Stanford University, p 69. http://protege.stanford.edu/conference/2006/

12.

Marquet G, Dameron O, Saikali S, Mosser J, Burgun A (2007) Grading glioma tumors using OWL-DL and NCI thesaurus. In: AMIA Annual Symposium Proceedings, volume 2007, American Medical Informatics Association, pp 508–512. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2655830

13.

Kleihues P, Sobin LH: World Health Organization classification of tumors. Cancer 88: 2887–2887, 2000CrossRef

14.

Keet CM: Open World Assumption.. In: Encyclopedia of Systems Biology. Springer, Berlin, 2013, pp 1567–1567

15.

Tutac AE, Racoceanu D, Putti T, Xiong W, Leow WK, Cretu V: Knowledge-Guided Semantic Indexing of Breast Cancer Histopathology Images. 2008 International Conference on BioMedical Engineering and Informatics 2: 107–112, 2008. https://doi.org/10.1109/BMEI.2008.166 https://doi.org/10.1109/BMEI.2008.166 CrossRef

16.

Tutac AE, Cretu VI, Racoceanu D (2010) Spatial representation and reasoning in breast cancer grading ontology. In: 2010 International Joint Conference on Computational Cybernetics and Technical Informatics (ICCC-CONTI), pp 89–94. https://doi.org/10.1109/ICCCYB.2010.5491320

17.

Massicano F, Sasso A, Tomaz H, Oleynik M, Nobrega C, Patrão DFC (2015) An Ontology for TNM Clinical Stage Inference. In: ONTOBRAS

18.

Franca F, Schulz S, Bronsert P, Novais P, Boeker M (2015) Feasibility of an ontology driven tumor-node-metastasis classifier application: A study on colorectal cancer. In: 2015 International Symposium on Innovations in Intelligent SysTems and Applications (INISTA), pp 1–7. https://doi.org/10.1109/INISTA.2015.7276757

19.

Zillner S (2009) Towards the Ontology-based Classification of Lymphoma Patients using Semantic Image Annotations. In: SWAT4LS, Citeseer

20.

Boeker M, França F, Bronsert P, Schulz S: TNM-O: ontology support for staging of malignant tumours. Journal of Biomedical Semantics 7: 64, 2016. https://doi.org/10.1186/s13326-016-0106-9 CrossRefPubMedPubMedCentral

21.

Boeker M, Faria R, Schulz S (2014) A Proposal for an Ontology for the Tumor-Node-Metastasis Classification of Malignant Tumors: a Study on Breast Tumors, et al., Ontologies and Data in Life Sciences (ODLS 2014), IMISEREPORTS Leipzig. http://www.onto-med.de/obml/ws2014/odls2014report.pdf

22.

Seneviratne O, Rashid SM, Chari S, McCusker JP, Bennett KP, Hendler JA, McGuinness DL (2018) Knowledge Integration for Disease Characterization: A Breast Cancer Example. arXiv:1807.07991

23.

Meriem B, Yamina T, Pathology A (2012) Interpretation breast cancer imaging by using ontology, Cyber Journals: Multidisciplinary Journals in Science and Technology. Journal of Selected Areas in Bioengineering (JSAB). pp 1–6. http://www.cyberjournals.com/Papers/Mar2012/06.pdf

24.

Racoceanu D, Capron F: Towards semantic-driven high-content image analysis: An operational instantiation for mitosis detection in digital histopathology. Comput Med Imaging Graph 42: 2–15, 2015. https://doi.org/10.1016/j.compmedimag.2014.09.004 CrossRefPubMed

25.

Möller M., Sonntag D, Erñst P: A Spatio-anatomical Medical Ontology and Automatic Plausibility Checks Berlin: Springer, 2013, pp 41–55. https://doi.org/10.1007/978-3-642-29764-9-3

26.

Zillner S, Sonntag D: Image metadata reasoning for improved clinical decision support. Network Modeling Analysis in Health Informatics and Bioinformatics 1: 37–46, 2012. https://doi.org/10.1007/s13721-012-0003-9 CrossRef

27.

Gimenez F, Xu J, Liu Y, Liu TT, Beaulieu CF, Rubin DL, Napel S (2011) On the Feasibility of Predicting Radiological Observations from Computational Imaging Features of Liver Lesions in CT Scans, in: 2011 First IEEE International Conference on Healthcare Informatics, Imaging and Systems Biology (HISB), pp 346–350. https://doi.org/10.1109/HISB.2011.37

28.

Kurtz C, Depeursinge A, Napel S, Beaulieu CF, Rubin DL: On combining image-based and ontological semantic dissimilarities for medical image retrieval applications. Med Image Anal 18: 1082–1100, 2014. https://doi.org/10.1016/j.media.2014.06.009 CrossRef

29.

Bulu H, Rubin DL (2015) Java Application Programming Interface (API) for Annotation Imaging Markup (AIM). 1–12. https://sourceforge.net/projects/aimapi/, [Online; accessed 02-March-2015.]

30.

Rubin DL: Finding the meaning in images: annotation and image markup. Philos Psychiatry Psychol 18: 311–318, 2011. https://doi.org/10.1353/ppp.2011.0045 CrossRef

31.

Radlex (2016) Radlex, http://www.radlex.org. [Online; accessed 02-October-2016.]

32.

Kokciyan N, Turkay R, Uskudarli S, Yolum P, Bakir B, Acar B: Semantic description of liver ct images: An ontological approach. IEEE J Biomed Health Inform 18: 1363–1369, 2014. https://doi.org/10.1109/JBHI.2014.2298880 CrossRefPubMed

33.

Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, et al: The Cancer Imaging Archive (tcia): Maintaining and Operating a Public Information Repository. J Digit Imaging 26: 1045–1057, 2013. https://doi.org/10.1007/s10278-013-9622-7 CrossRefPubMedPubMedCentral

34.

Blomqvist E, Seil Sepour A, Presutti V: Ontology Testing - Methodology and Tool.. In: Proceedings of the 18th International Conference on Knowledge Engineering and Knowledge Management, EKAW12. Springer, Berlin, 2012, pp 216–226

35.

Depeursinge A, Kurtz C, Beaulieu C, Napel S, Rubin D: Predicting Visual Semantic Descriptive Terms from Radiological Image Data: Preliminary Results With Liver Lesions in CT. IEEE Trans Med Imaging 33: 1669–1676, 2014. https://doi.org/10.1109/TMI.2014.2321347 CrossRefPubMedPubMedCentral

Title: Automatic Staging of Cancer Tumors Using AIM Image Annotations and Ontologies
Authors: E. F. Luque
N. Miranda
D. L. Rubin
D. A. Moreira
Publication date: 01-04-2020
Publisher: Springer International Publishing
Published in: Journal of Imaging Informatics in Medicine / Issue 2/2020
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI: https://doi.org/10.1007/s10278-019-00251-x

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Automatic Staging of Cancer Tumors Using AIM Image Annotations and Ontologies

Abstract

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 2/2020

Musculoskeletal Radiology Reports: Overlooked or Valuable?

Deep Learning for Low-Dose CT Denoising Using Perceptual Loss and Edge Detection Layer

Crowdsourcing pneumothorax annotations using machine learning annotations on the NIH chest X-ray dataset

Three-Dimensional Texture Feature Analysis of Pulmonary Nodules in CT Images: Lung Cancer Predictive Models Based on Support Vector Machine Classifier

A Block Adaptive Near-Lossless Compression Algorithm for Medical Image Sequences and Diagnostic Quality Assessment

Assessment of Dynamic Change of Coronary Artery Geometry and Its Relationship to Coronary Artery Disease, Based on Coronary CT Angiography