Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
European Radiology
Published in: European Radiology 11/2020

Open Access 01-11-2020 | Biomarkers | Imaging Informatics and Artificial Intelligence

Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform

Authors: Isabella Fornacon-Wood, Hitesh Mistry, Christoph J. Ackermann, Fiona Blackhall, Andrew McPartlin, Corinne Faivre-Finn, Gareth J. Price, James P. B. O’Connor

Published in: European Radiology | Issue 11/2020

Login to get access

Abstract

Objective

To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.

Methods

The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset.

Results

The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.

Conclusion

IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics.

Key Points

• Reliability of radiomic features varies between feature calculation platforms and with choice of software version.
• Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised.
• IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features.
Appendix
Available only for authorised users
Literature
1.
Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577CrossRef
2.
Lambin P, Leijenaar RT, Deist TM et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749–762CrossRef
3.
Aerts HJWL (2016) The potential of radiomic-based phenotyping in precision medicine: a review. JAMA Oncol 2:1636–1642CrossRef
4.
O’Connor JPB, Aboagye EO, Adams JE et al (2017) Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol 14:169–186CrossRef
5.
Traverso A, Wee L, Dekker A, Gillies R (2018) Repeatability and reproducibility of radiomic features: a systematic review. Int J Radiat Oncol Biol Phys 102:1143–1158CrossRef
6.
Midya A, Chakraborty J, Gönen M, Do RKG, Simpson AL (2018) Influence of CT acquisition and reconstruction parameters on radiomic feature reproducibility. J Med Imaging (Bellingham) 5:011020
7.
Ger RB, Meier JG, Pahlka RB et al (2019) Effects of alterations in positron emission tomography imaging parameters on radiomics features. PLoS One 14:e0221877CrossRef
8.
Pavic M, Bogowicz M, Würms X et al (2018) Influence of inter-observer delineation variability on radiomics stability in different tumor sites. Acta Oncol 57:1070–1074CrossRef
9.
Haga A, Takahashi W, Aoki S et al (2018) Classification of early stage non-small cell lung cancers on computed tomographic images into histological types using radiomic features: interobserver delineation variability analysis. Radiol Phys Technol 11:27–35CrossRef
10.
He L, Huang Y, Ma Z, Liang C, Liang C, Liu Z (2016) Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule. Sci Rep 6:1–10
11.
Aerts HJWL, Velazquez ER, Leijenaar RTH et al (2014) Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 5:4006CrossRef
12.
Apte AP, Iyer A, Crispin-Ortuzar M et al (2018) Technical note: extension of CERR for computational radiomics: a comprehensive MATLAB platform for reproducible radiomics research. Med Phys 45:3713–3720CrossRef
13.
Zhang L, Fried DV, Fave XJ, Hunter LA, Yang J, Court LE (2015) IBEX: An open infrastructure software platform to facilitate collaborative work in radiomics. Med Phys 42:1341–1353
14.
van Griethuysen JJM, Fedorov A, Parmar C et al (2017) Computational radiomics system to decode the radiographic phenotype. Cancer Res 77:e104–e107
15.
Nioche C, Orlhac F, Boughdad S et al (2018) Lifex: a freeware for radiomic feature calculation in multimodality imaging to accelerate advances in the characterization of tumor heterogeneity. Cancer Res 78:4786–4789CrossRef
16.
Court LE, Fave X, Mackin D, Lee J, Yang J, Zhang L (2016) Computational resources for radiomics. Transl Cancer Res 5:340–348
17.
Wilkinson MD, Dumontier M, Aalbersberg IJ et al (2016) Comment: the FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3:160018CrossRef
18.
Foy JJ, Robinson KR, Li H, Giger ML, Al-Hallaq H, Armato SG (2018) Variation in algorithm implementation across radiomics software. J Med Imaging (Bellingham) 5:044505
19.
Bogowicz M, Leijenaar RTH, Tanadini-Lang S et al (2017) Post-radiochemotherapy PET radiomics in head and neck cancer – the influence of radiomics implementation on the reproducibility of local control tumor models. Radiother Oncol 125:385–391CrossRef
20.
Liang ZG, Tan HQ, Zhang F et al (2019) Comparison of radiomics tools for image analyses and clinical prediction in nasopharyngeal carcinoma. Br J Radiol 92:20190271CrossRef
21.
Zwanenburg A, Leger S, Vallières M, Löck S (2019) Image biomarker standardisation initiative arXiv Prepr arXiv161207003
22.
Hatt M, Vallieres M, Visvikis D, Zwanenburg A (2018) IBSI: an international community radiomics standardization initiative. J Nucl Med 59:287–287
23.
Faivre-Finn C, Snee M, Ashcroft L et al (2017) Concurrent once-daily versus twice-daily chemoradiotherapy in patients with limited-stage small-cell lung cancer (CONVERT): an open-label, phase 3, randomised, superiority trial. Lancet Oncol 18:1116–1125CrossRef
24.
Szczypiński PM, Strzelecki M, Materka A, Klepaczko A (2009) MaZda-a software package for image texture analysis. Comput Methods Programs Biomed 94:66–76CrossRef
25.
Fang YHD, Lin CY, Shih MJ et al (2014) Development and evaluation of an open-source software package “cGITA” for quantifying tumor heterogeneity with molecular images. Biomed Res Int 2014:248505PubMedPubMedCentral
26.
Dinapoli N, Alitto AR, Vallati M et al (2015) Moddicom: a complete and easily accessible library for prognostic evaluations relying on image features. Conf Proc IEEE Eng Med Biol Soc 2015:771–774
27.
Echegaray S, Bakr S, Rubin DL, Napel S (2018) Quantitative image feature engine (QIFE): an open-source, modular engine for 3D quantitative feature extraction from volumetric medical images. J Digit Imaging 31:403–414CrossRef
28.
Götz M, Nolden M, Maier-Hein K (2019) MITK Phenotyping: an open-source toolchain for image-based personalized medicine with radiomics. Radiother Oncol 131:108–111CrossRef
29.
Pfaehler E, Zwanenburg A, de Jong JR, Boellaard R (2019) RACAT: an open source and easy to use radiomics calculator tool. PLoS One 14:1–26CrossRef
30.
Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15:155–163CrossRef
31.
32.
33.
34.
Yip SSF, Aerts HJWL (2016) Applications and limitations of radiomics. Phys Med Biol 61:R150–R166CrossRef
35.
Larue RTHM, van Timmeren JE, de Jong EEC et al (2017) Influence of gray level discretization on radiomic feature stability for different CT scanners, tube currents and slice thicknesses: a comprehensive phantom study. Acta Oncol 56:1544–1553CrossRef
36.
Shafiq-Ul-Hassan M, Latifi K, Ullah G, Gillies R, Moros E (2018) Voxel size and gray level normalization of CT radiomic features in lung cancer. Sci Rep 8:10545
37.
O’Connor JPB (2017) Rethinking the role of clinical imaging. eLife 6:e30563CrossRef
Metadata
Title
Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform
Authors
Isabella Fornacon-Wood
Hitesh Mistry
Christoph J. Ackermann
Fiona Blackhall
Andrew McPartlin
Corinne Faivre-Finn
Gareth J. Price
James P. B. O’Connor
Publication date
01-11-2020
Publisher
Springer Berlin Heidelberg
Keyword
Biomarkers
Published in
European Radiology / Issue 11/2020
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-020-06957-9

Other articles of this Issue 11/2020

European Radiology 11/2020 Go to the issue

Musculoskeletal

Natural history of new horizontal meniscal tears in individuals at risk for and with mild to moderate osteoarthritis: data from osteoarthritis initiative

Cardiac

Influence of contrast agent and spatial resolution on myocardial strain results using feature tracking MRI

Cardiac

Advanced myocardial characterization in hypertrophic cardiomyopathy: feasibility of CMR-based feature tracking strain analysis in a case-control study

Musculoskeletal

CT fluoroscopy-guided transforaminal and intra-articular facet steroid injections for the treatment of cervical radiculopathy: injectate distribution patterns and association with clinical outcome

Imaging Informatics and Artificial Intelligence

Radiomics analysis of dual-energy CT-derived iodine maps for diagnosing metastatic cervical lymph nodes in patients with papillary thyroid cancer

Breast

Breast-specific gamma imaging or ultrasonography as adjunct imaging diagnostics in women with mammographically dense breasts