Top

06-09-2024 | Imaging Informatics and Artificial Intelligence

Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews

Authors: Burak Kocak, Ali Keles, Fadime Kose, Abdurrezzak Sendur

Abstract

Purpose

This study aims to comprehensively evaluate the quality of radiomics research by examining unique papers from reviews using the radiomics quality score (RQS).

Methods

A literature search was conducted in PubMed (last search date: April 14, 2024). Systematic or non-systematic reviews using the RQS to evaluate radiomic studies were potentially included. Exclusion was applied at two levels: first, at the review level, and second, at the study level (i.e., for the individual articles previously evaluated within the reviews). Score-wise and item-wise analyses were performed, along with trend, multivariable, and subgroup analyses based on baseline study characteristics and validation methods.

Results

A total of 1574 unique papers (published online between 1999 and 2023) from 89 reviews were included in the final analysis. The median RQS percentage was 31% with an IQR of 25% (25th–75th percentiles, 14–39%). A positive correlation between median RQS percentage and publication year (2014–2023) was found, with Kendall’s tau coefficient of 0.908 (p < 0.001), suggesting an improvement in quality over time. The quality of radiomics publications significantly varied according to different subfields of radiology (p < 0.001). Around one-third of the publications (32%) lacked a separate validation set. Papers with internal validation (54%) dominated those with external validation (14%). Higher-quality validation practices were significantly associated with better RQS percentage scores, independent of the validation’s effect on the final score. Item-wise analysis revealed significant shortcomings in several areas.

Conclusion

Radiomics research quality is low but improving according to RQS. Significant variation exists across radiology subfields. Critical areas were identified for targeted improvement.

Clinical relevance statement

Our study shows that the quality of radiomics research is generally low but improving over time, with item-wise analysis highlighting critical areas needing improvement. It also reveals that the quality of radiomics research differs across subfields and validation methods.

Key Points

Overall quality of radiomics research remains low and highly variable, although a significant positive trend suggests an improvement in quality over time.
Considerable variations exist in the quality of radiomics publications across different subfields of radiology and validation types.
The item-wise analysis highlights several critical areas requiring attention, emphasizing the need for targeted improvements.

Graphical abstract

Available only for authorised users

Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446. https://doi.org/10.1016/j.ejca.2011.11.036CrossRefPubMedPubMedCentral

Zhong J, Lu J, Zhang G et al (2023) An overview of meta-analyses on radiomics: more evidence is needed to support clinical translation. Insights Imaging 14:111. https://doi.org/10.1186/s13244-023-01437-2CrossRefPubMedPubMedCentral

Huang EP, O’Connor JPB, McShane LM et al (2023) Criteria for the translation of radiomics into clinically useful tests. Nat Rev Clin Oncol 20:69–82. https://doi.org/10.1038/s41571-022-00707-0CrossRefPubMed

Cobo M, Menéndez Fernández-Miranda P, Bastarrika G, Lloret Iglesias L (2023) Enhancing radiomics and deep learning systems through the standardization of medical imaging workflows. Sci Data 10:732. https://doi.org/10.1038/s41597-023-02641-xCrossRefPubMedPubMedCentral

Fusco R, Granata V, Grazzini G et al (2022) Radiomics in medical imaging: pitfalls and challenges in clinical management. Jpn J Radiol 40:919–929. https://doi.org/10.1007/s11604-022-01271-4CrossRefPubMed

Rizzo S, Botta F, Raimondi S et al (2018) Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp 2:36. https://doi.org/10.1186/s41747-018-0068-zCrossRefPubMedPubMedCentral

Whybra P, Zwanenburg A, Andrearczyk V et al (2024) The image biomarker standardization initiative: standardized convolutional filters for reproducible radiomics and enhanced clinical insights. Radiology 310:e231319. https://doi.org/10.1148/radiol.231319CrossRefPubMed

Zwanenburg A, Vallières M, Abdalah MA et al (2020) The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 295:328–338. https://doi.org/10.1148/radiol.2020191145CrossRefPubMed

Kocak B, Baessler B, Bakas S et al (2023) CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII. Insights Imaging 14:75. https://doi.org/10.1186/s13244-023-01415-8CrossRefPubMedPubMedCentral

10.

Kocak B, Borgheresi A, Ponsiglione A et al (2024) Explanation and elaboration with examples for CLEAR (CLEAR-E3): an EuSoMII radiomics auditing group initiative. Eur Radiol Exp 8:72. https://doi.org/10.1186/s41747-024-00471-zCrossRefPubMedPubMedCentral

11.

Lambin P, Leijenaar RTH, Deist TM et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749–762. https://doi.org/10.1038/nrclinonc.2017.141CrossRefPubMed

12.

Zhang S, Liu R, Wang Y et al (2024) Ultrasound-base radiomics for discerning lymph node metastasis in thyroid cancer: a systematic review and meta-analysis. Acad Radiol S1076-6332:00154–00155. https://doi.org/10.1016/j.acra.2024.03.012CrossRef

13.

O’Sullivan NJ, Temperley HC, Horan MT et al (2024) Role of radiomics as a predictor of disease recurrence in ovarian cancer: a systematic review. Abdom Radiol. https://doi.org/10.1007/s00261-024-04330-8

14.

Vacca S, Scicolone R, Gupta A et al (2024) Atherosclerotic carotid artery disease radiomics: a systematic review with meta-analysis and radiomic quality score assessment. Eur J Radiol 177:111547. https://doi.org/10.1016/j.ejrad.2024.111547CrossRefPubMed

15.

Koçak B, Akinci D’Antonoli T, Cuocolo R (2024) Exploring radiomics research quality scoring tools: a comparative analysis of METRICS and RQS. Diagn Interv Radiol. https://doi.org/10.4274/dir.2024.242793

16.

Spadarella G, Stanzione A, Akinci D’Antonoli T et al (2023) Systematic review of the radiomics quality score applications: an EuSoMII radiomics auditing group initiative. Eur Radiol 33:1884–1894. https://doi.org/10.1007/s00330-022-09187-3CrossRefPubMed

17.

Kocak B, Akinci D’Antonoli T, Mercaldo N et al (2024) METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII. Insights Imaging 15:8. https://doi.org/10.1186/s13244-023-01572-wCrossRefPubMedPubMedCentral

Title: Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews
Authors: Burak Kocak
Ali Keles
Fadime Kose
Abdurrezzak Sendur
Publication date: 06-09-2024
Publisher: Springer Berlin Heidelberg
Published in: European Radiology
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI: https://doi.org/10.1007/s00330-024-11057-z

Keynote webinar | Spotlight on modern management of frailty

Springer Medicine

Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews

Abstract

Purpose

Methods

Results

Conclusion

Clinical relevance statement

Key Points

Graphical abstract

Keynote webinar | Spotlight on modern management of frailty

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusion

Clinical relevance statement

Key Points

Graphical abstract

Please log in to get access to this content