Top

Published in:

01-06-2022 | Magnetic Resonance Imaging | Hollow Organ GI

Rectal MRI radiomics inter- and intra-reader reliability: should we worry about that?

Authors: Henry C. Kwok, Charlotte Charbel, Sofia Danilova, Joao Miranda, Natalie Gangai, Iva Petkovska, Jayasree Chakraborty, Natally Horvat

Published in: Abdominal Radiology | Issue 6/2022

Abstract

Purpose

The aim of this review paper is to summarize the current literature regarding inter- and intra-reader reliability of radiomics on rectal MRI.

Methods

Original studies examining treatment response prediction in patients with rectal cancer following neoadjuvant therapy using rectal MRI-based radiomics between January 2010 and December 2021 were identified via a PubMed/Medline search. Studies in which intra- and/or inter-reader reliability had been reported were included in this review.

Results

Thirteen studies were selected, with an average number of patients of 145 (range, 20–649). All included studies evaluated T2-weighted imaging (T2WI) and/or diffusion-weighted imaging (DWI) sequences, while 3/13 (23%) also evaluated the contrast-enhanced T1-weighted imaging (T1WI) sequence. Most of the selected studies involved two readers (10/13, 77%), 6/13 (46%) studies used baseline MRI only, 1/13 (8%) study used restaging MRI only, and 6/13 (46%) used both. Segmentation was performed manually in 10/13 (77%) studies, and in a slight majority of studies (7/13, 54%), the entire tumor volume (3D VOI) was segmented, while 4/13 (31%) studies segmented the 2D ROI and 2/13 (15%) segmented both. Intraclass correlation coefficient (ICC) on intra-reader agreement varied from 0.73 to 0.93. ICC to assess inter-reader varied from 0.60 to 0.99. Overall, features obtained from baseline rectal MRI, using 3D VOI and first-order features, had higher agreement.

Conclusion

Based on our qualitative assessment of a small number of non-dedicated studies, there seems to be good reliability, particularly among low-order features extracted from the entire tumor volume using baseline MRI; however, direct evidence remains scarce. More targeted research in this area is required to quantitatively verify reliability, and before these novel radiomic techniques can be clinically adopted.

Graphical abstract

Available only for authorised users

Habr-Gama A, Perez RO, Nadalin W, Sabbaga J, Ribeiro U, Jr., Silva e Sousa AH, Jr., Campos FG, Kiss DR, Gama-Rodrigues J (2004) Operative versus nonoperative treatment for stage 0 distal rectal cancer following chemoradiation therapy: long-term results. Ann Surg 240 (4):711–717; discussion 717–718

[2] Maas M, Beets-Tan RG, Lambregts DM, Lammering G, Nelemans PJ, Engelen SM, van Dam RM, Jansen RL, Sosef M, Leijtens JW, Hulsewe KW, Buijsen J, Beets GL (2011) Wait-and-see policy for clinical complete responders after chemoradiation for rectal cancer. J Clin Oncol 29 (35):4633-4640. https://doi.org/10.1200/JCO.2011.37.7176CrossRefPubMed

Li J, Liu H, Yin J, Liu S, Hu J, Du F, Yuan J, Lv B, Fan J, Leng S, Zhang X (2015) Wait-and-see or radical surgery for rectal cancer patients with a clinical complete response after neoadjuvant chemoradiotherapy: a cohort study. Oncotarget 6 (39):42354–42361. https://doi.org/10.18632/oncotarget.6093

[4] Renehan AG, Malcomson L, Emsley R, Gollins S, Maw A, Myint AS, Rooney PS, Susnerwala S, Blower A, Saunders MP, Wilson MS, Scott N, O'Dwyer ST (2016) Watch-and-wait approach versus surgical resection after chemoradiotherapy for patients with rectal cancer (the OnCoRe project): a propensity-score matched cohort analysis. Lancet Oncol 17 (2):174-183. https://doi.org/10.1016/S1470-2045(15)00467-2CrossRefPubMed

[5] Kasi A, Abbasi S, Handa S, Al-Rajabi R, Saeed A, Baranda J, Sun W (2020) Total Neoadjuvant Therapy vs Standard Therapy in Locally Advanced Rectal Cancer: A Systematic Review and Meta-analysis. JAMA Netw Open 3 (12):e2030097. https://doi.org/10.1001/jamanetworkopen.2020.30097CrossRefPubMedPubMedCentral

[6] Guillem JG, Chessin DB, Shia J, Moore HG, Mazumdar M, Bernard B, Paty PB, Saltz L, Minsky BD, Weiser MR, Temple LK, Cohen AM, Wong WD (2005) Clinical examination following preoperative chemoradiation for rectal cancer is not a reliable surrogate end point. J Clin Oncol 23 (15):3475-3479. https://doi.org/10.1200/JCO.2005.06.114CrossRefPubMed

[7] Kawai K, Ishihara S, Nozawa H, Hata K, Kiyomatsu T, Morikawa T, Fukayama M, Watanabe T (2017) Prediction of Pathological Complete Response Using Endoscopic Findings and Outcomes of Patients Who Underwent Watchful Waiting After Chemoradiotherapy for Rectal Cancer. Dis Colon Rectum 60 (4):368-375. https://doi.org/10.1097/DCR.0000000000000742CrossRefPubMed

Park SH, Cho SH, Choi SH, Jang JK, Kim MJ, Kim SH, Lim JS, Moon SK, Park JH, Seo N, Cancer KSoARSGfR (2020) MRI Assessment of Complete Response to Preoperative Chemoradiation Therapy for Rectal Cancer: 2020 Guide for Practice from the Korean Society of Abdominal Radiology. Korean J Radiol 21 (7):812-828. https://doi.org/10.3348/kjr.2020.0483

[9] Kim SH, Lee JM, Park HS, Eun HW, Han JK, Choi BI (2009) Accuracy of MRI for predicting the circumferential resection margin, mesorectal fascia invasion, and tumor response to neoadjuvant chemoradiotherapy for locally advanced rectal cancer. J Magn Reson Imaging 29 (5):1093-1101. https://doi.org/10.1002/jmri.21742CrossRefPubMed

10.

[10] Nahas SC, Nahas CSR, Cama GM, de Azambuja RL, Horvat N, Marques CFS, Menezes MR, Junior UR, Cecconello I (2019) Diagnostic performance of magnetic resonance to assess treatment response after neoadjuvant therapy in patients with locally advanced rectal cancer. Abdom Radiol (NY). https://doi.org/10.1007/s00261-019-01894-8CrossRef

11.

[11] Bates DDB, Golia Pernicka JS, Fuqua JL, Paroder V, Petkovska I, Zheng J, Capanu M, Schilsky J, Gollub MJ (2020) Diagnostic accuracy of b800 and b1500 DWI-MRI of the pelvis to detect residual rectal adenocarcinoma: a multi-reader study. Abdom Radiol (NY) 45 (2):293-300. https://doi.org/10.1007/s00261-019-02283-xCrossRef

12.

Horvat N, Veeraraghavan H, Khan M, Blazic I, Zheng J, Capanu M, Sala E, Garcia-Aguilar J, Gollub MJ, Petkovska I (2018) MR Imaging of Rectal Cancer: Radiomics Analysis to Assess Treatment Response after Neoadjuvant Therapy. Radiology:172300. https://doi.org/10.1148/radiol.2018172300

13.

[13] Cusumano D, Meijer G, Lenkowicz J, Chiloiro G, Boldrini L, Masciocchi C, Dinapoli N, Gatta R, Casà C, Damiani A, Barbaro B, Gambacorta MA, Azario L, De Spirito M, Intven M, Valentini V (2021) A field strength independent MR radiomics model to predict pathological complete response in locally advanced rectal cancer. Radiol Med 126 (3):421-429. https://doi.org/10.1007/s11547-020-01266-zCrossRefPubMed

14.

[14] Petkovska I, Tixier F, Ortiz EJ, Golia Pernicka JS, Paroder V, Bates DD, Horvat N, Fuqua J, Schilsky J, Gollub MJ, Garcia-Aguilar J, Veeraraghavan H (2020) Clinical utility of radiomics at baseline rectal MRI to predict complete response of rectal cancer after chemoradiation therapy. Abdom Radiol (NY). https://doi.org/10.1007/s00261-020-02502-wCrossRef

15.

[15] Horvat N, Miranda J, El Homsi M, Peoples JJ, Long NM, Simpson AL, Do RKG (2021) A primer on texture analysis in abdominal radiology. Abdom Radiol (NY). https://doi.org/10.1007/s00261-021-03359-3CrossRefPubMed

16.

[16] Rizzo S, Botta F, Raimondi S, Origgi D, Fanciullo C, Morganti AG, Bellomi M (2018) Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp 2 (1):36. https://doi.org/10.1186/s41747-018-0068-zCrossRefPubMedPubMedCentral

17.

[17] Lubner MG, Smith AD, Sandrasegaran K, Sahani DV, Pickhardt PJ (2017) CT Texture Analysis: Definitions, Applications, Biologic Correlates, and Challenges. Radiographics 37 (5):1483-1503. https://doi.org/10.1148/rg.2017170056CrossRefPubMed

18.

[18] Bashir U, Siddique MM, Mclean E, Goh V, Cook GJ (2016) Imaging Heterogeneity in Lung Cancer: Techniques, Applications, and Challenges. AJR Am J Roentgenol 207 (3):534-543. https://doi.org/10.2214/AJR.15.15864CrossRefPubMed

19.

Miranda J, Tan GXV, Fernandes MC, Yildirim O, Sims JA, Araujo-Filho JAB, de M Machado FA, Assuncao-Jr AN, Nomura CH, Horvat N (2021) Rectal MRI radiomics for predicting pathological complete response: Where we are. Clin Imaging 82:141–149. https://doi.org/10.1016/j.clinimag.2021.10.005

20.

[20] Li ZY, Wang XD, Li M, Liu XJ, Ye Z, Song B, Yuan F, Yuan Y, Xia CC, Zhang X, Li Q (2020) Multi-modal radiomics model to predict treatment response to neoadjuvant chemotherapy for locally advanced rectal cancer. World J Gastroenterol 26 (19):2388-2402. https://doi.org/10.3748/wjg.v26.i19.2388CrossRefPubMedPubMedCentral

21.

[21] Schurink NW, van Kranen SR, Roberti S, van Griethuysen JJM, Bogveradze N, Castagnoli F, Khababi NE, Bakers FCH, de Bie SH, Bosma GPT, Cappendijk VC, Geenen RWF, Neijenhuis PA, Peterson GM, Veeken CJ, Vliegen RFA, Beets-Tan RGH, Lambregts DMJ (2021) Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility. Eur Radiol. https://doi.org/10.1007/s00330-021-08251-8CrossRefPubMedPubMedCentral

22.

[22] Palmisano A, Di Chiara A, Esposito A, Rancoita PMV, Fiorino C, Passoni P, Albarello L, Rosati R, Del Maschio A, De Cobelli F (2020) MRI prediction of pathological response in locally advanced rectal cancer: when apparent diffusion coefficient radiomics meets conventional volumetry. Clin Radiol 75 (10):798.e791-798.e711. https://doi.org/10.1016/j.crad.2020.06.023CrossRef

23.

Petresc B, Lebovici A, Caraiani C, Feier DS, Graur F, Buruian MM (2020) Pre-Treatment T2-WI Based Radiomics Features for Prediction of Locally Advanced Rectal Cancer Non-Response to Neoadjuvant Chemoradiotherapy: A Preliminary Study. Cancers (Basel) 12 (7). https://doi.org/10.3390/cancers12071894

24.

[24] van Griethuysen JJM, Lambregts DMJ, Trebeschi S, Lahaye MJ, Bakers FCH, Vliegen RFA, Beets GL, Aerts HJWL, Beets-Tan RGH (2020) Radiomics performs comparable to morphologic assessment by expert radiologists for prediction of response to neoadjuvant chemoradiotherapy on baseline staging MRI in rectal cancer. Abdom Radiol (NY) 45 (3):632-643. https://doi.org/10.1007/s00261-019-02321-8CrossRef

25.

[25] Aker M, Ganeshan B, Afaq A, Wan S, Groves AM, Arulampalam T (2019) Magnetic Resonance Texture Analysis in Identifying Complete Pathological Response to Neoadjuvant Treatment in Locally Advanced Rectal Cancer. Dis Colon Rectum 62 (2):163-170. https://doi.org/10.1097/DCR.0000000000001224CrossRefPubMed

26.

[26] Cui Y, Yang X, Shi Z, Yang Z, Du X, Zhao Z, Cheng X (2019) Radiomics analysis of multiparametric MRI for prediction of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Eur Radiol 29 (3):1211-1220. https://doi.org/10.1007/s00330-018-5683-9CrossRefPubMed

27.

[27] Meng X, Xia W, Xie P, Zhang R, Li W, Wang M, Xiong F, Liu Y, Fan X, Xie Y, Wan X, Zhu K, Shan H, Wang L, Gao X (2019) Preoperative radiomic signature based on multiparametric magnetic resonance imaging for noninvasive evaluation of biological characteristics in rectal cancer. Eur Radiol 29 (6):3200-3209. https://doi.org/10.1007/s00330-018-5763-xCrossRefPubMed

28.

[28] Liu Z, Zhang XY, Shi YJ, Wang L, Zhu HT, Tang Z, Wang S, Li XT, Tian J, Sun YS (2017) Radiomics Analysis for Evaluation of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer. Clin Cancer Res 23 (23):7253-7262. https://doi.org/10.1158/1078-0432.CCR-17-1038CrossRefPubMed

29.

[29] Choi MH, Oh SN, Rha SE, Choi JI, Lee SH, Jang HS, Kim JG, Grimm R, Son Y (2016) Diffusion-weighted imaging: Apparent diffusion coefficient histogram analysis for detecting pathologic complete response to chemoradiotherapy in locally advanced rectal cancer. J Magn Reson Imaging 44 (1):212-220. https://doi.org/10.1002/jmri.25117CrossRefPubMed

30.

[30] Nougaret S, Vargas HA, Lakhman Y, Sudre R, Do RK, Bibeau F, Azria D, Assenat E, Molinari N, Pierredon MA, Rouanet P, Guiu B (2016) Intravoxel Incoherent Motion-derived Histogram Metrics for Assessment of Response after Combined Chemotherapy and Radiation Therapy in Rectal Cancer: Initial Experience and Comparison between Single-Section and Volumetric Analyses. Radiology 280 (2):446-454. https://doi.org/10.1148/radiol.2016150702CrossRefPubMed

31.

[31] Yang C, Jiang ZK, Liu LH, Zeng MS (2020) Pre-treatment ADC image-based random forest classifier for identifying resistant rectal adenocarcinoma to neoadjuvant chemoradiotherapy. Int J Colorectal Dis 35 (1):101-107. https://doi.org/10.1007/s00384-019-03455-3CrossRefPubMed

32.

[32] Li Z, Ma X, Shen F, Lu H, Xia Y, Lu J (2021) Evaluating treatment response to neoadjuvant chemoradiotherapy in rectal cancer using various MRI-based radiomics models. BMC Med Imaging 21 (1):30. https://doi.org/10.1186/s12880-021-00560-0CrossRefPubMedPubMedCentral

33.

[33] Cui Y, Yang X, Shi Z, Yang Z, Du X, Zhao Z, Cheng X (2018) Radiomics analysis of multiparametric MRI for prediction of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Eur Radiol. https://doi.org/10.1007/s00330-018-5683-9CrossRefPubMedPubMedCentral

34.

[34] Meng Y, Zhang Y, Dong D, Li C, Liang X, Zhang C, Wan L, Zhao X, Xu K, Zhou C, Tian J, Zhang H (2018) Novel radiomic signature as a prognostic biomarker for locally advanced rectal cancer. J Magn Reson Imaging. https://doi.org/10.1002/jmri.25968CrossRefPubMed

35.

Meng Y, Zhang C, Zou S, Zhao X, Xu K, Zhang H, Zhou C (2018) MRI texture analysis in predicting treatment response to neoadjuvant chemoradiotherapy in rectal cancer. Oncotarget 9 (15):11999–12008. https://doi.org/10.18632/oncotarget.23813

36.

[36] Jha AK, Mithun S, Jaiswar V, Sherkhane UB, Purandare NC, Prabhash K, Rangarajan V, Dekker A, Wee L, Traverso A (2021) Repeatability and reproducibility study of radiomic features on a phantom and human cohort. Scientific Reports 11 (1):2055. https://doi.org/10.1038/s41598-021-81526-8CrossRefPubMedPubMedCentral

37.

[37] Moradmand H, Aghamiri SMR, Ghaderi R (2020) Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma. 21 (1):179-190. https://doi.org/10.1002/acm2.12795CrossRef

38.

[38] Traverso A, Wee L, Dekker A, Gillies R (2018) Repeatability and Reproducibility of Radiomic Features: A Systematic Review. International journal of radiation oncology, biology, physics 102 (4):1143-1158. https://doi.org/10.1016/j.ijrobp.2018.05.053CrossRefPubMedPubMedCentral

Title: Rectal MRI radiomics inter- and intra-reader reliability: should we worry about that?
Authors: Henry C. Kwok
Charlotte Charbel
Sofia Danilova
Joao Miranda
Natalie Gangai
Iva Petkovska
Jayasree Chakraborty
Natally Horvat
Publication date: 01-06-2022
Publisher: Springer US
Keywords: Magnetic Resonance Imaging
Magnetic Resonance Imaging
Rectal Cancer
Rectal Cancer
Published in: Abdominal Radiology / Issue 6/2022
Print ISSN: 2366-004X
Electronic ISSN: 2366-0058
DOI: https://doi.org/10.1007/s00261-022-03503-7

Live Webinar | 27-06-2024 | 18:00 (CEST)

Keynote webinar | Spotlight on medication adherence

Reserve your place

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

WHO estimates that half of all patients worldwide are non-adherent to their prescribed medication. The consequences of poor adherence can be catastrophic, on both the individual and population level.

Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.

Prof. Kevin Dolgin

Prof. Florian Limbourg

Prof. Anoop Chauhan

Developed by: Springer Medicine

Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusion

Graphical abstract

Please log in to get access to this content

Other articles of this Issue 6/2022

Uterine fibroid-like tumors: spectrum of MR imaging findings and their differential diagnosis

Many faces of torsion in pediatric female pelvis

The diagnostic performance of contrast-enhanced CT versus extracellular contrast agent-enhanced MRI in detecting hepatocellular carcinoma: direct comparison and a meta-analysis

Characterization of mesenteric and portal hemodynamics using 4D flow MRI: the effects of meals and diurnal variation

Pancreatic enlargement in a patient receiving therapy with vasodilators for pulmonary arterial hypertension: a case report

MR findings of polypoid endometriosis of female genital organs: report of three cases

Keynote webinar | Spotlight on medication adherence

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

At a glance: The STEP trials