Top

Open Access 06-11-2024 | Glioma | Original Article

The Segment Anything foundation model achieves favorable brain tumor auto-segmentation accuracy in MRI to support radiotherapy treatment planning

Authors: PD Dr. Florian Putz, Sogand Beirami, Manuel Alexander Schmidt, Matthias Stefan May, Johanna Grigo, Thomas Weissmann, Philipp Schubert, Daniel Höfler, Ahmed Gomaa, Ben Tkhayat Hassen, Sebastian Lettmaier, Benjamin Frey, Udo S. Gaipl, Luitpold V. Distel, Sabine Semrau, Christoph Bert, Rainer Fietkau, Yixing Huang

Published in: Strahlentherapie und Onkologie

Abstract

Background

Promptable foundation auto-segmentation models like Segment Anything (SA, Meta AI, New York, USA) represent a novel class of universal deep learning auto-segmentation models that could be employed for interactive tumor auto-contouring in RT treatment planning.

Methods

Segment Anything was evaluated in an interactive point-to-mask auto-segmentation task for glioma brain tumor auto-contouring in 16,744 transverse slices from 369 MRI datasets (BraTS 2020 dataset). Up to nine interactive point prompts were automatically placed per slice. Tumor boundaries were auto-segmented on contrast-enhanced T1w sequences. Out of the three auto-contours predicted by SA, accuracy was evaluated for the contour with the highest calculated IoU (Intersection over Union, “oracle mask,” simulating interactive model use with selection of the best tumor contour) and for the tumor contour with the highest model confidence (“suggested mask”).

Results

Mean best IoU (mbIoU) using the best predicted tumor contour (oracle mask) in full MRI slices was 0.762 (IQR 0.713–0.917). The best 2D mask was achieved after a mean of 6.6 interactive point prompts (IQR 5–9). Segmentation accuracy was significantly better for high- compared to low-grade glioma cases (mbIoU 0.789 vs. 0.668). Accuracy was worse using the suggested mask (0.572). Stacking best tumor segmentations from transverse MRI slices, mean 3D Dice score for tumor auto-contouring was 0.872, which was improved to 0.919 by combining axial, sagittal, and coronal contours.

Conclusion

The Segment Anything foundation segmentation model can achieve high accuracy for glioma brain tumor segmentation in MRI datasets. The results suggest that foundation segmentation models could facilitate RT treatment planning when properly integrated in a clinical application.

Niyazi M, Brada M, Chalmers AJ, Combs SE, Erridge SC, Fiorentino A, Grosu AL, Lagerwaard FJ, Minniti G, Mirimanoff RO, Ricardi U, Short SC, Weber DC, Belka C (2016) ESTRO-ACROP guideline “target delineation of glioblastomas”. Radiother Oncol 118(1):35–42. https://doi.org/10.1016/j.radonc.2015.12.003CrossRefPubMed

Kruser TJ, Bosch WR, Badiyan SN, Bovi JA, Ghia AJ, Kim MM, Solanki AA, Sachdev S, Tsien C, Wang TJC, Mehta MP, McMullen KP (2019) NRG brain tumor specialists consensus guidelines for glioblastoma contouring. J Neurooncol 143(1):157–166. https://doi.org/10.1007/s11060-019-03152-9CrossRefPubMedPubMedCentral

McCarroll RE, Beadle BM, Balter PA, Burger H, Cardenas CE, Dalvie S, Followill DS, Kisling KD, Mejia M, Naidoo K, Nelson CL, Peterson CB, Vorster K, Wetter J, Zhang L, Court LE, Yang J (2018) Retrospective Validation and Clinical Implementation of Automated Contouring of Organs at Risk in the Head and Neck: A Step Toward Automated Radiation Treatment Planning for Low- and Middle-Income Countries. J Glob Oncol 4:1–11. https://doi.org/10.1200/jgo.18.00055CrossRefPubMed

Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 18(2):203–211. https://doi.org/10.1038/s41592-020-01008-zCrossRefPubMed

Isensee F, Jäger PF, Full PM, Vollmuth P (2020) Maier-Hein KH nnU-Net for brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020. October, vol 4. Lima, Peru, pp 118–132

Huang Y, Khodabakhshi Z, Gomaa A, Schmidt M, Fietkau R, Guckenberger M, Andratschke N, Bert C, Tanadini-Lang S, Putz F (2024) Multicenter privacy-preserving model training for deep learning brain metastases autosegmentation. Radiother Oncol 198:110419. https://doi.org/10.1016/j.radonc.2024.110419CrossRefPubMed

Xie C, Wang J, Zhang Z, Zhou Y, Xie L, Yuille A (2017) Adversarial examples for semantic segmentation and object detection. In: Proceedings of the IEEE international conference on computer vision, pp 1369–1378

Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg AC, Lo W‑Y (2023) Segment anything. Arxiv Prepr Arxiv 230402643. https://doi.org/10.48550/arXiv.2304.02643

Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner E, Weber MA, Arbel T, Avants BB, Ayache N, Buendia P, Collins DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demiralp C, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Pereira S, Precup D, Price SJ, Raviv TR, Reza SM, Ryan M, Sarikaya D, Schwartz L, Shin HC, Shotton J, Silva CA, Sousa N, Subbanna NK, Szekely G, Taylor TJ, Thomas OM, Tustison NJ, Unal G, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, Zikic D, Prastawa M, Reyes M, Van Leemput K (2015) The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024. https://doi.org/10.1109/tmi.2014.2377694CrossRefPubMed

10.

Henry T, Carré A, Lerousseau M, Estienne T, Robert C, Paragios N, Deutsch E (2020) Brain tumor segmentation with self-ensembled, deeply-supervised 3D U‑net neural networks: a BraTS 2020 challenge solution. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020. Lima, Peru, pp 327–339

11.

Niyazi M, Andratschke N, Bendszus M, Chalmers AJ, Erridge SC, Galldiks N, Lagerwaard FJ, Navarria P, Munck Af Rosenschöld P, Ricardi U, van den Bent MJ, Weller M, Belka C, Minniti G (2023) ESTRO-EANO guideline on target delineation and radiotherapy details for glioblastoma. Radiother Oncol 184:109663. https://doi.org/10.1016/j.radonc.2023.109663CrossRefPubMed

12.

Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin JC, Pujol S, Bauer C, Jennings D, Fennessy F, Sonka M, Buatti J, Aylward S, Miller JV, Pieper S, Kikinis R (2012) 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging 30(9):1323–1341. https://doi.org/10.1016/j.mri.2012.05.001CrossRefPubMedPubMedCentral

13.

Putz F, Mengling V, Perrin R, Masitho S, Weissmann T, Rosch J, Bauerle T, Janka R, Cavallaro A, Uder M, Amarteifio P, Doussin S, Schmidt MA, Dorfler A, Semrau S, Lettmaier S, Fietkau R, Bert C (2020) Magnetic resonance imaging for brain stereotactic radiotherapy : A review of requirements and pitfalls. Strahlenther Onkol 196(5):444–456. https://doi.org/10.1007/s00066-020-01604-0CrossRef

14.

Wang Y, Jiang T (2013) Understanding high grade glioma: molecular mechanism, therapy and comprehensive management. Cancer Lett 331(2):139–146. https://doi.org/10.1016/j.canlet.2012.12.024CrossRefPubMed

15.

Seker-Polat F, Pinarbasi Degirmenci N, Solaroglu I, Bagci-Onder T (2022) Tumor Cell Infiltration into the Brain in Glioblastoma: From Mechanisms to Clinical Perspectives. Cancers. https://doi.org/10.3390/cancers14020443CrossRefPubMedPubMedCentral

16.

Back M, Jayamanne D, Brazier D, Newey A, Bailey D, Schembri G, Hsiao E, Khasraw M, Wong M, Kastelan M, Brown C, Wheeler H (2020) Pattern of failure in anaplastic glioma patients with an IDH1/2 mutation. Strahlenther Onkol 196(1):31–39. https://doi.org/10.1007/s00066-019-01467-0CrossRef

17.

Huang Y, Bert C, Sommer P, Frey B, Gaipl U, Distel LV, Weissmann T, Uder M, Schmidt MA, Dörfler A, Maier A, Fietkau R, Putz F (2022) Deep learning for brain metastasis detection and segmentation in longitudinal MRI data. Med Phys. https://doi.org/10.1002/mp.15863CrossRefPubMed

18.

Neve OM, Chen Y, Tao Q, Romeijn SR, de Boer NP, Grootjans W, Kruit MC, Lelieveldt BPF, Jansen JC, Hensen EF, Verbist BM, Staring M (2022) Fully Automated 3D Vestibular Schwannoma Segmentation with and without Gadolinium-based Contrast Material: A Multicenter, Multivendor Study. Radiol Artif Intell 4(4):e210300. https://doi.org/10.1148/ryai.210300CrossRefPubMedPubMedCentral

19.

Erdur AC, Rusche D, Scholz D, Kiechle J, Fischer S, Llorián-Salvador Ó, Buchner JA, Nguyen MQ, Etzel L, Weidner J, Metz MC, Wiestler B, Schnabel J, Rueckert D, Combs SE, Peeken JC (2024) Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives. Strahlenther Onkol. https://doi.org/10.1007/s00066-024-02262-2CrossRef

20.

Buchner JA, Kofler F, Etzel L, Mayinger M, Christ SM, Brunner TB, Wittig A, Menze B, Zimmer C, Meyer B, Guckenberger M, Andratschke N, El Shafie RA, Debus J, Rogers S, Riesterer O, Schulze K, Feldmann HJ, Blanck O, Zamboglou C, Ferentinos K, Wolff R, Eitz KA, Combs SE, Bernhardt D, Wiestler B, Peeken JC (2023) Development and external validation of an MRI-based neural network for brain metastasis segmentation in the AURORA multicenter study. Radiother Oncol 178:109425. https://doi.org/10.1016/j.radonc.2022.11.014CrossRefPubMed

21.

Lu SL, Xiao FR, Cheng JC, Yang WC, Cheng YH, Chang YC, Lin JY, Liang CH, Lu JT, Chen YF, Hsu FM (2021) Randomized multi-reader evaluation of automated detection and segmentation of brain tumors in stereotactic radiosurgery with deep neural networks. Neuro-Oncology 23(9):1560–1568. https://doi.org/10.1093/neuonc/noab071CrossRefPubMedPubMedCentral

22.

Ronneberger O, Fischer P, U‑net BT (2015) Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI. Springer, Munich, Germany, pp 234–241 (18th International Conference)

23.

Huang Y, Gomaa A, Höfler D, Schubert P, Gaipl U, Frey B, Fietkau R, Bert C, Putz F (2024) Principles of artificial intelligence in radiooncology. Strahlentherapie Onkol. https://doi.org/10.1007/s00066-024-02272-0CrossRefPubMed

24.

Putz F, Lamrani A, Weissmann T, Mansoorian S, Frey B, Bert C, Fietkau R (2021) Die OAR Turing-Test Studie: Ein verblindeter Vergleich von expertenbasierter Organ at Risk Erstellung und drei kommerziellen Autosegmentierungslösungen. Strahlenther Onkol 197(1):1–246. https://doi.org/10.1007/s00066-021-01791-4CrossRef

25.

Wen PY, Macdonald DR, Reardon DA, Cloughesy TF, Sorensen AG, Galanis E, Degroot J, Wick W, Gilbert MR, Lassman AB, Tsien C, Mikkelsen T, Wong ET, Chamberlain MC, Stupp R, Lamborn KR, Vogelbaum MA, van den Bent MJ, Chang SM (2010) Updated response assessment criteria for high-grade gliomas: response assessment in neuro-oncology working group. J Clin Oncol 28(11):1963–1972. https://doi.org/10.1200/jco.2009.26.3541CrossRefPubMed

26.

OpenAI (2023) GPT-4 Technical Report. Arxiv Prepr Arxiv:230308774. https://doi.org/10.48550/arXiv.2303.08774

Title: The Segment Anything foundation model achieves favorable brain tumor auto-segmentation accuracy in MRI to support radiotherapy treatment planning
Authors: PD Dr. Florian Putz
Sogand Beirami
Manuel Alexander Schmidt
Matthias Stefan May
Johanna Grigo
Thomas Weissmann
Philipp Schubert
Daniel Höfler
Ahmed Gomaa
Ben Tkhayat Hassen
Sebastian Lettmaier
Benjamin Frey
Udo S. Gaipl
Luitpold V. Distel
Sabine Semrau
Christoph Bert
Rainer Fietkau
Yixing Huang
Publication date: 06-11-2024
Publisher: Springer Berlin Heidelberg
Keywords: Glioma
Glioma
Glioma
Brain Tumor
Radiation Treatment Planning
Published in: Strahlentherapie und Onkologie
Print ISSN: 0179-7158
Electronic ISSN: 1439-099X
DOI: https://doi.org/10.1007/s00066-024-02313-8

Keynote series | Spotlight on menopause

Springer Medicine

The Segment Anything foundation model achieves favorable brain tumor auto-segmentation accuracy in MRI to support radiotherapy treatment planning

Abstract

Background

Methods

Results

Conclusion

Keynote series | Spotlight on menopause

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content