Top

Strahlentherapie und Onkologie

Published in:

01-02-2012 | Original article

Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy

Authors: I. Fotina, C. Lütgendorf-Caucig, M. Stock, R. Pötter, D. Georg

Published in: Strahlentherapie und Onkologie | Issue 2/2012

Abstract

Background and purpose

Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review.

Patients and methods

In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters.

Results

It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed.

Conclusion

A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.

Allozi R, Li XA, White J et al (2010) Tools for consensus analysis of experts’ contours for radiotherapy structure definitions. Radiother Oncol 97:572–578PubMedCrossRef

Altorjai G, Fotina I, Lütgendorf-Caucig C et al (2011) Cone-beam ct-based delineation of stereotactic lung targets: the influence of image modality and target size on interobserver variability. Int J Radiat Oncol Biol Phys [Epub ahead of print]

Berthelet E, Liu MC, Agranovich A et al (2002) Computed tomography determination of prostate volume and maximum dimensions: a study of interobserver variability. Radiother Oncol 63:37–40PubMedCrossRef

Breen SL, Publicover J, De Silva S et al (2007) Intraobserver and inter-observer variability in GTV delineation on FDG-PET-CT images of head and neck cancers. Int J Radiat Oncol Biol Phys 68:763–770PubMedCrossRef

Castro Pena P, Kirova YM, Campana F et al (2009) Anatomical, clinical and radiological delineation of target volumes in breast cancer radiotherapy planning: individual variability, questions and answers. Br J Radiol 82:595–599CrossRef

Dimopoulos JC, De Vos V, Berger D et al (2009) Inter-observer comparison of target delineation for MRI-assisted cervical cancer brachytherapy: application of the GYN GEC-ESTRO recommendations. Radiother Oncol 91:166–172PubMedCrossRef

Eliasziw M, Young SL, Woodbury MG et al (1994) Statistical methodology for the concurrent assessment of interrater and intrarater reliability: using goniometric measurements as an example. Phys Ther 74:777–788PubMed

Fuller CD, Nijkamp J, Duppen JC et al (2011) Prospective randomized double-blind pilot study of site-specific consensus atlas implementation for rectal cancer target volume delineation in the cooperative group setting. Int J Radiat Oncol Biol Phys 79:481–489PubMedCrossRef

Genovesi D, Cèfaro GA, Vinciguerra A et al (2011) Interobserver variability of clinical target volume delineation in supra-diaphragmatic Hodgkin’s disease: a multi-institutional experience. Strahlenther Onkol 187:357–366PubMedCrossRef

10.

Giezen M, Kouwenhoven E, Scholten AN et al (2011) Magnetic resonance imaging- versus computed tomography-based target volume delineation of the glandular breast tissue in breast-conserving therapy: an exploratory study. Int J Radiat Oncol Biol Phys 81:804–811PubMedCrossRef

11.

Giraud P, Elles S, Helfre S et al (2002) Conformal radiotherapy for lung cancer: different delineation of the gross tumour volume (GTV) by radiologists and radiation oncologists. Radiother Oncol 62:27–36PubMedCrossRef

12.

Goldner G, Dimopoulos J, Kirisits C, Pötter R (2009) Moderate dose escalation in three-dimensional conformal localized prostate cancer radiotherapy: single-institutional experience in 398 patients comparing 66 Gy versus 70 Gy versus 74 Gy. Strahlenther Onkol 185:438–445PubMedCrossRef

13.

Grabarz D, Panzarella T, Bezjak A et al (2011) Quantifying interobserver variation in target definition in Palliative Radiotherapy. Int J Radiat Oncol Biol Phys 80:1498–1504PubMedCrossRef

14.

Guckenberger M, Ok S, Polat B et al (2010) Toxicity after intensity-modulated, image-guided radiotherapy for prostate cancer. Strahlenther Onkol 186:535–43 (Erratum: Strahlenther Onkol 2010;186:705)

15.

International Commission on Radiation Units and Measurements. ICRU Report 62 (1999) Prescribing, recording, and reporting photon beam therapy (Supplement to ICRU Report 50). ICRU, Bethesda, MD, USA

16.

Hentschel B, Oehler W, Strauss D et al (2011) Definition of the CTV prostate in CT and MRI by using CT-MRI image fusion in IMRT planning for prostate cancer. Strahlenther Onkol 187:183–190PubMedCrossRef

17.

Jansen EP, Nijkamp J, Gubanski M et al (2010) Interobserver variation of clinical target volume delineation in gastric cancer. Int J Radiat Oncol Biol Phys 77:1166–1170PubMedCrossRef

18.

Kouwenhoven E, Giezen M, Struikmans H (2009) Measuring the similarity of target volume delineations independent of the number of observers. Phys Med Biol 54:2863–2873PubMedCrossRef

19.

Krengli M, Cannillo B, Turri L et al (2010) Target volume delineation for preoperative radiotherapy of rectal cancer: inter-observer variability and potential impact of FDG-PET/CT imaging. Technol Cancer Res Treat 9:393–398PubMed

20.

Lawton CA, Michalski J, El-Naqa I et al (2009) Variation in the definition of clinical target volumes for pelvic nodal conformal radiation therapy for prostate cancer. Int J Radiat Oncol Biol Phys 74:377–382PubMedCrossRef

21.

Leunens G, Menten J, Weltens C et al (1993) Quality assessment of medical decision making in radiation oncology: variability in target volume delineation for brain tumours. Radiother Oncol 29:169–175PubMedCrossRef

22.

Li XA, Tai A, Arthur DW, Buchholz TA et al (2009) Variability of target and normal structure delineation for breast cancer radiotherapy: an RTOG Multi-Institutional and Multiobserver Study. Int J Radiat Oncol Biol Phys 73:944–951PubMedCrossRef

23.

Logue JP, Sharrock CL, Cowan RA et al (1998) Clinical variability of target volume description in conformal radiotherapy planning. Int J Radiat Oncol Biol Phys 41:929–9231PubMedCrossRef

24.

Louie AV, Rodrigues G, Olsthoorn J et al (2010) Inter-observer and intra-observer reliability for lung cancer target volume delineation in the 4D-CT era. Radiother Oncol 95:166–171PubMedCrossRef

25.

Lütgendorf-Caucig C, Fotina I, Stock M et al (2011) Feasibility of CBCT-based target and normal structure delineation in prostate cancer radiotherapy: multi-observer and image multi-modality study. Radiother Oncol 98:154–161PubMedCrossRef

26.

Metwally H, Courbon F, David I et al (2011) Coregistration of prechemotherapy PET-CT for planning pediatric Hodgkin’s disease radiotherapy significantly diminishes interobserver variability of clinical target volume definition. Int J Radiat Oncol Biol Phys 80:793–799PubMedCrossRef

27.

Mitchell JR, Karlik SJ, Lee DH et al (1996) The variability of manual and computer assisted quantification of multiple sclerosis lesion volumes. Med Phys 23:85–97PubMedCrossRef

28.

Njeh CF (2008) Tumor delineation: The weakest link in the search for accuracy in radiotherapy. J Med Phys 33:136–140PubMedCrossRef

29.

Petersen RP, Truong PT, Kader HA et al (2007) Target volume delineation for partial breast radiotherapy planning: Clinical characteristics associated with low interobserver concordance. Int J Radiat Oncol Biol Phys 69:41–48PubMedCrossRef

30.

Pinkawa M, Holy R, Piroth MD et al (2010) Intensity-modulated radiotherapy for prostate cancer implementing molecular imaging with 18 F-choline PET-CT to define a simultaneous integrated boost. Strahlenther Onkol 186:600–606PubMedCrossRef

31.

Pinkawa M, Piroth MD, Holy R et al (2011) Combination of dose escalation with technological advances (intensity-modulated and image-guided radiotherapy) is not associated with increased morbidity for patients with prostate cancer. Strahlenther Onkol 187:479–484PubMedCrossRef

32.

Rasch C, Barillot I, Remeijer P et al (1999) Definition of the prostate in CT and MRI: a multi-observer study. Int J Radiat Oncol Biol Phys 43:57–66PubMedCrossRef

33.

Rasch C, Steenbakkers R, Herk M van (2005) Target definition in prostate, head, and neck. Semin Radiat Oncol 15:136–145PubMedCrossRef

34.

Rasch CR, Steenbakkers RJ, Fitton I et al (2010) Decreased 3D observer variation with matched CT-MRI, for target delineation in nasopharynx cancer. Radiat Oncol 5:21PubMedCrossRef

35.

Senan S, Koste J de, Samson M et al (1999) Evaluation of a target contouring protocol for 3D conformal radiotherapy in non-small cell lung cancer. Radiother Oncol 53:247–255PubMedCrossRef

36.

Smith WL, Lewis C, Bauman G et al (2007) Prostate volume contouring: a 3D analysis of segmentation using 3DTRUS, CT, and MR. Int J Radiat Oncol Biol Phys 67:1238–1247PubMedCrossRef

37.

Song W, Chiu B, Bauman G et al (2006) Prostate contouring uncertainty in megavoltage computed tomography images acquired with a helical tomotherapy unit during image-guided radiation therapy. Int J Radiat Oncol Biol Phys 65:595–607PubMedCrossRef

38.

Steenbakkers RJ, Duppen JC, Fitton I et al (2006) Reduction of observer variation using matched CT-PET for lung cancer delineation: a three-dimensional analysis. Int J Radiat Oncol Biol Phys 64:435–434PubMedCrossRef

39.

Stock M, Dörr W, Stromberger C et al (2010) Investigations on parotid gland recovery after IMRT in head and neck tumor patients. Strahlenther Onkol 186:665–671PubMedCrossRef

40.

Stroom JC, Heijmen BJ (2002) Geometrical uncertainties, radiotherapy planning margins, and the ICRU-62 report. Radiother Oncol 64:75–83PubMedCrossRef

41.

Struikmans H, Wárlám-Rodenhuis C, Stam T et al (2005) Interobserver variability of clinical target volume delineation of glandular breast tissue and of boost volume in tangential breast irradiation. Radiother Oncol 76:293–299PubMedCrossRef

42.

Tong S, Cardinal HN, McLoughlin RF et al (1998) Intra- and interobserver variability and reliability of prostate volume measurement via two-dimensional and three-dimensional ultrasound imaging. Ultrasound Med Biol 24:673–681PubMedCrossRef

43.

Tyng CJ, Chojniak R, Pinto PN et al (2009) Conformal radiotherapy for lung cancer: interobservers’ variability in the definition of gross tumor volume between radiologists and radiotherapists. Radiat Oncol 4:28PubMedCrossRef

44.

Usmani N, Sloboda R, Kamal W et al (2011) Can images obtained with high field strength magnetic resonance imaging reduce contouring variability of the prostate? Int J Radiat Oncol Biol Phys 80:728–734PubMedCrossRef

45.

Baardwijk A van, Bosmans G, Boersma et al (2007) PET-CT-based auto-contouring in non-small-cell lung cancer correlates with pathology and reduces interobserver variability in the delineation of the primary tumor and involved nodal volumes. Int J Radiat Oncol Biol Phys 68:771–778PubMedCrossRef

46.

Mourik AM van, Elkhuizen PH, Minkema D et al (2010) Multiinstitutional study on target volume delineation variation in breast radiotherapy in the presence of guidelines. Radiother Oncol 94:286–291PubMedCrossRef

47.

Villeirs GM, Van Vaerenbergh K, Vakaet L et al (2005) Interobserver delineation variation using CT versus combined CT + MRI in intensity-modulated radiotherapy for prostate cancer. Strahlenther Onkol 181:424–430PubMedCrossRef

48.

Vorwerk H, Beckmann G, Bremer M et al (2009) The delineation of target volumes for radiotherapy of lung cancer patients. Radiother Oncol 91:455–460PubMedCrossRef

49.

Weiss E, Hess CF (2003) The impact of gross tumor volume (GTV) and clinical target volume (CTV) definition on the total accuracy in radiotherapy theoretical aspects and practical experiences. Strahlenther Onkol 179:21–30PubMedCrossRef

50.

Weiss E, Richter S, Krauss T et al (2003) Conformal radiotherapy planning of cervix carcinoma: differences in the delineation of the clinical target volume. A comparison between gynaecologic and radiation oncologists. Radiother Oncol 67:87–95PubMedCrossRef

51.

Weiss E, Wu J, Sleeman W et al (2010) Clinical evaluation of soft tissue organ boundary visualization on cone-beam computed tomographic imaging. Int J Radiat Oncol Biol Phys 78:929–936PubMedCrossRef

52.

Weltens C, Menten J, Feron M et al (2001) Interobserver variations in gross tumor volume delineation of brain tumors on computed tomography and impact of magnetic resonance imaging. Radiother Oncol 60:49–59PubMedCrossRef

53.

White EA, Brock KK, Jaffray DA et al (2009) Inter-observer variability of prostate delineation on cone beam computerised tomography images. Clin Oncol 21:32–38CrossRef

Title: Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy
Authors: I. Fotina
C. Lütgendorf-Caucig
M. Stock
R. Pötter
D. Georg
Publication date: 01-02-2012
Publisher: Springer Berlin Heidelberg
Published in: Strahlentherapie und Onkologie / Issue 2/2012
Print ISSN: 0179-7158
Electronic ISSN: 1439-099X
DOI: https://doi.org/10.1007/s00066-011-0027-6

At a glance: The STEP trials

Springer Medicine

Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy

Abstract

Background and purpose

Patients and methods

Results

Conclusion

At a glance: The STEP trials

Springer Medicine

Abstract

Background and purpose

Patients and methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 2/2012

Evaluation of time, attendance of medical staff, and resources during radiotherapy for breast cancer patients

Radiotherapy with and without temozolomide in elderly patients with glioblastoma

RTOG-Studie 0324: Cetuximab bei Patienten mit nicht-kleinzelligem Bronchialkarzinom

Capecitabin plus Oxaliplatin verglichen mit Fluorouracil und Folinsäure als adjuvante Therapie bei Kolonkarzinom im Stadium III

Resection plus whole-brain irradiation versus resection plus whole-brain irradiation plus boost for the treatment of single brain metastasis

Radiosensitizing effect of epothilone B on human epithelial cancer cells