Skip to main content
Top
Published in: Radiation Oncology 1/2014

Open Access 01-12-2014 | Research

Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer: a multi-institutional contouring experience

Authors: Luciana Caravatta, Gabriella Macchia, Gian Carlo Mattiucci, Aldo Sainato, Nunzia LV Cernusco, Giovanna Mantello, Monica Di Tommaso, Marianna Trignani, Antonino De Paoli, Gianni Boz, Maria L Friso, Vincenzo Fusco, Marta Di Nicola, Alessio G Morganti, Domenico Genovesi

Published in: Radiation Oncology | Issue 1/2014

Login to get access

Abstract

Background

An observational multi-institutional study has been conducted aimed to evaluate the inter-observer variability in clinical target volume (CTV) delineation among different radiation oncologists in radiotherapy treatment of pancreatic cancer.

Methods

A multi-institutional contouring dummy-run of two different cases of pancreatic cancer treated by postoperative and preoperative radiotherapy (RT) was performed. Clinical history, diagnostics, and planning CT imaging were available on AIRO website (http://​www.​radioterapiaital​ia.​it). Participants were requested to delineate CTVs according to their skills and knowledge. Aiming to quantify interobserver variability of CTVs delineations, the total volume, craniocaudal, laterolateral, and anteroposterior diameters were calculated. Descriptive statistic was calculated. The 95% Confidence Interval (95% CI) for coefficient of variation (CV) was estimated. The Dice Similarity Index (DSI) was used to evaluate the spatial overlap accuracy of the different CTVs compared with the CTVs of a national reference Centre considered as a benchmark. The mean DSI (mDSI) was calculated and reported.

Results

A total of 18 radiation oncologists from different Institutes submitted the targets. Less variability was observed for the Elective CTV rather than the Boost CTV, in both cases. The estimated CV were 28.8% (95% CI: 21.2 - 45.0%) and 20.0% (95% CI: 14.9 - 30.6%) for the Elective CTV, in adjuvant (Case 1) and neoadjuvant (Case 2) case, respectively. The mDSI value was 0.68 for the Elective CTVs in both cases (range 0.19 - 0.79 in postoperative vs range 0.35 - 0.79 in preoperative case). The mDSI was increased to 0.71 (Case 1) and 0.72 (Case 2) if the observers with a worse agreement have been excluded. On the other hand, a CV of 42.4% (95% CI: 30.1 - 72.4%) and 63.8% (95% CI: 43.9 - 119.2%) with a mDSI value of 0.44 and 0.52, were calculated for the Boost CTV in Case 1 and Case 2, respectively.

Conclusions

The CV and mDSI obtained values for Elective CTVs showed an acceptable agreement among participants either in postoperative as well in preoperative setting. Additional strategies to reduce the variability in Boost CTV delineation need to be found and promoted.
Appendix
Available only for authorised users
Literature
1.
go back to reference Morganti AG, Massaccesi M, La Torre G, Caravatta L, Piscopo A, Tambaro R, Sofo L, Sallustio G, Ingrosso M, Macchia G, Deodato F, Picardi V, Ippolito E, Cellini N, Valentini V: A systematic review of resectability and survival after concurrent chemoradiation in primarily unresectable pancreatic cancer. Ann Surg Oncol 2010, 17: 194-205. 10.1245/s10434-009-0762-4CrossRefPubMed Morganti AG, Massaccesi M, La Torre G, Caravatta L, Piscopo A, Tambaro R, Sofo L, Sallustio G, Ingrosso M, Macchia G, Deodato F, Picardi V, Ippolito E, Cellini N, Valentini V: A systematic review of resectability and survival after concurrent chemoradiation in primarily unresectable pancreatic cancer. Ann Surg Oncol 2010, 17: 194-205. 10.1245/s10434-009-0762-4CrossRefPubMed
2.
go back to reference Goodman KA, Hajj C: Role of radiation therapy in the management of pancreatic cancer. J Surg Oncol 2013, 107: 86-96. 10.1002/jso.23137CrossRefPubMed Goodman KA, Hajj C: Role of radiation therapy in the management of pancreatic cancer. J Surg Oncol 2013, 107: 86-96. 10.1002/jso.23137CrossRefPubMed
3.
go back to reference Asiyanbola B, Gleisner A, Herman JM, Choti MA, Wolfgang CL, Swartz M, Edil BH, Schulick RD, Cameron JL, Pawlik TM: Determining pattern of recurrence following pancreaticoduodenectomy and adjuvant 5-flurouracil-based chemoradiation therapy: effect of number of metastatic lymph nodes and lymph node ratio. J Gastrointest Surg 2009, 13: 752-759. 10.1007/s11605-008-0762-xCrossRefPubMed Asiyanbola B, Gleisner A, Herman JM, Choti MA, Wolfgang CL, Swartz M, Edil BH, Schulick RD, Cameron JL, Pawlik TM: Determining pattern of recurrence following pancreaticoduodenectomy and adjuvant 5-flurouracil-based chemoradiation therapy: effect of number of metastatic lymph nodes and lymph node ratio. J Gastrointest Surg 2009, 13: 752-759. 10.1007/s11605-008-0762-xCrossRefPubMed
4.
go back to reference Yoshida T, Matsumoto T, Sasaki A, Shibata K, Aramaki M, Kitano S: Outcome of paraaortic node-positive pancreatic head and bile duct adenocarcinoma. Am J Surg 2004, 187: 736-740. 10.1016/j.amjsurg.2003.07.031CrossRefPubMed Yoshida T, Matsumoto T, Sasaki A, Shibata K, Aramaki M, Kitano S: Outcome of paraaortic node-positive pancreatic head and bile duct adenocarcinoma. Am J Surg 2004, 187: 736-740. 10.1016/j.amjsurg.2003.07.031CrossRefPubMed
5.
go back to reference Hishinuma S, Ogata Y, Tomikawa M, Ozawa I, Hirabayashi K, Igarashi S: Patterns of recurrence after curative resection of pancreatic cancer, based on autopsy findings. J Gastrointest Surg 2006, 10: 511-518. 10.1016/j.gassur.2005.09.016CrossRefPubMed Hishinuma S, Ogata Y, Tomikawa M, Ozawa I, Hirabayashi K, Igarashi S: Patterns of recurrence after curative resection of pancreatic cancer, based on autopsy findings. J Gastrointest Surg 2006, 10: 511-518. 10.1016/j.gassur.2005.09.016CrossRefPubMed
6.
go back to reference Crane CH, Antolak JA, Rosen II, Forster KM, Evans DB, Janjan NA, Charnsangavej C, Pisters PW, Lenzi R, Papagikos MA, Wolff RA: Phase I study of concomitant gemcitabine and IMRT for patients with unresectable adenocarcinoma of the pancreatic head. Int J Gastrointest Cancer 2001, 30: 123-132. 10.1385/IJGC:30:3:123CrossRefPubMed Crane CH, Antolak JA, Rosen II, Forster KM, Evans DB, Janjan NA, Charnsangavej C, Pisters PW, Lenzi R, Papagikos MA, Wolff RA: Phase I study of concomitant gemcitabine and IMRT for patients with unresectable adenocarcinoma of the pancreatic head. Int J Gastrointest Cancer 2001, 30: 123-132. 10.1385/IJGC:30:3:123CrossRefPubMed
7.
go back to reference Yovino S, Poppe M, Jabbour S, David V, Garofalo M, Pandya N, Alexander R, Hanna N, Regine WF: Intensity-modulated radiation therapy significantly improves acute gastrointestinal toxicity in pancreatic and ampullary cancers. Int J Radiat Oncol Biol Phys 2011, 79: 158-162. 10.1016/j.ijrobp.2009.10.043CrossRefPubMed Yovino S, Poppe M, Jabbour S, David V, Garofalo M, Pandya N, Alexander R, Hanna N, Regine WF: Intensity-modulated radiation therapy significantly improves acute gastrointestinal toxicity in pancreatic and ampullary cancers. Int J Radiat Oncol Biol Phys 2011, 79: 158-162. 10.1016/j.ijrobp.2009.10.043CrossRefPubMed
8.
go back to reference Caravatta L, Macchia G, Deodato F, Felicetti M, Cellini F, Ciabattoni A, Buwenge M, Picardi V, Cilla S, Scapati A, Valentini V, Morganti AG: Intensity-Modulated radiotherapy in the treatment of pancreatic adenocarcinoma: a review. EMJ Oncol 2013, 1: 90-97. Caravatta L, Macchia G, Deodato F, Felicetti M, Cellini F, Ciabattoni A, Buwenge M, Picardi V, Cilla S, Scapati A, Valentini V, Morganti AG: Intensity-Modulated radiotherapy in the treatment of pancreatic adenocarcinoma: a review. EMJ Oncol 2013, 1: 90-97.
9.
go back to reference NCCN: Clinical Practice Guidelines in Oncology, Pancreatic Adenocarcinoma. 2013. NCCN: Clinical Practice Guidelines in Oncology, Pancreatic Adenocarcinoma. 2013.
10.
go back to reference Fiorino C, Reni M, Bolognesi A, Cattaneo GM, Calandrino R: Intra- and inter-observer variability in contouring prostate and seminal vesicles: implications for conformal treatment planning. Radiother Oncol 1998, 47: 285-292. 10.1016/S0167-8140(98)00021-8CrossRefPubMed Fiorino C, Reni M, Bolognesi A, Cattaneo GM, Calandrino R: Intra- and inter-observer variability in contouring prostate and seminal vesicles: implications for conformal treatment planning. Radiother Oncol 1998, 47: 285-292. 10.1016/S0167-8140(98)00021-8CrossRefPubMed
11.
go back to reference Brunner TB, Merkel S, Grabenbauer GG, Meyer T, Baum U, Papadopoulos T, Sauer R, Hohenberger W: Definition of elective lymphatic target volume in ductal carcinoma of the pancreatic head based on histopathologic analysis. Int J Radiat Oncol Biol Phys 2005, 62: 1021-1029. 10.1016/j.ijrobp.2004.12.018CrossRefPubMed Brunner TB, Merkel S, Grabenbauer GG, Meyer T, Baum U, Papadopoulos T, Sauer R, Hohenberger W: Definition of elective lymphatic target volume in ductal carcinoma of the pancreatic head based on histopathologic analysis. Int J Radiat Oncol Biol Phys 2005, 62: 1021-1029. 10.1016/j.ijrobp.2004.12.018CrossRefPubMed
12.
go back to reference Goodman KA, Regine WF, Dawson LA, Ben-Josef E, Haustermans K, Bosch WR, Turian J, Abrams RA: Radiation Therapy Oncology Group consensus panel guidelines for the delineation of the clinical target volume in the postoperative treatment of pancreatic head cancer. Int J Radiat Oncol Biol Phys 2012, 83: 901-908. 10.1016/j.ijrobp.2012.01.022CrossRefPubMed Goodman KA, Regine WF, Dawson LA, Ben-Josef E, Haustermans K, Bosch WR, Turian J, Abrams RA: Radiation Therapy Oncology Group consensus panel guidelines for the delineation of the clinical target volume in the postoperative treatment of pancreatic head cancer. Int J Radiat Oncol Biol Phys 2012, 83: 901-908. 10.1016/j.ijrobp.2012.01.022CrossRefPubMed
13.
go back to reference Sun W, Leong CN, Zhang Z, Lu JJ: Proposing the lymphatic target volume for elective radiation therapy for pancreatic cancer: a pooled analysis of clinical evidence. Radiat Oncol 2010, 5: 28. 10.1186/1748-717X-5-28PubMedCentralCrossRefPubMed Sun W, Leong CN, Zhang Z, Lu JJ: Proposing the lymphatic target volume for elective radiation therapy for pancreatic cancer: a pooled analysis of clinical evidence. Radiat Oncol 2010, 5: 28. 10.1186/1748-717X-5-28PubMedCentralCrossRefPubMed
14.
go back to reference Caravatta L, Sallustio G, Pacelli F, Padula GD, Deodato F, Macchia G, Massaccesi M, Picardi V, Cilla S, Marinelli A, Cellini N, Valentini V, Morganti AG: Clinical target volume delineation including elective nodal irradiation in preoperative and definitive radiotherapy of pancreatic cancer. Radiat Oncol 2012,12(7):86.CrossRef Caravatta L, Sallustio G, Pacelli F, Padula GD, Deodato F, Macchia G, Massaccesi M, Picardi V, Cilla S, Marinelli A, Cellini N, Valentini V, Morganti AG: Clinical target volume delineation including elective nodal irradiation in preoperative and definitive radiotherapy of pancreatic cancer. Radiat Oncol 2012,12(7):86.CrossRef
15.
go back to reference Associazione Italiana Radioterapia Oncologica (AIRO) - Gruppo di Studio per i Tumori Gastrointestinali: Tumori Gastrointestinali - Indicazioni e Criteri Guida. Cap 2012, 3: 82. Associazione Italiana Radioterapia Oncologica (AIRO) - Gruppo di Studio per i Tumori Gastrointestinali: Tumori Gastrointestinali - Indicazioni e Criteri Guida. Cap 2012, 3: 82.
16.
go back to reference Japan Pancreas Society: Classification of pancreatic carcinoma. Tokyo: Kanehara; 2003. 2nd English Japan Pancreas Society: Classification of pancreatic carcinoma. Tokyo: Kanehara; 2003. 2nd English
17.
go back to reference Genovesi D, Ausili-Cèfaro G, Vinciguerra A, Augurio A, Di Tommaso M, Marchese R, Ricardi U, Filippi AR, Girinsky T, Di Biagio K, Belfiglio M, Barbieri E, Valentini V: Interobserver variability of clinical target volume delineation in supra-diaphragmatic Hodgkin’s disease: a multi-institutional experience. Strahlenther Onkol 2011, 187: 357-366. 10.1007/s00066-011-2221-yCrossRefPubMed Genovesi D, Ausili-Cèfaro G, Vinciguerra A, Augurio A, Di Tommaso M, Marchese R, Ricardi U, Filippi AR, Girinsky T, Di Biagio K, Belfiglio M, Barbieri E, Valentini V: Interobserver variability of clinical target volume delineation in supra-diaphragmatic Hodgkin’s disease: a multi-institutional experience. Strahlenther Onkol 2011, 187: 357-366. 10.1007/s00066-011-2221-yCrossRefPubMed
18.
go back to reference Genovesi D, Ausili-Cèfaro G, Trignani M, Vinciguerra A, Augurio A, Di Tommaso M, Perrotti F, De Paoli A, Olmi P, Valentini V, Di Nicola M: Interobserver variability of clinical target volume delineation in soft-tissue sarcomas. Cancer/Radiothérapie Available online 17 January 2014. In press Genovesi D, Ausili-Cèfaro G, Trignani M, Vinciguerra A, Augurio A, Di Tommaso M, Perrotti F, De Paoli A, Olmi P, Valentini V, Di Nicola M: Interobserver variability of clinical target volume delineation in soft-tissue sarcomas. Cancer/Radiothérapie Available online 17 January 2014. In press
19.
go back to reference Dice LR: Measures of the amount of ecologic association between species. Ecology 1945, 26: 297-302. 10.2307/1932409CrossRef Dice LR: Measures of the amount of ecologic association between species. Ecology 1945, 26: 297-302. 10.2307/1932409CrossRef
20.
go back to reference Fotina I, Lütgendorf-Caucig C, Stock M, Pötter R, Georg D: Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy. Strahlenther Onkol 2012, 188: 160-167. 10.1007/s00066-011-0027-6CrossRefPubMed Fotina I, Lütgendorf-Caucig C, Stock M, Pötter R, Georg D: Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy. Strahlenther Onkol 2012, 188: 160-167. 10.1007/s00066-011-0027-6CrossRefPubMed
21.
go back to reference Fokas E, Eccles C, Patel N, Chu KY, Warren S, McKenna WG, Brunner TB: A treatment planning comparison of four target volume contouring guidelines for locally advanced pancreatic cancer radiotherapy. Radiother Oncol 2013,107(2):200-206. 10.1016/j.radonc.2013.04.010CrossRefPubMed Fokas E, Eccles C, Patel N, Chu KY, Warren S, McKenna WG, Brunner TB: A treatment planning comparison of four target volume contouring guidelines for locally advanced pancreatic cancer radiotherapy. Radiother Oncol 2013,107(2):200-206. 10.1016/j.radonc.2013.04.010CrossRefPubMed
22.
go back to reference Abrams RA, Winter KA, Regine WF, Safran H, Hoffman JP, Lustig R, Konski AA, Benson AB, Macdonald JS, Rich TA, Willett CG: Failure to adhere to protocol specified radiation therapy guidelines was associated with decreased survival in RTOG 9704--a phase III trial of adjuvant chemotherapy and chemoradiotherapy for patients with resected adenocarcinoma of the pancreas. Int J Radiat Oncol Biol Phys 2012,82(2):809-816. 10.1016/j.ijrobp.2010.11.039PubMedCentralCrossRefPubMed Abrams RA, Winter KA, Regine WF, Safran H, Hoffman JP, Lustig R, Konski AA, Benson AB, Macdonald JS, Rich TA, Willett CG: Failure to adhere to protocol specified radiation therapy guidelines was associated with decreased survival in RTOG 9704--a phase III trial of adjuvant chemotherapy and chemoradiotherapy for patients with resected adenocarcinoma of the pancreas. Int J Radiat Oncol Biol Phys 2012,82(2):809-816. 10.1016/j.ijrobp.2010.11.039PubMedCentralCrossRefPubMed
23.
go back to reference Willett CG, Moughan J, O’Meara E, Galvin JM, Crane CH, Winter K, Manfredi D, Rich TA, Rabinovitch R, Lustig R, Machtay M, Curran WJ: Compliance with therapeutic guidelines in Radiation Therapy Oncology Group prospective gastrointestinal clinical trials. Radiother Oncol 2012,105(1):9-13. 10.1016/j.radonc.2012.09.011PubMedCentralCrossRefPubMed Willett CG, Moughan J, O’Meara E, Galvin JM, Crane CH, Winter K, Manfredi D, Rich TA, Rabinovitch R, Lustig R, Machtay M, Curran WJ: Compliance with therapeutic guidelines in Radiation Therapy Oncology Group prospective gastrointestinal clinical trials. Radiother Oncol 2012,105(1):9-13. 10.1016/j.radonc.2012.09.011PubMedCentralCrossRefPubMed
24.
go back to reference Metha VK: Radiation Oncology: An Evidence-Based Approach. Edited by: Lu JJ, Brady LW. Berlin Heidelberg: Springer-Verlag; 2008. Metha VK: Radiation Oncology: An Evidence-Based Approach. Edited by: Lu JJ, Brady LW. Berlin Heidelberg: Springer-Verlag; 2008.
25.
go back to reference Halperin EC, Perez CA, Brady LW: Principles and Practice of Radiation Oncology. 5th edition. Philadelphia, Pa: Lippincott Williams & Wilkins; 2008. Halperin EC, Perez CA, Brady LW: Principles and Practice of Radiation Oncology. 5th edition. Philadelphia, Pa: Lippincott Williams & Wilkins; 2008.
26.
go back to reference Brunner TB, Baum U, Grabenbauer GG, Sauer R, Lambrecht U: Large topographic variability of upper abdominal lymphatics and the consequences for radiation treatment planning. Radiother Oncol 2006, 81: 190-195. 10.1016/j.radonc.2006.09.007CrossRefPubMed Brunner TB, Baum U, Grabenbauer GG, Sauer R, Lambrecht U: Large topographic variability of upper abdominal lymphatics and the consequences for radiation treatment planning. Radiother Oncol 2006, 81: 190-195. 10.1016/j.radonc.2006.09.007CrossRefPubMed
27.
go back to reference Batumalai V, Koh ES, Delaney GP, Holloway LC, Jameson MG, Papadatos G, Lonergan DM: Interobserver variability in clinical target volume delineation in tangential breast irradiation: a comparison between radiation oncologists and radiation therapists. Clin Oncol (R Coll Radiol) 2011, 23: 108-113. 10.1016/j.clon.2010.10.004CrossRef Batumalai V, Koh ES, Delaney GP, Holloway LC, Jameson MG, Papadatos G, Lonergan DM: Interobserver variability in clinical target volume delineation in tangential breast irradiation: a comparison between radiation oncologists and radiation therapists. Clin Oncol (R Coll Radiol) 2011, 23: 108-113. 10.1016/j.clon.2010.10.004CrossRef
28.
go back to reference Crippa S, Partelli S, Falconi M: Extent of surgical resections for intraductal papillary mucinous neoplasms. World J Gastrointest Surg 2010, 2: 347-351. 10.4240/wjgs.v2.i10.347PubMedCentralCrossRefPubMed Crippa S, Partelli S, Falconi M: Extent of surgical resections for intraductal papillary mucinous neoplasms. World J Gastrointest Surg 2010, 2: 347-351. 10.4240/wjgs.v2.i10.347PubMedCentralCrossRefPubMed
29.
go back to reference Yamazaki H, Nishiyama K, Tanaka E, Koiwai K, Shikama N, Ito Y, Arahira S, Tamamoto T, Shibata T, Tamaki Y, Kodaira T, Oguchi M: Dummy run for a phase II multi-institute trial of chemoradiotherapy for unresectable pancreatic cancer: inter-observer variance in contour delineation. Anticancer Res 2007, 27: 2965-2971.PubMed Yamazaki H, Nishiyama K, Tanaka E, Koiwai K, Shikama N, Ito Y, Arahira S, Tamamoto T, Shibata T, Tamaki Y, Kodaira T, Oguchi M: Dummy run for a phase II multi-institute trial of chemoradiotherapy for unresectable pancreatic cancer: inter-observer variance in contour delineation. Anticancer Res 2007, 27: 2965-2971.PubMed
Metadata
Title
Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer: a multi-institutional contouring experience
Authors
Luciana Caravatta
Gabriella Macchia
Gian Carlo Mattiucci
Aldo Sainato
Nunzia LV Cernusco
Giovanna Mantello
Monica Di Tommaso
Marianna Trignani
Antonino De Paoli
Gianni Boz
Maria L Friso
Vincenzo Fusco
Marta Di Nicola
Alessio G Morganti
Domenico Genovesi
Publication date
01-12-2014
Publisher
BioMed Central
Published in
Radiation Oncology / Issue 1/2014
Electronic ISSN: 1748-717X
DOI
https://doi.org/10.1186/1748-717X-9-198

Other articles of this Issue 1/2014

Radiation Oncology 1/2014 Go to the issue