Skip to main content
Top
Published in: European Journal of Nuclear Medicine and Molecular Imaging 5/2016

01-05-2016 | Original Article

Impact of consensus contours from multiple PET segmentation methods on the accuracy of functional volume delineation

Authors: A. Schaefer, M. Vermandel, C. Baillet, A. S. Dewalle-Vignion, R. Modzelewski, P. Vera, L. Massoptier, C. Parcq, D. Gibon, T. Fechter, U. Nemer, I. Gardin, U. Nestle

Published in: European Journal of Nuclear Medicine and Molecular Imaging | Issue 5/2016

Login to get access

Abstract

Purpose

The aim of this study was to evaluate the impact of consensus algorithms on segmentation results when applied to clinical PET images. In particular, whether the use of the majority vote or STAPLE algorithm could improve the accuracy and reproducibility of the segmentation provided by the combination of three semiautomatic segmentation algorithms was investigated.

Methods

Three published segmentation methods (contrast-oriented, possibility theory and adaptive thresholding) and two consensus algorithms (majority vote and STAPLE) were implemented in a single software platform (Artiview®). Four clinical datasets including different locations (thorax, breast, abdomen) or pathologies (primary NSCLC tumours, metastasis, lymphoma) were used to evaluate accuracy and reproducibility of the consensus approach in comparison with pathology as the ground truth or CT as a ground truth surrogate.

Results

Variability in the performance of the individual segmentation algorithms for lesions of different tumour entities reflected the variability in PET images in terms of resolution, contrast and noise. Independent of location and pathology of the lesion, however, the consensus method resulted in improved accuracy in volume segmentation compared with the worst-performing individual method in the majority of cases and was close to the best-performing method in many cases. In addition, the implementation revealed high reproducibility in the segmentation results with small changes in the respective starting conditions. There were no significant differences in the results with the STAPLE algorithm and the majority vote algorithm.

Conclusion

This study showed that combining different PET segmentation methods by the use of a consensus algorithm offers robustness against the variable performance of individual segmentation methods and this approach would therefore be useful in radiation oncology. It might also be relevant for other scenarios such as the merging of expert recommendations in clinical routine and trials or the multiobserver generation of contours for standardization of automatic contouring.
Literature
1.
go back to reference Miller TR, Grigsby PW. Measurement of tumor volume by PET to evaluate prognosis in patients with advanced cervical cancer treated by radiation therapy. Int J Radiat Oncol Biol Phys. 2002;53:353–9.CrossRefPubMed Miller TR, Grigsby PW. Measurement of tumor volume by PET to evaluate prognosis in patients with advanced cervical cancer treated by radiation therapy. Int J Radiat Oncol Biol Phys. 2002;53:353–9.CrossRefPubMed
2.
go back to reference Fernando S, Kong F, Kessler M, Chetty I, Narayan S, Tatro D, et al. Using FDG-PET to delineate gross tumor and internal target volumes. Int J Radiat Oncol Biol Phys. 2005;63 Suppl 1:S400-1.CrossRef Fernando S, Kong F, Kessler M, Chetty I, Narayan S, Tatro D, et al. Using FDG-PET to delineate gross tumor and internal target volumes. Int J Radiat Oncol Biol Phys. 2005;63 Suppl 1:S400-1.CrossRef
3.
go back to reference Zasadny KR, Kison PV, Francis IR, Wahl RL. FDG-PET determination of metabolically active tumor volume and comparison with CT. Clin Positron Imaging. 1998;1:123–9. Zasadny KR, Kison PV, Francis IR, Wahl RL. FDG-PET determination of metabolically active tumor volume and comparison with CT. Clin Positron Imaging. 1998;1:123–9.
4.
go back to reference Bryant AS, Cerfolio RJ. The maximum standardized uptake values on integrated FDG-PET/CT is useful in differentiating benign from malignant pulmonary nodules. Ann Thorac Surg. 2006;81:1016–20.CrossRef Bryant AS, Cerfolio RJ. The maximum standardized uptake values on integrated FDG-PET/CT is useful in differentiating benign from malignant pulmonary nodules. Ann Thorac Surg. 2006;81:1016–20.CrossRef
5.
go back to reference Black QC, Grills IS, Kestin LL, Wong CY, Wong JW, Martinez AA, et al. Defining a radiotherapy target with positron emission tomography. Int J Radiat Oncol Biol Phys. 2004;60:1272–82.CrossRefPubMed Black QC, Grills IS, Kestin LL, Wong CY, Wong JW, Martinez AA, et al. Defining a radiotherapy target with positron emission tomography. Int J Radiat Oncol Biol Phys. 2004;60:1272–82.CrossRefPubMed
6.
go back to reference Daisne JF, Duprez T, Weynand B, Lonneux M, Hamoir M, Reychler H, et al. Tumor volume in pharyngolaryngeal squamous cell carcinoma: Comparison between CT, MR imaging, and FDG PET and validation with surgical specimen. Radiology. 2004;233:93–100.CrossRefPubMed Daisne JF, Duprez T, Weynand B, Lonneux M, Hamoir M, Reychler H, et al. Tumor volume in pharyngolaryngeal squamous cell carcinoma: Comparison between CT, MR imaging, and FDG PET and validation with surgical specimen. Radiology. 2004;233:93–100.CrossRefPubMed
7.
go back to reference Daisne JF, Sibomana M, Bol A, Doumont T, Lonneux M, Grégoire V. Tri-dimensional automatic segmentation of PET volumes based on measured source-to-background ratios: influence of the reconstruction algorithms. Radiother Oncol. 2003;69:247–50.CrossRefPubMed Daisne JF, Sibomana M, Bol A, Doumont T, Lonneux M, Grégoire V. Tri-dimensional automatic segmentation of PET volumes based on measured source-to-background ratios: influence of the reconstruction algorithms. Radiother Oncol. 2003;69:247–50.CrossRefPubMed
8.
go back to reference Erdi YE, Mawlawi O, Larson SM, Imbriaco M, Yeung H, Finn R, et al. Segmentation of lung lesion volume by adaptive positron emission tomography image thresholding. Cancer. 1997;80(12 Suppl):2505–9.CrossRefPubMed Erdi YE, Mawlawi O, Larson SM, Imbriaco M, Yeung H, Finn R, et al. Segmentation of lung lesion volume by adaptive positron emission tomography image thresholding. Cancer. 1997;80(12 Suppl):2505–9.CrossRefPubMed
9.
go back to reference Erdi YE, Wessels BW, Loew MH, Erdi AK. Threshold estimation in single photon emission computed tomography and planar imaging for clinical radioimmunotherapy. Cancer Res. 1995;55(23 Suppl):S5823–6. Erdi YE, Wessels BW, Loew MH, Erdi AK. Threshold estimation in single photon emission computed tomography and planar imaging for clinical radioimmunotherapy. Cancer Res. 1995;55(23 Suppl):S5823–6.
10.
go back to reference Jentzen W, Freudenberg L, Eising EG, Heinze M, Brandau W, Bockisch A. Segmentation of PET volumes by iterative image thresholding. J Nucl Med. 2007;48:108–14.PubMed Jentzen W, Freudenberg L, Eising EG, Heinze M, Brandau W, Bockisch A. Segmentation of PET volumes by iterative image thresholding. J Nucl Med. 2007;48:108–14.PubMed
11.
go back to reference Nestle U, Kremp S, Schaefer-Schuler A, Sebastian-Welsch C, Hellwig D, Rube C, et al. Comparison of different methods for delineation of 18F-FDG PET-positive tissue for target volume definition in radiotherapy of patients with non-small cell lung cancer. J Nucl Med. 2005;46:1342–8.PubMed Nestle U, Kremp S, Schaefer-Schuler A, Sebastian-Welsch C, Hellwig D, Rube C, et al. Comparison of different methods for delineation of 18F-FDG PET-positive tissue for target volume definition in radiotherapy of patients with non-small cell lung cancer. J Nucl Med. 2005;46:1342–8.PubMed
12.
go back to reference Nestle U, Schaefer-Schuler A, Kremp S, Groeschel A, Hellwig D, Rube C, et al. Target volume definition for 18 F-FDG PET-positive lymph nodes in radiotherapy of patients with non-small cell lung cancer. Eur J Nucl Med Mol Imaging. 2007;34:453–62.CrossRefPubMed Nestle U, Schaefer-Schuler A, Kremp S, Groeschel A, Hellwig D, Rube C, et al. Target volume definition for 18 F-FDG PET-positive lymph nodes in radiotherapy of patients with non-small cell lung cancer. Eur J Nucl Med Mol Imaging. 2007;34:453–62.CrossRefPubMed
13.
go back to reference Schaefer A, Kremp S, Hellwig D, Rube C, Kirsch CM, Nestle U. A contrast-oriented algorithm for FDG-PET-based delineation of tumour volumes for the radiotherapy of lung cancer: derivation from phantom measurements and validation in patient data. Eur J Nucl Med Mol Imaging. 2008;35:1989–99.CrossRefPubMed Schaefer A, Kremp S, Hellwig D, Rube C, Kirsch CM, Nestle U. A contrast-oriented algorithm for FDG-PET-based delineation of tumour volumes for the radiotherapy of lung cancer: derivation from phantom measurements and validation in patient data. Eur J Nucl Med Mol Imaging. 2008;35:1989–99.CrossRefPubMed
14.
go back to reference Riddell C, Brigger P, Carson RE, Bacharach SL. The watershed algorithm: a method to segment noisy PET transmission images. IEEE Trans Nucl Sci. 1999;46:713–9.CrossRef Riddell C, Brigger P, Carson RE, Bacharach SL. The watershed algorithm: a method to segment noisy PET transmission images. IEEE Trans Nucl Sci. 1999;46:713–9.CrossRef
15.
go back to reference Tylski P, Bonniaud G, Decenciere E, Stawiaski J, Coulot J, Lefkopoulos D, et al. 18F-FDG PET images segmentation using morphological watershed: a phantom study. IEEE Nucl Sci Symp Conf Rec. 2006;4:2063–7. Tylski P, Bonniaud G, Decenciere E, Stawiaski J, Coulot J, Lefkopoulos D, et al. 18F-FDG PET images segmentation using morphological watershed: a phantom study. IEEE Nucl Sci Symp Conf Rec. 2006;4:2063–7.
16.
go back to reference Nissen I, Yaqub M, Lammertsma A, Lee J, Geets X, Boellaard R. A novel supervised watershed method for segmentation of tumors with heterogeneous tracer uptake in PET. Proceedings of the Society of Nuclear Medicine Annual Meeting, St. Louis, Mo, 7–11 June 2014. Abstract 260. Nissen I, Yaqub M, Lammertsma A, Lee J, Geets X, Boellaard R. A novel supervised watershed method for segmentation of tumors with heterogeneous tracer uptake in PET. Proceedings of the Society of Nuclear Medicine Annual Meeting, St. Louis, Mo, 7–11 June 2014. Abstract 260.
17.
go back to reference Geets X, Lee JA, Bol A, Lonneux M, Grégoire V. A gradient-based method for segmenting FDG-PET images: methodology and validation. Eur J Nucl Med Mol Imaging. 2007;34:1427–38.CrossRefPubMed Geets X, Lee JA, Bol A, Lonneux M, Grégoire V. A gradient-based method for segmenting FDG-PET images: methodology and validation. Eur J Nucl Med Mol Imaging. 2007;34:1427–38.CrossRefPubMed
18.
go back to reference Zhu W, Jiang T. Automation segmentation of PET image for brain tumors. IEEE Nucl Sci Symp Conf Rec. 2003;4:2627–9. Zhu W, Jiang T. Automation segmentation of PET image for brain tumors. IEEE Nucl Sci Symp Conf Rec. 2003;4:2627–9.
19.
go back to reference Belhassen S, Zaidi H. A novel fuzzy C-means algorithm for unsupervised heterogeneous tumor quantification in PET. Med Phys. 2010;37:1309–24.CrossRefPubMed Belhassen S, Zaidi H. A novel fuzzy C-means algorithm for unsupervised heterogeneous tumor quantification in PET. Med Phys. 2010;37:1309–24.CrossRefPubMed
21.
go back to reference Dewalle-Vignion AS, Betrouni N, Makni N, Huglo D, Rousseau J, Vermandel M. A new method based on both fuzzy set and possibility theories for tumor volume segmentation on PET images. Conf Proc IEEE Eng Med Biol Soc. 2008;2008:3122–5. Dewalle-Vignion AS, Betrouni N, Makni N, Huglo D, Rousseau J, Vermandel M. A new method based on both fuzzy set and possibility theories for tumor volume segmentation on PET images. Conf Proc IEEE Eng Med Biol Soc. 2008;2008:3122–5.
25.
go back to reference Shepherd T, Teras M, Beichel R, Boellaard R, Bruynooghe M, Dicken V, et al. Comparative study with new accuracy metrics for target volume contouring in PET image guided radiation therapy. IEEE Trans Med Imaging. 2012;31:2006–24. doi:10.1109/TMI.2012.2202322. Shepherd T, Teras M, Beichel R, Boellaard R, Bruynooghe M, Dicken V, et al. Comparative study with new accuracy metrics for target volume contouring in PET image guided radiation therapy. IEEE Trans Med Imaging. 2012;31:2006–24. doi:10.​1109/​TMI.​2012.​2202322.
26.
go back to reference Østergaard LR, Larsen OV. Applying voting to segmentation of MR images. In: Amin A, Dori D, Pudil P, Freeman H, editors. Advances in pattern recognition. Berlin: Springer; 1998. p. 795–804. Østergaard LR, Larsen OV. Applying voting to segmentation of MR images. In: Amin A, Dori D, Pudil P, Freeman H, editors. Advances in pattern recognition. Berlin: Springer; 1998. p. 795–804.
28.
go back to reference Lam L, Suen CY. Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans Syst Man Cybern A Syst Humans. 1997;27:553–68.CrossRef Lam L, Suen CY. Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans Syst Man Cybern A Syst Humans. 1997;27:553–68.CrossRef
32.
go back to reference Vauclin S, Doyeux K, Hapdey S, Edet-Sanson A, Vera P, Gardin I. Development of a generic thresholding algorithm for the delineation of 18 FDG-PET-positive tissue: application to the comparison of three thresholding models. Phys Med Biol. 2009;54:6901–16.CrossRefPubMed Vauclin S, Doyeux K, Hapdey S, Edet-Sanson A, Vera P, Gardin I. Development of a generic thresholding algorithm for the delineation of 18 FDG-PET-positive tissue: application to the comparison of three thresholding models. Phys Med Biol. 2009;54:6901–16.CrossRefPubMed
33.
go back to reference Doll C, Parcq C, Modzelewski R, Dewalle-Vignion AS, Christ U, Loquin K, et al. PET-based target volume delineation in radiation therapy planning: are different implementations of the same automatic delineation method really equal? Proceedings of the Annual Congress of the European Association of Nuclear Medicine (EANM'13), October 2013, Lyon, France. Vol. 40 (2 suppl), p. S244 Doll C, Parcq C, Modzelewski R, Dewalle-Vignion AS, Christ U, Loquin K, et al. PET-based target volume delineation in radiation therapy planning: are different implementations of the same automatic delineation method really equal? Proceedings of the Annual Congress of the European Association of Nuclear Medicine (EANM'13), October 2013, Lyon, France. Vol. 40 (2 suppl), p. S244
34.
go back to reference Schaefer A, Nestle U, Kremp S, Hellwig D, Grgic A, Buchholz HG, et al. Multi-centre calibration of an adaptive thresholding method for PET-based delineation of tumour volumes in radiotherapy planning of lung cancer. Nuklearmedizin. 2012;51:101–10. doi:10.3413/Nukmed-0452-11-12.CrossRefPubMed Schaefer A, Nestle U, Kremp S, Hellwig D, Grgic A, Buchholz HG, et al. Multi-centre calibration of an adaptive thresholding method for PET-based delineation of tumour volumes in radiotherapy planning of lung cancer. Nuklearmedizin. 2012;51:101–10. doi:10.​3413/​Nukmed-0452-11-12.CrossRefPubMed
35.
go back to reference Zadeh LA. Fuzzy sets as the basis for a theory of possibility. Fuzzy Sets Systems. 1978;1:3–28.CrossRef Zadeh LA. Fuzzy sets as the basis for a theory of possibility. Fuzzy Sets Systems. 1978;1:3–28.CrossRef
36.
go back to reference Kittler J, Alkoot FM. Sum versus vote fusion in multiple classifier systems. IEEE Trans Pattern Anal Mach Intell. 2003;25:110–5.CrossRef Kittler J, Alkoot FM. Sum versus vote fusion in multiple classifier systems. IEEE Trans Pattern Anal Mach Intell. 2003;25:110–5.CrossRef
37.
go back to reference Dewalle-Vignion AS, Betrouni N, Baillet C, Vermandel M. Is STAPLE algorithm confident to assess segmentation methods in PET imaging? Phys Med Biol. 2015. In press. Dewalle-Vignion AS, Betrouni N, Baillet C, Vermandel M. Is STAPLE algorithm confident to assess segmentation methods in PET imaging? Phys Med Biol. 2015. In press.
38.
go back to reference Schaefer A, Kim YJ, Kremp S, Mai S, Fleckenstein J, Bohnenberger H, et al. PET-based delineation of tumour volumes in lung cancer: comparison with pathological findings. Eur J Nucl Med Mol Imaging. 2013;40:1233–44. doi:10.1007/s00259-013-2407-x.CrossRefPubMed Schaefer A, Kim YJ, Kremp S, Mai S, Fleckenstein J, Bohnenberger H, et al. PET-based delineation of tumour volumes in lung cancer: comparison with pathological findings. Eur J Nucl Med Mol Imaging. 2013;40:1233–44. doi:10.​1007/​s00259-013-2407-x.CrossRefPubMed
40.
go back to reference Zou KH, Warfield SK, Bharatha A, Tempany CM, Kaus MR, Haker SJ, et al. Statistical validation of image segmentation quality based on a spatial overlap index. Acad Radiol. 2004;11:178–89.CrossRefPubMedPubMedCentral Zou KH, Warfield SK, Bharatha A, Tempany CM, Kaus MR, Haker SJ, et al. Statistical validation of image segmentation quality based on a spatial overlap index. Acad Radiol. 2004;11:178–89.CrossRefPubMedPubMedCentral
41.
44.
go back to reference Steenbakkers RJ, Duppen JC, Fitton I, Deurloo KE, Zijp L, Uitterhoeve AL, et al. Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a ‘Big Brother’ evaluation. Radiother Oncol. 2005;77:182–90. doi:10.1016/j.radonc.2005.09.017.CrossRefPubMed Steenbakkers RJ, Duppen JC, Fitton I, Deurloo KE, Zijp L, Uitterhoeve AL, et al. Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a ‘Big Brother’ evaluation. Radiother Oncol. 2005;77:182–90. doi:10.​1016/​j.​radonc.​2005.​09.​017.CrossRefPubMed
45.
go back to reference Erasmus JJ, Gladish GW, Broemeling L, Sabloff BS, Truong MT, Herbst RS, et al. Interobserver and intraobserver variability in measurement of non-small-cell carcinoma lung lesions: implications for assessment of tumor response. J Clin Oncol. 2003;21:2574–82. doi:10.1200/JCO.2003.01.144.CrossRefPubMed Erasmus JJ, Gladish GW, Broemeling L, Sabloff BS, Truong MT, Herbst RS, et al. Interobserver and intraobserver variability in measurement of non-small-cell carcinoma lung lesions: implications for assessment of tumor response. J Clin Oncol. 2003;21:2574–82. doi:10.​1200/​JCO.​2003.​01.​144.CrossRefPubMed
46.
go back to reference McGurk RJ. Consensus segmentation for positron emission tomography: development and applications in radiation therapy. Duke University. 2013. McGurk RJ. Consensus segmentation for positron emission tomography: development and applications in radiation therapy. Duke University. 2013.
48.
go back to reference Kuncheva LI. Combining pattern classifiers: methods and algorithms. Hoboken, NJ: Wiley. 2004.CrossRef Kuncheva LI. Combining pattern classifiers: methods and algorithms. Hoboken, NJ: Wiley. 2004.CrossRef
50.
go back to reference Hatt M, Cheze le Rest C, Descourt P, Dekker A, De Ruysscher D, Oellers M, et al. Accurate automatic delineation of heterogeneous functional volumes in positron emission tomography for oncology applications. Int J Radiat Oncol Biol Phys. 2010;77:301–8. doi:10.1016/j.ijrobp.2009.08.018.CrossRefPubMed Hatt M, Cheze le Rest C, Descourt P, Dekker A, De Ruysscher D, Oellers M, et al. Accurate automatic delineation of heterogeneous functional volumes in positron emission tomography for oncology applications. Int J Radiat Oncol Biol Phys. 2010;77:301–8. doi:10.​1016/​j.​ijrobp.​2009.​08.​018.CrossRefPubMed
51.
go back to reference Papadimitroulas P, Loudos G, Le Maitre A, Hatt M, Tixier F, Efthimiou N, et al. Investigation of realistic PET simulations incorporating tumor patient’s specificity using anthropomorphic models: creation of an oncology database. Med Phys. 2013;40:112506. doi:10.1118/1.4826162.CrossRefPubMed Papadimitroulas P, Loudos G, Le Maitre A, Hatt M, Tixier F, Efthimiou N, et al. Investigation of realistic PET simulations incorporating tumor patient’s specificity using anthropomorphic models: creation of an oncology database. Med Phys. 2013;40:112506. doi:10.​1118/​1.​4826162.CrossRefPubMed
Metadata
Title
Impact of consensus contours from multiple PET segmentation methods on the accuracy of functional volume delineation
Authors
A. Schaefer
M. Vermandel
C. Baillet
A. S. Dewalle-Vignion
R. Modzelewski
P. Vera
L. Massoptier
C. Parcq
D. Gibon
T. Fechter
U. Nemer
I. Gardin
U. Nestle
Publication date
01-05-2016
Publisher
Springer Berlin Heidelberg
Published in
European Journal of Nuclear Medicine and Molecular Imaging / Issue 5/2016
Print ISSN: 1619-7070
Electronic ISSN: 1619-7089
DOI
https://doi.org/10.1007/s00259-015-3239-7

Other articles of this Issue 5/2016

European Journal of Nuclear Medicine and Molecular Imaging 5/2016 Go to the issue