Skip to main content
Top
Published in: Journal of Translational Medicine 1/2011

Open Access 01-12-2011 | Methodology

Multiplex serum biomarker assessments: technical and biostatistical issues

Authors: Lisa H Butterfield, Douglas M Potter, John M Kirkwood

Published in: Journal of Translational Medicine | Issue 1/2011

Login to get access

Abstract

Background

Identification of predictive and prognostic biomarkers for patients with disease and undergoing different therapeutic options is a very active area of investigation. Many of these studies seek biomarkers among circulating proteins accessed in blood. Many levels of standardization in materials and procedures have been identified which can impact the resulting data.

Methods

Here, we have observed unexpected variability in levels of commonly tested analytes in serum which were processed and stored under standardized conditions. We have identified apparent changes in cytokine, chemokine and growth factor levels detected by multiplex Luminex assay in melanoma patient and healthy donor serum samples, over storage time at -80°C. Controls included Luminex kit standards, multiplexed cytokine standards and WHO cytokine controls. Data were analyzed by Wilcoxon rank-sum testing and Spearman's test for correlations.

Results

The interpretation of these changes is confounded by lot-to-lot kit standard curve reagent changes made by a single manufacturer of Luminex kits.

Conclusions

This study identifies previously unknown sources of variation in a commonly used biomarker assay, and suggests additional levels of controls needed for identification of true changes in circulating protein levels.
Appendix
Available only for authorised users
Literature
1.
go back to reference Dancey JE, Dobbin KK, Groshen S, Jessup JM, Hruszkewycz AH, Koehler M, Parchment R, Ratain MJ, Shankar LK, Stadler WM, True LD, Gravell A, Grever MR, Biomarkers Task Force of the NCI Investigational Drug Steering Committee: Guidelines for the development and incorporation of biomarker studies in early clinical trials of novel agents. Clin Cancer Res. 2010, 16: 1745-55. 10.1158/1078-0432.CCR-09-2167.CrossRefPubMed Dancey JE, Dobbin KK, Groshen S, Jessup JM, Hruszkewycz AH, Koehler M, Parchment R, Ratain MJ, Shankar LK, Stadler WM, True LD, Gravell A, Grever MR, Biomarkers Task Force of the NCI Investigational Drug Steering Committee: Guidelines for the development and incorporation of biomarker studies in early clinical trials of novel agents. Clin Cancer Res. 2010, 16: 1745-55. 10.1158/1078-0432.CCR-09-2167.CrossRefPubMed
2.
go back to reference Kirkwood JM, Manola J, Ibrahim J, Sondak V, Ernstoff MS, Rao U, Eastern Cooperative Oncology Group: A pooled analysis of eastern cooperative oncology group and intergroup trials of adjuvant high-dose interferon for melanoma. Clin Cancer Res. 2004, 10: 1670-1677. 10.1158/1078-0432.CCR-1103-3.CrossRefPubMed Kirkwood JM, Manola J, Ibrahim J, Sondak V, Ernstoff MS, Rao U, Eastern Cooperative Oncology Group: A pooled analysis of eastern cooperative oncology group and intergroup trials of adjuvant high-dose interferon for melanoma. Clin Cancer Res. 2004, 10: 1670-1677. 10.1158/1078-0432.CCR-1103-3.CrossRefPubMed
3.
go back to reference Rosenberg SA, Restifo NP, Yang JC, Morgan RA, Dudley ME: Adoptive cell transfer: a clinical path to effective cancer immunotherapy. Nat Rev Cancer. 2008, 8: 299-308. 10.1038/nrc2355.PubMedCentralCrossRefPubMed Rosenberg SA, Restifo NP, Yang JC, Morgan RA, Dudley ME: Adoptive cell transfer: a clinical path to effective cancer immunotherapy. Nat Rev Cancer. 2008, 8: 299-308. 10.1038/nrc2355.PubMedCentralCrossRefPubMed
4.
go back to reference Butterfield LH, Disis ML, Fox BA, Lee PP, Khleif SN, Thurin M, Trinchieri G, Wang E, Wigginton J, Chaussabel D, Coukos G, Dhodapkar M, Hakansson L, Janetzki S, Kleen TO, Kirkwood JM, Maccalli C, Maecker H, Maio M, Malyguine A, Masucci G, Palucka AK, Potter DM, Ribas A, Rivoltini L, Schendel D, Seliger B, Selvan S, Slingluff CL, Stroncek DF, Streicher H, Wu X, Zeskind B, Zhao Y, Zocca M-B, Zwierzina H, Marincola FM: A systematic approach to biomarker discovery; Preamble to "the iSBTc-FDA taskforce on immunotherapy biomarkers. J Transl Med. 2008, 6: 81-10.1186/1479-5876-6-81.PubMedCentralCrossRefPubMed Butterfield LH, Disis ML, Fox BA, Lee PP, Khleif SN, Thurin M, Trinchieri G, Wang E, Wigginton J, Chaussabel D, Coukos G, Dhodapkar M, Hakansson L, Janetzki S, Kleen TO, Kirkwood JM, Maccalli C, Maecker H, Maio M, Malyguine A, Masucci G, Palucka AK, Potter DM, Ribas A, Rivoltini L, Schendel D, Seliger B, Selvan S, Slingluff CL, Stroncek DF, Streicher H, Wu X, Zeskind B, Zhao Y, Zocca M-B, Zwierzina H, Marincola FM: A systematic approach to biomarker discovery; Preamble to "the iSBTc-FDA taskforce on immunotherapy biomarkers. J Transl Med. 2008, 6: 81-10.1186/1479-5876-6-81.PubMedCentralCrossRefPubMed
5.
go back to reference Tahara H, Sato M, Thurin M, Wang E, Butterfield LH, Disis ML, Fox BA, Lee PP, Khleif SN, Wigginton JM, Ambs S, Akutsu Y, Chaussabel D, Doki Y, Eremin O, Fridman WH, Hirohashi Y, Imai K, Jacobson J, Jinushil M, Kanamoto A, Kashani-Sabet M, Kato K, Kawakami Y, Kirkwood JM, Kleen TO, Lehmann PV, Liotta L, Lotze MT, Malyguine A, Masucci G, Matsubara H, Nakamura K, Nishikawa H, Palucka AK, Petricoin EF, Pos Z, Ribas A, Rivoltini L, Sato N, Shiku H, Slingluff CL, Streicher H, Stronchek DF, Takeuchi H, Toyota M, Wada H, Wu X, Wulfkuhle J, Yaguchi T, Zeskind B, Zhao Y, Zocca M-B, Marincola FM: Emerging concepts in biomarker discovery: The US-Japan workshop on immunological molecular markers in oncology. J Transl Med. 2009, 7: 45-10.1186/1479-5876-7-45.PubMedCentralCrossRefPubMed Tahara H, Sato M, Thurin M, Wang E, Butterfield LH, Disis ML, Fox BA, Lee PP, Khleif SN, Wigginton JM, Ambs S, Akutsu Y, Chaussabel D, Doki Y, Eremin O, Fridman WH, Hirohashi Y, Imai K, Jacobson J, Jinushil M, Kanamoto A, Kashani-Sabet M, Kato K, Kawakami Y, Kirkwood JM, Kleen TO, Lehmann PV, Liotta L, Lotze MT, Malyguine A, Masucci G, Matsubara H, Nakamura K, Nishikawa H, Palucka AK, Petricoin EF, Pos Z, Ribas A, Rivoltini L, Sato N, Shiku H, Slingluff CL, Streicher H, Stronchek DF, Takeuchi H, Toyota M, Wada H, Wu X, Wulfkuhle J, Yaguchi T, Zeskind B, Zhao Y, Zocca M-B, Marincola FM: Emerging concepts in biomarker discovery: The US-Japan workshop on immunological molecular markers in oncology. J Transl Med. 2009, 7: 45-10.1186/1479-5876-7-45.PubMedCentralCrossRefPubMed
6.
go back to reference Butterfield LH, Disis ML, Khleif SN, Balwit JM, Marincola F: Immuno-oncology biomarkers 2010 and beyond: Perspectives from the iSBTc/SITC Biomarker Task Force. J Transl Med. 2010, 8: 130-10.1186/1479-5876-8-130.PubMedCentralCrossRefPubMed Butterfield LH, Disis ML, Khleif SN, Balwit JM, Marincola F: Immuno-oncology biomarkers 2010 and beyond: Perspectives from the iSBTc/SITC Biomarker Task Force. J Transl Med. 2010, 8: 130-10.1186/1479-5876-8-130.PubMedCentralCrossRefPubMed
7.
go back to reference Butterfield LH, Palucka AK, Britten CM, Dhodapkar MV, Hakansson L, Janetzki S, Kawakami Y, Kleen T-O, Lee PP, Macalli C, Maecker HT, Maino VC, Maio M, Malyguine A, Masucci G, Pawelec G, Potter DM, Rivoltini L, Salazar LG, Schendel DJ, Slingluff CL, Song W, Stroncek DF, Tahara H, Thurin M, Trinchieri G, van Der Burg SH, Whiteside TL, Wigginton JM, Marincola F, Khleif S, Fox BA, Disis ML: Recommendations from the iSBTc-SITC/FDA/NCI Workshop on Immunotherapy Biomarkers. Clin Cancer Res. 2011, 17: 3064-3076. 10.1158/1078-0432.CCR-10-2234.PubMedCentralCrossRefPubMed Butterfield LH, Palucka AK, Britten CM, Dhodapkar MV, Hakansson L, Janetzki S, Kawakami Y, Kleen T-O, Lee PP, Macalli C, Maecker HT, Maino VC, Maio M, Malyguine A, Masucci G, Pawelec G, Potter DM, Rivoltini L, Salazar LG, Schendel DJ, Slingluff CL, Song W, Stroncek DF, Tahara H, Thurin M, Trinchieri G, van Der Burg SH, Whiteside TL, Wigginton JM, Marincola F, Khleif S, Fox BA, Disis ML: Recommendations from the iSBTc-SITC/FDA/NCI Workshop on Immunotherapy Biomarkers. Clin Cancer Res. 2011, 17: 3064-3076. 10.1158/1078-0432.CCR-10-2234.PubMedCentralCrossRefPubMed
8.
go back to reference Yurkovetsky ZR, Kirkwood JM, Edington HD, Marrangoni AM, Velikokhatnaya L, Winans MT, Gorelik E, Lokshin AE: Multiplex analysis of serum cytokines in melanoma patients treated with interferon-alpha2b. Clin Cancer Res. 2007, 13: 2422-2428. 10.1158/1078-0432.CCR-06-1805.CrossRefPubMed Yurkovetsky ZR, Kirkwood JM, Edington HD, Marrangoni AM, Velikokhatnaya L, Winans MT, Gorelik E, Lokshin AE: Multiplex analysis of serum cytokines in melanoma patients treated with interferon-alpha2b. Clin Cancer Res. 2007, 13: 2422-2428. 10.1158/1078-0432.CCR-06-1805.CrossRefPubMed
9.
go back to reference Hutchinson PE, Osborne JE, Pringle JH: Higher serum 25-hydroxy vitamin D3 levels at presentation are associated with improved survival from melanoma, but there is no evidence that later prevailing levels are protective. J Clin Oncol. 2010, 28: e492-493. 10.1200/JCO.2010.29.6095.CrossRefPubMed Hutchinson PE, Osborne JE, Pringle JH: Higher serum 25-hydroxy vitamin D3 levels at presentation are associated with improved survival from melanoma, but there is no evidence that later prevailing levels are protective. J Clin Oncol. 2010, 28: e492-493. 10.1200/JCO.2010.29.6095.CrossRefPubMed
10.
go back to reference Gajewski TF, Fuertes M, Spaapen R, Zheng Y, Kline J: Molecular profiling to identify relevant immune resistance mechanisms in the tumor microenvironment. Curr Opin Immunol. 2011, 23: 286-292. 10.1016/j.coi.2010.11.013.PubMedCentralCrossRefPubMed Gajewski TF, Fuertes M, Spaapen R, Zheng Y, Kline J: Molecular profiling to identify relevant immune resistance mechanisms in the tumor microenvironment. Curr Opin Immunol. 2011, 23: 286-292. 10.1016/j.coi.2010.11.013.PubMedCentralCrossRefPubMed
11.
go back to reference Ma J, Lin JY, Alloo A, Wilson BJ, Schatton T, Zhan Q, Murphy GF, Waaga-Gasser AM, Gasser M, Stephen Hodi F, Frank NY, Frank MH: Isolation of tumorigenic circulating melanoma cells. Biochem Biophys Res Commun. 2010, 402: 711-717. 10.1016/j.bbrc.2010.10.091.PubMedCentralCrossRefPubMed Ma J, Lin JY, Alloo A, Wilson BJ, Schatton T, Zhan Q, Murphy GF, Waaga-Gasser AM, Gasser M, Stephen Hodi F, Frank NY, Frank MH: Isolation of tumorigenic circulating melanoma cells. Biochem Biophys Res Commun. 2010, 402: 711-717. 10.1016/j.bbrc.2010.10.091.PubMedCentralCrossRefPubMed
12.
go back to reference Liu Y, He J, Xie X, Su G, Teitz-Tennenbaum S, Sabel MS, Lubman DM: Serum autoantibody profiling using a natural glycoprotein microarray for the prognosis of early melanoma. J Proteome Res. 2010, 9: 6044-6051. 10.1021/pr100856k.PubMedCentralCrossRefPubMed Liu Y, He J, Xie X, Su G, Teitz-Tennenbaum S, Sabel MS, Lubman DM: Serum autoantibody profiling using a natural glycoprotein microarray for the prognosis of early melanoma. J Proteome Res. 2010, 9: 6044-6051. 10.1021/pr100856k.PubMedCentralCrossRefPubMed
13.
go back to reference Kirkwood JM, Lee S, Moschos SJ, Albertini MR, Michalak JC, Sander C, whiteside TL, Butterfield LH, Weiner L: Immunogenicity and antitumor effects of vaccination with peptide vaccine+/-granulocyte-monocyte colony-stimulating factor and/or IFN-alpha2b in advanced metastatic melanoma: Eastern Cooperative Oncology Group Phase II Trial E1696. Clin Cancer Res. 2009, 15: 1443-1451. 10.1158/1078-0432.CCR-08-1231.PubMedCentralCrossRefPubMed Kirkwood JM, Lee S, Moschos SJ, Albertini MR, Michalak JC, Sander C, whiteside TL, Butterfield LH, Weiner L: Immunogenicity and antitumor effects of vaccination with peptide vaccine+/-granulocyte-monocyte colony-stimulating factor and/or IFN-alpha2b in advanced metastatic melanoma: Eastern Cooperative Oncology Group Phase II Trial E1696. Clin Cancer Res. 2009, 15: 1443-1451. 10.1158/1078-0432.CCR-08-1231.PubMedCentralCrossRefPubMed
14.
go back to reference Gould Rothberg BE, Rimm DL: Biomarkers: the useful and the not so useful--an assessment of molecular prognostic markers for cutaneous melanoma. J Invest Dermatol. 2010, 130: 1971-1987. 10.1038/jid.2010.149.CrossRefPubMed Gould Rothberg BE, Rimm DL: Biomarkers: the useful and the not so useful--an assessment of molecular prognostic markers for cutaneous melanoma. J Invest Dermatol. 2010, 130: 1971-1987. 10.1038/jid.2010.149.CrossRefPubMed
16.
go back to reference Bull M, Lee D, Stucky J, Chiu YL, Rubin A, Horton H, McElrath MJ: Defining blood processing parameters for optimal detection of cryopreserved antigen-specific responses for HIV vaccine trials. J Immunol Meth. 2007, 322: 57-69. 10.1016/j.jim.2007.02.003.CrossRef Bull M, Lee D, Stucky J, Chiu YL, Rubin A, Horton H, McElrath MJ: Defining blood processing parameters for optimal detection of cryopreserved antigen-specific responses for HIV vaccine trials. J Immunol Meth. 2007, 322: 57-69. 10.1016/j.jim.2007.02.003.CrossRef
17.
go back to reference Kierstead LS, Dubey S, Meyer B, Tobery TW, Mogg R, Fernandez VR, Long R, Guan L, Gaunt C, Collins K, Sykes KJ, Mehrotra DV, Chirmule N, Shiver JW, Casimiro DR: Enhanced rates and magnitude of immune responses detected against an HIV vaccine: effect of using an optimized process for isolating PBMC. AIDS Res Hum Retroviruses. 2007, 23: 86-92. 10.1089/aid.2006.0129.CrossRefPubMed Kierstead LS, Dubey S, Meyer B, Tobery TW, Mogg R, Fernandez VR, Long R, Guan L, Gaunt C, Collins K, Sykes KJ, Mehrotra DV, Chirmule N, Shiver JW, Casimiro DR: Enhanced rates and magnitude of immune responses detected against an HIV vaccine: effect of using an optimized process for isolating PBMC. AIDS Res Hum Retroviruses. 2007, 23: 86-92. 10.1089/aid.2006.0129.CrossRefPubMed
18.
go back to reference McKenna KC, Beatty KM, Bilonick RA, Schoenfield L, Lathrop KL, Singh AD: Activated CD11b+CD15+ granulocytes increase in the blood of patients with uveal melanoma. Invest Ophthalmol Vis Sci. 2009, 50: 4295-4303. 10.1167/iovs.08-3012.CrossRefPubMed McKenna KC, Beatty KM, Bilonick RA, Schoenfield L, Lathrop KL, Singh AD: Activated CD11b+CD15+ granulocytes increase in the blood of patients with uveal melanoma. Invest Ophthalmol Vis Sci. 2009, 50: 4295-4303. 10.1167/iovs.08-3012.CrossRefPubMed
19.
go back to reference Boaz MJ, Hayes P, Tarragona T, Seamons L, Cooper A, Birungi J, Kitandwe P, Semaganda A, Kaleebu P, Stevens G, Anzala O, Farah B, Ogola S, Indangasi J, Mhlanga P, Van Eeden M, Thakar M, Pujari A, Mishra S, Goonetilleke N, Moore S, Mahmoud A, Sathyamoorthy P, Mahalingam J, Narayanan PR, Ramanathan VD, Cox JH, Dally L, Gill DK, Gilmour J: Concordant proficiency in measurement of T-cell immunity in human immunodeficiency virus vaccine clinical trials by peripheral blood mononuclear cell and enzyme-linked immunospot assays in laboratories from three continents. Clin Vacc Immunol. 2009, 16: 147-155. 10.1128/CVI.00326-08.CrossRef Boaz MJ, Hayes P, Tarragona T, Seamons L, Cooper A, Birungi J, Kitandwe P, Semaganda A, Kaleebu P, Stevens G, Anzala O, Farah B, Ogola S, Indangasi J, Mhlanga P, Van Eeden M, Thakar M, Pujari A, Mishra S, Goonetilleke N, Moore S, Mahmoud A, Sathyamoorthy P, Mahalingam J, Narayanan PR, Ramanathan VD, Cox JH, Dally L, Gill DK, Gilmour J: Concordant proficiency in measurement of T-cell immunity in human immunodeficiency virus vaccine clinical trials by peripheral blood mononuclear cell and enzyme-linked immunospot assays in laboratories from three continents. Clin Vacc Immunol. 2009, 16: 147-155. 10.1128/CVI.00326-08.CrossRef
21.
23.
go back to reference Potter DM, Butterfield LH, Divito SJ, Sander C, Kirkwood JM: Pitfalls in retrospective analyses of biomarkers: a case study with metastatic melanoma patients. 2011, Potter DM, Butterfield LH, Divito SJ, Sander C, Kirkwood JM: Pitfalls in retrospective analyses of biomarkers: a case study with metastatic melanoma patients. 2011,
24.
go back to reference de Jager W, Bourcier K, Rijkers GT, Prakken BJ, Seyfert-Margolis V: Prerequisites for cytokine measurements in clinical trials with multiplex immunoassays. BMC Immunol. 2009, 10: 52-10.1186/1471-2172-10-52.PubMedCentralCrossRefPubMed de Jager W, Bourcier K, Rijkers GT, Prakken BJ, Seyfert-Margolis V: Prerequisites for cytokine measurements in clinical trials with multiplex immunoassays. BMC Immunol. 2009, 10: 52-10.1186/1471-2172-10-52.PubMedCentralCrossRefPubMed
25.
go back to reference Zhou XB, Fragala MS, McElhaney JEB, Kuchel GA: Conceptual and methodological issues relevant to cytokine and inflammatory marker measurements in clinical research. Curr Opin Clin Nutr Metabolic Care. 2010, 13: 541-547. 10.1097/MCO.0b013e32833cf3bc.CrossRef Zhou XB, Fragala MS, McElhaney JEB, Kuchel GA: Conceptual and methodological issues relevant to cytokine and inflammatory marker measurements in clinical research. Curr Opin Clin Nutr Metabolic Care. 2010, 13: 541-547. 10.1097/MCO.0b013e32833cf3bc.CrossRef
26.
go back to reference Butterfield LH, Gooding W, Whiteside TL: Development of a potency assay for human dendritic cells: IL-12p70 production. J Immunother. 2008, 31: 89-100. 10.1097/CJI.0b013e318158fce0.CrossRefPubMed Butterfield LH, Gooding W, Whiteside TL: Development of a potency assay for human dendritic cells: IL-12p70 production. J Immunother. 2008, 31: 89-100. 10.1097/CJI.0b013e318158fce0.CrossRefPubMed
27.
go back to reference Khan SS, Smith MS, Reda D, Suffredini AF, McCoy JP: Multiplex bead array assays for detection of soluble cytokines: comparisons of sensitivity and quantitative values among kits from multiple manufacturers. Cytometry B Clin Cytom. 2004, 61: 35-39.CrossRefPubMed Khan SS, Smith MS, Reda D, Suffredini AF, McCoy JP: Multiplex bead array assays for detection of soluble cytokines: comparisons of sensitivity and quantitative values among kits from multiple manufacturers. Cytometry B Clin Cytom. 2004, 61: 35-39.CrossRefPubMed
Metadata
Title
Multiplex serum biomarker assessments: technical and biostatistical issues
Authors
Lisa H Butterfield
Douglas M Potter
John M Kirkwood
Publication date
01-12-2011
Publisher
BioMed Central
Published in
Journal of Translational Medicine / Issue 1/2011
Electronic ISSN: 1479-5876
DOI
https://doi.org/10.1186/1479-5876-9-173

Other articles of this Issue 1/2011

Journal of Translational Medicine 1/2011 Go to the issue