Skip to main content
Top
Published in: Trials 1/2018

Open Access 01-12-2018 | Methodology

A simulation study on estimating biomarker–treatment interaction effects in randomized trials with prognostic variables

Authors: Bernhard Haller, Kurt Ulm

Published in: Trials | Issue 1/2018

Login to get access

Abstract

Background

To individualize treatment decisions based on patient characteristics, identification of an interaction between a biomarker and treatment is necessary. Often such potential interactions are analysed using data from randomized clinical trials intended for comparison of two treatments. Tests of interactions are often lacking statistical power and we investigated if and how a consideration of further prognostic variables can improve power and decrease the bias of estimated biomarker–treatment interactions in randomized clinical trials with time-to-event outcomes.

Methods

A simulation study was performed to assess how prognostic factors affect the estimate of the biomarker–treatment interaction for a time-to-event outcome, when different approaches, like ignoring other prognostic factors, including all available covariates or using variable selection strategies, are applied. Different scenarios regarding the proportion of censored observations, the correlation structure between the covariate of interest and further potential prognostic variables, and the strength of the interaction were considered.

Results

The simulation study revealed that in a regression model for estimating a biomarker–treatment interaction, the probability of detecting a biomarker–treatment interaction can be increased by including prognostic variables that are associated with the outcome, and that the interaction estimate is biased when relevant prognostic variables are not considered. However, the probability of a false-positive finding increases if too many potential predictors are included or if variable selection is performed inadequately.

Conclusions

We recommend undertaking an adequate literature search before data analysis to derive information about potential prognostic variables and to gain power for detecting true interaction effects and pre-specifying analyses to avoid selective reporting and increased false-positive rates.
Appendix
Available only for authorised users
Literature
1.
2.
go back to reference Chen JJ, Lu TP, Chen YC, Lin WJ. Predictive biomarkers for treatment selection: statistical considerations. Biomark Med. 2015; 9(11):1121–35.CrossRefPubMed Chen JJ, Lu TP, Chen YC, Lin WJ. Predictive biomarkers for treatment selection: statistical considerations. Biomark Med. 2015; 9(11):1121–35.CrossRefPubMed
3.
go back to reference Rothwell PM. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet. 2005; 365(9454):176–86.CrossRefPubMed Rothwell PM. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet. 2005; 365(9454):176–86.CrossRefPubMed
4.
go back to reference Karapetis CS, Khambata-Ford S, Jonker DJ, O’Callaghan CJ, Tu D, Tebbutt NC, et al.K-ras mutations and benefit from cetuximab in advanced colorectal cancer. N Engl J Med. 2008; 359(17):1757–65.CrossRefPubMed Karapetis CS, Khambata-Ford S, Jonker DJ, O’Callaghan CJ, Tu D, Tebbutt NC, et al.K-ras mutations and benefit from cetuximab in advanced colorectal cancer. N Engl J Med. 2008; 359(17):1757–65.CrossRefPubMed
5.
go back to reference Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet. 2000; 355(9209):1064–9.CrossRefPubMed Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet. 2000; 355(9209):1064–9.CrossRefPubMed
6.
go back to reference Naggara O, Raymond J, Guilbert F, Roy D, Weill A, Altman DG. Analysis by categorizing or dichotomizing continuous variables is inadvisable: an example from the natural history of unruptured aneurysms. Am J Neuroradiol. 2011; 32(3):437–40.CrossRefPubMed Naggara O, Raymond J, Guilbert F, Roy D, Weill A, Altman DG. Analysis by categorizing or dichotomizing continuous variables is inadvisable: an example from the natural history of unruptured aneurysms. Am J Neuroradiol. 2011; 32(3):437–40.CrossRefPubMed
7.
go back to reference Royston P, Sauerbrei W. Interactions between treatment and continuous covariates: a step toward individualizing therapy. J Clin Oncol. 2008; 26(9):1397–9.CrossRefPubMed Royston P, Sauerbrei W. Interactions between treatment and continuous covariates: a step toward individualizing therapy. J Clin Oncol. 2008; 26(9):1397–9.CrossRefPubMed
8.
go back to reference Royston P, Sauerbrei W. A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials. Stat Med. 2004; 23(16):2509–25.CrossRefPubMed Royston P, Sauerbrei W. A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials. Stat Med. 2004; 23(16):2509–25.CrossRefPubMed
9.
go back to reference Tian L, Alizadeh AA, Gentles AJ, Tibshirani R. A simple method for estimating interactions between a treatment and a large number of covariates. J Am Stat Assoc. 2014; 109(508):1517–32.CrossRefPubMedPubMedCentral Tian L, Alizadeh AA, Gentles AJ, Tibshirani R. A simple method for estimating interactions between a treatment and a large number of covariates. J Am Stat Assoc. 2014; 109(508):1517–32.CrossRefPubMedPubMedCentral
10.
go back to reference Committee for Proprietary Medicinal Products. Points to consider on adjustment for baseline covariates. Stat Med. 2004; 23(5):701.CrossRef Committee for Proprietary Medicinal Products. Points to consider on adjustment for baseline covariates. Stat Med. 2004; 23(5):701.CrossRef
11.
go back to reference Langner I, Bender R, Lenz-Tönjes R, Küchenhoff H, Blettner M. Bias of maximum-likelihood estimates in logistic and Cox regression models: a comparative simulation study. Sonderforschungsbereich 386. Ludwig-Maximilians-Universität München; 2003. Langner I, Bender R, Lenz-Tönjes R, Küchenhoff H, Blettner M. Bias of maximum-likelihood estimates in logistic and Cox regression models: a comparative simulation study. Sonderforschungsbereich 386. Ludwig-Maximilians-Universität München; 2003.
12.
go back to reference Harrell F. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. New York: Springer; 2001.CrossRef Harrell F. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. New York: Springer; 2001.CrossRef
13.
go back to reference Royston P, Sauerbrei W. Multivariable model-building: a pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables, Vol. 777. Chichester: Wiley; 2008.CrossRef Royston P, Sauerbrei W. Multivariable model-building: a pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables, Vol. 777. Chichester: Wiley; 2008.CrossRef
14.
go back to reference Akaike H. A new look at the statistical model identification. IEEE Trans Autom Control. 1974; 19(6):716–23.CrossRef Akaike H. A new look at the statistical model identification. IEEE Trans Autom Control. 1974; 19(6):716–23.CrossRef
15.
go back to reference Babyak MA. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med. 2004; 66(3):411–21.PubMed Babyak MA. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med. 2004; 66(3):411–21.PubMed
16.
go back to reference Cox DR. Regression models and life tables (with discussion). J Royal Stat Soc. 1972; 34:187–220. Cox DR. Regression models and life tables (with discussion). J Royal Stat Soc. 1972; 34:187–220.
17.
go back to reference Therneau TM, Grambsch PM. Modeling survival data: extending the Cox model. New York: Springer; 2013. Therneau TM, Grambsch PM. Modeling survival data: extending the Cox model. New York: Springer; 2013.
18.
go back to reference Vach W. Regression models as a tool in medical research. Boca Raton: CRC Press; 2012. Vach W. Regression models as a tool in medical research. Boca Raton: CRC Press; 2012.
21.
go back to reference Polley M-YC, Freidlin B, Korn EL, Conley BA, Abrams JS, McShane LM. Statistical and practical considerations for clinical evaluation of predictive biomarkers. J Natl Cancer Inst. 2013; 105(22):1677–83.CrossRefPubMedPubMedCentral Polley M-YC, Freidlin B, Korn EL, Conley BA, Abrams JS, McShane LM. Statistical and practical considerations for clinical evaluation of predictive biomarkers. J Natl Cancer Inst. 2013; 105(22):1677–83.CrossRefPubMedPubMedCentral
24.
go back to reference Burton A, Altman DG, Royston P, Holder RL. The design of simulation studies in medical statistics. Stat Med. 2006; 25(24):4279–92.CrossRefPubMed Burton A, Altman DG, Royston P, Holder RL. The design of simulation studies in medical statistics. Stat Med. 2006; 25(24):4279–92.CrossRefPubMed
25.
go back to reference Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006; 25(1):127–41.CrossRefPubMed Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006; 25(1):127–41.CrossRefPubMed
26.
go back to reference Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol. 1995; 48(12):1495–501.CrossRefPubMed Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol. 1995; 48(12):1495–501.CrossRefPubMed
27.
go back to reference Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995; 48(12):1503–10.CrossRefPubMed Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995; 48(12):1503–10.CrossRefPubMed
28.
go back to reference Sainani KL. Multivariate regression: the pitfalls of automated variable selection. PM&R. 2013; 5(9):791–4.CrossRef Sainani KL. Multivariate regression: the pitfalls of automated variable selection. PM&R. 2013; 5(9):791–4.CrossRef
29.
go back to reference Schwarz G, et al. Estimating the dimension of a model. Annals Stat. 1978; 6(2):461–4.CrossRef Schwarz G, et al. Estimating the dimension of a model. Annals Stat. 1978; 6(2):461–4.CrossRef
30.
go back to reference Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol. 1996; 58(1):267–288. Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol. 1996; 58(1):267–288.
31.
go back to reference Liu Y, Jiang W, Chen BE. Testing for treatment-biomarker interaction based on local partial-likelihood. Stat Med. 2015; 34(27):3516–30.CrossRefPubMed Liu Y, Jiang W, Chen BE. Testing for treatment-biomarker interaction based on local partial-likelihood. Stat Med. 2015; 34(27):3516–30.CrossRefPubMed
32.
go back to reference European Medicines Agency. Guideline on multiplicity issues in clinical trials. 2017. http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/general/general_content_001220.jsp&mid=. European Medicines Agency. Guideline on multiplicity issues in clinical trials. 2017. http://​www.​ema.​europa.​eu/​ema/​index.​jsp?​curl=​pages/​regulation/​general/​general_​content_​001220.​jsp&​mid=​.​
33.
go back to reference Ioannidis JP. Why most published research findings are false. PLoS Med. 2005; 2(8):124.CrossRef Ioannidis JP. Why most published research findings are false. PLoS Med. 2005; 2(8):124.CrossRef
Metadata
Title
A simulation study on estimating biomarker–treatment interaction effects in randomized trials with prognostic variables
Authors
Bernhard Haller
Kurt Ulm
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Trials / Issue 1/2018
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/s13063-018-2491-0

Other articles of this Issue 1/2018

Trials 1/2018 Go to the issue