Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Research article

Assessing correlates of protection in vaccine trials: statistical solutions in the context of high vaccine efficacy

Authors: Andrea Callegaro, Fabian Tibaldi

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

The use of correlates of protection (CoPs) in vaccination trials offers significant advantages as useful clinical endpoint substitutes. Vaccines with very high vaccine efficacy (VE) are documented in the literature (VE ≥95%). The rare events (number of infections) observed in the vaccinated groups of these trials posed challenges when applying conventionally-used statistical methods for CoP assessment. In this paper, we describe the nature of these challenges, and propose easy-to-implement and uniquely-tailored statistical solutions for the assessment of CoPs in the specific context of high VE.

Methods

The Prentice criteria and meta-analytic frameworks are standard statistical methods for assessing vaccine CoPs, but can be problematic in high VE cases due to the rare events data available. As a result, lack of fit and the problem of infinite estimates may arise, in the former and latter methods respectively. The use of flexible models within the Prentice framework, and penalized-likelihood methods to solve the issue of infinite estimates can improve the performance of both methods in high VE settings.

Results

We have 1) devised flexible non-linear models to counteract the Prentice framework lack of fit, providing sufficient statistical power to the method, and 2) proposed the use of penalised likelihood approaches to make the meta-analytic framework applicable on randomized subgroups, such as regions. The performance of the proposed methods for high VE cases was evaluated by running simulations.

Conclusions

As vaccines with high efficacy are documented in the literature, there is a need to identify effective statistical solutions to assess CoPs. Our proposed adaptations are straight-forward and improve the performance of conventional statistical methods for high VE data, leading to more reliable CoP assessments in the context of high VE settings.
Literature
1.
go back to reference Nguipdop-Djomo P, Thomas SL, Fine PEM. Correlates of vaccine-induced protection: methods and implications. WHO/IVB/10.00. 2013; 181:1–55. Nguipdop-Djomo P, Thomas SL, Fine PEM. Correlates of vaccine-induced protection: methods and implications. WHO/IVB/10.00. 2013; 181:1–55.
2.
go back to reference Prentice RL. Surrogate endpoints in clinical trials: definition and operational criteria. Stat Med. 1989; 8(4):431–40.CrossRef Prentice RL. Surrogate endpoints in clinical trials: definition and operational criteria. Stat Med. 1989; 8(4):431–40.CrossRef
3.
go back to reference Follmann D. Augmented designs to assess immune response in vaccine trials. Biometrics. 2006; 62(4):1161–9.CrossRef Follmann D. Augmented designs to assess immune response in vaccine trials. Biometrics. 2006; 62(4):1161–9.CrossRef
4.
go back to reference Frangakis CE, Rubin DB. Principal stratification in causal inference. Biometrics. 2002; 58(1):21–9.CrossRef Frangakis CE, Rubin DB. Principal stratification in causal inference. Biometrics. 2002; 58(1):21–9.CrossRef
5.
go back to reference Gilbert PB, Qin L, Self SG. Evaluating a surrogate endpoint at three levels, with application to vaccine development. Stat Med. 2008; 27(23):4758–78.CrossRef Gilbert PB, Qin L, Self SG. Evaluating a surrogate endpoint at three levels, with application to vaccine development. Stat Med. 2008; 27(23):4758–78.CrossRef
6.
go back to reference Buyse M, Molenberghs G, Burzykowski T, Renard D, Geys H. The validation of surrogate endpoints in meta-analysis of randomized experiments. Biostatistics. 2000; 1:49–67.CrossRef Buyse M, Molenberghs G, Burzykowski T, Renard D, Geys H. The validation of surrogate endpoints in meta-analysis of randomized experiments. Biostatistics. 2000; 1:49–67.CrossRef
7.
go back to reference Daniels MJ, Hughes MD. Meta-analysis for the evaluation of potential surrogate markers. Stat Med. 1997; 16:1965–82.CrossRef Daniels MJ, Hughes MD. Meta-analysis for the evaluation of potential surrogate markers. Stat Med. 1997; 16:1965–82.CrossRef
8.
go back to reference Gail MH, Pfeiffer R, Houwelingen HCV, Carroll R. On meta-analytic assessment of surrogate outcomes. Biostatistics. 2000; 1:231–46.CrossRef Gail MH, Pfeiffer R, Houwelingen HCV, Carroll R. On meta-analytic assessment of surrogate outcomes. Biostatistics. 2000; 1:231–46.CrossRef
9.
go back to reference Alonso A, Van der Elst W, Molenberghs G, Buyse M, Burzykowski T. On the relationship between the causal-inference and meta-analytic paradigms for the validation of surrogate endpoints. Biometrics. 2015; 71(1):15–24.CrossRef Alonso A, Van der Elst W, Molenberghs G, Buyse M, Burzykowski T. On the relationship between the causal-inference and meta-analytic paradigms for the validation of surrogate endpoints. Biometrics. 2015; 71(1):15–24.CrossRef
10.
go back to reference Qin L, Gilbert PB, Corey L, McElrath MJ, Self SG. A framework for assessing immunological correlates of protection in vaccine trials. J Infect Dis. 2007; 196(9):1304–12.CrossRef Qin L, Gilbert PB, Corey L, McElrath MJ, Self SG. A framework for assessing immunological correlates of protection in vaccine trials. J Infect Dis. 2007; 196(9):1304–12.CrossRef
11.
go back to reference Rubin DB. Causal inference using potential outcomes: Design, modeling, decisions. J Am Stat Assoc. 2005; 100(469):322–31.CrossRef Rubin DB. Causal inference using potential outcomes: Design, modeling, decisions. J Am Stat Assoc. 2005; 100(469):322–31.CrossRef
12.
go back to reference Mitra M, Shah N, Ghosh A, Chatterjee S, Kaur I, Bhattacharya N, Basu S. Efficacy and safety of vi-tetanus toxoid conjugated typhoid vaccine (pedatyph) in indian children: school based cluster randomized study. Hum Vaccines Immunotherapeutics. 2016; 12(4):939–45.CrossRef Mitra M, Shah N, Ghosh A, Chatterjee S, Kaur I, Bhattacharya N, Basu S. Efficacy and safety of vi-tetanus toxoid conjugated typhoid vaccine (pedatyph) in indian children: school based cluster randomized study. Hum Vaccines Immunotherapeutics. 2016; 12(4):939–45.CrossRef
13.
go back to reference Lin FYC, Ho VA, Khiem HB, et al.The efficacy of a salmonella typhi vi conjugate vaccine in two-to-five-year-old children. N Engl J Med. 2001; 344(17):1263–9.CrossRef Lin FYC, Ho VA, Khiem HB, et al.The efficacy of a salmonella typhi vi conjugate vaccine in two-to-five-year-old children. N Engl J Med. 2001; 344(17):1263–9.CrossRef
14.
go back to reference Wei M, Meng F, Wang S, Li J, et al.Two-year efficacy, immunogenicity, and safety of vigoo enterovirus 71 vaccine in healthy chinese children: a randomised open-label study. J Infect Dis. 2017; jiw502:56–63.CrossRef Wei M, Meng F, Wang S, Li J, et al.Two-year efficacy, immunogenicity, and safety of vigoo enterovirus 71 vaccine in healthy chinese children: a randomised open-label study. J Infect Dis. 2017; jiw502:56–63.CrossRef
15.
go back to reference Phua KB, Lim FS, Lau YL, Nelson EAS, et al.Rotavirus vaccine RIX4414 efficacy sustained during the third year of life: a randomized clinical trial in an asian population. Vaccine. 2012; 30(30):4552–7.CrossRef Phua KB, Lim FS, Lau YL, Nelson EAS, et al.Rotavirus vaccine RIX4414 efficacy sustained during the third year of life: a randomized clinical trial in an asian population. Vaccine. 2012; 30(30):4552–7.CrossRef
16.
go back to reference Black S, Shinefield H, Fireman B, Lewis E, et al.Efficacy, safety and immunogenicity of heptavalent pneumococcal conjugate vaccine in children. Pediatr Infect Dis J. 2000; 19(3):187–95.CrossRef Black S, Shinefield H, Fireman B, Lewis E, et al.Efficacy, safety and immunogenicity of heptavalent pneumococcal conjugate vaccine in children. Pediatr Infect Dis J. 2000; 19(3):187–95.CrossRef
17.
go back to reference Prymula R, Bergsaker MR, Esposito S, Gothefors L, et al.Protection against varicella with two doses of combined measles-mumps-rubella-varicella vaccine versus one dose of monovalent varicella vaccine: a multicentre, observer-blind, randomised, controlled trial. Lancet. 2014; 383(9925):1313–24.CrossRef Prymula R, Bergsaker MR, Esposito S, Gothefors L, et al.Protection against varicella with two doses of combined measles-mumps-rubella-varicella vaccine versus one dose of monovalent varicella vaccine: a multicentre, observer-blind, randomised, controlled trial. Lancet. 2014; 383(9925):1313–24.CrossRef
18.
go back to reference Burzykowski T, Molenberghs G, Buyse M. The Evaluation of Surrogate Endpoints. New York: Springer; 2005.CrossRef Burzykowski T, Molenberghs G, Buyse M. The Evaluation of Surrogate Endpoints. New York: Springer; 2005.CrossRef
19.
go back to reference Freedman LS, Graubard BI, Schatzkin A. Statistical validation of intermediate endpoints for chronic disease. Stat Med. 1992; 11:167–78.CrossRef Freedman LS, Graubard BI, Schatzkin A. Statistical validation of intermediate endpoints for chronic disease. Stat Med. 1992; 11:167–78.CrossRef
20.
go back to reference Qu Y, Case M. Quantifying the effect of the surrogate marker by information gain. Biometrics. 2007; 63(3):958–63.CrossRef Qu Y, Case M. Quantifying the effect of the surrogate marker by information gain. Biometrics. 2007; 63(3):958–63.CrossRef
21.
go back to reference Alonso A, Molenberghs G. Surrogate marker evaluation from an information theory perspective. Biometrics. 2007; 63:180–6.CrossRef Alonso A, Molenberghs G. Surrogate marker evaluation from an information theory perspective. Biometrics. 2007; 63:180–6.CrossRef
22.
go back to reference Houwelingen H. C. v., Arends LR, Stijnen T. Advanced methods in meta-analysis: Multivariate approach and meta-regression. Stat Med. 2002; 21(4):589–624.CrossRef Houwelingen H. C. v., Arends LR, Stijnen T. Advanced methods in meta-analysis: Multivariate approach and meta-regression. Stat Med. 2002; 21(4):589–624.CrossRef
23.
go back to reference Tibaldi F, Abrahantes JC, et al. Simplified hierarchical linear models for the evaluation of surrogate endpoints. J Stat Comput Simul. 2003; 73:643–58.CrossRef Tibaldi F, Abrahantes JC, et al. Simplified hierarchical linear models for the evaluation of surrogate endpoints. J Stat Comput Simul. 2003; 73:643–58.CrossRef
24.
go back to reference Del Paal B. A comparison of different methods for modelling rare events data. PhD thesis, Ghent University, Ghent, Belgium. 2013. Del Paal B. A comparison of different methods for modelling rare events data. PhD thesis, Ghent University, Ghent, Belgium. 2013.
25.
go back to reference Kim HJ. Binary regression with a class of skewed t link models. Commun Stat. 2002; 31(10):1863–6.CrossRef Kim HJ. Binary regression with a class of skewed t link models. Commun Stat. 2002; 31(10):1863–6.CrossRef
26.
go back to reference Dunning AJ. A model for immunological correlates of protection. Stat Med. 2006; 25(9):1485–97.CrossRef Dunning AJ. A model for immunological correlates of protection. Stat Med. 2006; 25(9):1485–97.CrossRef
27.
go back to reference Dunning AJ, Kensler J, Coudeville L, Bailleux F. Some extensions in continuous models for immunological correlates of protection. BMC Med Res Methodol. 2015; 15(1):107.CrossRef Dunning AJ, Kensler J, Coudeville L, Bailleux F. Some extensions in continuous models for immunological correlates of protection. BMC Med Res Methodol. 2015; 15(1):107.CrossRef
28.
go back to reference Firth D. Bias reduction of maximum likelihood estimates. Biometrika. 1993; 80:27–38.CrossRef Firth D. Bias reduction of maximum likelihood estimates. Biometrika. 1993; 80:27–38.CrossRef
29.
go back to reference Gelman A, Jakulin A, Pittau MG, Su YS. A weakly informative default prior distribution for logistic and other regression models. Ann Appl Stat. 1993; 2(4):1360–83.CrossRef Gelman A, Jakulin A, Pittau MG, Su YS. A weakly informative default prior distribution for logistic and other regression models. Ann Appl Stat. 1993; 2(4):1360–83.CrossRef
30.
go back to reference Heinze G, Ploner M, Dunkler D, Southworth H. logisf: Firth’s bias reduced logistic regression. R package version 1.21. 2013; 1. Heinze G, Ploner M, Dunkler D, Southworth H. logisf: Firth’s bias reduced logistic regression. R package version 1.21. 2013; 1.
31.
go back to reference Gelman A, Su YS. arm: Data analysis using regression and multilevel/hierarchical models. R package version 1.8-6. 2015;1. Gelman A, Su YS. arm: Data analysis using regression and multilevel/hierarchical models. R package version 1.8-6. 2015;1.
32.
go back to reference Slifka MK, Amanna I. How advances in immunology provide insight into improving vaccine efficacy. Vaccine. 2014; 32(25):2948–57.CrossRef Slifka MK, Amanna I. How advances in immunology provide insight into improving vaccine efficacy. Vaccine. 2014; 32(25):2948–57.CrossRef
33.
go back to reference Naud PS, Roteli-Martins CM, De Carvalho NS, Teixeira JC, de Borba PC. Sustained efficacy, immunogenicity, and safety of the HPV-16/18 AS04-adjuvanted vaccine: final analysis of a long-term follow-up study up to 9.4 years post-vaccination. Hum Vaccin Immunother. 2014; 10(8):2147–62.CrossRef Naud PS, Roteli-Martins CM, De Carvalho NS, Teixeira JC, de Borba PC. Sustained efficacy, immunogenicity, and safety of the HPV-16/18 AS04-adjuvanted vaccine: final analysis of a long-term follow-up study up to 9.4 years post-vaccination. Hum Vaccin Immunother. 2014; 10(8):2147–62.CrossRef
34.
go back to reference Siber GR. Methods for estimating serological correlates of protection. Dev Biol Stand. 1997; 89:283–96.PubMed Siber GR. Methods for estimating serological correlates of protection. Dev Biol Stand. 1997; 89:283–96.PubMed
35.
go back to reference Chan IS, Li S, Matthews H, Chan C, Vessey R, Sadoff J, et al.Use of statistical models for evaluating antibody response as a correlate of protection against varicella. Stat Med. 2002; 21(22):3411–1430.CrossRef Chan IS, Li S, Matthews H, Chan C, Vessey R, Sadoff J, et al.Use of statistical models for evaluating antibody response as a correlate of protection against varicella. Stat Med. 2002; 21(22):3411–1430.CrossRef
36.
go back to reference Alonso A, Van der Elst W, Meyvisch P. Assessing a surrogate predictive value: a causal inference approach. Stat Med. 2017; 36(7):1083–98.CrossRef Alonso A, Van der Elst W, Meyvisch P. Assessing a surrogate predictive value: a causal inference approach. Stat Med. 2017; 36(7):1083–98.CrossRef
Metadata
Title
Assessing correlates of protection in vaccine trials: statistical solutions in the context of high vaccine efficacy
Authors
Andrea Callegaro
Fabian Tibaldi
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0687-y

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue