Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Acute Lymphoblastic Leukemia | Research article

Integrating expert opinion with clinical trial data to extrapolate long-term survival: a case study of CAR-T therapy for children and young adults with relapsed or refractory acute lymphoblastic leukemia

Authors: Shannon Cope, Dieter Ayers, Jie Zhang, Katharine Batt, Jeroen P. Jansen

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

Long-term clinical outcomes are necessary to assess the cost-effectiveness of new treatments over a lifetime horizon. Without long-term clinical trial data, current practice to extrapolate survival beyond the trial period involves fitting alternative parametric models to the observed survival. Choosing the most appropriate model is based on how well each model fits to the observed data. Supplementing trial data with feedback from experts may improve the plausibility of survival extrapolations. We demonstrate the feasibility of formally integrating long-term survival estimates from experts with empirical clinical trial data to provide more credible extrapolated survival curves.

Methods

The case study involved relapsed or refractory B-cell pediatric and young adult acute lymphoblastic leukemia (r/r pALL) regarding long-term survival for tisagenlecleucel (chimeric antigen receptor T-cell [CAR-T]) with evidence from the phase II ELIANA trial. Seven pediatric oncologists and hematologists experienced with CAR-T therapies were recruited. Relevant evidence regarding r/r pALL and tisagenlecleucel provided a common basis for expert judgments. Survival rates and related uncertainty at 2, 3, 4, and 5 years were elicited from experts using a web-based application adapted from Sheffield Elicitation Framework. Estimates from each expert were combined with observed data using time-to-event parametric models that accounted for experts’ uncertainty, producing an overall distribution of survival over time. These results were validated based on longer term follow-up (median duration 24.2 months) from ELIANA following the elicitation.

Results

Extrapolated survival curves based on ELIANA trial without expert information were highly uncertain, differing substantially depending on the model choice. Survival estimates between 2 to 5 years from individual experts varied with a fair amount of uncertainty. However, incorporating expert estimates improved the precision in the extrapolated survival curves. Predictions from a Gompertz model, which experts believed was most appropriate, suggested that more than half of the ELIANA patients treated with tisagenlecleucel will survive up to 5 years. Expert estimates at 24 months were validated by longer follow-up.

Conclusions

This study provides an example of how expert opinion can be elicited and synthesized with observed survival data using a transparent and formal procedure, capturing expert uncertainty, and ensuring projected long-term survival is clinically plausible.
Appendix
Available only for authorised users
Literature
1.
go back to reference Latimer NR. Survival analysis for economic evaluations alongside clinical trials--extrapolation with patient-level data: inconsistencies, limitations, and a practical guide. Med Decis Mak. 2013;33(6):743–54.CrossRef Latimer NR. Survival analysis for economic evaluations alongside clinical trials--extrapolation with patient-level data: inconsistencies, limitations, and a practical guide. Med Decis Mak. 2013;33(6):743–54.CrossRef
2.
go back to reference Jackson C, Stevens J, Ren S, Latimer N, Bojke L, Manca A, Sharples L. Extrapolating survival from randomized trials using external data: a review of methods. Med Decis Mak. 2017;37(4):377–90.CrossRef Jackson C, Stevens J, Ren S, Latimer N, Bojke L, Manca A, Sharples L. Extrapolating survival from randomized trials using external data: a review of methods. Med Decis Mak. 2017;37(4):377–90.CrossRef
3.
go back to reference Grigore B, Peters J, Hyde C, Stein K. A comparison of two methods for expert elicitation in health technology assessments. BMC Med Res Methodol. 2016;16:85.PubMedPubMedCentralCrossRef Grigore B, Peters J, Hyde C, Stein K. A comparison of two methods for expert elicitation in health technology assessments. BMC Med Res Methodol. 2016;16:85.PubMedPubMedCentralCrossRef
4.
go back to reference Bojke L, Grigore B, Jankovic D, Peters J, Soares M, Stein K. Informing reimbursement decisions using cost-effectiveness modelling: a guide to the process of generating elicited priors to capture model uncertainties. PharmacoEconomics. 2017;35:867–77.PubMedCrossRef Bojke L, Grigore B, Jankovic D, Peters J, Soares M, Stein K. Informing reimbursement decisions using cost-effectiveness modelling: a guide to the process of generating elicited priors to capture model uncertainties. PharmacoEconomics. 2017;35:867–77.PubMedCrossRef
5.
go back to reference Guyot P, Ades AE, Beasley M, Lueza B, Pignon J-P, Welton NJ. Extrapolation of survival curves from Cancer trials using external information. Med Decis Mak. 2017;37(4):353–66.PubMedPubMedCentralCrossRef Guyot P, Ades AE, Beasley M, Lueza B, Pignon J-P, Welton NJ. Extrapolation of survival curves from Cancer trials using external information. Med Decis Mak. 2017;37(4):353–66.PubMedPubMedCentralCrossRef
6.
go back to reference Howlader N, Noone AM, Krapcho M, Miller D, Bishop K, Kosary CL, Yu M, Ruhl J, Tatalovich Z, Mariotto A, et al. SEER Cancer Statistics Review, 1975-2014. Bethesda: National Cancer Institute; 2017. Howlader N, Noone AM, Krapcho M, Miller D, Bishop K, Kosary CL, Yu M, Ruhl J, Tatalovich Z, Mariotto A, et al. SEER Cancer Statistics Review, 1975-2014. Bethesda: National Cancer Institute; 2017.
7.
go back to reference Chessells JM, Veys P, Kempski H, Henley P, Leiper A, Webb D, Hann IM. Long-term follow-up of relapsed childhood acute lymphoblastic leukaemia. Br J Haematol. 2003;123(3):396–405.PubMedCrossRef Chessells JM, Veys P, Kempski H, Henley P, Leiper A, Webb D, Hann IM. Long-term follow-up of relapsed childhood acute lymphoblastic leukaemia. Br J Haematol. 2003;123(3):396–405.PubMedCrossRef
8.
go back to reference Oudot C, Auclerc MF, Levy V, Porcher R, Piguet C, Perel Y, Gandemer V, Debre M, Vermylen C, Pautard B, et al. Prognostic factors for leukemic induction failure in children with acute lymphoblastic leukemia and outcome after salvage therapy: the FRALLE 93 study. J Clin Oncol. 2008;26(9):1496–503.PubMedCrossRef Oudot C, Auclerc MF, Levy V, Porcher R, Piguet C, Perel Y, Gandemer V, Debre M, Vermylen C, Pautard B, et al. Prognostic factors for leukemic induction failure in children with acute lymphoblastic leukemia and outcome after salvage therapy: the FRALLE 93 study. J Clin Oncol. 2008;26(9):1496–503.PubMedCrossRef
9.
go back to reference Reismuller B, Peters C, Dworzak MN, Potschger U, Urban C, Meister B, Schmitt K, Dieckmann K, Gadner H, Attarbaschi A, et al. Outcome of children and adolescents with a second or third relapse of acute lymphoblastic leukemia (ALL): a population-based analysis of the Austrian ALL-BFM (Berlin-Frankfurt-Munster) study group. J Pediatr Hematol Oncol. 2013;35(5):e200–4.PubMedCrossRef Reismuller B, Peters C, Dworzak MN, Potschger U, Urban C, Meister B, Schmitt K, Dieckmann K, Gadner H, Attarbaschi A, et al. Outcome of children and adolescents with a second or third relapse of acute lymphoblastic leukemia (ALL): a population-based analysis of the Austrian ALL-BFM (Berlin-Frankfurt-Munster) study group. J Pediatr Hematol Oncol. 2013;35(5):e200–4.PubMedCrossRef
10.
go back to reference Maude SL, Frey N, Shaw PA, Aplenc R, Barrett DM, Bunin NJ, Chew A, Gonzalez VE. Chimeric antigen receptor T cells for sustained remissions in leukemia. N Engl J Med. 2014;371(16):1507–17.PubMedPubMedCentralCrossRef Maude SL, Frey N, Shaw PA, Aplenc R, Barrett DM, Bunin NJ, Chew A, Gonzalez VE. Chimeric antigen receptor T cells for sustained remissions in leukemia. N Engl J Med. 2014;371(16):1507–17.PubMedPubMedCentralCrossRef
11.
go back to reference Hettle R, Corbett M, Hinde S, Hodgson R, Jones-Diette J, Woolacott N, Palmer S. In: York CCUo, editor. Exploring the assessment and appraisal of regenerative medicines and cell therapy products. Heslington, York: CRD and CHE Technology Assessment Group, University of York; 2016. Hettle R, Corbett M, Hinde S, Hodgson R, Jones-Diette J, Woolacott N, Palmer S. In: York CCUo, editor. Exploring the assessment and appraisal of regenerative medicines and cell therapy products. Heslington, York: CRD and CHE Technology Assessment Group, University of York; 2016.
12.
go back to reference Grupp SA, Laetsch TW, Buechner J, Bittencourt H, Maude SL, Verneris MR, Myers GD, Boyer MW, Rives S, De Moerloose B, et al. Analysis of a global registration trial of the efficacy and safety of CTL-19 in pediatric and young adults with relapsed/refractory acute lymphoblastic leukemia (ALL). Blood-J Hematol. 2016:128(22). https://biblio.ugent.be/publication/8525747. Grupp SA, Laetsch TW, Buechner J, Bittencourt H, Maude SL, Verneris MR, Myers GD, Boyer MW, Rives S, De Moerloose B, et al. Analysis of a global registration trial of the efficacy and safety of CTL-19 in pediatric and young adults with relapsed/refractory acute lymphoblastic leukemia (ALL). Blood-J Hematol. 2016:128(22). https://​biblio.​ugent.​be/​publication/​8525747.
14.
go back to reference O'Hagan A, Buck C, Daneshkhah A, Eiser J, Garthwaite P, Jenkinson D, Oakley J, Rakow T. Uncertain judgements: eliciting experts’ probabilities. Statistics in Practice. Chichester: Wiley; 2006. O'Hagan A, Buck C, Daneshkhah A, Eiser J, Garthwaite P, Jenkinson D, Oakley J, Rakow T. Uncertain judgements: eliciting experts’ probabilities. Statistics in Practice. Chichester: Wiley; 2006.
15.
go back to reference Johnson SR, Tomlinson GA, Hawker GA, Granton JT, Feldman BM. Methods to elicit beliefs for Bayesian priors: a systematic review. J Clin Epidemiol. 2010;63(4):355–69.PubMedCrossRef Johnson SR, Tomlinson GA, Hawker GA, Granton JT, Feldman BM. Methods to elicit beliefs for Bayesian priors: a systematic review. J Clin Epidemiol. 2010;63(4):355–69.PubMedCrossRef
16.
go back to reference Maude SL, Pulsipher MA, Boyer MW, Grupp SA, Davies SM, Phillips CL, Verneris MR, August KJ, Schlis K, Driscoll TA, et al. Efficacy and safety of CTL019 in the first US phase II multicenter trial in pediatric relapsed/refractory acute lymphoblastic leukemia: results of an interim analysis. Blood-J Hematol. 2016;128:2801. Maude SL, Pulsipher MA, Boyer MW, Grupp SA, Davies SM, Phillips CL, Verneris MR, August KJ, Schlis K, Driscoll TA, et al. Efficacy and safety of CTL019 in the first US phase II multicenter trial in pediatric relapsed/refractory acute lymphoblastic leukemia: results of an interim analysis. Blood-J Hematol. 2016;128:2801.
17.
go back to reference Grupp SA, Maude SL, Shaw PA, Aplenc R, Barrett DM, Callahan C, Lacey SF, Levine BL, Melenhorst JJ, Motley L, et al. Durable remissions in children with relapsed/refractory ALL treated with T-cells engineered with a CD19-targeted chimeric antigen receptor (CLT019). Blood-J Hematol. 2015;126:681. Grupp SA, Maude SL, Shaw PA, Aplenc R, Barrett DM, Callahan C, Lacey SF, Levine BL, Melenhorst JJ, Motley L, et al. Durable remissions in children with relapsed/refractory ALL treated with T-cells engineered with a CD19-targeted chimeric antigen receptor (CLT019). Blood-J Hematol. 2015;126:681.
18.
go back to reference Ouwens MJNM, Philips Z, Jansen JP. Network meta-analysis of parametric survival curves. Res Synth Methods. 2011;1(3–4):258–71. Ouwens MJNM, Philips Z, Jansen JP. Network meta-analysis of parametric survival curves. Res Synth Methods. 2011;1(3–4):258–71.
20.
go back to reference Jansen JP, Cope S. Meta-regression models to address heterogeneity and inconsistency in network meta-analysis of survival outcomes. BMC Med Res Methodol. 2012;12(1):152.PubMedPubMedCentralCrossRef Jansen JP, Cope S. Meta-regression models to address heterogeneity and inconsistency in network meta-analysis of survival outcomes. BMC Med Res Methodol. 2012;12(1):152.PubMedPubMedCentralCrossRef
21.
go back to reference Dempster AP. The direct use of likelihood for significance testing. Stat Comput. 1997;7(4):247–52.CrossRef Dempster AP. The direct use of likelihood for significance testing. Stat Comput. 1997;7(4):247–52.CrossRef
22.
go back to reference Spiegelhalter D, Best N, Carlin B, Van Der Linde A. Bayesian measures of model complexity and fit. J R Stat Soc Series B Stat Methodol. 2002;64(4):583–639.CrossRef Spiegelhalter D, Best N, Carlin B, Van Der Linde A. Bayesian measures of model complexity and fit. J R Stat Soc Series B Stat Methodol. 2002;64(4):583–639.CrossRef
23.
go back to reference Grupp SA, Maude SL, Rives S, Baruchel A, Boyer MW, Bittencourt H, Bader P, Büchner J, Laetsch TW, Stefanski H, et al. Updated analysis of the efficacy and safety of tisagenlecleucel in pediatric and young adult patients with relapsed/refractory acute lymphoblastic leukemia. In: 60th American Society of Hematology Annual Meeting: December 1–4 2018; San Diego, California; 2018. Abstract 895. Grupp SA, Maude SL, Rives S, Baruchel A, Boyer MW, Bittencourt H, Bader P, Büchner J, Laetsch TW, Stefanski H, et al. Updated analysis of the efficacy and safety of tisagenlecleucel in pediatric and young adult patients with relapsed/refractory acute lymphoblastic leukemia. In: 60th American Society of Hematology Annual Meeting: December 1–4 2018; San Diego, California; 2018. Abstract 895.
24.
go back to reference Jeha S, Gaynon PS, Razzouk BI, Franklin J, Kadota R, Shen V, Luchtman-Jones L, Rytting M, Bomgaars LR, Rheingold S, et al. Phase II study of clofarabine in pediatric patients with refractory or relapsed acute lymphoblastic leukemia. J Clin Oncol. 2006;24(12):1917–23.PubMedCrossRef Jeha S, Gaynon PS, Razzouk BI, Franklin J, Kadota R, Shen V, Luchtman-Jones L, Rytting M, Bomgaars LR, Rheingold S, et al. Phase II study of clofarabine in pediatric patients with refractory or relapsed acute lymphoblastic leukemia. J Clin Oncol. 2006;24(12):1917–23.PubMedCrossRef
25.
go back to reference von Stackelberg A, Locatelli F, Zugmaier G, Handgretinger R, Trippett TM, Rizzari C, Bader P, O'Brien MM, Brethon B, Bhojwani D, et al. Phase I/phase II study of Blinatumomab in pediatric patients with relapsed/refractory acute lymphoblastic leukemia. J Clin Oncol. 2016;34(36):4381–9.CrossRef von Stackelberg A, Locatelli F, Zugmaier G, Handgretinger R, Trippett TM, Rizzari C, Bader P, O'Brien MM, Brethon B, Bhojwani D, et al. Phase I/phase II study of Blinatumomab in pediatric patients with relapsed/refractory acute lymphoblastic leukemia. J Clin Oncol. 2016;34(36):4381–9.CrossRef
26.
go back to reference Windish DM, Huot SJ, Green ML. Medicine residents’ understanding of the biostatistics and results in the medical literature. JAMA. 2007;298(9):1010–22.PubMedCrossRef Windish DM, Huot SJ, Green ML. Medicine residents’ understanding of the biostatistics and results in the medical literature. JAMA. 2007;298(9):1010–22.PubMedCrossRef
27.
go back to reference Bollschweiler E. Benefits and limitations of Kaplan-Meier calculations of survival chance in cancer surgery. Langenbeck's Arch Surg. 2003;388(4):239–44.CrossRef Bollschweiler E. Benefits and limitations of Kaplan-Meier calculations of survival chance in cancer surgery. Langenbeck's Arch Surg. 2003;388(4):239–44.CrossRef
28.
go back to reference Pocock SJ, Clayton TC, Altman DG. Survival plots of time-to-event outcomes in clinical trials: good practice and pitfalls. Lancet. 2002;359(9318):1686–9.PubMedCrossRef Pocock SJ, Clayton TC, Altman DG. Survival plots of time-to-event outcomes in clinical trials: good practice and pitfalls. Lancet. 2002;359(9318):1686–9.PubMedCrossRef
30.
go back to reference Armstrong G, Chen Y, Yasui Y, Leisenring W, Gibson TM, Mertens A, Stovall M, Oeffinger KC, Bhatia S. Reduction in late mortality among 5-year survivors of childhood cancer. N Engl J Med. 2015;374(9):833–42.CrossRef Armstrong G, Chen Y, Yasui Y, Leisenring W, Gibson TM, Mertens A, Stovall M, Oeffinger KC, Bhatia S. Reduction in late mortality among 5-year survivors of childhood cancer. N Engl J Med. 2015;374(9):833–42.CrossRef
31.
go back to reference Royston P, Parmar MK. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Stat Med. 2002;21(15):2175–97.PubMedCrossRef Royston P, Parmar MK. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Stat Med. 2002;21(15):2175–97.PubMedCrossRef
Metadata
Title
Integrating expert opinion with clinical trial data to extrapolate long-term survival: a case study of CAR-T therapy for children and young adults with relapsed or refractory acute lymphoblastic leukemia
Authors
Shannon Cope
Dieter Ayers
Jie Zhang
Katharine Batt
Jeroen P. Jansen
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0823-8

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue