Top

Published in:

Open Access 01-12-2018 | Review

Lessons learned from IDeAl — 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials

Authors: Ralf-Dieter Hilgers, Malgorzata Bogdan, Carl-Fredrik Burman, Holger Dette, Mats Karlsson, Franz König, Christoph Male, France Mentré, Geert Molenberghs, Stephen Senn

Published in: Orphanet Journal of Rare Diseases | Issue 1/2018

Abstract

Background

IDeAl (Integrated designs and analysis of small population clinical trials) is an EU funded project developing new statistical design and analysis methodologies for clinical trials in small population groups. Here we provide an overview of IDeAl findings and give recommendations to applied researchers.

Method

The description of the findings is broken down by the nine scientific IDeAl work packages and summarizes results from the project’s more than 60 publications to date in peer reviewed journals. In addition, we applied text mining to evaluate the publications and the IDeAl work packages’ output in relation to the design and analysis terms derived from in the IRDiRC task force report on small population clinical trials.

Results

The results are summarized, describing the developments from an applied viewpoint. The main result presented here are 33 practical recommendations drawn from the work, giving researchers a comprehensive guidance to the improved methodology. In particular, the findings will help design and analyse efficient clinical trials in rare diseases with limited number of patients available. We developed a network representation relating the hot topics developed by the IRDiRC task force on small population clinical trials to IDeAl’s work as well as relating important methodologies by IDeAl’s definition necessary to consider in design and analysis of small-population clinical trials. These network representation establish a new perspective on design and analysis of small-population clinical trials.

Conclusion

IDeAl has provided a huge number of options to refine the statistical methodology for small-population clinical trials from various perspectives. A total of 33 recommendations developed and related to the work packages help the researcher to design small population clinical trial. The route to improvements is displayed in IDeAl-network representing important statistical methodological skills necessary to design and analysis of small-population clinical trials. The methods are ready for use.

CHMP. Guideline on clinical trials in small populations. 2007. [Cited: February 1,2013.] www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003615.pdf.

Austin CP, et al. Future of rare diseases research 2017-2027: an IRDiRC perspective. Clin Transl Sci. 2018;11(1):21–7. https://doi.org/10.1111/cts.12500.CrossRefPubMed

Hilgers RD, Roes KC, Stallard N. Directions for new developments on statistical design and analysis of small population group trials. Orphanet J Rare Dis. 2016;11(1):78. https://doi.org/10.1186/s13023-016-0464-5.CrossRefPubMedPubMedCentral

Senn SJ. A note regarding meta-analysis of sequential trials with stopping for efficacy. Pharm Stat. 2014;13(6):371–5. https://doi.org/10.1002/pst.1639.CrossRefPubMed

Bauer P, König F. Adaptive paediatric investigation plans, a small step to improve regulatory decision making in drug development for children. Pharm Stat. 2016:384–6. https://doi.org/10.1002/pst.1762.CrossRefPubMed

Lendrem D, et al. R&D productivity rides again? Pharm Stat. 2015;14(1):–3. https://doi.org/10.1002/pst.1653.CrossRefPubMed

Jobjörnsson S, et al. Late-stage pharmaceutical R&D and pricing policies under two-stage regulation. J Health Econ. 2016;50:298–311. https://doi.org/10.1016/j.jhealeco.2016.06.002.CrossRefPubMed

Miller F, Burman CF. A decision theoretical modeling for Phase III investments and drug licensing. J Biopharm Stat. 2017;18:1–24. https://doi.org/10.1080/10543406.2017.1377729. CrossRef

Ondra T, et al. Optimizing trial designs for targeted therapies. PLoS ONE. 2017;11(9):e0163726. https://doi.org/10.1371/journal.pone.0163726.CrossRef

10.

Ondra T, Jobjörnsson S, Beckman RA, Burman CF, König F, Stallard N, Posch M. Optimized adaptive enrichment designs. Stat Methods Med Res. 2017; https://doi.org/10.1177/0962280217747312.

11.

Burman CF. From optimal design theory to optimizing designs of clinical trials. In: Fackle-Fornius E, editor. Festschrift in Honor of Hans Nyquist on the Occasion of His 65th Birthday. Stockholm: Stockholm Univ.; 2015.

12.

Karlsson KE, et al. Comparisons of Analysis Methods for Proof-of-Concept Trials. CPT Pharmacometrics Syst Pharmacol. 2013;2:e23. https://doi.org/10.1038/psp.2012.24.CrossRefPubMedPubMedCentral

13.

Keizer RJ, Karlsson MO, Hooker AC. Modeling and Simulation Workbench for NONMEM: Tutorial on Pirana, PsN, and Xpose. CPT Pharmacometrics Syst Pharmacol. 2013;2:e50. https://doi.org/10.1038/psp.2013.24.CrossRefPubMedPubMedCentral

14.

Danhof M, et al. Mechanism-based pharmacokinetic-pharmacodynamic (PK-PD) modeling in translational drug research. Trends Pharmacol Sci. 2008;29:186–91. https://doi.org/10.1016/j.tips.2008.01.007.CrossRefPubMed

15.

Marshall S, et al. Role of mechanistically-based pharmacokinetic/pharmacodynamic models in drug development: a case study of a therapeutic protein. Clin Pharmacokinet. 2006;45:177–97.CrossRefPubMed

16.

Lesko LJ. Drug Research and Translational Bioinformatics. Clin Pharmacol Ther. 2012;91:960–2. https://doi.org/10.1038/clpt.2012.45.CrossRefPubMed

17.

Ueckert S, Karlsson MO, Hooker AC. Accelerating Monte Carlo power studies through parametric power estimation. J Pharmacokinet Pharmacodyn. 2016;43:223–34. https://doi.org/10.1007/s10928-016-9468-y.CrossRefPubMedPubMedCentral

18.

Lindbom L, Ribbing J, Jonsson EN. Perl-speaks-NONMEM (PsN) - a Perl module for NONMEM related programming. Comput Methods Programs Biomed. 2004;75:85–94. https://doi.org/10.1016/j.cmpb.2003.11.003.CrossRefPubMed

19.

Jonsson EN, Karlsson MO. Xpose--an S-PLUS based population pharmacokinetic/pharmacodynamic model building aid for NONMEM. Comput Methods Programs Biomed. 1999;58:51–64.CrossRefPubMed

20.

Harling, K., et al. Perl speaks NONMEM ( PsN ) and Xpose. Abstracts of the Annual Meeting of the Population Approach Group in Europe. 2016. https://www.pagemeeting.org/default.asp?abstract=5916.

21.

Deng, C., Plan, EL. and Karlsson, MO. Influence of clinical trial design to detect drug effect in systems with within subject variability. Abstracts of the Annual Meeting of the Population Approach Group in Europe. 2015. www.page-meeting.org/?abstract=3549.

22.

Dosne AG, et al. Improving the estimation of parameter uncertainty distributions in nonlinear mixed effects models using sampling importance resampling. J Pharmacokinet Pharmacodyn. 2016;43:583–96. https://doi.org/10.1007/s10928-016-9487-8.CrossRefPubMedPubMedCentral

23.

Dosne AG, Niebecker R, Karlsson MO. dOFV distributions: a new diagnostic for the adequacy of parameter uncertainty in nonlinear mixed-effects models applied to the bootstrap. J Pharmacokinet Pharmacodyn. 2016;43:597–608. https://doi.org/10.1007/s10928-016-9496-7.CrossRefPubMedPubMedCentral

24.

Aoki Y, Nordgren R, Hooker AC. Preconditioning of Nonlinear Mixed Effects Models for Stabilisation of Variance-Covariance Matrix Computations. AAPS J. 2016;18(2):505–18. https://doi.org/10.1208/s12248-016-9866-5.CrossRefPubMedPubMedCentral

25.

Aoki, Y., et al. Averaged Model Based Decision Making for Dose Selection Studies. Abstracts of the Annual Meeting of the Population Approach Group in Europe. 2014. www.page-meeting.org/?abstract=3121.

26.

Strömberg, EA. and Hooker, AC. Simulated model based adaptive optimal design of adult to children bridging study using FDA stopping criteria. Design and Analysis of Experiments in Healthcare. 2015. https://www.newton.ac.uk/event/daew07.

27.

Strömberg EA, Hooker AC. The effect of using a robust optimality criterion in model based adaptive optimization. J Pharmacokinet Pharmacodyn. 2017;44(4):317–24. https://doi.org/10.1007/s10928-017-9521-5.CrossRefPubMedPubMedCentral

28.

Wang S, et al. A proof-of-principle example for identifying drug effect from a mechanistic model with a more parsimonious model. J Pharmacokinet Pharmacodyn. 2016;43:S35.

29.

Wellhagen, GJ., Karlsson, MO. and Kjellsson, MC. Quantifying drug effects in phase 2a anti-diabetic studies?: Power of four HbA1c models. Abstracts of the Annual Meeting of the Population Approach Group in Europe. 2015. https://www.pagemeeting.org/?abstract=3631.

30.

Ueckert S, Mentré F. A new method for evaluation of the Fisher information matrix for discrete mixed effect models using Monte Carlo sampling and adaptive Gaussian quadrature. Comput Stat Data Anal. 2017:203–19. https://doi.org/10.1016/j.csda.2016.10.011.CrossRef

31.

Riviere MK, Ueckert S, Mentré F. An MCMC method for the evaluation of the Fisher information matrix for non-linear mixed effect models. Biostatistics. 2016;17(4):737–50. https://doi.org/10.1093/biostatistics/kxw020.CrossRefPubMed

32.

Lestini G, Dumont C, Mentré F. Influence of the size of cohorts in adaptive design for nonlinear mixed effects models: an evaluation by simulation for a pharmacokinetic and pharmacodynamic model for a biomarker in oncology. Pharm Res. 2015;32(10):3159–69. https://doi.org/10.1007/s11095-015-1693-3.CrossRefPubMedPubMedCentral

33.

Dumont C, Chenel M, Mentré F. Two-stage adaptive designs in nonlinear mixed effects models: application to pharmacokinetics in children. Commun Stat Simul Comput. 2016;45(5):1511–25. https://doi.org/10.1080/03610918.2014.930901.CrossRef

34.

Loingeville, F., et al. A new approach for robust designs in longitudinal studies accounting for parameter and model uncertainties - Application to count data. Biostatistics. submitted.

35.

Lonergan M, et al. Defining drug response for stratified medicine. Drug Discov Today. 2017;22(1):173–9. https://doi.org/10.1016/j.drudis.2016.10.016.CrossRefPubMed

36.

Bogdan M, et al. SLOPE-Adaptive Variable Selection via Convex Optimization. Ann Appl Stat. 2015;9(3):1103–40. https://doi.org/10.1214/15-AOAS842.CrossRefPubMedPubMedCentral

37.

Brzyski D, et al. Group SLOPE - adaptive selection of groups of predictors. J Am Stat Assoc. 2018; https://doi.org/10.1080/01621459.2017.1411269.

38.

Brzyski D, et al. Controlling the rate of GWAS false discoveries. Genetics. 2017;205(1):61–75. https://doi.org/10.1534/genetics.116.193987.CrossRefPubMed

39.

Szulc P, et al. Joint Genotype- and Ancestry-based Genome-wide Association Studies in Admixed Populations. Genet Epidemiol. 2017;41(6):555–66. https://doi.org/10.1002/gepi.22056.CrossRefPubMed

40.

Sobczyk, P., Bogdan, M. and Josse, J. varclust - Variable Subspace Clustering package in R. 2016 https://github.com/psobczyk/varclust.

41.

Sobczyk, P., Bogdan, M. and Josse, J. Bayesian dimensionality reduction with PCA using penalized semi-integrated likelihood. 2016. [Cited: May 04, 2018.] https://arxiv.org/abs/1606.05333.

42.

Sabino, G., et al. Patient- Centered Outcome Measures in the Field of Rare Diseases. 2016. [Cited: May 04, 2018.] http://www.irdirc.org/wpcontent/uploads/2017/12/PCOM_Post-Workshop_Report_Final.pdf.

43.

Van der Elst W, Molenberghs G, Alonso A. Exploring the relationship between the causal-inference and meta-analytic paradigms for the evaluation of surrogate endpoints. Stat Med. 2015:1281–98. https://doi.org/10.1002/sim.6807.CrossRefPubMed

44.

Alonso Abad, A., et al. A causal-inference approach for the validation of surrogate endpoints based on information theory and sensitivity analysis. Biometrics 2016, p. DOI: https://doi.org/10.1111/biom.12483.CrossRefPubMed

45.

Alonso Abad A, Van der Elst W, Meyvish P. Assessing a surrogate predictive value: a causal inference approach. Stat Med. 2017:1083–98. https://doi.org/10.1002/sim.7197.CrossRefPubMed

46.

Hermans L, et al. Fast, closed-form, and efficient estimators for hierarchical models with AR(1) covariance and unequal cluster size. Commun Stat. 2017; https://doi.org/10.1080/03610918.2017.1316395. CrossRef

47.

Dette H, et al. Equivalence of regression curves. J Am Stat Assoc. 2018; https://doi.org/10.1080/01621459.2017.1281813.CrossRef

48.

Flórez Poveda, A., et al. Closed-form Estimators to Evaluate Surrogate Endpoints in a Meta-analysis with Normally Distributed Endpoints. Submitted for publication. 2017, Vol. Submitted.

49.

Nassiri V, et al. Fast precision estimation in high-dimensional multivariate joint models. Biom J. 2017;59(6):1221–31. https://doi.org/10.1002/bimj.201600241.CrossRefPubMed

50.

Alonso Abad A, et al. Applied Surrogate Endpoint Evaluation with SAS and R. Boca Raton: Chapman&Hall/CRC; 2017.

51.

Van der Elst W, et al. Unbalanced cluster sizes and rates of convergence in mixed-effect models for clustered data. J Stat Comput Stimul. 2016:2123–39. https://doi.org/10.1080/00949655.2015.1103738.CrossRef

52.

Van der Elst W, Molenberghs G. Surrogate Endpoints in Rare Diseases. In: Alonso A, et al., editors. Applied Surrogate Endpoint Evaluation Methods with SAS and R. Boca Raton: Chapman & Hall/Crc; 2016. p. 257–74.

53.

Van der Elst W, et al. Estimating the reliability of repeatedly measured endpoints based on linear mixed-effectsmodels. A tutorial. Pharm Stat. 2016;15(6):486–93. https://doi.org/10.1002/pst.1787.CrossRefPubMed

54.

Langer S. The modified distribution of the t-test statistic under the infuence of selection bias based on random allocation rule. Mathematical Faculty, RWTH Aachen University. Aachen: Mathematical Faculty; 2014. Master Thesis

55.

Uschner D, Hilgers RD, Heussen N. The Impact of selection bias in randomized multi-arm parallel group clinical trials. PLoS One. 2018;13(1):e0192065. https://doi.org/10.1371/journal.pone.0192065.CrossRefPubMedPubMedCentral

56.

Rückbeil M, Hilgers RD, Heussen N. Assessing the impact of selection bias on test decisions in trials with a time-to-event outcome. Stat Med. 2017;36(17):2656–68. https://doi.org/10.1002/sim.7299.CrossRefPubMedPubMedCentral

57.

Tamm M, Hilgers RD. Chronological Bias in Randomized Clinical Trials Arising from Different Types of Unobserved Time Trends. Methods Inf Med. 2014;53:501–10. https://doi.org/10.3414/ME14-01-0048.CrossRefPubMed

58.

Schindler, D. Assessment of Randomization Procedures in the Presence of Selection and Chronological Bias. Statistik, TU Dortmund. Dortmund : s.n., 2016. PhD Thesis.

59.

Hilgers RD, et al. ERDO - A framework to select an appropriate randomization procedure for clinical trials. BMC Med Res. 2017;17(1):159. https://doi.org/10.1186/s12874-017-0428-z.CrossRef

60.

Uschner, D., et al. randomizeR: An R Package for the Assessment and Implementation of Randomization in Clinical Trials. Statistical Software Letters. 2017 (accepted).

61.

Kennes LN, Rosenberger WF, Hilgers RD. Inference for blocked randomization under a selection bias model. Biometrics. 2015;71(4):979–84. DOI.org/10.1111/biom.12334 CrossRefPubMed

62.

Kennes LN, et al. The impact of selection bias on test decisions in randomized clinical trials. Stat Med. 2011;30(21):2573–81. https://doi.org/10.1002/sim.4279.CrossRefPubMed

63.

Tamm M, et al. In uence of Selection Bias on the Test Decision - A Simulation Study. Methods Inf Med. 2012;51:138–43. https://doi.org/10.3414/ME11-01-0043. CrossRefPubMed

64.

Bauer P, et al. Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls. Stat Med. 2016;35(3):325–47. https://doi.org/10.1002/sim.6472.CrossRefPubMed

65.

Jonker, AH., et al. Small Population Clinical Trials: Challenges in the Field of Rare Diseases. 2016. [Cited: May 04, 2018.] http://www.irdirc.org/wpcontent/uploads/2017/12/SPCT_Report.pdf.

66.

Hlavin G, et al. Evidence, eminence and extrapolation. Stat Med. 2016;35(13):2117–32. https://doi.org/10.1002/sim.6865.CrossRefPubMedPubMedCentral

67.

König F, et al. Sharing clinical trial data on patient level: Opportunities and challenges. Biom J. 2015;57(1):8–28. https://doi.org/10.1002/bimj.201300283.CrossRef

68.

ICH. Topic E 10: Choice of Control Group in Clinical Trials. NOTE FOR GUIDANCE ON CHOICE OF CONTROL GROUP IN CLINICAL TRIALS. 2001. [Cited: July 10, 2017.] http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500002925.pdf. CPMP/ICH/364/96.

69.

Eichler HG, et al. ‘Treshold-crossing’: A useful way to establish the counterfactual in clinical trials? Clin Pharmacol Ther. 2016:699–712. https://doi.org/10.1002/cpt.515.CrossRef

70.

Klinglmueller F, Posch M, Koenig F. Adaptive graph-based multiple testingprocedures. Pharm Stat. 2014;13:345–56. https://doi.org/10.1002/pst.1640.CrossRefPubMedPubMedCentral

71.

Hofer MP, et al. Marketing authorisation of orphan medicines in Europe 2000-2013: a 14-year experience. Nat Rev Drug Discov. 2018;23(2):424–33. https://doi.org/10.1016/j.drudis.2017.10.012.CrossRef

72.

Magirr D, et al. Sample Size Reassessment and Hypothesis Testing in Adaptive Survival Trials. PLoS One. 2016;11(2):e0146465. https://doi.org/10.1371/journal.pone.0146465.CrossRefPubMedPubMedCentral

73.

Brueckner, M., Posch, M. and Koenig, F. Group-sequential permutation tests for time-toevent data. submitted. 2017.

74.

Bird SM, et al. Statistical issues in first-in-human studies on BIA 10-2474: Neglected comparison of protocol against practice. Pharm Stat. 2017;16(2):100–6. https://doi.org/10.1002/pst.1801.CrossRefPubMedPubMedCentral

75.

Gewandter JS, et al. Research designs for proof-of-concept chronic pain clinical trials: IMMPACT recommendations. Pain. 2014;155(9):1683–95. https://doi.org/10.1016/j.pain.2014.05.025.CrossRefPubMedPubMedCentral

76.

Araujo A, Julious S, Senn SJ. Understanding Variation in Sets of N-of-1 Trials. PLoS One. 2016;11(12):e0167167. https://doi.org/10.1371/journal.pone.0167167.CrossRefPubMedPubMedCentral

77.

Senn SJ. Sample size considerations for n-of-1 trials. Stat Methods Med Res. 2017; https://doi.org/10.1177/0962280217726801.

78.

Collignon O, Monnez JM. Clustering of the values of a response variable and simultaneous covariate selection using a stepwise algorithm. Applied Mathematics. 2016;7:1639–48. https://doi.org/10.4236/am.2016.715141. CrossRef

79.

Senn SJ. Mastering variation: variance components and personalised medicine. Stat Med. 2016;35:966–77. https://doi.org/10.1002/sim.6739.CrossRefPubMed

80.

Liu W, et al. Assessing nonsuperiority, noninferiority, or equivalence when comparing two regression models over a restricted covariate region. Biometrics. 2009;65(4):1279–87. https://doi.org/10.1111/j.1541-0420.2008.01192.x.CrossRefPubMed

81.

Gsteiger S, Bretz F, Liu W. Simultaneous Confidence Bands for Nonlinear Regression Models with Application to Population Pharmacokinetic Analyses. J Biopharm Stat. 2011;21(4):708–25. https://doi.org/10.1080/10543406.2011.551332.CrossRefPubMed

82.

Möllenhoff, K. CRAN. Bootstrap Test for Similarity of Dose Response Curves Concerning the Maximum Absolute Deviation. 2016. https://cran.rproject.org/web/packages/TestingSimilarity/index.html.

83.

Ting N. Dose Finding in Drug Development. New York: Springer; 2006.CrossRef

84.

Bretz F, et al. Assessing the similarity of dose response and target doses in two nonoverlapping subgroups. Stat Med. 2018;37(5):722–38. https://doi.org/10.1002/sim.7546.CrossRefPubMed

85.

Schorning K, et al. Model Selection versus Model Averaging in Dose Finding Studies. Stat Med. 2016:4021–40. https://doi.org/10.1002/sim.6991.CrossRefPubMed

86.

Dette H, Schorning K. Optimal designs for comparing curves. Ann Stat. 2016;44(3):1103–30. https://doi.org/10.1214/15-AOS1399.CrossRefPubMedPubMedCentral

87.

Dette H, Schorning K, Konstantinou M. Optimal designs for comparing regression models with correlated observations. Comput Stat Data Anal. 2017;113:273–86. https://doi.org/10.1016/j.csda.2016.06.017.CrossRef

88.

Hecksteden A, et al. Individual response to exercise training – a statistical perspective. J Appl Physiol. 2015;118(12):1450–9. https://doi.org/10.1152/japplphysiol.00714.2014.CrossRefPubMed

89.

Reetz K, et al. Progression characteristics of the European Friedreich’s Ataxia Consortium for Translational Studies (EFACTS). Lancet Neurol. 2016:1346–54. https://doi.org/10.1016/S1474-4422(16)30287-3.CrossRefPubMed

90.

Cornu C, et al. Experimental designs for small randomized clinical trials: an algorithm for choice. Orphanet J Rare Dis. 2013;8:48. https://doi.org/10.1186/1750-1172-8-48.CrossRefPubMedPubMedCentral

91.

Hilgers RD, et al. Design and analysis of clinical trials for small rare disease populations. J Rare Dis Res Treat. 2016;1(3):53–60.CrossRef

92.

EMA. Certification procedures for micro-, small- and medium-sized enterprises (SMEs). 2007. [Cited: 08 02, 2017.] http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/general/general_content_000300.jsp&mid=WC0b01ac058007f4bd.

93.

Auffray C, et al. Making sense of big data in health research: Towards an European Union action plan. Genome Med. 2016;8:71. https://doi.org/10.1186/s13073-016-0323-y.CrossRefPubMedPubMedCentral

Title: Lessons learned from IDeAl — 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials
Authors: Ralf-Dieter Hilgers
Malgorzata Bogdan
Carl-Fredrik Burman
Holger Dette
Mats Karlsson
Franz König
Christoph Male
France Mentré
Geert Molenberghs
Stephen Senn
Publication date: 01-12-2018
Publisher: BioMed Central
Published in: Orphanet Journal of Rare Diseases / Issue 1/2018
Electronic ISSN: 1750-1172
DOI: https://doi.org/10.1186/s13023-018-0820-8

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Lessons learned from IDeAl — 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials

Abstract

Background

Method

Results

Conclusion

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Method

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2018

A nomenclature and classification for the congenital myasthenic syndromes: preparing for FAIR data in the genomic era

Dentinogenesis imperfecta type II in Swedish children and adolescents

Gynecologic assessment of 19 adult females with cartilage-hair hypoplasia – high rate of HPV positivity

Meta-analysis of the clinical and immunopathological characteristics and treatment outcomes in epidermolysis bullosa acquisita patients

Federating patients identities: the case of rare diseases

Epidemiology of Sanfilippo syndrome: results of a systematic literature review