Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2020 | Research article

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy

Authors: Rachel Phillips, Odile Sauzet, Victoria Cornelius

Published in: BMC Medical Research Methodology | Issue 1/2020

Abstract

Background

Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs).

Methods

A scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded.

Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification.

Results

Forty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods).

Conclusions

This review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed.

Trial registration

PROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442

Available only for authorised users

An adverse event is defined as ‘any untoward medical occurrence that may present during treatment with a pharmaceutical product but which does not necessarily have a causal relationship with this treatment’. An adverse drug reaction (ADR) is defined as ‘a response to a drug which is noxious and unintended …’ where a causal relationship is ‘at least a reasonable possibility’.

Edwards IR, Biriell C. Harmonisation in pharmacovigilance. Drug Saf. 1994;10(2):93–102.PubMedCrossRef

The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH): ICH Harmonised Tripartite Guideline. E2A Clinical Safety Data Management: Definitions and Standards for Expedited Reporting. 1994.

O'Neill RT. Regulatory perspectives on data monitoring. Stat Med. 2002;21(19):2831–42.PubMedCrossRef

Phillips R, Hazell L, Sauzet O, Cornelius V. Analysis and reporting of adverse events in randomised controlled trials: a review. BMJ Open. 2019;9(2):e024537.PubMedPubMedCentralCrossRef

Seltzer JH, Li J, Wang W. Interdisciplinary safety evaluation and quantitative safety monitoring: introduction to a series of papers. Ther Innov Regul Sci. 2019;0(0):2168479018793130.

Ioannidis JA, Evans SW, Gøtzsche PC, et al. Better reporting of harms in randomized trials: an extension of the consort statement. Ann Intern Med. 2004;141(10):781–8.PubMedCrossRef

Zorzela L, Loke YK, Ioannidis JP, Golder S, Santaguida P, Altman DG, Moher D, Vohra S. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;352.

Lineberry N, Berlin JA, Mansi B, Glasser S, Berkwits M, Klem C, Bhattacharya A, Citrome L, Enck R, Fletcher J, et al. Recommendations to improve adverse event reporting in clinical trial publications: A joint pharmaceutical industry/journal editor perspective. BMJ. 2016;355(linn033422):i5078.CrossRefPubMed

Crowe BJ, Xia HA, Berlin JA, Watson DJ, Shi H, Lin SL, Kuebler J, Schriver RC, Santanello NC, Rochester G, et al. Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team. Clin Trials. 2009;6(5):430–40.PubMedCrossRef

10.

Communication from the Commission. Detailed guidance on the collection, verification and presentation of adverse event/reaction reports arising from clinical trials on medicinal products for human use (‘CT-3’). Off J Eur Union. 2011; C 172/1.

11.

Food and Drug Administration. Guidance for Industry and Investigators. Safety Reporting Requirements for INDs and BA/BE studies: Edited by Administration USDoHaHSFaD, (CDER) CfDEaR, (CBER); CfBEaR; 2012. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/safety-reporting-requirements-indsinvestigational-new-drug-applications-and-babe.

12.

Food and Drug Administration. Safety assessment for IND safety reporting guidance for industry: Food and Drug Administration; 2015. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/safety-assessment-ind-safety-reporting-guidance-industry.

13.

Singh S, Loke YK. Drug safety assessment in clinical trials: methodological challenges and opportunities. 2012;13(138). https://doi.org/10.1186/1745-6215-13-138.

14.

Zink RC, Marchenko O, Sanchez-Kam M, Ma H, Jiang Q. Sources of safety data and statistical strategies for design and analysis:clinical trials. Ther Innov Regul Sci. 2018;52(2):141–58.PubMedCrossRef

15.

Siddiqui O. Statistical methods to analyze adverse events data of randomized clinical trials. J Biopharm Stat. 2009;19(5):889–99.PubMedCrossRef

16.

Ma H, Ke C, Jiang Q, Snapinn S. Statistical considerations on the evaluation of imbalances of adverse events in randomized clinical trials. Ther Innov Regul Sci. 2015;49(6):957–65.PubMedCrossRef

17.

Food and Drug Administration. Attachment B: Clinical Safety Review of an NDA or BLA of the Good Review Practice. Clinical Review Template (MAPP 6010.3 Rev. 1). Silver Spring: Food and Drug Administration; 2010.

18.

Unkel S, Amiri M, Benda N, Beyersmann J, Knoerzer D, Kupas K, Langer F, Leverkus F, Loos A, Ose C, et al. On estimands and the analysis of adverse events in the presence of varying follow-up times within the benefit assessment of therapies. Pharm Stat. 2019;18(2):166–83.PubMedCrossRef

19.

Favier R, Crépin S. The reporting of harms in publications on randomized controlled trials funded by the “Programme Hospitalier de Recherche Clinique,” a French academic funding scheme. Clin Trials. 2018;0(0):1740774518760565.

20.

Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.CrossRef

21.

O'Brien PC, Fleming TR. A multiple testing procedure for clinical trials. Biometrics. 1979;35(3):549–56.PubMedCrossRef

22.

DeMets DL, Lan G. The alpha spending function approach to interim data analyses. In: Thall PF, editor. Recent Advances in Clinical Trial Design and Analysis. Boston: Springer US; 1995. p. 1–27.

23.

Tricco AC, Lillie E, Zarin W, et al. Prisma extension for scoping reviews (prisma-scr): checklist and explanation. Ann Intern Med. 2018.

24.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.PubMedPubMedCentralCrossRef

25.

Amit O, Heiberger RM, Lane PW. Graphical approaches to the analysis of safety data from clinical trials. Pharm Stat. 2008;7(1):20–35.PubMedCrossRef

26.

Chuang-Stein C, Le V, Chen W. Recent advancements in the analysis and presentation of safety data. Drug Information Journal. 2001;35(2):377–97.CrossRef

27.

Chuang-Stein C, Xia HA. The practice of pre-marketing safety assessment in drug development. J Biopharm Stat. 2013;23(1):3–25.PubMedCrossRef

28.

Karpefors M, Weatherall J. The tendril plot—a novel visual summary of the incidence, significance and temporal aspects of adverse events in clinical trials. J Am Med Inform Assoc. 2018;25(8):1069–73.PubMedCrossRefPubMedCentral

29.

Southworth H. Detecting outliers in multivariate laboratory data. J Biopharm Stat. 2008;18(6):1178–83.PubMedCrossRef

30.

Trost DC, Freston JW. Vector analysis to detect hepatotoxicity signals in drug development. Ther Innov Regul Science. 2008;42(1):27–34.

31.

Zink RC, Wolfinger RD, Mann G. Summarizing the incidence of adverse events using volcano plots and time intervals. Clin Trials. 2013;10(3):398–406.PubMedCrossRef

32.

Bolland K, Whitehead J. Formal approaches to safety monitoring of clinical trials in life-threatening conditions. Stat Med. 2000;19(21):2899–917.PubMedCrossRef

33.

Fleishman AN, Parker RA. Stopping guidelines for harm in a study designed to establish the safety of a marketed drug. J Biopharm Stat. 2012;22(2):338–50.PubMedCrossRef

34.

Lieu TA, Kulldorff M, Davis RL, Lewis EM, Weintraub E, Yih K, Yin R, Brown JS, Platt R. Real-time vaccine safety surveillance for the early detection of adverse events. Med Care. 2007;45(10 SUPPL. 2):S89–95.PubMedCrossRef

35.

Liu JP. Rethinking statistical approaches to evaluating drug safety. Yonsei Med J. 2007;48(6):895–900.PubMedPubMedCentralCrossRef

36.

Shih MC, Lai TL, Heyse JF, Chen J. Sequential generalized likelihood ratio tests for vaccine safety evaluation. Stat Med. 2010;29(26):2698–708.PubMedPubMedCentralCrossRef

37.

Agresti AaK B. Multivariate tests comparing binomial probabilities, with application to safety studies for drugs. Appl Stat. 2005;54(4):691–706.

38.

Bristol DR, Patel HI. A Markovian model for comparing incidences of side effects. Stat Med. 1990;9(7):803–9.PubMedCrossRef

39.

Chuang-Stein C, Mohberg NR, Musselman DM. Organization and analysis of safety data using a multivariate approach. Stat Med. 1992;11(8):1075–89.PubMedCrossRef

40.

Huang L, Zalkikar J, Tiwari R. Likelihood ratio based tests for longitudinal drug safety data. Stat Med. 2014;33(14):2408–24.PubMedCrossRef

41.

Mehrotra DV, Adewale AJ. Flagging clinical adverse experiences: reducing false discoveries without materially compromising power for detecting true signals. Stat Med. 2012;31(18):1918–30.PubMedCrossRef

42.

Mehrotra DV, Heyse JF. Use of the false discovery rate for evaluating clinical safety data. Stat Methods Med Res. 2004;13(3):227–38.PubMedCrossRef

43.

Allignol A, Beyersmann J, Schmoor C. Statistical issues in the analysis of adverse events in time-to-event data. Pharm Stat. 2016;15(4):297–305.PubMedCrossRef

44.

Borkowf CB. Constructing binomial confidence intervals with near nominal coverage by adding a single imaginary failure or success. Stat Med. 2006;25(21):3679–95.PubMedCrossRef

45.

Evans SJW, Nitsch D. Statistics: Analysis and Presentation of Safety Data. In: Talbot J, Aronson JK, editors. Stephens' Detection and Evaluation of Adverse Drug Reactions: Principles and Practice. 6th ed: Wiley; 2012. p. 349–88..

46.

Gong Q, Tong B, Strasak A, Fang L. Analysis of safety data in clinical trials using a recurrent event approach. Pharm Stat. 2014;13(2):136–44.PubMedCrossRef

47.

Hengelbrock J, Gillhaus J, Kloss S, Leverkus F. Safety data from randomized controlled trials: applying models for recurrent events. Pharm Stat. 2016;15(4):315–23.PubMedCrossRef

48.

Lancar R, Kramar A, Haie-Meder C. Non-parametric methods for analysing recurrent complications of varying severity. Stat Med. 1995;14(24):2701–12.PubMedCrossRef

49.

Leon-Novelo LG, Zhou X, Bekele BN, Muller P. Assessing toxicities in a clinical trial: Bayesian inference for ordinal data nested within categories. Biometrics. 2010;66(3):966–74.PubMedPubMedCentralCrossRef

50.

Liu GF, Wang J, Liu K, Snavely DB. Confidence intervals for an exposure adjusted incidence rate difference with applications to clinical trials. Stat Med. 2006;25(8):1275–86.PubMedCrossRef

51.

Nishikawa M, Tango T, Ogawa M. Non-parametric inference of adverse events under informative censoring. Stat Med. 2006;25(23):3981–4003.PubMedCrossRef

52.

O'Gorman TW, Woolson RF, Jones MP. A comparison of two methods of estimating a common risk difference in a stratified analysis of a multicenter clinical trial. Control Clin Trials. 1994;15(2):135–53.PubMedCrossRef

53.

Rosenkranz G. Analysis of adverse events in the presence of discontinuations. Drug Inform J. 2006;40(1):79–87.CrossRef

54.

Sogliero-Gilbert G, Ting N, Zubkoff L. A statistical comparison of drug safety in controlled clinical trials: The Genie score as an objective measure of lab abnormalities. Ther Innov Regul Sci. 1991;25(1):81–96.

55.

Wang J, Quartey G. Nonparametric estimation for cumulative duration of adverse events. Biom J. 2012;54(1):61–74.PubMedCrossRef

56.

Wang J, Quartey G. A semi-parametric approach to analysis of event duration and prevalence. Comput Stat Data Anal. 2013;67:248–57.CrossRef

57.

Berry DA. Monitoring accumulating data in a clinical trial. Biometrics. 1989;45(4):1197–211.PubMedCrossRef

58.

French JL, Thomas N, Wang C. Using historical data with Bayesian methods in early clinical trial monitoring. Stat Biopharm Res. 2012;4(4):384–94.CrossRef

59.

Yao B, Zhu L, Jiang Q, Xia HA. Safety monitoring in clinical trials. Pharmaceutics. 2013;5(1):94–106.PubMedPubMedCentralCrossRef

60.

Zhu L, Yao B, Xia HA, Jiang Q. Statistical monitoring of safety in clinical trials. Stat Biopharm Res. 2016;8(1):88–105.CrossRef

61.

Berry SM, Berry DA. Accounting for multiplicities in assessing drug safety: a three-level hierarchical mixture model. Biometrics. 2004;60(2):418–26.PubMedCrossRef

62.

Chen W, Zhao N, Qin G, Chen J. A bayesian group sequential approach to safety signal detection. J Biopharm Stat. 2013;23(1):213–30.PubMedCrossRef

63.

Gould AL. Detecting potential safety issues in clinical trials by Bayesian screening. Biom J. 2008;50(5):837–51.PubMedCrossRef

64.

Gould AL. Detecting potential safety issues in large clinical or observational trials by bayesian screening when event counts arise from poisson distributions. J Biopharm Stat. 2013;23(4):829–47.PubMedCrossRef

65.

McEvoy BW, Nandy RR, Tiwari RC. Bayesian approach for clinical trial safety data using an Ising prior. Biometrics. 2013;69(3):661–72.PubMedCrossRef

66.

Xia HA, Ma H, Carlin BP. Bayesian hierarchical modeling for detecting safety signals in clinical trials. J Biopharm Stat. 2011;21(5):1006–29.PubMedCrossRef

67.

Whone A, Luz M, Boca M, Woolley M, Mooney L, Dharia S, Broadfoot J, Cronin D, Schroers C, Barua NU, et al. Randomized trial of intermittent intraputamenal glial cell line-derived neurotrophic factor in Parkinson’s disease. Brain. 2019;142(3):512–25.PubMedPubMedCentralCrossRef

68.

Wang W, Whalen E, Munsaka M, Li J, Fries M, Kracht K, Sanchez-Kam M, Singh K, Zhou K. On quantitative methods for clinical safety monitoring in drug development. Stat Biopharm Res. 2018;10(2):85–97.CrossRef

69.

Phillips R, Cornelius V. Understanding current practice, identifying barriers and exploring priorities for adverse event analysis in randomised controlled trials: an online, cross-sectional survey of statisticians from academia and industry. BMJ Open. 2020;10(6):e036875.PubMedPubMedCentralCrossRef

70.

Cornelius VR, Sauzet O, Williams JE, Ayis S, Farquhar-Smith P, Ross JR, Branford RA, Peacock JL. Adverse event reporting in randomised controlled trials of neuropathic pain: considerations for future practice. PAIN. 2013;154(2):213–20.PubMedCrossRef

71.

Hum SW, Golder S, Shaikh N. Inadequate harms reporting in randomized control trials of antibiotics for pediatric acute otitis media: a systematic review. Drug Saf. 2018;41(10):933–8.PubMedCrossRef

72.

Patson N, Mukaka M, Otwombe KN, Kazembe L, Mathanga DP, Mwapasa V, Kabaghe AN, Eijkemans MJC, Laufer MK, Chirwa T. Systematic review of statistical methods for safety data in malaria chemoprevention in pregnancy trials. Malar J. 2020;19(1):119.PubMedPubMedCentralCrossRef

73.

Food and Drug Administration. Guidance for industry e9 statistical principles for clinical trials: Food and Drug Administration. Food and Drug Administration; 1998. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/e9-statistical-principlesclinical-trials.

74.

Harrell F. Continuous Learning from Data: No Multiplicities from Computing and Using Bayesian Posterior Probabilities as Often as Desired. In: Statistical Thinking, vol. 2019; 2018. https://www.fharrell.com/post/bayes-seq/.

75.

Spiegelhalter DJ, Abrams KR, Myles JP. An Overview of the Bayesian Approach. In: Bayesian Approaches to Clinical Trials and Health-Care Evaluation: John Wiley & Sons, Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK: Wiley; 2004.

76.

Gelman A, Hill J, Yajima M. Why We (Usually) Don't Have to Worry About Multiple Comparisons. J Res Educ Effect. 2012;5(2):189–211.

77.

Spiegelhalter DJ. Incorporating Bayesian ideas into health-care evaluation. Stat Sci. 2004;19(1):156–74.CrossRef

78.

Ball G, Piller LB. Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information part 2: statistical considerations. Contemp Clin Trials. 2011;32:S5–7.PubMedCrossRef

79.

Gould AL, Wang WB. Monitoring potential adverse event rate differences using data from blinded trials: the canary in the coal mine. Stat Med. 2017;36(1):92–104.PubMedCrossRef

Title: Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy
Authors: Rachel Phillips
Odile Sauzet
Victoria Cornelius
Publication date: 01-12-2020
Publisher: BioMed Central
Published in: BMC Medical Research Methodology / Issue 1/2020
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-020-01167-9

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy

Abstract

Background

Methods

Results

Conclusions

Trial registration

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Trial registration

Please log in to get access to this content

Other articles of this Issue 1/2020

What feedback do reviewers give when reviewing qualitative manuscripts? A focused mapping review and synthesis

Don’t dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning

Linkage of the CHHiP randomised controlled trial with primary care data: a study investigating ways of supplementing cancer trials and improving evidence-based practice

Group penalized generalized estimating equation for correlated event-related potentials and biomarker selection

Overlapping-sample Mendelian randomisation with multiple exposures: a Bayesian approach

Compbdt: an R program to compare two binary diagnostic tests subject to a paired design