Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Drug Safety
Published in: Drug Safety 7/2017

01-07-2017 | Leading Article

Summarising the Evidence for Drug Safety: A Methodological Discussion of Different Meta-Analysis Approaches

Authors: Guillermo Prada-Ramallal, Bahi Takkouche, Adolfo Figueiras

Published in: Drug Safety | Issue 7/2017

Login to get access

Abstract

Evidence on drug safety obtained from randomised clinical trials is very limited due to, among other reasons, their relatively small sample size. Hence, combining the results of available studies can prove particularly useful. This paper reviews the different data sources for summarising drug safety outcomes, according to study design, publication of data, and origin of the information. It then discusses the various types of overviews that can be used in the study of treatment harms, focusing on meta-analyses of aggregate data and meta-analyses of individual patient data, with their advantages and drawbacks, such as publication bias and heterogeneity. Although the different approaches available for combining the results are of great utility in assessing treatment harms, none of them is free from limitations. Therefore, it might be appropriate to perform an analysis of sensitivity to assess whether the results are sensitive to the technique that has been used.
Literature
1.
Bouvy JC, De Bruin ML, Koopmanschap MA. Epidemiology of adverse drug reactions in Europe: a review of recent observational studies. Drug Saf. 2015;38:437–53.CrossRefPubMedPubMedCentral
2.
Clark DW, Coulter DM, Besag FM. Randomized controlled trials and assessment of drug safety. Drug Saf. 2008;31:1057–61.CrossRefPubMed
3.
Reynolds RF, Lem JA, Gatto NM, et al. Is the large simple trial design used for comparative, post-approval safety research? A review of a clinical trials registry and the published literature. Drug Saf. 2011;34:799–820.CrossRefPubMed
4.
5.
CIOMS Working Group X. Evidence synthesis and meta-analysis: report of CIOMS Working Group X. Geneva: Council for International Organizations of Medical Sciences (CIOMS); 2016.
6.
Ioannidis JP, Lau J. Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA. 2001;285:437–43.CrossRefPubMed
7.
Chou R, Helfand M. Challenges in systematic reviews that assess treatment harms. Ann Intern Med. 2005;142:1090–9.CrossRefPubMed
8.
Nuesch E, Trelle S, Reichenbach S, et al. The effects of excluding patients from the analysis in randomized controlled trials: meta-epidemiological study. BMJ. 2009;339:b3244.CrossRefPubMedPubMedCentral
9.
Hammad TA, Pinheiro SP, Neyarapally GA. Secondary use of randomized controlled trials to evaluate drug safety: a review of methodological considerations. Clin Trials. 2011;8:559–70.CrossRefPubMed
10.
Moride Y, Abenhaim L. Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research. J Clin Epidemiol. 1994;47:731–7.CrossRefPubMed
11.
12.
Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet. 2004;363:1728–31.CrossRefPubMed
13.
Vandenbroucke JP. The HRT controversy: observational studies and RCTs fall in line. Lancet. 2009;373:1233–5.CrossRefPubMed
14.
Papanikolaou PN, Christidi GD, Ioannidis JP. Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ. 2006;174:635–41.CrossRefPubMedPubMedCentral
15.
16.
Reeves BC, Deeks JJ, Higgins JPT, et el. Chapter 13: Including non-randomized studies. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011). The Cochrane Collaboration; 2011. Available from http://​www.​cochrane-handbook.​org.
17.
Golder S, Loke YK, Bland M. Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med. 2011;8:e1001026.CrossRefPubMedPubMedCentral
18.
Bailey C, Peddie D, Wickham ME, et al. Adverse drug event reporting systems-a systematic review. Br J Clin Pharmacol. 2016;82:17–29.CrossRefPubMed
19.
Saini P, Loke YK, Gamble C, et al. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ. 2014;349:g6501.CrossRefPubMedPubMedCentral
20.
Rücker G, Schwarzer G, Carpenter J, et al. Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta-analysis with zero cells. Stat Med. 2009;28:721–38.CrossRefPubMed
21.
Song F, Eastwood AJ, Gilbody S, et al. Publication and related biases. Health Technol Assess. 2000;4:1–115.
22.
Morrison A, Moulton K, Clark M, Polisena, et al. English-language restriction when conducting systematic review-based meta-analyses: systematic review of published studies. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2009.
23.
24.
25.
Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007:MR000005.
26.
Kotecha D, Manzano L, Krum H, et al. Beta-Blockers in Heart Failure Collaborative Group. Effect of age and sex on efficacy and tolerability of β blockers in patients with heart failure with reduced ejection fraction: individual patient data meta-analysis. BMJ. 2016;353:i1855.CrossRefPubMedPubMedCentral
27.
Reichenpfader U, Gartlehner G, Morgan LC, et al. Sexual dysfunction associated with second-generation antidepressants in patients with major depressive disorder: results from a systematic review with network meta-analysis. Drug Saf. 2014;37:19–31.CrossRefPubMed
28.
Veroniki AA, Straus SE, Ashoor HM, et al. Comparative safety and effectiveness of cognitive enhancers for Alzheimer’s dementia: protocol for a systematic review and individual patient data network meta-analysis. BMJ Open. 2016;6:e010251.CrossRefPubMedPubMedCentral
29.
Golder S, Loke YK, Wright K, Norman G. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Med. 2016;13:e1002127.CrossRefPubMedPubMedCentral
30.
FDA Amendments Act. Public Law 110-85. 2007. Ref Type: Statute.
31.
Law MR, Kawasumi Y, Morgan SG. Despite law, fewer than one in eight completed studies of drugs and biologics are reported on time on ClinicalTrials.gov. Health Aff (Millwood). 2011;30:2338–45.CrossRef
32.
33.
Aalaei-Andabili SH, Alavian SM. Important steps for a reliable meta-analysis. Lancet Infect Dis. 2012;12:663.CrossRefPubMed
34.
Sterne JAC, Egger M, Moher D. Chapter 10: Addressing reporting biases. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011). The Cochrane Collaboration; 2011. Available from http://​www.​cochrane-handbook.​org.
35.
Terrin N, Schmid CH, Lau J. In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J Clin Epidemiol. 2005;58:894–901.CrossRefPubMed
36.
37.
Loke YK, Mattishent K. If nothing happens, is everything all right? Distinguishing genuine reassurance from a false sense of security. CMAJ. 2015;187:15–6.CrossRefPubMedPubMedCentral
38.
Chou R, Aronson N, Atkins D, et al. AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol. 2010;63:502–12.CrossRefPubMed
39.
40.
Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005;58:323–37.CrossRefPubMed
41.
Figueiras A, Ferreira MT, Gestal JJ. Farmacovigilancia. In: Fernández-Crehuet J, Gestal JJ, Delgado M, et al., editors. Piédrola Gil. Medicina Preventiva y Salud Pública. 12ª Edición. Barcelona: Elsevier Masson; 2015. pp. 1093–104.
42.
Van Walraven C, Austin P. Administrative database research has unique characteristics that can risk biased results. J Clin Epidemiol. 2012;65:126–31.CrossRefPubMed
43.
Takahashi Y, Nishida Y, Asai S. Utilization of health care databases for pharmacoepidemiology. Eur J Clin Pharmacol. 2012;68:123–9.CrossRefPubMed
44.
Hennessy S. Use of health care databases in pharmacoepidemiology. Basic Clin Pharmacol Toxicol. 2006;98:311–3.CrossRefPubMed
45.
Blettner M, Sauerbrei W, Schlehofer B, et al. Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol. 1999;28:1–9.CrossRefPubMed
46.
Blettner M, Schlattmann P. Meta-analysis in epidemiology. In: Ahrens W, Pigeot I, editors. Handbook of epidemiology. Berlin: Springer; 2005. pp. 829–59.CrossRef
47.
Nordmann AJ, Kasenda B, Briel M. Meta-analyses: what they can and cannot do. Swiss Med Wkly. 2012;142:w13518.PubMed
48.
Colquhoun HL, Levac D, O’Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67:1291–4.CrossRefPubMed
49.
Riley RD, Simmonds MC, Look MP. Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. J Clin Epidemiol. 2007;60:431–9.PubMed
50.
Friedenreich CM. Methods for pooled analyses of epidemiologic studies. Epidemiology. 1993;4:295–302.CrossRefPubMed
51.
52.
Song F, Loke YK, Walsh T, et al. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ. 2009;338:b1147.CrossRefPubMedPubMedCentral
53.
Cameron C, Fireman B, Hutton B, et al. Network meta-analysis incorporating randomized controlled trials and non-randomized comparative cohort studies for assessing the safety and effectiveness of medical treatments: challenges and opportunities. Syst Rev. 2015;4:147.CrossRefPubMedPubMedCentral
54.
Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56:455–63.CrossRefPubMed
55.
56.
Tobías A, Saez M, Kogevinas M. Meta-analysis of results and individual patient data in epidemiologal studies. J Mod Appl Stat Methods. 2004;1:176–85.CrossRef
57.
Oakes M. On meta-analysis. In: Statistical inference. Chestnut Hill: Epidemiology Resources Inc; 1990. pp. 157–63.
58.
59.
Feinstein AR. Meta-analysis: statistical alchemy for the 21st century. J Clin Epidemiol. 1995;48:71–9.CrossRefPubMed
60.
61.
Ioannidis JP, Lau J. Pooling research results: benefits and limitations of meta-analysis. Jt Comm J Qual Improv. 1999;25:462–9.PubMed
62.
63.
Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med. 2007;356:2457–71.CrossRefPubMed
64.
Hu M, Cappelleri JC, Lan KK. Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes. Clin Trials. 2007;4:329–40.CrossRefPubMed
65.
66.
Higgins JPT. Heterogeneity in meta-analysis should be expected and appropriately quantified. Int J Epidemiol. 2008;37:1158–60.CrossRefPubMed
67.
Berlin JA. Benefits of heterogeneity in meta-analysis of data from epidemiologic studies. Am J Epidemiol. 1995;142:383–7.CrossRefPubMed
68.
Takkouche B, Khudyakov P, Costa-Bouzas J, et al. Confidence intervals for heterogeneity measures in meta-analysis. Am J Epidemiol. 2013;178:993–1004.CrossRefPubMedPubMedCentral
69.
70.
Berlin JA, Santanna J, Schmid CH, et al. Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Stat Med. 2002;21:371–87.CrossRefPubMed
71.
Riley RD, Lambert PC, Staessen JA, et al. Meta-analysis of continuous outcomes combining individual patient data and aggregate data. Stat Med. 2008;27:1870–93.CrossRefPubMed
72.
Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;340:c221.CrossRefPubMed
73.
Simmonds MC, Higgins JP, Stewart LA, et al. Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Clin Trials. 2005;2:209–17.CrossRefPubMed
74.
Stewart LA, Parmar MK. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet. 1993;341:418–22.CrossRefPubMed
75.
Stewart LA, Clarke MJ. Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Stat Med. 1995;14:2057–79.CrossRefPubMed
76.
Stewart LA, Clarke M, Rovers M, et al. PRISMA-IPD Development Group. Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data: the PRISMA-IPD Statement. JAMA. 2015;313:1657–65.CrossRefPubMed
77.
Stewart LA, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof. 2002;25:76–97.CrossRefPubMed
78.
Burgess S, White IR, Resche-Rigon M, et al. Combining multiple imputation and meta-analysis with individual participant data. Stat Med. 2013;32:4499–514.CrossRefPubMedPubMedCentral
79.
Lambert PC, Sutton AJ, Abrams KR, et al. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol. 2002;55:86–94.CrossRefPubMed
80.
Ahmed I, Sutton AJ, Riley RD. Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey. BMJ. 2012;344:d7762.CrossRefPubMed
81.
Abo-Zaid G, Guo B, Deeks JJ, et al. Individual participant data meta-analyses should not ignore clustering. J Clin Epidemiol. 2013;66(865–873):e4.
82.
Debray TP, Moons KG, van Valkenhoef G, et al. Get real in individual participant data (IPD) meta-analysis: a review of the methodology. Res Synth Methods. 2015;6:293–309.CrossRefPubMedPubMedCentral
83.
Wolfson M, Wallace SE, Masca N, et al. DataSHIELD: resolving a conflict in contemporary bioscience–performing a pooled analysis of individual-level data without sharing the data. Int J Epidemiol. 2010;39:1372–82.CrossRefPubMedPubMedCentral
84.
Zorzela L, Loke YK, Ioannidis JP, et al. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;352:i157.CrossRefPubMed
85.
Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005;293:2362–6.CrossRefPubMed
Metadata
Title
Summarising the Evidence for Drug Safety: A Methodological Discussion of Different Meta-Analysis Approaches
Authors
Guillermo Prada-Ramallal
Bahi Takkouche
Adolfo Figueiras
Publication date
01-07-2017
Publisher
Springer International Publishing
Published in
Drug Safety / Issue 7/2017
Print ISSN: 0114-5916
Electronic ISSN: 1179-1942
DOI
https://doi.org/10.1007/s40264-017-0518-1

Other articles of this Issue 7/2017

Drug Safety 7/2017 Go to the issue

Original Research Article

Drospirenone-Containing Oral Contraceptive Pills and the Risk of Venous Thromboembolism: An Assessment of Risk in First-Time Users and Restarters

Commentary

The Power of the Case Narrative - Can it be Brought to Bear on Duplicate Detection?

Original Research Article

The Quality of Clinical Information in Adverse Drug Reaction Reports by Patients and Healthcare Professionals: A Retrospective Comparative Analysis

Original Research Article

Patient Reporting in the EU: Analysis of EudraVigilance Data

Original Research Article

Using Probabilistic Record Linkage of Structured and Unstructured Data to Identify Duplicate Cases in Spontaneous Adverse Event Reporting Systems

Original Research Article

Impact of Safety-Related Regulations on Codeine Use in Children: A Quasi-Experimental Study Using Taiwan’s National Health Insurance Research Database