Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2018

Open Access 01-12-2018 | Research article

Primary versus secondary source of data in observational studies and heterogeneity in meta-analyses of drug effects: a survey of major medical journals

Authors: Guillermo Prada-Ramallal, Fatima Roque, Maria Teresa Herdeiro, Bahi Takkouche, Adolfo Figueiras

Published in: BMC Medical Research Methodology | Issue 1/2018

Login to get access

Abstract

Background

The data from individual observational studies included in meta-analyses of drug effects are collected either from ad hoc methods (i.e. “primary data”) or databases that were established for non-research purposes (i.e. “secondary data”). The use of secondary sources may be prone to measurement bias and confounding due to over-the-counter and out-of-pocket drug consumption, or non-adherence to treatment. In fact, it has been noted that failing to consider the origin of the data as a potential cause of heterogeneity may change the conclusions of a meta-analysis. We aimed to assess to what extent the origin of data is explored as a source of heterogeneity in meta-analyses of observational studies.

Methods

We searched for meta-analyses of drugs effects published between 2012 and 2018 in general and internal medicine journals with an impact factor > 15. We evaluated, when reported, the type of data source (primary vs secondary) used in the individual observational studies included in each meta-analysis, and the exposure- and outcome-related variables included in sensitivity, subgroup or meta-regression analyses.

Results

We found 217 articles, 23 of which fulfilled our eligibility criteria. Eight meta-analyses (8/23, 34.8%) reported the source of data. Three meta-analyses (3/23, 13.0%) included the method of outcome assessment as a variable in the analysis of heterogeneity, and only one compared and discussed the results considering the different sources of data (primary vs secondary).

Conclusions

In meta-analyses of drug effects published in seven high impact general medicine journals, the origin of the data, either primary or secondary, is underexplored as a source of heterogeneity.
Appendix
Available only for authorised users
Literature
1.
Terris DD, Litaker DG, Koroukian SM. Health state information derived from secondary databases is affected by multiple sources of bias. J Clin Epidemiol. 2007;60:734–41.CrossRefPubMedCentralPubMed
2.
Schneeweiss S. Understanding secondary databases: a commentary on “sources of bias for health state characteristics in secondary databases”. J Clin Epidemiol. 2007;60:648–50.CrossRefPubMedCentralPubMed
3.
Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005;58:323–37.CrossRefPubMed
4.
Knottnerus JA, Tugwell P. Requirements for utilizing health care-based data sources for research. J Clin Epidemiol. 2011;64:1051–3.CrossRefPubMed
5.
6.
Blettner M, Schlattmann P. Meta-analysis in epidemiology. In: Ahrens W, Pigeot I, editors. Handbook of epidemiology. Berlin: Springer; 2005. p. 829–59.CrossRef
7.
Higgins JPT. Heterogeneity in meta-analysis should be expected and appropriately quantified. Int J Epidemiol. 2008;37:1158–60.CrossRefPubMed
8.
Prada-Ramallal G, Takkouche B, Figueiras A. Diverging conclusions from the same meta-analysis in drug safety: source of data (primary versus secondary) takes a toll. Drug Saf. 2017;40:351–8.CrossRefPubMed
9.
Madigan D, Ryan PB, Schuemie M, Stang PE, Overhage JM, Hartzema AG, et al. Evaluating the impact of database heterogeneity on observational study results. Am J Epidemiol. 2013;178:645–51.CrossRefPubMedCentralPubMed
10.
11.
Faggion CM Jr, Bakas NP, Wasiak J. A survey of prevalence of narrative and systematic reviews in five major medical journals. BMC Med Res Methodol. 2017;17:176.CrossRefPubMedCentralPubMed
12.
Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors' implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ. 2012;344:e4178.CrossRefPubMedCentralPubMed
13.
Blanc X, Collet TH, Auer R, Fischer R, Locatelli I, Iriarte P, et al. Publication trends of shared decision making in 15 high impact medical journals: a full-text review with bibliometric analysis. BMC Med Inform Decis Mak. 2014;14:71.CrossRefPubMedCentralPubMed
14.
Rehal S, Morris TP, Fielding K, Carpenter JR, Phillips PP. Non-inferiority trials: are they inferior? A systematic review of reporting in major medical journals. BMJ Open. 2016;6:e012594.CrossRefPubMedCentralPubMed
15.
Callaham M, Wears RL, Weber E. Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA. 2002;287:2847–50.CrossRefPubMed
16.
Kim SY, Park JE, Lee YJ, Seo HJ, Sheen SS, Hahn S, et al. Testing a tool for assessing the risk of bias for nonrandomized studies showed moderate reliability and promising validity. J Clin Epidemiol. 2013;66:408–14.CrossRefPubMed
17.
18.
Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of observational studies in epidemiology (MOOSE) group. JAMA. 2000;283:2008–12.CrossRefPubMed
19.
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Epidemiology. 2007;18:800–4.CrossRefPubMed
20.
Weiss J, Freeman M, Low A, Fu R, Kerfoot A, Paynter R, et al. Benefits and harms of intensive blood pressure treatment in adults aged 60 years or older: a systematic review and meta-analysis. Ann Intern Med. 2017;166:419–29.CrossRefPubMed
21.
Bally M, Dendukuri N, Rich B, Nadeau L, Helin-Salmivaara A, Garbe E, et al. Risk of acute myocardial infarction with NSAIDs in real world use: bayesian meta-analysis of individual patient data. BMJ. 2017;357:j1909.CrossRefPubMedCentralPubMed
22.
Sordo L, Barrio G, Bravo MJ, Indave BI, Degenhardt L, Wiessing L, et al. Mortality risk during and after opioid substitution treatment: systematic review and meta-analysis of cohort studies. BMJ. 2017;357:j1550.CrossRefPubMedCentralPubMed
23.
Tariq R, Singh S, Gupta A, Pardi DS, Khanna S. Association of Gastric Acid Suppression with Recurrent Clostridium difficile infection: a systematic review and meta-analysis. JAMA Intern Med. 2017;177:784–91.CrossRefPubMedCentralPubMed
24.
Maruthur NM, Tseng E, Hutfless S, Wilson LM, Suarez-Cuervo C, Berger Z, et al. Diabetes medications as monotherapy or metformin-based combination therapy for type 2 diabetes: a systematic review and meta-analysis. Ann Intern Med. 2016;164:740–51.CrossRefPubMed
25.
Paul S, Saxena A, Terrin N, Viveiros K, Balk EM, Wong JB. Hepatitis B virus reactivation and prophylaxis during solid tumor chemotherapy: a systematic review and meta-analysis. Ann Intern Med. 2016;164:30–40.CrossRefPubMed
26.
Li L, Li S, Deng K, Liu J, Vandvik PO, Zhao P, et al. Dipeptidyl peptidase-4 inhibitors and risk of heart failure in type 2 diabetes: systematic review and meta-analysis of randomised and observational studies. BMJ. 2016;352:i610.CrossRefPubMedCentralPubMed
27.
Molnar AO, Fergusson D, Tsampalieros AK, Bennett A, Fergusson N, Ramsay T, et al. Generic immunosuppression in solid organ transplantation: systematic review and meta-analysis. BMJ. 2015;350:h3163.CrossRefPubMedCentralPubMed
28.
Ziff OJ, Lane DA, Samra M, Griffith M, Kirchhof P, Lip GY, et al. Safety and efficacy of digoxin: systematic review and meta-analysis of observational and controlled trial data. BMJ. 2015;351 h4451. Erratum in: BMJ 2015;351:h4937.
29.
Collaborative Group On Epidemiological Studies Of Ovarian Cancer, Beral V, Gaitskell K, Hermon C, Moser K, Reeves G, et al. Menopausal hormone use and ovarian cancer risk: individual participant meta-analysis of 52 epidemiological studies. Lancet. 2015;385:1835–42.CrossRefPubMedCentral
30.
Bellemain-Appaix A, Kerneis M, O'Connor SA, Silvain J, Cucherat M, Beygui F, et al. Reappraisal of thienopyridine pretreatment in patients with non-ST elevation acute coronary syndrome: a systematic review and meta-analysis. BMJ. 2014;349:g6269.CrossRefPubMedCentralPubMed
31.
Grigoriadis S, Vonderporten EH, Mamisashvili L, Tomlinson G, Dennis CL, Koren G, et al. Prenatal exposure to antidepressants and persistent pulmonary hypertension of the newborn: systematic review and meta-analysis. BMJ. 2014;348:f6932.CrossRefPubMedCentralPubMed
32.
Li L, Shen J, Bala MM, Busse JW, Ebrahim S, Vandvik PO, et al. Incretin treatment and risk of pancreatitis in patients with type 2 diabetes mellitus: systematic review and meta-analysis of randomised and non-randomised studies. BMJ. 2014;348:g2366.CrossRefPubMedCentralPubMed
33.
Kalil AC, Van Schooneveld TC, Fey PD, Rupp ME. Association between vancomycin minimum inhibitory concentration and mortality among patients with Staphylococcus aureus bloodstream infections: a systematic review and meta-analysis. JAMA. 2014;312:1552–64.CrossRefPubMed
34.
Stegeman BH, de Bastos M, Rosendaal FR, van Hylckama Vlieg A, Helmerhorst FM, Stijnen T, et al. Different combined oral contraceptives and the risk of venous thrombosis: systematic review and network meta-analysis. BMJ. 2013;347:f5298.CrossRefPubMedCentralPubMed
35.
Maneiro JR, Salgado E, Gomez-Reino JJ. Immunogenicity of monoclonal antibodies against tumor necrosis factor used in chronic immune-mediated inflammatory conditions: systematic review and meta-analysis. JAMA Intern Med. 2013;173:1416–28.CrossRefPubMed
36.
Hartling L, Abou-Setta AM, Dursun S, Mousavi SS, Pasichnyk D, Newton AS. Antipsychotics in adults with schizophrenia: comparative effectiveness of first-generation versus second-generation medications: a systematic review and meta-analysis. Ann Intern Med. 2012;157:498–511.CrossRefPubMed
37.
Hsu J, Santesso N, Mustafa R, Brozek J, Chen YL, Hopkins JP, et al. Antivirals for treatment of influenza: a systematic review and meta-analysis of observational studies. Ann Intern Med. 2012;156:512–24.CrossRefPubMedPubMedCentral
38.
Caldeira D, Alarcão J, Vaz-Carneiro A, Costa J. Risk of pneumonia associated with use of angiotensin converting enzyme inhibitors and angiotensin receptor blockers: systematic review and meta-analysis. BMJ. 2012;345:e4260.CrossRefPubMedCentralPubMed
39.
MacArthur GJ, Minozzi S, Martin N, Vickerman P, Deren S, Bruneau J, et al. Opiate substitution treatment and HIV transmission in people who inject drugs: systematic review and meta-analysis. BMJ. 2012;345:e5945.CrossRefPubMedCentralPubMed
40.
Mantha S, Karp R, Raghavan V, Terrin N, Bauer KA, Zwicker JI. Assessing the risk of venous thromboembolic events in women taking progestin-only contraception: a meta-analysis. BMJ. 2012;345:e4944.CrossRefPubMedCentralPubMed
41.
Silvain J, Beygui F, Barthélémy O, Pollack C Jr, Cohen M, Zeymer U, et al. Efficacy and safety of enoxaparin versus unfractionated heparin during percutaneous coronary intervention: systematic review and meta-analysis. BMJ. 2012;344:e553.CrossRefPubMedCentralPubMed
42.
McKnight RF, Adida M, Budge K, Stockton S, Goodwin GM, Geddes JR. Lithium toxicity profile: a systematic review and meta-analysis. Lancet. 2012;379:721–8.CrossRef
43.
Egger M, Davey Smith G, Schneider M. Systematic reviews of observational studies. In: Egger M, Davey Smith G, Altman DG, editors. Systematic reviews in health care: meta-analysis in context. 2nd ed. London: BMJ Publishing Group; 2001. p. 211–27.CrossRef
44.
Glasziou PP, Sanders SL. Investigating causes of heterogeneity in systematic reviews. Stat Med. 2002;21:1503–11.CrossRefPubMed
45.
Seeger J, Daniel GW. Commercial Insurance Databases. In: Strom BL, Kimmel SE, Hennessy S, editors. Pharmacoepidemiology. 5th ed. Chichester, John Wiley & Sons; 2012. p. 189–208.CrossRef
46.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700.CrossRefPubMedCentralPubMed
47.
Zorzela L, Loke YK, Ioannidis JP, Golder S, Santaguida P, Altman DG, et al. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;352:i157.CrossRefPubMed
48.
Cohen JM, Wood ME, Hernandez-Diaz S, Nordeng H. Agreement between paternal self-reported medication use and records from a national prescription database. Pharmacoepidemiol Drug Saf. 2018;27:413–21.CrossRefPubMed
49.
Gamble JM, McAlister FA, Johnson JA, Eurich DT. Quantifying the impact of drug exposure misclassification due to restrictive drug coverage in administrative databases: a simulation cohort study. Value Health. 2012;15:191–7.CrossRefPubMed
50.
Upadhya KK, Santelli JS, Raine-Bennett TR, Kottke MJ, Grossman D. Over-the-counter access to oral contraceptives for adolescents. J Adolesc Health. 2017;60:634–40.CrossRefPubMed
51.
Raebel MA, Schmittdiel J, Karter AJ, Konieczny JL, Steiner JF. Standardizing terminology and definitions of medication adherence and persistence in research employing electronic databases. Med Care. 2013;51(Suppl 3):S11–21.CrossRefPubMedCentralPubMed
52.
Wu AC, Butler MG, Li L, Fung V, Kharbanda EO, Larkin EK, et al. Primary adherence to controller medications for asthma is poor. Ann Am Thorac Soc. 2015;12:161–6.CrossRefPubMedCentralPubMed
53.
Fallis BA, Dhalla IA, Klemensberg J, Bell CM. Primary medication non-adherence after discharge from a general internal medicine service. PLoS One. 2013;8:e61735.CrossRefPubMedCentralPubMed
54.
Anderson KL, Dothard EH, Huang KE, Feldman SR. Frequency of primary nonadherence to acne treatment. JAMA Dermatol. 2015;151:623–6.CrossRefPubMed
55.
Fischer MA, Stedman MR, Lii J, Vogeli C, Shrank WH, Brookhart MA, et al. Primary medication non-adherence: analysis of 195,930 electronic prescriptions. J Gen Intern Med. 2010;25:284–90.CrossRefPubMedCentralPubMed
56.
Tamblyn R, Eguale T, Huang A, Winslade N, Doran P. The incidence and determinants of primary nonadherence with prescribed medication in primary care: a cohort study. Ann Intern Med. 2014;160:441–50.CrossRefPubMed
57.
Kolandaivelu K, Leiden BB, O'Gara PT, Bhatt DL. Non-adherence to cardiovascular medications. Eur Heart J. 2014;35:3267–76.CrossRefPubMed
58.
Kirkeby MJ, Hansen CD, Andersen JH. Socio-economic differences in use of prescribed and over-the-counter medicine for pain and psychological problems among Danish adolescents--a longitudinal study. Eur J Pediatr. 2014;173:1147–55.CrossRefPubMed
59.
Mukherjee K, Kamal KM. Sociodemographic determinants of out-of-pocket expenditures for patients using prescription drugs for rheumatoid arthritis. Am Health Drug Benefits. 2017;10:7–15.PubMedPubMedCentral
60.
Karter AJ, Parker MM, Solomon MD, Lyles CR, Adams AS, Moffet HH, et al. Effect of out-of-pocket cost on medication initiation, adherence, and persistence among patients with type 2 diabetes: The Diabetes Study of Northern California (DISTANCE). Health Serv Res. 2018;53:1227–47.CrossRefPubMed
61.
Leong A, Dasgupta K, Bernatsky S, Lacaille D, Avina-Zubieta A, Rahme E. Systematic review and meta-analysis of validation studies on a diabetes case definition from health administrative records. PLoS One. 2013;8:e75256.CrossRefPubMedCentralPubMed
62.
Takahashi Y, Nishida Y, Asai S. Utilization of health care databases for pharmacoepidemiology. Eur J Clin Pharmacol. 2012;68:123–9.CrossRefPubMed
63.
Prada-Ramallal G, Takkouche B, Figueiras A. Summarising the evidence for drug safety: a methodological discussion of different meta-analysis approaches. Drug Saf. 2017;40:547–58.CrossRefPubMed
64.
65.
66.
67.
Gerber S, Tallon D, Trelle S, Schneider M, Jüni P, Egger M. Bibliographic study showed improving methodology of meta-analyses published in leading journals 1993-2002. J Clin Epidemiol. 2007;60:773–80.CrossRefPubMed
68.
Petropoulou M, Nikolakopoulou A, Veroniki AA, Rios P, Vafaei A, Zarin W, et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol. 2017;82:20–8.CrossRefPubMed
69.
Metadata
Title
Primary versus secondary source of data in observational studies and heterogeneity in meta-analyses of drug effects: a survey of major medical journals
Authors
Guillermo Prada-Ramallal
Fatima Roque
Maria Teresa Herdeiro
Bahi Takkouche
Adolfo Figueiras
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2018
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-018-0561-3

Other articles of this Issue 1/2018

BMC Medical Research Methodology 1/2018 Go to the issue

Research article

Examiners’ influence on the measured active and passive extension deficit in finger joints affected by Dupuytren disease

Research article

Application of the dyadic data analysis in behavioral medicine research: marital satisfaction and anxiety in infertile couples

Research article

A transdisciplinary team approach to scoping reviews: the case of pediatric polypharmacy

Research article

Adjustment for unmeasured confounding through informative priors for the confounder-outcome relation

Research article

Assessment of research waste part 1: an exemplar from examining study design, surrogate and clinical endpoints in studies of calcium intake and vitamin D supplementation

Debate

Writing a discussion section: how to integrate substantive and statistical expertise