Top

Published in:

Open Access 01-12-2015 | Protocol

Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol for a systematic review

Authors: Evan Mayo-Wilson, Susan Hutfless, Tianjing Li, Gillian Gresham, Nicole Fusco, Jeffrey Ehmsen, James Heyward, Swaroop Vedula, Diana Lock, Jennifer Haythornthwaite, Jennifer L. Payne, Theresa Cowley, Elizabeth Tolbert, Lori Rosman, Claire Twose, Elizabeth A. Stuart, Hwanhee Hong, Peter Doshi, Catalina Suarez-Cuervo, Sonal Singh, Kay Dickersin

Published in: Systematic Reviews | Issue 1/2015

Abstract

Background

Systematic reviews should provide trustworthy guidance to decision-makers, but their credibility is challenged by the selective reporting of trial results and outcomes. Some trials are not published, and even among clinical trials that are published partially (e.g., as conference abstracts), many are never published in full. Although there are many potential sources of published and unpublished data for systematic reviews, there are no established methods for choosing among multiple reports or data sources about the same trial.

Methods

We will conduct systematic reviews of the effectiveness and safety of two interventions following the Institute of Medicine (IOM) guidelines: (1) gabapentin for neuropathic pain and (2) quetiapine for bipolar depression. For the review of gabapentin, we will include adult participants with neuropathic pain who do not require ventilator support. For the review of quetiapine, we will include adult participants with acute bipolar depression (excluding mixed or rapid cycling episodes). We will compare these drugs (used alone or in combination with other interventions) with placebo or with the same intervention alone; direct comparisons with other medications will be excluded. For each review, we will conduct highly sensitive electronic searches, and the results of the searches will be assessed by two independent reviewers. Outcomes, study characteristics, and risk of bias ratings will be extracted from multiple reports by two individuals working independently, stored in a publicly available database (Systematic Review Data Repository) and analyzed using commonly available statistical software. In each review, we will conduct a series of meta-analyses using data from different sources to determine how the results are affected by the inclusion of data from multiple published sources (e.g., journal articles and conference abstracts) as well as unpublished aggregate data (e.g., “clinical study reports”) and individual participant data (IPD). We will identify patient-centered outcomes in each report and identify differences in the reporting of these outcomes across sources.

Systematic review registration

CRD42015014037, CRD42015014038

Available only for authorised users

Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007(2):MR000005. doi:10.1002/14651858.MR000005.pub3.

Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):1–193. doi:10.3310/hta14080. iii, ix-xi.CrossRef

Goodman S, Dickersin K. Metabias: a challenge for comparative effectiveness research. Ann Intern Med. 2011;155(1):61–2. doi:10.7326/0003-4819-155-1-201107050-00010.CrossRefPubMed

Vedula SS, Li T, Dickersin K. Differences in reporting of analyses in internal company documents versus published trial reports: comparisons in industry-sponsored trials in off-label uses of gabapentin. PLoS Med. 2013;10(1):e1001378. doi:10.1371/journal.pmed.1001378.CrossRefPubMedPubMedCentral

Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66. doi:10.1016/S0140-6736(13)62296-5.CrossRefPubMedPubMedCentral

Lefebvre C, Glanville J, Wieland LS, Coles B, Weightman AL. Methodological developments in searching for studies for systematic reviews: past, present and future? Syst Rev. 2013;2:78. doi:10.1186/2046-4053-2-78.CrossRefPubMedPubMedCentral

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7(1):1–76.PubMed

Scherer RW, Huynh L, Ervin AM, Taylor J, Dickersin K. ClinicalTrials.gov registration can supplement information in abstracts for systematic reviews: a comparison study. BMC Med Res Methodol. 2013;13:79. doi:10.1186/1471-2288-13-79.CrossRefPubMedPubMedCentral

Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60. doi:10.1056/NEJMsa065779.CrossRefPubMed

10.

Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev. 2014;4:CD008965. doi:10.1002/14651858.CD008965.pub4.

11.

Fu R, Selph S, McDonagh M, Peterson K, Tiwari A, Chou R, et al. Effectiveness and harms of recombinant human bone morphogenetic protein-2 in spine fusion: a systematic review and meta-analysis. Ann Intern Med. 2013;158(12):890–902. doi:10.7326/0003-4819-158-12-201306180-00006.CrossRefPubMed

12.

Simmonds MC, Brown JV, Heirs MK, Higgins JP, Mannion RJ, Rodgers MA, et al. Safety and effectiveness of recombinant human bone morphogenetic protein-2 for spinal fusion: a meta-analysis of individual-participant data. Ann Intern Med. 2013;158(12):877–89. doi:10.7326/0003-4819-158-12-201306180-00005.CrossRefPubMed

13.

Stewart LA, Parmar MK. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet. 1993;341(8842):418–22.CrossRefPubMed

14.

Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;340:c221. doi:10.1136/bmj.c221.CrossRefPubMed

15.

Rodgers MA, Brown JV, Heirs MK, Higgins JP, Mannion RJ, Simmonds MC, et al. Reporting of industry funded study outcome data: comparison of confidential and published data on the safety and effectiveness of rhBMP-2 for spinal fusion. BMJ. 2013;346:f3981. doi:10.1136/bmj.f3981.CrossRefPubMedPubMedCentral

16.

Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington, DC: The National Academies Press; 2011.

17.

Gabriel SE, Normand SL. Getting the methods right--the foundation of patient-centered outcomes research. N Engl J Med. 2012;367(9):787–90. doi:10.1056/NEJMp1207437.CrossRefPubMed

18.

Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361(20):1963–71. doi:10.1056/NEJMsa0906126.CrossRefPubMed

19.

Turner EH, Knoepflmacher D, Shapley L. Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration database. PLoS Med. 2012;9(3):e1001189. doi:10.1371/journal.pmed.1001189.CrossRefPubMedPubMedCentral

20.

Elbourne DR, Altman DG, Higgins JP, Curtin F, Worthington HV, Vail A. Meta-analyses involving cross-over trials: methodological issues. Int J Epidemiol. 2002;31(1):140–9.CrossRefPubMed

21.

Li T, Yu T, Hawkins BS, Dickersin K. Design, analysis, and reporting of crossover trials for inclusion in a meta-analysis. PLoS One. 2015;10(8):e0133023. doi:10.1371/journal.pone.0133023.CrossRefPubMedPubMedCentral

22.

Turk DC, Dworkin RH, Allen RR, Bellamy N, Brandenburg N, Carr DB, et al. Core outcome domains for chronic pain clinical trials: IMMPACT recommendations. Pain. 2003;106(3):337–45.CrossRefPubMed

23.

Moore RA, Wiffen PJ, Derry S, Toelle T, Rice AS. Gabapentin for chronic neuropathic pain and fibromyalgia in adults. Cochrane Database Syst Rev. 2014;4:CD007938. doi:10.1002/14651858.CD007938.pub3.

24.

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. doi:10.1371/journal.pmed.1000097.CrossRefPubMedPubMedCentral

25.

Glanville J, Duffy S, McCool R, Varley D. Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? J Med Lib Assoc. 2014;102(3):177–83.CrossRef

26.

Turner EH. How to access and process FDA drug approval packages for use in research. BMJ. 2013;347:f5992. doi:10.1136/bmj.f5992.CrossRefPubMed

27.

Saldanha IJ, Dickersin K, Wang X, Li T. Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability. 2014.

28.

Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues. N Engl J Med. 2011;364(9):852–60. doi:10.1056/NEJMsa1012065.CrossRefPubMedPubMedCentral

29.

Loke YK, Price D, Herxheimer A. Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol. 2007;7:32. doi:10.1186/1471-2288-7-32.CrossRefPubMedPubMedCentral

30.

FDA. COSTART: coding symbols for thesaurus of adverse reaction terms. Fourth Edition: Office of Management and Operations 1993.

31.

ICH. Introductory Guide MedDRA Version 17.0: International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) 2014.

32.

Schroll JB, Maund E, Gotzsche PC. Challenges in coding adverse events in clinical trials: a systematic review. PLoS One. 2012;7(7):e41174. doi:10.1371/journal.pone.0041174.CrossRefPubMedPubMedCentral

33.

Software OCR. ABBYY FineReader version 12. 2013.

34.

Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011.

35.

Little RJ, D'Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, et al. The prevention and treatment of missing data in clinical trials. N Engl J Med. 2012;367(14):1355–60. doi:10.1056/NEJMsr1203730.CrossRefPubMedPubMedCentral

36.

Mavridis D, Chaimani A, Efthimiou O, Leucht S, Salanti G. Addressing missing outcome data in meta-analysis. Evid Based Ment Health. 2014;17(3):85–9. doi:10.1136/eb-2014-101900.CrossRefPubMed

37.

Mavridis D, White IR, Higgins JP, Cipriani A, Salanti G. Allowing for uncertainty due to missing continuous outcome data in pairwise and network meta-analysis. Stat Med. 2014;34(5):721–41. doi:10.1002/sim.6365.CrossRefPubMed

38.

Little RJA. Pattern-mixture models for multivariate incomplete data. J Am Stat Assoc. 1993;88:125–34.

39.

White IR, Welton NJ, Wood AM, Ades AE, Higgins JP. Allowing for uncertainty due to missing data in meta-analysis—part 2: hierarchical models. Stat Med. 2008;27(5):728–45. doi:10.1002/sim.3007.CrossRefPubMed

40.

Li T, Hutfless S, Scharfstein D, Daniels M, Hogan J, Little R, et al. Standards should be applied in the prevention and handling of missing data for patient-centered outcomes research: a systematic review and expert consensus. J Clin Epidemiol. 2014;67(1):15–32.CrossRefPubMedPubMedCentral

41.

PCORI. The PCORI Methodology Report. Appendix A: methodology standards. 2013.

42.

Little RJA, Rubin DB. Statistical analysis with missing data. 2nd ed. Hoboken: Wiley; 2002.CrossRef

43.

StataCorp. Stata: Release 13. College Station, TX: StataCorp LP; 2013.

44.

Horton NJ, Lipsitz SR. Multiple imputation in practice: comparison of software packages for regression models with missing variables. The American Statistician, American Statistical Association. 2001;55:244–54.

45.

Graham JW, Olchowski AE, Gilreath TD. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prev Sci. 2007;8:206–13.CrossRefPubMed

46.

Moons KG, Donders RA, Stijnen T, Harrell Jr FE. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol. 2006;59(10):1092–101. doi:10.1016/j.jclinepi.2006.01.009.CrossRefPubMed

47.

Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393. doi:10.1136/bmj.b2393.CrossRefPubMedPubMedCentral

48.

White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–99. doi:10.1002/sim.4067.CrossRefPubMed

49.

Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60. doi:10.1136/bmj.327.7414.557.CrossRefPubMedPubMedCentral

50.

Riley RD, Simmonds MC, Look MP. Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. J Clin Epidemiol. 2007;60(5):431–9. doi:10.1016/j.jclinepi.2006.09.009.CrossRefPubMed

51.

Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F, Thijs L, et al. Meta-analysis of continuous outcomes combining individual patient data and aggregate dat. Stat Med. 2008;27(11):1870–93.CrossRefPubMed

52.

Stewart LA, Clarke MJ. Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Stat Med. 1995;14(19):2057–79.CrossRefPubMed

53.

Higgins JP, Whitehead A, Turner RM, Omar RZ, Thompson SG. Meta-analysis of continuous outcome data from individual patients. Stat Med. 2001;20(15):2219–41. doi:10.1002/sim.918.CrossRefPubMed

54.

Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG. A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Stat Med. 2000;19(24):3417–32.CrossRefPubMed

55.

DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–88.CrossRefPubMed

56.

Thompson SG, Higgins JP. Treating individuals 4: can meta-analysis help target interventions at individuals most likely to benefit? Lancet. 2005;365(9456):341–6. doi:10.1016/S0140-6736(05)17790-3.CrossRefPubMed

57.

Yawn BP, Wollan PC, Weingarten TN, Watson JC, Hooten WM, Melton 3rd LJ. The prevalence of neuropathic pain: clinical evaluation compared with screening tools in a community population. Pain Med. 2009;10(3):586–93. doi:10.1111/j.1526-4637.2009.00588.x.CrossRefPubMedPubMedCentral

58.

Gordois A, Scuffham P, Shearer A, Oglesby A, Tobian JA. The health care costs of diabetic peripheral neuropathy in the US. Diabetes Care. 2003;26(6):1790–5.CrossRefPubMed

59.

Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):617–27. doi:10.1001/archpsyc.62.6.617.CrossRefPubMedPubMedCentral

60.

Merikangas KR, Jin R, He JP, Kessler RC, Lee S, Sampson NA, et al. Prevalence and correlates of bipolar spectrum disorder in the world mental health survey initiative. Arch Gen Psychiatry. 2011;68(3):241–51. doi:10.1001/archgenpsychiatry.2011.12.CrossRefPubMedPubMedCentral

61.

Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20(3 Suppl):21–35.CrossRefPubMed

62.

Calabrese JR, Hirschfeld RM, Frye MA, Reed ML. Impact of depressive symptoms compared with manic symptoms in bipolar disorder: results of a U.S. community-based sample. J Clin Psychiatry. 2004;65(11):1499–504.CrossRefPubMed

63.

Hirschfeld RM, Vornik LA. Recognition and diagnosis of bipolar disorder. J Clin Psychiatry. 2004;65 Suppl 15:5–9.PubMed

64.

Judd LL, Akiskal HS, Schettler PJ, Coryell W, Endicott J, Maser JD, et al. A prospective investigation of the natural history of the long-term weekly symptomatic status of bipolar II disorder. Arch Gen Psychiatry. 2003;60(3):261–9.CrossRefPubMed

65.

Judd LL, Akiskal HS, Schettler PJ, Endicott J, Maser J, Solomon DA, et al. The long-term natural history of the weekly symptomatic status of bipolar I disorder. Arch Gen Psychiatry. 2002;59(6):530–7.CrossRefPubMed

66.

Ilgen MA, Bohnert AS, Ignacio RV, McCarthy JF, Valenstein MM, Kim HM, et al. Psychiatric diagnoses and risk of suicide in veterans. Arch Gen Psychiatry. 2010;67(11):1152–8. doi:10.1001/archgenpsychiatry.2010.129.CrossRefPubMed

67.

Holma KM, Haukka J, Suominen K, Valtonen HM, Mantere O, Melartin TK et al. Differences in incidence of suicide attempts between bipolar I and II disorders and major depressive disorder. Bipolar Disorders. 2014:n/a-n/a. doi:10.1111/bdi.12195.

68.

Cipriani A, Barbui C, Salanti G, Rendell J, Brown R, Stockton S, et al. Comparative efficacy and acceptability of antimanic drugs in acute mania: a multiple-treatments meta-analysis. Lancet. 2011;378(9799):1306–15. doi:10.1016/S0140-6736(11)60873-8.CrossRefPubMed

69.

Jensen NH, Rodriguiz RM, Caron MG, Wetsel WC, Rothman RB, Roth BL. N-desalkylquetiapine, a potent norepinephrine reuptake inhibitor and partial 5-HT1A agonist, as a putative mediator of quetiapine's antidepressant activity. Neuropsychopharmacology. 2008;33(10):2303–12. doi:10.1038/sj.npp.1301646.CrossRefPubMed

70.

Sanford M, Keating G. Quetiapine: a review of its use in the management of bipolar depression. CNS Drugs. 2012;26(5):435–60.CrossRefPubMed

71.

Nivoli AM, Colom F, Murru A, Pacchiarotti I, Castro-Loli P, Gonzalez-Pinto A, et al. New treatment guidelines for acute bipolar depression: a systematic review. J Affect Disord. 2011;129(1–3):14–26. doi:10.1016/j.jad.2010.05.018.CrossRefPubMed

72.

NICE. Bipolar disorder: the management of bipolar disorder in adults, children and adolescents, in primary and secondary care. London: National Institute for Health and Care Excellence; 2014.

73.

Newcomer JW. Second-generation (atypical) antipsychotics and metabolic effects: a comprehensive literature review. CNS Drugs. 2005;19 Suppl 1:1–93.PubMed

74.

Ray WA, Chung CP, Murray KT, Hall K, Stein CM. Atypical antipsychotic drugs and the risk of sudden cardiac death. N Engl J Med. 2009;360(3):225–35. doi:10.1056/NEJMoa0806994.CrossRefPubMedPubMedCentral

75.

Hawley CJ, Finneberg N, Roberts AG, Baldwin D, Sahadevan A, Sharman V. The use of the Simpson Angus Scale for the assessment of movement disorder: a training guide. Int J Psych Clin Pract. 2003;7:249–57.CrossRef

76.

Rautaharju PM, Surawicz B. AHA/ACCF/HRS recommendations for the standardization and interpretation of the electrocardiogram: Part IV. Circulation. 2009;119:e241–e50.CrossRefPubMed

77.

Young RC, Biggs JT, Ziegler VE, Meyer DA. A rating scale for mania: reliability, validity and sensitivity. Br J Psychiatry. 1978;133:429–35.CrossRefPubMed

78.

Lukasiewicz M, Gerard S, Besnard A, Falissard B, Perrin E, Sapin H, et al. Young Mania Rating Scale: how to interpret the numbers? Determination of a severity threshold and of the minimal clinically significant difference in the EMBLEM cohort. Int J Methods Psychiatr Res. 2013;22(1):46–58. doi:10.1002/mpr.1379.CrossRefPubMed

Title: Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol for a systematic review
Authors: Evan Mayo-Wilson
Susan Hutfless
Tianjing Li
Gillian Gresham
Nicole Fusco
Jeffrey Ehmsen
James Heyward
Swaroop Vedula
Diana Lock
Jennifer Haythornthwaite
Jennifer L. Payne
Theresa Cowley
Elizabeth Tolbert
Lori Rosman
Claire Twose
Elizabeth A. Stuart
Hwanhee Hong
Peter Doshi
Catalina Suarez-Cuervo
Sonal Singh
Kay Dickersin
Publication date: 01-12-2015
Publisher: BioMed Central
Published in: Systematic Reviews / Issue 1/2015
Electronic ISSN: 2046-4053
DOI: https://doi.org/10.1186/s13643-015-0134-z

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol for a systematic review

Abstract

Background

Methods

Systematic review registration

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Systematic review registration

Please log in to get access to this content

Other articles of this Issue 1/2015

Association between BDNF levels and suicidal behaviour: a systematic review protocol

Better duplicate detection for systematic reviewers: evaluation of Systematic Review Assistant-Deduplication Module

Epidemiology of hepatitis A virus in Africa among persons aged 1–10 years: a systematic review protocol

Red blood cell transfusion and mortality effect in aneurysmal subarachnoid hemorrhage: a systematic review and meta-analysis protocol

Contributory factors to patient safety incidents in primary care: protocol for a systematic review

Meta-analysis of test accuracy studies: an exploratory method for investigating the impact of missing thresholds