Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Systematic Reviews
Published in: Systematic Reviews 1/2017

Open Access 01-12-2017 | Research

Comparison of two independent systematic reviews of trials of recombinant human bone morphogenetic protein-2 (rhBMP-2): the Yale Open Data Access Medtronic Project

Authors: Jeffrey Low, Joseph S. Ross, Jessica D. Ritchie, Cary P. Gross, Richard Lehman, Haiqun Lin, Rongwei Fu, Lesley A. Stewart, Harlan M. Krumholz

Published in: Systematic Reviews | Issue 1/2017

Login to get access

Abstract

Background

It is uncertain whether the replication of systematic reviews, particularly those with the same objectives and resources, would employ similar methods and/or arrive at identical findings. We compared the results and conclusions of two concurrent systematic reviews undertaken by two independent research teams provided with the same objectives, resources, and individual participant-level data.

Methods

Two centers in the USA and UK were each provided with participant-level data on 17 multi-site clinical trials of recombinant human bone morphogenetic protein-2 (rhBMP-2). The teams were blinded to each other’s methods and findings until after publication. We conducted a retrospective structured comparison of the results of the two systematic reviews. The main outcome measures included (1) trial inclusion criteria; (2) statistical methods; (3) summary efficacy and risk estimates; and (4) conclusions.

Results

The two research teams’ meta-analyses inclusion criteria were broadly similar but differed slightly in trial inclusion and research methodology. They obtained similar results in summary estimates of most clinical outcomes and adverse events. Center A incorporated all trials into summary estimates of efficacy and harms, while Center B concentrated on analyses stratified by surgical approach. Center A found a statistically significant, but small, benefit whereas Center B reported no advantage. In the analysis of harms, neither showed an increased cancer risk at 48 months, although Center B reported a significant increase at 24 months. Conclusions reflected these differences in summary estimates of benefit balanced with small but potentially important risk of harm.

Conclusions

Two independent groups given the same research objectives, data, resources, funding, and time produced broad general agreement but differed in several areas. These differences, the importance of which is debatable, indicate the value of the availability of data to allow for more than a single approach and a single interpretation of the data.

Systematic review registration

Appendix
Available only for authorised users
Literature
1.
Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005;294(2):218–28.CrossRefPubMed
2.
Stewart LA, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof. 2002;25(1):76–97.CrossRefPubMed
3.
Wang JG, Staessen JA, Franklin SS, Fagard R, Gueyffier F. Systolic and diastolic blood pressure lowering as determinants of cardiovascular outcome. Hypertension. 2005;45(5):907–13.CrossRefPubMed
4.
Stewart LA, Parmar MK. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet. 1993;341(8842):418–22.CrossRefPubMed
5.
Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F, Thijs L, Boutitie F. Meta-analysis of continuous outcomes combining individual patient data and aggregate data. Stat Med. 2008;27(11):1870–93.CrossRefPubMed
6.
Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;340:c221.CrossRefPubMed
7.
Jeng GT, Scott JR, Burmeister LF. A comparison of meta-analytic results using literature vs individual patient data. Paternal cell immunization for recurrent miscarriage. JAMA. 1995;274(10):830–6.CrossRefPubMed
8.
Berlin JA, Santanna J, Schmid CH, Szczech LA, Feldman HI. Anti-lymphocyte antibody induction therapy study G: individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Stat Med. 2002;21(3):371–87.CrossRefPubMed
9.
McCormack K, Grant A, Scott N, Collaboration EUHT. Value of updating a systematic review in surgery using individual patient data. Br J Surg. 2004;91(4):495–9.CrossRefPubMed
10.
Ioannidis JP. Why science is not necessarily self-correcting. Perspect Psychol Sci. 2012;7(6):645–54.CrossRefPubMed
11.
12.
13.
14.
Nisen P, Rockhold F. Access to patient-level data from GlaxoSmithKline clinical trials. N Engl J Med. 2013;369(5):475–8.CrossRefPubMed
15.
16.
17.
Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9. W264.CrossRefPubMed
18.
Thompson R, Bandera E, Burley V, Cade J, Forman D, Freudenheim J, Greenwood D, Jacobs D, Kalliecharan R, Kushi L, et al. Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer. Public Health Nutr. 2008;11(10):1006–14.CrossRefPubMed
19.
Anonymous. Worldwide collaborative observational study and meta-analysis on allogenic leukocyte immunotherapy for recurrent spontaneous abortion. Recurrent Miscarriage Immunotherapy Trialists Group. [Erratum appears in Am J Reprod Immunol 1994 Oct;32(3):255]. Am J Reprod Immunol. 1994;32(2):55–72.CrossRef
20.
Pladevall-Vila M, Delclos GL, Varas C, Guyer H, Brugues-Tarradellas J, Anglada-Arisa A. Controversy of oral contraceptives and risk of rheumatoid arthritis: meta-analysis of conflicting studies and review of conflicting meta-analyses with special emphasis on analysis of heterogeneity. Am J Epidemiol. 1996;144(1):1–14.CrossRefPubMed
21.
Hopayian K, Mugford M. Conflicting conclusions from two systematic reviews of epidural steroid injections for sciatica: which evidence should general practitioners heed? Br J Gen Pract. 1999;49(438):57–61.PubMedPubMedCentral
22.
23.
24.
25.
Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344:d7202.CrossRefPubMed
26.
Fu R, Selph S, McDonagh M, Peterson K, Tiwari A, Chou R, Helfand M. Effectiveness and Harms of Recombinant Human Bone Morphogenetic Protein-2 in Spine Fusion: A Systematic Review and Meta-analysis. Ann Intern Med. 2013;158:890–902.
27.
Simmonds MC, Brown JV, Heirs MK, Higgins JP, Mannion RJ, Rodgers MA, Stewart LA. Safety and effectiveness of recombinant human bone morphogenetic protein-2 for spinal fusion: a meta-analysis of individual-participant data. Ann Intern Med. 2013;158(12):877–89.CrossRefPubMed
28.
Fu XH, Li MN, Zheng GH, Le YQ, Wang L. Waste recombinant DNA: effectiveness of thermo-treatment to manage potential gene pollution. Environ Pollut. 2009;157(8-9):2536–41.CrossRefPubMed
29.
Fu R, Selph S, McDonagh M, Peterson K, Tiwari A, Chou R, Helfand M. Effectiveness and harms of recombinant human bone morphogenetic protein-2 in spine fusion: a systematic review and meta-analysis. Ann Intern Med. 2013;158(12):890–902.CrossRefPubMed
30.
31.
32.
33.
Glassman SD, Carreon LY, Djurasovic M, Campbell MJ, Puno RM, Johnson JR, Dimar JR. RhBMP-2 versus iliac crest bone graft for lumbar spine fusion: a randomized, controlled trial in patients over sixty years of age. Spine. 2008;33(26):2843–9.CrossRefPubMed
34.
Resnick D, Bozic KJ. Meta-analysis of trials of recombinant human bone morphogenetic protein-2: what should spine surgeons and their patients do with this information? Ann Intern Med. 2013;158(12):912–3.CrossRefPubMed
35.
Riew KD, Carragee EJ. Commentary: despite reports of catastrophic complications, why recombinant human bone morphogenetic protein-2 should be available for use in anterior cervical spine surgery. Spine J. 2012;12(10):900–1.CrossRefPubMed
36.
Cahill KS, Chi JH, Day A, Claus EB. Prevalence, complications, and hospital charges associated with use of bone-morphogenetic proteins in spinal fusion procedures. JAMA. 2009;302(1):58–66.CrossRefPubMed
37.
Laine C, Guallar E, Mulrow C, Taichman DB, Cornell JE, Cotton D, Griswold ME, Localio AR, Meibohm AR, Stack CB, et al. Closing in on the truth about recombinant human bone morphogenetic protein-2: evidence synthesis, data sharing, peer review, and reproducible research. Ann Intern Med. 2013;158(12):916–8.CrossRefPubMed
38.
Hsu WK. Recombinant human bone morphogenetic protein-2 in spine surgery. JBJS Rev. 2014;2(6).
39.
Carragee EJ, Hurwitz EL, Weiner BK. A critical review of recombinant human bone morphogenetic protein-2 trials in spinal surgery: emerging safety concerns and lessons learned. The Spine Journal. 2011;11(6):471–91.CrossRefPubMed
Metadata
Title
Comparison of two independent systematic reviews of trials of recombinant human bone morphogenetic protein-2 (rhBMP-2): the Yale Open Data Access Medtronic Project
Authors
Jeffrey Low
Joseph S. Ross
Jessica D. Ritchie
Cary P. Gross
Richard Lehman
Haiqun Lin
Rongwei Fu
Lesley A. Stewart
Harlan M. Krumholz
Publication date
01-12-2017
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2017
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/s13643-017-0422-x

Other articles of this Issue 1/2017

Systematic Reviews 1/2017 Go to the issue

Research

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a prospective study

Protocol

The macro-economic determinants of health and health inequalities—umbrella review protocol

Protocol

The dynamics of HIV transmission in out of school young heterosexual men in South Africa: a systematic scoping review protocol

Methodology

Putting research in place: an innovative approach to providing contextualized evidence synthesis for decision makers

Protocol

Contacting of authors by systematic reviewers: protocol for a cross-sectional study and a survey

Protocol

Prevalence, predictors and adverse outcomes of adolescent pregnancy in sub-Saharan Africa: a protocol of a systematic review