Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2012

Open Access 01-12-2012 | Research article

Individual participant data meta-analysis of prognostic factor studies: state of the art?

Published in: BMC Medical Research Methodology | Issue 1/2012

Login to get access

Abstract

Background

Prognostic factors are associated with the risk of a subsequent outcome in people with a given disease or health condition. Meta-analysis using individual participant data (IPD), where the raw data are synthesised from multiple studies, has been championed as the gold-standard for synthesising prognostic factor studies. We assessed the feasibility and conduct of this approach.

Methods

A systematic review to identify published IPD meta-analyses of prognostic factors studies, followed by detailed assessment of a random sample of 20 articles published from 2006. Six of these 20 articles were from the IMPACT (International Mission for Prognosis and Analysis of Clinical Trials in traumatic brain injury) collaboration, for which additional information was also used from simultaneously published companion papers.

Results

Forty-eight published IPD meta-analyses of prognostic factors were identified up to March 2009. Only three were published before 2000 but thereafter a median of four articles exist per year, with traumatic brain injury the most active research field. Availability of IPD offered many advantages, such as checking modelling assumptions; analysing variables on their continuous scale with the possibility of assessing for non-linear relationships; and obtaining results adjusted for other variables. However, researchers also faced many challenges, such as large cost and time required to obtain and clean IPD; unavailable IPD for some studies; different sets of prognostic factors in each study; and variability in study methods of measurement. The IMPACT initiative is a leading example, and had generally strong design, methodological and statistical standards. Elsewhere, standards are not always as high and improvements in the conduct of IPD meta-analyses of prognostic factor studies are often needed; in particular, continuous variables are often categorised without reason; publication bias and availability bias are rarely examined; and important methodological details and summary results are often inadequately reported.

Conclusions

IPD meta-analyses of prognostic factors are achievable and offer many advantages, as displayed most expertly by the IMPACT initiative. However such projects face numerous logistical and methodological obstacles, and their conduct and reporting can often be substantially improved.

Literature
  1. Steyerberg EW, Mushkudiani N, Perel P, Butcher I, Lu J, McHugh GS, Murray GD, Marmarou A, Roberts I, Habbema JDF, et al: Predicting outcome after traumatic brain injury: Development and international validation of prognostic scores based on admission characteristics. PLoS Med. 2008, 5 (8): 1251-1261.View Article
  2. Look MP, van Putten WL, Duffy MJ, Harbeck N, Christensen IJ, Thomssen C, Kates R, Spyratos F, Ferno M, Eppenberger-Castori S, et al: Pooled analysis of prognostic impact of urokinase-type plasminogen activator and its inhibitor PAI-1 in 8377 breast cancer patients. J Natl Cancer Inst. 2002, 94 (2): 116-128. 10.1093/jnci/94.2.116.View ArticlePubMed
  3. Sauerbrei W: Covariates. Wiley Encyclopedia of Clinical Trials. 2007, 1-13.
  4. Maas AI, Marmarou A, Murray GD, Teasdale SG, Steyerberg EW: Prognosis and clinical trial design in traumatic brain injury: the IMPACT study. J Neurotrauma. 2007, 24 (2): 232-238. 10.1089/neu.2006.0024.View ArticlePubMed
  5. Royston P, Altman DG, Sauerbrei W: Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006, 25 (1): 127-141. 10.1002/sim.2331.View ArticlePubMed
  6. Royston P, Moons KGM, Altman DG, Vergouwe Y: Prognosis and prognostic research: developing a prognostic model. Br Med J. 2009, 338 (b604): 1373-1377.
  7. Cianfrocca M, Goldstein LJ: Prognostic and predictive factors in early-stage breast cancer. Oncologist. 2004, 9 (6): 606-616. 10.1634/theoncologist.9-6-606.View ArticlePubMed
  8. Riley RD, Abrams KR, Sutton AJ, Lambert PC, Jones DR, Heney D, Burchill SA: Reporting of prognostic markers: current problems and development of guidelines for evidence-based practice in the future. Br J Cancer. 2003, 88 (8): 1191-1198. 10.1038/sj.bjc.6600886.View ArticlePubMedPubMed Central
  9. Holländer N, Sauerbrei W: On statistical approaches for the multivariable analysis of prognostic factor studies. Advances in statistical methods for the health sciences. Edited by: Auget J-L, Balakrishnan N, Mesbah M, Molenberghs G. 2006, Birkhäuser, Boston, 19-38.
  10. Kyzas PA, Loizou KT, Ioannidis JP: Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst. 2005, 97 (14): 1043-1055. 10.1093/jnci/dji184.View ArticlePubMed
  11. Sutcliffe P, Hummel S, Simpson E, Young T, Rees A, Wilkinson A, Hamdy F, Clarke N, Staffurth J: Use of classical and novel biomarkers as prognostic risk factors for localised prostate cancer: a systematic review. Health Technol Assess. 2009, 13 (5): 1-219. iii, xi-xiiiView Article
  12. Stewart LA, Parmar MK: Meta-analysis of the literature or of individual patient data: is there a difference?. Lancet. 1993, 341 (8842): 418-422. 10.1016/0140-6736(93)93004-K.View ArticlePubMed
  13. Riley RD, Sauerbrei W, Altman DG: Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond. Br J Cancer. 2009, 100 (8): 1219-1229. 10.1038/sj.bjc.6604999.View ArticlePubMedPubMed Central
  14. Trivella M, Pezzella F, Pastorino U, Harris AL, Altman DG: Microvessel density as a prognostic factor in non-small-cell lung carcinoma: a meta-analysis of individual patient data. Lancet Oncol. 2007, 8 (6): 488-499. 10.1016/S1470-2045(07)70145-6.View ArticlePubMed
  15. Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C: Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol. 1999, 28 (1): 1-9. 10.1093/ije/28.1.1.View ArticlePubMed
  16. Riley RD, Lambert PC, Abo-Zaid G: Meta-analysis of individual participant data: conduct, rationale and reporting. BMJ. 2010, 340: c221-10.1136/bmj.c221.View ArticlePubMed
  17. Altman DG, Trivella M, Pezzella F, Harris AL, Pastorino U: Systematic review of multiple studies of prognosis: the feasibility of obtaining individual patient data. I. Advances in statistical methods for the health sciences. Edited by: Auget J-L, Balakrishnan N, Mesbah M, Molenberghs G. 2006, Birkhäuser, Boston, 3-18.
  18. Meert AP, Paesmans M, Martin B, Delmotte P, Berghmans T, Verdebout JM, Lafitte JJ, Mascaux C, Sculier JP: The role of microvessel density on the survival of patients with lung cancer: a systematic review of the literature with meta-analysis. Br J Cancer. 2002, 87 (7): 694-701. 10.1038/sj.bjc.6600551.View ArticlePubMedPubMed Central
  19. Riley RD: Commentary: Like it and lump it? Meta-analysis using individual participant data. Int J Epidemiol. 2010, 39 (5): 1359-1361. 10.1093/ije/dyq129.View ArticlePubMed
  20. Ioannidis JP, Rosenberg PS, Goedert JJ, O'Brien TR: Commentary: meta-analysis of individual participants' data in genetic epidemiology. Am J Epidemiol. 2002, 156 (3): 204-210. 10.1093/aje/kwf031.View ArticlePubMed
  21. Moons KG, Altman DG, Vergouwe Y, Royston P: Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009, 338: b606-10.1136/bmj.b606.View ArticlePubMed
  22. Hemingway H, Riley RD, Altman DG: Ten steps towards improving prognosis research. BMJ. 2009, 339: b4184-10.1136/bmj.b4184.View ArticlePubMed
  23. Riley RD, Simmonds MC, Look MP: Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. J Clin Epidemiol. 2007, 60 (5): 431-439.View ArticlePubMed
  24. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB: Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000, 283 (15)): 2008-2012.View ArticlePubMed
  25. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF: Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999, 354 (9193)): 1896-1900.View ArticlePubMed
  26. Marmarou A, Lu J, Butcher I, McHugh GS, Mushkudiani NA, Murray GD, Steyerberg EW, Maas AI: IMPACT database of traumatic brain injury: design and description. J Neurotrauma. 2007, 24 (2): 239-250. 10.1089/neu.2006.0036.View ArticlePubMed
  27. MeRGE collaborators: Independent prognostic importance of a restrictive left ventricular filling pattern after myocardial infarction: an individual patient meta-analysis. Circulation. 2008, 117 (20)): 2591-2598.
  28. Murray GD, Butcher I, McHugh GS, Lu J, Mushkudiani NA, Maas AIR, Marmarou A, Steyerberg EW: Multivariable prognostic analysis in traumatic brain injury: Results from the IMPACT study. J Neurotrauma. 2007, 24 (2): p329-p337. 10.1089/neu.2006.0035.View Article
  29. Butcher I, Maas AIR, Lu J, Marmarou A, Murray GD, Mushkudiani NA, McHugh GS, Steyerberg EW: Prognostic value of admission blood pressure in traumatic brain injury: Results from the IMPACT study. J Neurotrauma. 2007, 24 (2): 294-302. 10.1089/neu.2006.0032.View ArticlePubMed
  30. Van Beek JGM, Mushkudiani NA, Steyerberg EW, Butcher I, McHugh GS, Lu J, Marmarou A, Murray GD, Maas AIR: Prognostic value of admission laboratory parameters in traumatic brain injury: Results from the IMPACT study. J Neurotrauma. 2007, 24 (2): 315-328. 10.1089/neu.2006.0034.View ArticlePubMed
  31. Maas AIR, Steyerberg EW, Butcher I, Dammers R, Lu J, Marmarou A, Mushkudiani NA, McHugh GS, Murray GD: Prognostic value of computerized tomography scan characteristics in traumatic brain injury: Results from the IMPACT study. J Neurotrauma. 2007, 24 (2): 303-314. 10.1089/neu.2006.0033.View ArticlePubMed
  32. Mushkudiani NA, Engel DC, Steyerberg EW, Butcher I, Lu J, Marmarou A, Slieker F, McHugh GS, Murray GD, Maas AIR: Prognostic value of demographic characteristics in traumatic brain injury: Results from the IMPACT study. J Neurotrauma. 2007, 24 (2): 259-269. 10.1089/neu.2006.0028.View ArticlePubMed
  33. McHugh GS, Engel DC, Butcher I, Steyerberg EW, Lu J, Mushkudiani N, Hernandez AV, Marmarou A, Maas AIR, Murray GD: Prognostic value of secondary insults in traumatic brain injury: Results from the IMPACT study. J Neurotrauma. 2007, 24 (2): 287-293. 10.1089/neu.2006.0031.View ArticlePubMed
  34. Butcher I, McHugh GS, Lu J, Steyerberg EW, Hernandez AV, Mushkudiani N, Maas AI, Marmarou A, Murray GD: Prognostic value of cause of injury in traumatic brain injury: results from the IMPACT study. J Neurotrauma. 2007, 24 (2): 281-286. 10.1089/neu.2006.0030.View ArticlePubMed
  35. Marmarou A, Lu J, Butcher I, McHugh GS, Murray GD, Steyerberg EW, Mushkudiani NA, Choi S, Maas AI: Prognostic value of the Glasgow Coma Scale and pupil reactivity in traumatic brain injury assessed pre-hospital and on enrollment: an IMPACT analysis. J Neurotrauma. 2007, 24 (2): 270-280. 10.1089/neu.2006.0029.View ArticlePubMed
  36. Yap YG, Duong T, Bland M, Malik M, Torp-Pedersen C, Kober L, Connolly SJ, Gallagher MM, Camm AJ: Potential demographic and baselines variables for risk stratification of high-risk post-myocardial infarction patients in the era of implantable cardioverter-defibrillator - A prognostic indicator. Int J Cardiol. 2008, 126 (1): 101-107. 10.1016/j.ijcard.2007.03.122.View ArticlePubMed
  37. Thakkinstian A, Dmitrienko S, Gerbase-DeLima M, McDaniel DO, Inigo P, Chow KM, McEvoy M, Ingsathit A, Trevillian P, Barber WH, et al: Association between cytokine gene polymorphisms and outcomes in renal transplantation: A meta-analysis of individual patient data. Nephrol Dial Transplant. 2008, 23 (9): 3017-3023. 10.1093/ndt/gfn185.View ArticlePubMed
  38. Koopman L, Hoes AW, Glasziou PP, Appelman CL, Burke P, McCormick DP, Damoiseaux RA, Le SN, Rovers MM: Antibiotic therapy to prevent the development of asymptomatic middle ear effusion in children with acute otitis media: A meta-analysis of individual patient data. Arch Otolaryngol Head Neck Surg. 2008, 134 (2): 128-132. 10.1001/archoto.2007.3.View ArticlePubMed
  39. Goetz CG, Wuu J, McDermott MP, Adler CH, Fahn S, Freed CR, Hauser RA, Olanow WC, Shoulson I, Tandon PK, et al: Placebo response in Parkinson's disease: Comparisons among 11 trials covering medical and surgical interventions. Mov Disord. 2008, 23 (5): 690-699. 10.1002/mds.21894.View ArticlePubMed
  40. MeRGE collaborators: Independence of restrictive filling pattern and LV ejection fraction with mortality in heart failure: An individual patient meta-analysis. Eur J Heart Fail. 2008, 10 (8)): 786-792.
  41. Sylaja PN, Dong W, Grotta JC, Miller MK, Tomita K, Hamilton S, Semba C, Hill MD: Safety outcomes of Alteplase among acute ischemic stroke patients with special characteristics. Neurocrit Care. 2007, 6 (3): 181-185. 10.1007/s12028-007-0018-8.View ArticlePubMed
  42. Schaich M, Schlenk RF, Al-Ali HK, Dohner H, Ganser A, Heil G, Illmer T, Krahl R, Krauter J, Sauerland C, et al: Prognosis of acute myeloid leukemia patients up to 60 years of age exhibiting trisomy 8 within a non-complex karyotype: Individual patient data-based meta-analysis of the German Acute Myeloid Leukemia Intergroup. Haematologica. 2007, 92 (6): 763-770. 10.3324/haematol.11100.View ArticlePubMed
  43. Rovers MM, Glasziou P, Appelman CL, Burke P, McCormick DP, Damoiseaux RA, Little P, Le SN, Hoes AW: Predictors of pain and/or fever at 3 to 7 days for children with acute otitis media not treated initially with antibiotics: A meta-analysis of individual patient data. Pediatrics. 2007, 119 (3): 579-585. 10.1542/peds.2006-2092.View ArticlePubMed
  44. Noordzij JP, Lee SL, Bernet VJ, Payne RJ, Cohen SM, McLeod IK, Hier MP, Black MJ, Kerr PD, Richards ML, et al: Early Prediction of Hypocalcemia after Thyroidectomy using Parathyroid Hormone: An Analysis of Pooled Individual Patient Data from Nine Observational Studies. J Am Coll Surg. 2007, 205 (6): 748-754. 10.1016/j.jamcollsurg.2007.06.298.View ArticlePubMed
  45. Lanterna LA, Ruigrok Y, Alexander S, Tang J, Biroli F, Dunn LT, Poon WS: Meta-analysis of APOE genotype and subarachnoid hemorrhage: Clinical outcome and delayed ischemia. Neurology. 2007, 69 (8): 766-775. 10.1212/01.wnl.0000267640.03300.6b.View ArticlePubMed
  46. Downing M, Lau F, Lesperance M, Karlson N, Shaw J, Kuziemsky C, Bernard S, Hanson L, Olajide L, Head B, et al: Meta-analysis of survival prediction with Palliative Performance Scale. J Palliat Care. 2007, 23 (4): 245-252.PubMed
  47. Warkentin TE, Sheppard J-A, Sigouin CS, Kohlmann T, Eichler P, Greinacher A: Gender imbalance and risk factor interactions in heparin-induced thrombocytopenia. Blood. 2006, 108 (9): 2937-2941. 10.1182/blood-2005-11-012450.View ArticlePubMed
  48. McHugh GS, Butcher I, Steyerberg EW, Lu J, Mushkudiani N, Marmarou A, Maas AI, Murray GD: Statistical approaches to the univariate prognostic analysis of the IMPACT database on traumatic brain injury. J Neurotrauma. 2007, 24 (2): 251-258. 10.1089/neu.2006.0026.View ArticlePubMed
  49. Hayden JA, Cote P, Bombardier C: Evaluation of the quality of prognosis studies in systematic reviews. Ann Intern Med. 2006, 144 (6): 427-437.View ArticlePubMed
  50. Cochrane Handbook for Systematic Reviews of Interventions. Edited by: Higgins JPT, Green S. 2008, John Wiley & Sons, Chichester
  51. McKibbon KA: Evidence-based practice. Bull Med Libr Assoc. 1998, 86 (3): 396-401.PubMedPubMed Central
  52. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F: Methods for Meta-analysis in Medical Research. 2000, John Wiley, London
  53. Thompson SG, Kaptoge S, White I, Wood A, Perry P, Danesh J: Statistical methods for the time-to-event analysis of individual participant data from multiple epidemiological studies. Int J Epidemiol. 2010, 39 (5): 1345-1359. 10.1093/ije/dyq063.View ArticlePubMedPubMed Central
  54. Higgins JP, Thompson SG, Deeks JJ, Altman DG: Measuring inconsistency in meta-analyses. BMJ. 2003, 327 (7414): 557-560. 10.1136/bmj.327.7414.557.View ArticlePubMedPubMed Central
  55. DerSimonian R, Laird N: Meta-analysis in clinical trials. Control Clin Trials. 1986, 7 (3): 177-188. 10.1016/0197-2456(86)90046-2.View ArticlePubMed
  56. Egger M, Davey Smith G, Schneider M, Minder C: Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997, 315 (7109): 629-634. 10.1136/bmj.315.7109.629.View ArticlePubMedPubMed Central
  57. Riley RD, Steyerberg EW: Meta-analysis of a binary outcome using individual participant data and aggregate data. Journal of Research Synthesis Methods. 2010, 1: 2-9. 10.1002/jrsm.4.View ArticlePubMed
  58. Broeze KA, Opmeer BC, Bachmann LM, Broekmans FJ, Bossuyt PM, Coppus SF, Johnson NP, Khan KS, ter Riet G, van der Veen F, et al: Individual patient data meta-analysis of diagnostic and prognostic studies in obstetrics, gynaecology and reproductive medicine. BMC Med Res Methodol. 2009, 9: 22-10.1186/1471-2288-9-22.View ArticlePubMedPubMed Central
  59. Sauerbrei W: Prognostic Factors - Confusion caused by bad quality of design, analysis and reporting of many studies. Current Research in Head and Neck CancerAdvances in Oto-Rhino-Laryngology. Volume 62. Edited by: Bier H. 2005, Karger, Basel, 184-200.
  60. Sauerbrei W, Blettner M, Royston P: On alcohol consumption and all-cause mortality. J Clin Epidemiol. 2001, 54: 537-538. 10.1016/S0895-4356(00)00246-8.View Article
  61. Sauerbrei W, Holländer N, Riley RD, Altman DG: Evidence-based assessment and application of prognostic markers: the long way from single studies to meta-analysis. Communications in Statistics. 2006, 35: 1333-1342. 10.1080/03610920600629666.View Article
  62. Moher D, Liberati A, Tetzlaff J, Altman DG, for the PRISMA Group): Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009, 339: b2535-10.1136/bmj.b2535.View ArticlePubMedPubMed Central
  63. McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM, Diagnostics ftSSotN-EWoC: REporting recommendations for tumor MARKer prognostic studies (REMARK). J Natl Cancer Inst. 2005, 97 (16)): 1180-1184.View ArticlePubMed
  64. Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG: A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Stat Med. 2000, 19 (24): 3417-3432. 10.1002/1097-0258(20001230)19:24<3417::AID-SIM614>3.0.CO;2-L.View ArticlePubMed
  65. Higgins JP, Whitehead A, Turner RM, Omar RZ, Thompson SG: Meta-analysis of continuous outcome data from individual patients. Stat Med. 2001, 20 (15): 2219-2241. 10.1002/sim.918.View ArticlePubMed
  66. Tudur-Smith C, Williamson PR, Marson AG: Investigating heterogeneity in an individual patient data meta-analysis of time to event outcomes. Stat Med. 2005, 24 (9): 1307-1319. 10.1002/sim.2050.View Article
  67. Steyerberg EW: Clinical prediction models: a practical approach to development, validation, and updating. 2009, Springer, New YorkView Article
  68. Riley RD, Higgins JP, Deeks JJ: Interpretation of random effects meta-analyses. BMJ. 2011, 342: d549-10.1136/bmj.d549.View ArticlePubMed
  69. Higgins JP, Thompson SG, Spiegelhalter DJ: A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society, Series A. 2009, 172: 137-159.View Article
  70. Ahmed I, Sutton AJ, Riley RD: Assessment of publication bias, selection bias and unavailable data in meta-analyses using individual participant data: a database survey. BMJ. 2012, 344: d7762-10.1136/bmj.d7762.View ArticlePubMed
  71. Kyzas PA, Denaxa-Kyza D, Ioannidis JP: Almost all articles on cancer prognostic markers report statistically significant results. Eur J Cancer. 2007, 43 (17): 2559-2579. 10.1016/j.ejca.2007.08.030.View ArticlePubMed
  72. Sterne JAC, Terrin N, Jones DR, Lau J, Carpenter J, Rücker G, Harbord RM, Schmid CH, A.J. S J.P.A. I, et al: Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011, 342: d4002-View Article
  73. McShane LM, Altman DG, Sauerbrei W: Identification of clinically useful cancer prognostic factors: what are we missing?. J Natl Cancer Inst. 2005, 97 (14): 1023-1025. 10.1093/jnci/dji193.View ArticlePubMed
    1. The pre-publication history for this paper can be accessed here:http://​www.​biomedcentral.​com/​1471-2288/​12/​56/​prepub
Metadata
Title
Individual participant data meta-analysis of prognostic factor studies: state of the art?
Publication date
01-12-2012
Published in
BMC Medical Research Methodology / Issue 1/2012
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-12-56

Other articles of this Issue 1/2012

BMC Medical Research Methodology 1/2012 Go to the issue