Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Systematic Reviews
Published in: Systematic Reviews 1/2014

Open Access 01-12-2014 | Methodology

Methodological issues and recommendations for systematic reviews of prognostic studies: an example from cardiovascular disease

Authors: Janine Dretzke, Joie Ensor, Sue Bayliss, James Hodgkinson, Marie Lordkipanidzé, Richard D Riley, David Fitzmaurice, David Moore

Published in: Systematic Reviews | Issue 1/2014

Login to get access

Abstract

Background

Prognostic factors are associated with the risk of future health outcomes in individuals with a particular health condition. The prognostic ability of such factors is increasingly being assessed in both primary research and systematic reviews. Systematic review methodology in this area is continuing to evolve, reflected in variable approaches to key methodological aspects. The aim of this article was to (i) explore and compare the methodology of systematic reviews of prognostic factors undertaken for the same clinical question, (ii) to discuss implications for review findings, and (iii) to present recommendations on what might be considered to be ‘good practice’ approaches.

Methods

The sample was comprised of eight systematic reviews addressing the same clinical question, namely whether ‘aspirin resistance’ (a potential prognostic factor) has prognostic utility relative to future vascular events in patients on aspirin therapy for secondary prevention. A detailed comparison of methods around study identification, study selection, quality assessment, approaches to analysis, and reporting of findings was undertaken and the implications discussed. These were summarised into key considerations that may be transferable to future systematic reviews of prognostic factors.

Results

Across systematic reviews addressing the same clinical question, there were considerable differences in the numbers of studies identified and overlap between included studies, which could only partially be explained by different study eligibility criteria. Incomplete reporting and differences in terminology within primary studies hampered study identification and selection process across reviews. Quality assessment was highly variable and only one systematic review considered a checklist for studies of prognostic questions. There was inconsistency between reviews in approaches towards analysis, synthesis, addressing heterogeneity and reporting of results.

Conclusions

Different methodological approaches may ultimately affect the findings and interpretation of systematic reviews of prognostic research, with implications for clinical decision-making.
Appendix
Available only for authorised users
Literature
1.
Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KGM, Steyerberg EW, Roberts I, Schroter S, Altman DG, Riley RD: Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. 2013, 346: e5595. 10.1136/bmj.e5595.CrossRefPubMedPubMedCentral
2.
Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, Malats N, Briggs A, Schroter S, Altman DG, Hemingway H: Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013, 10: e1001380. 10.1371/journal.pmed.1001380.CrossRefPubMedPubMedCentral
3.
Riley RD, Sauerbrei W, Altman DG: Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond. Br J Cancer. 2009, 100: 1219-1229. 10.1038/sj.bjc.6604999.CrossRefPubMedPubMedCentral
4.
Hayden JA, Chou R, Hogg-Johnson S, Bombardier C: Systematic reviews of low back pain prognosis had variable methods and results: guidance for future prognosis reviews. J Clin Epidemiol. 2009, 62: 781-796. 10.1016/j.jclinepi.2008.09.004.CrossRefPubMed
5.
Rector TS, Taylor BC, Wilt TJ: Chapter 12: systematic review of prognostic tests. J Gen Intern Med. 2012, 27: 94-101. 10.1007/s11606-011-1899-y.CrossRefPubMedCentral
6.
7.
Moons KGM, de Groot JAH, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, Reitsma JB, Collins GS: Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 2014, 11 (10): e1001744. 10.1371/journal.pmed.1001744. doi:10.1371/journal.pmed.1001744CrossRefPubMedPubMedCentral
8.
Dretzke J, Riley R, Lordkipanidzé M, Jowett S, O'Donnell J, Ensor J, Moloney E, Price M, Raichand S, Hodgkinson J, Bayliss S, Fitzmaurice D, Moore D: The prognostic and diagnostic utility of tests of platelet function for the detection of “aspirin resistance” in patients with established cardiovascular or cerebrovascular disease: a systematic review and economic evaluation. Health Technol Assess. 2014, in press
9.
Raichand S, Moore D, Riley RD, Lordkipanidze M, Dretzke J, O’Donnell J, Jowett S, Bayliss S, Fitzmaurice DA: Protocol for a systematic review of the diagnostic and prognostic utility of tests currently available for the detection of aspirin resistance in patients with established cardiovascular or cerebrovascular disease. Syst Rev. 2013, 2: 16. 10.1186/2046-4053-2-16.CrossRefPubMedPubMedCentral
10.
Canivano PL, Garcia YC: Resistance to aspirin: prevalence, mechanisms of action and association with thromboembolic events. A narrative review. Farm Hosp. 2010, 34: 32-43. 10.1016/j.farma.2009.08.002.CrossRef
11.
Krasopoulos G, Brister SJ, Scott BW, Buchanan MR: Aspirin resistance and risk of cardiovascular morbidity: systematic review and meta-analysis. BMJ. 2008, 336: 195-198. 10.1136/bmj.39430.529549.BE.CrossRefPubMedPubMedCentral
12.
Pusch G, Feher G, Kotai K, Tibold A, Gasztonyi B, Feher A, Papp E, Lupkovics G, Szapary L: Aspirin resistance: focus on clinical endpoints. J Cardiovasc Pharmacol. 2008, 52: 475-484. 10.1097/FJC.0b013e31818eee5f.CrossRefPubMed
13.
Snoep JD, Hovens MM, Eikenboom JC, Van Der Bom JG, Huisman MV: Association of laboratory-defined aspirin resistance with a higher risk of recurrent cardiovascular events: a systematic review and meta-analysis. Arch Intern Med. 2007, 167: 1593-1599. 10.1001/archinte.167.15.1593.CrossRefPubMed
14.
Sofi F, Marcucci R, Gori AM, Abbate R, Gensini GF: Residual platelet reactivity on aspirin therapy and recurrent cardiovascular events—a meta-analysis. Int J Cardiol. 2008, 128: 166-171. 10.1016/j.ijcard.2007.12.010.CrossRefPubMed
15.
Li J, Song M, Jian Z, Guo W, Chen G, Jiang G, Wang J, Wu X, Huang L: Laboratory aspirin resistance and the risk of major adverse cardiovascular events in patients with coronary heart disease on confirmed aspirin adherence. J Atheroscler Thromb. 2014, 21: 239-247. 10.5551/jat.19521.CrossRefPubMed
16.
Wisman PP, Roest M, Asselbergs FW, de Groot PG, Moll FL, van der Graaf Y, de Borst GJ: Platelet-reactivity tests identify patients at risk of secondary cardiovascular events: a systematic review and meta-analysis. J Thromb Haemost. 2014, 12: 736-747.CrossRefPubMed
17.
Geersing GJ, Bouwmeester W, Zuithoff P, Spijker R, Leeflang M, Moons KG: Search filters for finding prognostic and diagnostic prediction studies in MEDLINE to enhance systematic reviews. PLoS One. 2012, 7: e32844. 10.1371/journal.pone.0032844.CrossRefPubMedPubMedCentral
18.
Eikelboom JW, Hirsh J, Weitz JI, Johnston M, Yi Q, Yusuf S: Aspirin-resistant thromboxane biosynthesis and the risk of myocardial infarction, stroke, or cardiovascular death in patients at high risk for cardiovascular events. Circulation. 2002, 105: 1650-1655. 10.1161/01.CIR.0000013777.21160.07.CrossRefPubMed
19.
Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM, QUADAS-2 Group: QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011, 155: 529-536. 10.7326/0003-4819-155-8-201110180-00009.CrossRefPubMed
20.
Hayden JA, Cote P, Bombardier C: Evaluation of the quality of prognosis studies in systematic reviews. Ann Intern Med. 2006, 144: 427-437. 10.7326/0003-4819-144-6-200603210-00010.CrossRefPubMed
21.
22.
Hayden JA, van der Windt DA, Cartwright JL, Cote P, Bombardier C: Assessing bias in studies of prognostic factors. Ann Intern Med. 2013, 158: 280-286. 10.7326/0003-4819-158-4-201302190-00009.CrossRefPubMed
23.
Parmar MK, Torri V, Stewart L: Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med. 1998, 17: 2815-2834. 10.1002/(SICI)1097-0258(19981230)17:24<2815::AID-SIM110>3.0.CO;2-8.CrossRefPubMed
24.
Perneger TV: Estimating the relative hazard by the ratio of logarithms of event-free proportions. Contemp Clin Trials. 2008, 29: 762-766. 10.1016/j.cct.2008.06.002.CrossRefPubMed
25.
26.
Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkind MS, Go AS, Harrell FE, Hong Y, Howard BV, Howard VJ, Hsue PY, Kramer CM, McConnell JP, Normand SL, O’Donnell CJ, Smith SC, Wilson PW, American Heart Association Expert Panel on Subclinical Atherosclerotic Diseases and Emerging Risk Factors and the Stroke Council: Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation. 2009, 119: 2408-2416. 10.1161/CIRCULATIONAHA.109.192278.CrossRefPubMedPubMedCentral
27.
Chatterley T, Dennet L: Utilisation of search filters in systematic reviews of prognosis questions. Health Info Libr J. 2012, 29: 309-322. 10.1111/hir.12004.CrossRefPubMed
28.
Gum PA, Kottke-Marchant K, Welsh PA, White J, Topol EJ: A prospective, blinded determination of the natural history of aspirin resistance among stable patients with cardiovascular disease. J Am Coll Cardiol. 2003, 41: 961-965. 10.1016/S0735-1097(02)03014-0.CrossRefPubMed
29.
Buchanan MR, Schwartz L, Bourassa M, Brister SJ, Peniston CM: Results of the BRAT study–a pilot study investigating the possible significance of ASA nonresponsiveness on the benefits and risks of ASA on thrombosis in patients undergoing coronary artery bypass surgery. Can J Cardiol. 2000, 16: 1385-1390.PubMed
30.
Pamukcu B, Oflaz H, Oncul A, Umman B, Mercanoglu F, Ozcan M, Meric M, Nisanci Y: The role of aspirin resistance on outcome in patients with acute coronary syndrome and the effect of clopidogrel therapy in the prevention of major cardiovascular events. J Thromb Thrombolysis. 2006, 22: 103-110. 10.1007/s11239-006-8952-4.CrossRefPubMed
31.
Lefebvre C, Glanville J, Wieland LS, Coles B, Weightman AL: Methodological developments in searching for studies for systematic reviews: past, present and future?. Syst Rev. 2013, 2: 78. 10.1186/2046-4053-2-78.CrossRefPubMedPubMedCentral
32.
de Jonge RC, van Furth AM, Wassenaar M, Gemke RJ, Terwee CB: Predicting sequelae and death after bacterial meningitis in childhood: a systematic review of prognostic studies. BMC Infect Dis. 2010, 10: 232. 10.1186/1471-2334-10-232.CrossRefPubMedPubMedCentral
33.
Verkerk K, Luijsterburg PA, Miedema HS, Pool-Goudzwaard A, Koes BW: Prognostic factors for recovery in chronic nonspecific low back pain: a systematic review. Phys Ther. 2012, 92: 1093-1108. 10.2522/ptj.20110388.CrossRefPubMed
34.
35.
Huguet A, Hayden JA, Stinson J, McGrath PJ, Chambers CT, Tougas ME, Wozney L: Judging the quality of evidence in reviews of prognostic factor research: adapting the GRADE framework. Syst Rev. 2013, 2: 71. 10.1186/2046-4053-2-71.CrossRefPubMedPubMedCentral
36.
Dahlen JR, Price MJ, Parise H, Gurbel PA: Evaluating the clinical usefulness of platelet function testing: considerations for the proper application and interpretation of performance measures. Thromb Haemost. 2013, 109: 808-816.CrossRefPubMed
37.
Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DG: Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013, 10: e1001381. 10.1371/journal.pmed.1001381.CrossRefPubMedPubMedCentral
38.
Stewart LA, Tierney JF: To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof. 2002, 25: 76-97. 10.1177/0163278702025001006.CrossRefPubMed
39.
Abo-Zaid G, Sauerbrei W, Riley RD: Individual participant data meta-analysis of prognostic factor studies: state of the art?. BMC Med Res Methodol. 2012, 12: 56. 10.1186/1471-2288-12-56.CrossRefPubMedPubMedCentral
40.
Ensor J, Riley RD, Moore D, Bayliss S, Jowett S, Fitzmaurice DA: Protocol for a systematic review of prognostic models for the recurrence of venous thromboembolism (VTE) following treatment for a first unprovoked VTE. Syst Rev. 2013, 2: 91. 10.1186/2046-4053-2-91.CrossRefPubMedPubMedCentral
41.
Metadata
Title
Methodological issues and recommendations for systematic reviews of prognostic studies: an example from cardiovascular disease
Authors
Janine Dretzke
Joie Ensor
Sue Bayliss
James Hodgkinson
Marie Lordkipanidzé
Richard D Riley
David Fitzmaurice
David Moore
Publication date
01-12-2014
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2014
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/2046-4053-3-140

Other articles of this Issue 1/2014

Systematic Reviews 1/2014 Go to the issue

Protocol

Family-based interventions for substance misuse: a systematic review of systematic reviews—protocol

Protocol

Association of industry sponsorship and positive outcome in randomised controlled trials in general and abdominal surgery: protocol for a systematic review and empirical study

Research

Conflicts of interest and critiques of the use of systematic reviews in policymaking: an analysis of opinion articles

Protocol

Processes, contexts, and rationale for disinvestment: a protocol for a critical interpretive synthesis

Research

An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks

Research

The importance of baseline viral load when assessing relative efficacy in treatment-naïve HBeAg-positive chronic hepatitis B: a systematic review and network meta-analysis