Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2012

Open Access 01-12-2012 | Research article

Investigating clinical heterogeneity in systematic reviews: a methodologic review of guidance in the literature

Authors: Joel J Gagnier, David Moher, Heather Boon, Joseph Beyene, Claire Bombardier

Published in: BMC Medical Research Methodology | Issue 1/2012

Login to get access

Abstract

Background

While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. The objective of this study is to summarize and collate suggested methods for investigating clinical heterogeneity in systematic reviews.

Methods

We searched databases (Medline, EMBASE, CINAHL, Cochrane Library, and CONSORT, to December 2010) and reference lists and contacted experts to identify resources providing suggestions for investigating clinical heterogeneity between controlled clinical trials included in systematic reviews. We extracted recommendations, assessed resources for risk of bias, and collated the recommendations.

Results

One hundred and one resources were collected, including narrative reviews, methodological reviews, statistical methods papers, and textbooks. These resources generally had a low risk of bias, but there was minimal consensus among them. Resources suggested that planned investigations of clinical heterogeneity should be made explicit in the protocol of the review; clinical experts should be included on the review team; a set of clinical covariates should be chosen considering variables from the participant level, intervention level, outcome level, research setting, or others unique to the research question; covariates should have a clear scientific rationale; there should be a sufficient number of trials per covariate; and results of any such investigations should be interpreted with caution.

Conclusions

Though the consensus was minimal, there were many recommendations in the literature for investigating clinical heterogeneity in systematic reviews. Formal recommendations for investigating clinical heterogeneity in systematic reviews of controlled trials are required.
Appendix
Available only for authorised users
Literature
1.
2.
Pildal J, Hrobjartsson A, Jorgensen KJ, Hilden J, Altman DG, Gotzsche PC: Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007, 36 (4): 847-857. 10.1093/ije/dym087.PubMed
3.
Tugwell P, Robinson V, Grimshaw J, Santesso N: Systematic reviews and knowledge translation. Bulletin of the World Health Organization. 2006, 84: 643-651. 10.2471/BLT.05.026658.PubMedPubMedCentral
4.
Grimshaw JM, Santesso N, Cumpston M, Mayhew A, McGowan J: Knowledge for knowledge translation: the role of the cochrane collaboration. J Contin Educ Health Prof. 2006, 26: 55-62. 10.1002/chp.51.PubMed
5.
6.
7.
8.
Laupacis A, Strauss S, Systematic reviews, Systematic reviews: Time to address clinical and policy relevance as well as methodological rigor. Ann Int Med. 2007, 147 (4): 273-275.PubMed
9.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group: Preferred reporting items for systematic review and meta-analyses: the PRISMA statement. PLoS Medicine. 2009, 6 (&): e1000097-PubMedPubMedCentral
10.
Glasziou P, Meats E, Heneghan C, Shepperd S: What is missing from descriptions of treatment in trials and reviews?. BMJ. 2008, 336: 1472-1474. 10.1136/bmj.39590.732037.47.PubMedPubMedCentral
11.
Glasziou P, Chalmers I, Altman DG, et al: Taking healthcare interventions from trial to practice. BMJ. 2010, 341: 384-387.
12.
Chalmers I, Glasziou P: Avoidable waste in the production and reporting of research evidence. Lancet. 2009, 374: 86-89. 10.1016/S0140-6736(09)60329-9.PubMed
13.
Chalmers I, Glasziou P: Avoidable waste in the production and reporting of research evidence. Obstet Gynecol. 2009, 114 (6): 1341-1345. 10.1097/AOG.0b013e3181c3020d.PubMed
14.
Scott NA, Moga C, Barton P, Rashiq S, Schopflocher D, Taenzer P, Alberta Ambassador Program Team, et al: Creating clinically relevant knowledge from systematic reviews: The challenges of knowledge translation. J Eval Clin Pract. 2007, 13 (4): 681-688. 10.1111/j.1365-2753.2007.00830.x.PubMed
15.
Gagnier JJ, Bombardier C, Boon H, Moher D, Beyene J: An empirical study using permutation-based resampling in meta-regression. Systematic Reviews. 2012, 1: 18-10.1186/2046-4053-1-18.PubMedPubMedCentral
16.
Gagnier JJ, Morgenstern H, Moher D: Recommendations for investigating clinical heterogeneity in systematic reviews and meta-analyses. 2012, Under Review
17.
Arends LR, Hoes AW, Lubsen J, Grobbee DE, Stijnen T: Baseline risk as predictor of treatment benefit: Three clinical meta-re-analyses. Stat Med. 2000, 19: 3497-3518. 10.1002/1097-0258(20001230)19:24<3497::AID-SIM830>3.0.CO;2-H.PubMed
18.
Higgins J, Thompson S, Deeks J, Altman D: Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice. J Health Serv Res Policy. 2002, 7 (1): 51-61. 10.1258/1355819021927674.PubMed
19.
Schmid CH, Lau J, McIntosh MW, Cappelleri JC: An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Stat Med. 1998, 17 (17): 1923-1942. 10.1002/(SICI)1097-0258(19980915)17:17<1923::AID-SIM874>3.0.CO;2-6.PubMed
20.
Thompson SG: Why sources of heterogeneity in meta-analysis should be investigated. BMJ. 1994, 309 (6965): 1351-1355. 10.1136/bmj.309.6965.1351.PubMedPubMedCentral
21.
van den Ende CHM, Steultjens EMJ, Bouter LM, Dekker J: Clinical heterogeneity was a common problem in Cochrane reviews of physiotherapy and occupational therapy. J Clin Epidemiol. 2006, 59: 914-919. 10.1016/j.jclinepi.2005.12.014.PubMed
22.
Loke YK, Price D, Herxheimer A: Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol. 2007, 7: 32-10.1186/1471-2288-7-32.PubMedPubMedCentral
23.
Freemantle N, Mason J, Eccles M: Deriving treatment recommendations from evidence within randomized trials. The role and limitation of meta-analysis. Int J Technol Assess Health Care. 1999, 15 (2): 304-315.PubMed
24.
Huang JQ, Zheng GF, Irvine EJ, Karlberg J: Assessing heterogeneity in meta-analyses of Helicobacter pylori infection-related clinical studies: a critical appraisal. Chin J Dig Dis. 2004, 5 (3): 126-133. 10.1111/j.1443-9573.2004.00169.x.PubMed
25.
Lau J, Ioannidis JP, Schmid CH: Quantitative synthesis in systematic reviews. Ann Intern Med. 1997, 127 (9): 820-826.PubMed
26.
Bender R, Bunce C, Clarke M, et al: Attention should be given to multiplicity issues in systematic reviews. J Clin Epidemiol. 2008, 61 (9): 857-865. 10.1016/j.jclinepi.2008.03.004.PubMed
27.
van Tulder M, Furlan A, Bombardier C, Bouter L: Updated method guidelines for systematic reviews in the Cochrane collaboration back review group. Spine. 2003, 28 (12): 1290-1299.PubMed
28.
Anello C, Fleiss JL: Exploratory or analytic meta-analysis: should we distinguish between them?. J Clin Epidemiol. 1995, 48 (1): 109-116. 10.1016/0895-4356(94)00084-4. discussion 117-108PubMed
29.
Simmonds MC, Higgins JP, Stewart LA, Tierney JF, Clarke MJ, Thompson SG: Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Clin Trials. 2005, 2 (3): 209-217. 10.1191/1740774505cn087oa.PubMed
30.
Maxwell L, Santesso N, Tugwell PS, Wells GA, Judd M, Buchbinder R: Method guidelines for Cochrane Musculoskeletal Group systematic reviews. J Rheumatol. 2006, 33 (11): 2304-2311.PubMed
31.
Higgins JP, Thompson SG: Controlling the risk of spurious findings from meta-regression. Stat Med. 2004, 23 (11): 1663-1682. 10.1002/sim.1752.PubMed
32.
Song F, Sheldon TA, Sutton AJ, Abrams KR, Jones DR: Methods for exploring heterogeneity in meta-analysis. Eval Health Prof. 2001, 24 (2): 126-151.PubMed
33.
Glenton C, Underland V, Kho M, Pennick V, Oxman AD: Summaries of findings, descriptions of interventions, and information about adverse effects would make reviews more informative. J Clin Epidemiol. 2006, 59 (8): 770-778. 10.1016/j.jclinepi.2005.12.011.PubMed
34.
Dohoo I, Stryhn H, Sanchez J: Evaluation of underlying risk as a source of heterogeneity in meta-analyses: a simulation study of Bayesian and frequentist implementations of three models. Prev Vet Med. 2007, 81 (1–3): 38-55.PubMed
35.
Hall JA, Rosenthal R: Interpreting and evaluating meta-analysis. Eval Health Prof. 1995, 18 (4): 393-407. 10.1177/016327879501800404.PubMed
36.
Gerbarg ZB, Horwitz RI: Resolving conflicting clinical trials: guidelines for meta-analysis. J Clin Epidemiol. 1988, 41 (5): 503-509. 10.1016/0895-4356(88)90053-4.PubMed
37.
DerSimonian R, Laird N: Meta-analysis in clinical trials. Control Clin Trials. 1986, 7 (3): 177-188. 10.1016/0197-2456(86)90046-2.PubMed
38.
St-Pierre NR: Invited review: Integrating quantitative findings from multiple studies using mixed model methodology. J Dairy Sci. 2001, 84 (4): 741-755. 10.3168/jds.S0022-0302(01)74530-4.PubMed
39.
Cook DJ, Sackett DL, Spitzer WO: Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis. J Clin Epidemiol. 1995, 48 (1): 167-171. 10.1016/0895-4356(94)00172-M.PubMed
40.
Berlin JA, Santanna J, Schmid CH, Szczech LA, Feldman HI: Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Stat Med. 2002, 21 (3): 371-387. 10.1002/sim.1023.PubMed
41.
Walter SD: Variation in baseline risk as an explanation of heterogeneity in meta-analysis. Stat Med. 1997, 16 (24): 2883-2900. 10.1002/(SICI)1097-0258(19971230)16:24<2883::AID-SIM825>3.0.CO;2-B.PubMed
42.
Cheung MW: A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychol Methods. 2008, 13 (3): 182-202.PubMed
43.
Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F: Systematic reviews of trials and other studies. Health Technol Assess. 1998, 2 (19): 1-276.PubMed
44.
Rosenthal R, DiMatteo MR: Meta-analysis: recent developments in quantitative methods for literature reviews. Annu Rev Psychol. 2001, 52: 59-82. 10.1146/annurev.psych.52.1.59.PubMed
45.
Song F: Exploring heterogeneity in meta-analysis: is the L'Abbe plot useful?. J Clin Epidemiol. 1999, 52 (8): 725-730. 10.1016/S0895-4356(99)00066-9.PubMed
46.
Reade MC, Delaney A, Bailey MJ, Angus DC: Bench-to-bedside review: Avoiding pitfalls in critical care meta-analysis–funnel plots, risk estimates, types of heterogeneity, baseline risk and the ecologic fallacy. Crit Care. 2008, 12 (4): 220-10.1186/cc6941.PubMedPubMedCentral
47.
Xu H, Platt RW, Luo ZC, Wei S, Fraser WD: Exploring heterogeneity in meta-analyses: needs, resources and challenges. Paediatr Perinat Epidemiol. 2008, 22 (Suppl 1): 18-28.PubMed
48.
Olkin I: Diagnostic statistical procedures in medical meta-analyses. Stat Med. 1999, 18 (17–18): 2331-2341.PubMed
49.
Sterne JA, Egger M, Smith GD: Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ. 2001, 323 (7304): 101-105. 10.1136/bmj.323.7304.101.PubMedPubMedCentral
50.
Lipsey MW, Wilson DB: The way in which intervention studies have "personality" and why it is important to meta-analysis. Eval Health Prof. 2001, 24 (3): 236-254.PubMed
51.
Moher D, Jadad AR, Klassen TP: Guides for reading and interpreting systematic reviews: III. How did the authors synthesize the data and make their conclusions?. Arch Pediatr Adolesc Med. 1998, 152 (9): 915-920.PubMed
52.
Schmid JE, Koch GG, LaVange LM: An overview of statistical issues and methods of meta-analysis. J Biopharm Stat. 1991, 1 (1): 103-120. 10.1080/10543409108835008.PubMed
53.
Berlin JA: Invited commentary: benefits of heterogeneity in meta-analysis of data from epidemiologic studies. Am J Epidemiol. 1995, 142 (4): 383-387.PubMed
54.
Malling HJ, Thomsen AB, Andersen JS: Heterogeneity can impair the results of Cochrane meta-analyses despite accordance with statistical guidelines. Allergy. 2008, 63 (12): 1643-1645. 10.1111/j.1398-9995.2008.01908.x.PubMed
55.
Bravata DM, Shojania KG, Olkin I, Raveh A: CoPlot: a tool for visualizing multivariate data in medicine. Stat Med. 2008, 27 (12): 2234-2247. 10.1002/sim.3078.PubMed
56.
Baujat B, Mahe C, Pignon JP, Hill C: A graphical method for exploring heterogeneity in meta-analyses: application to a meta-analysis of 65 trials. Stat Med. 2002, 21 (18): 2641-2652. 10.1002/sim.1221.PubMed
57.
Higgins JP, Whitehead A: Borrowing strength from external trials in a meta-analysis. Stat Med. 1996, 15 (24): 2733-2749. 10.1002/(SICI)1097-0258(19961230)15:24<2733::AID-SIM562>3.0.CO;2-0.PubMed
58.
Michiels S, Baujat B, Mahe C, Sargent DJ, Pignon JP: Random effects survival models gave a better understanding of heterogeneity in individual patient data meta-analyses. J Clin Epidemiol. 2005, 58 (3): 238-245. 10.1016/j.jclinepi.2004.08.013.PubMed
59.
Thompson SG, Sharp SJ: Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999, 18 (20): 2693-2708. 10.1002/(SICI)1097-0258(19991030)18:20<2693::AID-SIM235>3.0.CO;2-V.PubMed
60.
Smith CT, Williamson PR, Marson AG: Investigating heterogeneity in an individual patient data meta-analysis of time to event outcomes. Stat Med. 2005, 24 (9): 1307-1319. 10.1002/sim.2050.PubMed
61.
Simmonds MC, Higgins JP: Covariate heterogeneity in meta-analysis: criteria for deciding between meta-regression and individual patient data. Stat Med. 2007, 26 (15): 2982-2999. 10.1002/sim.2768.PubMed
62.
Thompson SG, Higgins JP: How should meta-regression analyses be undertaken and interpreted?. Stat Med. 2002, 21 (11): 1559-1573. 10.1002/sim.1187.PubMed
63.
Thompson SG, Smith TC, Sharp SJ: Investigating underlying risk as a source of heterogeneity in meta-analysis. Stat Med. 1997, 16 (23): 2741-2758. 10.1002/(SICI)1097-0258(19971215)16:23<2741::AID-SIM703>3.0.CO;2-0.PubMed
64.
Frost C, Clarke R, Beacon H: Use of hierarchical models for meta-analysis: experience in the metabolic ward studies of diet and blood cholesterol. Stat Med. 1999, 18 (13): 1657-1676. 10.1002/(SICI)1097-0258(19990715)18:13<1657::AID-SIM155>3.0.CO;2-M.PubMed
65.
Naylor CD: Two cheers for meta-analysis: problems and opportunities in aggregating results of clinical trials. Cmaj. 1988, 138 (10): 891-895.PubMedPubMedCentral
66.
Schmid CH, Stark PC, Berlin JA, Landais P, Lau J: Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. J Clin Epidemiol. 2004, 57 (7): 683-697. 10.1016/j.jclinepi.2003.12.001.PubMed
67.
Higgins JP, Whitehead A, Turner RM, Omar RZ, Thompson SG: Meta-analysis of continuous outcome data from individual patients. Stat Med. 2001, 20 (15): 2219-2241. 10.1002/sim.918.PubMed
68.
Berkey CS, Anderson JJ, Hoaglin DC: Multiple-outcome meta-analysis of clinical trials. Stat Med. 1996, 15 (5): 537-557. 10.1002/(SICI)1097-0258(19960315)15:5<537::AID-SIM176>3.0.CO;2-S.PubMed
69.
Thompson SG, Turner RM, Warn DE: Multilevel models for meta-analysis, and their application to absolute risk differences. Stat Methods Med Res. 2001, 10 (6): 375-392. 10.1191/096228001682157616.PubMed
70.
Berkey CS, Hoaglin DC, Mosteller F, Colditz GA: A random-effects regression model for meta-analysis. Stat Med. 1995, 14 (4): 395-411. 10.1002/sim.4780140406.PubMed
71.
Warn DE, Thompson SG, Spiegelhalter DJ: Bayesian random effects meta-analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales. Stat Med. 2002, 21 (11): 1601-1623. 10.1002/sim.1189.PubMed
72.
Nixon RM, Bansback N, Brennan A: Using mixed treatment comparisons and meta-regression to perform indirect comparisons to estimate the efficacy of biologic treatments in rheumatoid arthritis. Stat Med. 2007, 26 (6): 1237-1254. 10.1002/sim.2624.PubMed
73.
Koopman L, van der Heijden GJ, Glasziou PP, Grobbee DE, Rovers MM: A systematic review of analytical methods used to study subgroups in (individual patient data) meta-analyses. J Clin Epidemiol. 2007, 60 (10): 1002-1009.PubMed
74.
Riley RD, Lambert PC, Staessen JA, et al: Meta-analysis of continuous outcomes combining individual patient data and aggregate data. Stat Med. 2008, 27 (11): 1870-1893. 10.1002/sim.3165.PubMed
75.
Thompson SG, Higgins JP: Treating individuals 4: can meta-analysis help target interventions at individuals most likely to benefit?. Lancet. 2005, 365 (9456): 341-346.PubMed
76.
Trikalinos TA, Ioannidis JP: Predictive modeling and heterogeneity of baseline risk in meta-analysis of individual patient data. J Clin Epidemiol. 2001, 54 (3): 245-252. 10.1016/S0895-4356(00)00311-5.PubMed
77.
Knapp G, Hartung J: Improved tests for a random effects meta-regression with a single covariate. Stat Med. 2003, 22 (17): 2693-2710. 10.1002/sim.1482.PubMed
78.
van Houwelingen HC, Arends LR, Stijnen T: Advanced methods in meta-analysis: multivariate approach and meta-regression. Stat Med. 2002, 21 (4): 589-624. 10.1002/sim.1040.PubMed
79.
Sharp SJ, Thompson SG: Analysing the relationship between treatment effect and underlying risk in meta-analysis: comparison and development of approaches. Stat Med. 2000, 19 (23): 3251-3274. 10.1002/1097-0258(20001215)19:23<3251::AID-SIM625>3.0.CO;2-2.PubMed
80.
Ghidey W, Lesaffre E, Stijnen T: Semi-parametric modelling of the distribution of the baseline risk in meta-analysis. Stat Med. 2007, 26 (30): 5434-5444. 10.1002/sim.3066.PubMed
81.
Cook RJ, Walter SD: A logistic model for trend in 2 x 2 x kappa tables with applications to meta-analyses. Biometrics. 1997, 53 (1): 352-357. 10.2307/2533120.PubMed
82.
Chang BH, Waternaux C, Lipsitz S: Meta-analysis of binary data: which within study variance estimate to use?. Stat Med. 2001, 20 (13): 1947-1956. 10.1002/sim.823.PubMed
83.
Davey Smith G, Egger M, Phillips AN: Meta-analysis. Beyond the grand mean?. BMJ. 1997, 315 (7122): 1610-1614. 10.1136/bmj.315.7122.1610.PubMedPubMedCentral
84.
Sidik K, Jonkman JN: A note on variance estimation in random effects meta-regression. J Pharm Stat. 2005, 15: 823-838.
85.
Sutton A: Recent development in meta-analysis. Stat Med. 2008, 27: 625-650. 10.1002/sim.2934.PubMed
86.
Bagnardi V, Quatto P, Corrao G: Flexible meta-regression functions for modelling aggregate dose-response data, with an application to alcohol and mortality. Am J Epidemiol. 2004, 159 (11): 1077-1086. 10.1093/aje/kwh142.PubMed
87.
Higgins JP, Thompson SG, Deeks JJ, Altman DG: Measuring inconsistency in meta-analyses. BMJ. 2007, 327: 557-560.
88.
Ioannidis JP: Interpretation of test of heterogeneity and bias in meta-analysis. J Eval Clin Pract. 2008, 14: 951-957. 10.1111/j.1365-2753.2008.00986.x.PubMed
89.
Glasziou PP, Sanders SL: Investigating causes of heterogeneity in systematic reviews. Stat Med. 2002, 21: 1503-11. 10.1002/sim.1183.PubMed
90.
Hatala R, Wyer P, Guyatt G, for the Evidence-Based Medicine Teaching Tips Working Group: Tips for learners of evidence-based medicine: 4. Assessing heterogeneity of primary studies in systematic reviews and whether to combine their results. CMAJ. 2005, 172 (5): 661-665.PubMedPubMedCentral
91.
Bailey KR: Inter-study differences: How should they influence the interpretation and analysis of results?. Stat Med. 1987, 6: 351-358. 10.1002/sim.4780060327.PubMed
92.
Khalid S, Khan RK, Kleijnen J, Antes G: Systematic Reviews to Support Evidence-based Medicine: How to Apply Findings of Health-Care Research. 2003, London: Royal Society of Medicine Press Ltd
93.
Sutton AJ, Jones DR, Sheldon TA, Song F: Methods for Meta-analysis in Medical Research. 2000, Chichester: John Wiley & Sons, Ltd
94.
Littell JC, Corcoran J, Pillai VK: Systematic Reviews and Meta-Analysis. 2008, Oxford: Oxford University Press
95.
Whitehead A: Meta-Analysis of Controlled Clinical Trials. 2002, Chichester: John Wiley & Sons, Ltd
96.
Furlan AD, Pennick V, Bombarider C, van Tulder M, from the Editorial Board of the Cochrane Back Review Group: 2009 Updated method guidelines for systematic reviews in the Cochrane back review group. Spine. 2009, 34 (18): 1929-1941. 10.1097/BRS.0b013e3181b1c99f.PubMed
97.
National Health and Medical Research Council: How to Review the Evidence: Systematic Identification and Review of the Scientific Literature. 2000, Australia: Commonwealth of Australia
98.
Centre for Reviews and Dissemination: CRD’s Guidance for Undertaking Reviews in Health Care. 2009, York: CRD
99.
Oxman AD, Guyatt GH: A consumer's guide to subgroup analyses. Ann Intern Med. 1992, 116 (1): 78-84.PubMed
100.
101.
Imperiale TF: Meta-analysis: when and how. Hepatology. 1999, 29 (6 Suppl): 26S-31S.PubMed
102.
Shekelle PG, Morton SC: Principles of metaanalysis. J Rheumatol. 2000, 27 (1): 251-252. discussion 252-53PubMed
103.
Nagin DS, Odgers CL: Group-based trajectory modeling in clinical research. Annu Rev Clin Pscyhol. 2010, 6: 109-138. 10.1146/annurev.clinpsy.121208.131413.
104.
Virgile G, Conto AA, Moja L, Gensini GL, Gusinu R: Heterogeneity and meta-analyses: do study results truly differ?. Intern Emerg Med. 2009, 4: 423-427. 10.1007/s11739-009-0296-6.
105.
Skipka G, Bender R: Intervention effects in the case of heterogeneity between three subgroups: Assessment within the framework of systematic reviews. Methods Inf Med. 2010, 49: 613-617. 10.3414/ME09-02-0054.PubMed
106.
Groenwold RHH, Rovers MM, Lubsen J, van der Heijden JMG: Subgroup effects despite homogenous heterogeneity test results. BMC Medical Research Methodology. 2010, 10: 43-10.1186/1471-2288-10-43.PubMedPubMedCentral
107.
Lockwood CM, DeFrancesco CA, Elliot DL, Beresford SAA, Toobert DJ: Mediation analyses: Applications in nutrition research and reading the literature. J Am Diet Assoc. 2010, 110: 753-763. 10.1016/j.jada.2010.02.005.PubMedPubMedCentral
108.
Baker W, White M, Cappelleri JC, Kluger J, Colman CI: Understanding heterogeneity in meta-analysis: the role of meta-regression. Int J Clin Pract. 2009, 63 (10): 1426-1434. 10.1111/j.1742-1241.2009.02168.x.PubMed
109.
Jones AP, Riley RD, Williamson PR, Whitehead A: Meta-analysis of individual patient data versus aggregate data from longitudinal clinical trials. Clin Trials. 2009, 6: 16-27. 10.1177/1740774508100984.PubMed
110.
Hemming K, Hutton JL, Maguire MJ, Marson AG: Meta-regression with partial information on summary trial or patient characteristics. Stat Med. 2008, 29: 1312-1324.
111.
Salanti G, Marinho V, Higgins JPT: A case study of multiple-treatments meta-analysis demonstrates covariates should be considered. J Clin Epidemiol. 2009, 62: 857-864. 10.1016/j.jclinepi.2008.10.001.PubMed
112.
Glasziou P, Chalmers I, Altman DG, Bastian H, Boutron I, Brice A, et al: Taking healthcare interventions from trial to practice. BMJ. 2010, 341: c3852-10.1136/bmj.c3852.PubMed
113.
Shadish WR: Meta-analysis and the exploration of causal mediating processes: A primer of examples, methods, and issues. Psychol Methods. 1996, 1: 47-65.
114.
Borenstein MA, Hedges LV, Higgins JPT, Rothstein HR: Introduction to Meta-Analysis. 2009, New Jersey: John Wiley and Sons
115.
116.
McIntosh MW: The population risk as an explanatory variable in research synthesis of clinical trials. Stats Med. 1996, 15: 1713-1728. 10.1002/(SICI)1097-0258(19960830)15:16<1713::AID-SIM331>3.0.CO;2-D.
117.
Boutitie F, Gueyffier F, Pocock SJ, Biossel JP: Assessing treatment-time interaction in clinical trials with time to event data: A meta-analysis of hypertension trials. Stat Med. 1998, 17: 2883-2903. 10.1002/(SICI)1097-0258(19981230)17:24<2883::AID-SIM900>3.0.CO;2-L.PubMed
118.
Booth A, Clarke M, Ghersi D, MOher D, Petticrew M, Stewart L: An international registry of systematic-review protocols. Lancet. 2011, 377 (9760): 108-109. 10.1016/S0140-6736(10)60903-8.PubMed
119.
Concato J, Peduzzi P, Holfold TR, et al: Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol. 1995, 48: 1495-1501. 10.1016/0895-4356(95)00510-2.PubMed
120.
Peduzzi P, Concato J, Feinstein AR, et al: Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995, 48: 1503-1510. 10.1016/0895-4356(95)00048-8.PubMed
121.
Peduzzi P, Concato J, Kemper E, et al: A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996, 49: 1373-1379. 10.1016/S0895-4356(96)00236-3.PubMed
122.
Harrell FE: Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. 2001, New York: Springer
123.
Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR: Bayesian methods in health technology assessment: A review. Health Technol Assess. 2000, 4: 1-130.PubMed
124.
Smith CT, Williamson PR, Marson AG: An overview of methods and empirical comparison of aggregate data and individual patient data results for investigating heterogeneity in meta-analysis to time-to-event data. J Eval Clin Pract. 2002, 55: 86-94.
125.
Lambert PC, Sutton AJ, Jones ADR: A comparison of patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol. 2002, 55: 86-94. 10.1016/S0895-4356(01)00414-0.PubMed
Metadata
Title
Investigating clinical heterogeneity in systematic reviews: a methodologic review of guidance in the literature
Authors
Joel J Gagnier
David Moher
Heather Boon
Joseph Beyene
Claire Bombardier
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2012
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-12-111

Other articles of this Issue 1/2012

BMC Medical Research Methodology 1/2012 Go to the issue

Research article

Constructing episodes of inpatient care: data infrastructure for population-based research

Research article

Investigating linkage rates among probabilistically linked birth and hospitalization records

Research article

The association between survey timing and patient-reported experiences with hospitals: results of a national postal survey

Research article

Weight change in control group participants in behavioural weight loss interventions: a systematic review and meta-regression study

Correspondence

Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers

Research article

A review of the reporting and handling of missing data in cohort studies with repeated assessment of exposure measures