Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2009

Open Access 01-12-2009 | Research article

Evidence synthesis as the key to more coherent and efficient research

Authors: Alexander J Sutton, Nicola J Cooper, David R Jones

Published in: BMC Medical Research Methodology | Issue 1/2009

Login to get access

Abstract

Background

Systematic review and meta-analysis currently underpin much of evidence-based medicine. Such methodologies bring order to previous research, but future research planning remains relatively incoherent and inefficient.

Methods

To outline a framework for evaluation of health interventions, aimed at increasing coherence and efficiency through i) making better use of information contained within the existing evidence-base when designing future studies; and ii) maximising the information available and thus potentially reducing the need for future studies.

Results

The framework presented insists that an up-to-date meta-analysis of existing randomised controlled trials (RCTs) should always be considered before future trials are conducted. Such a meta-analysis should inform critical design issues such as sample size determination. The contexts in which the use of individual patient data meta-analysis and mixed treatment comparisons modelling may be beneficial before further RCTs are conducted are considered. Consideration should also be given to how any newly planned RCTs would contribute to the totality of evidence through its incorporation into an updated meta-analysis. We illustrate how new RCTs can have very low power to change inferences of an existing meta-analysis, particularly when between study heterogeneity is taken into consideration.

Conclusion

While the collation of existing evidence as the basis for clinical practice is now routine, a more coherent and efficient approach to planning future RCTs to strengthen the evidence base needs to be developed. The framework presented is a proposal for how this situation can be improved.
Appendix
Available only for authorised users
Literature
2.
go back to reference Higgins JPT, Green S, editors: Cochrane Handbook for Systematic Reviews of Interventions 4.2.5. The Cochrane Library. 2005, Chichester, UK: John Wiley & Sons, Ltd, 3 Higgins JPT, Green S, editors: Cochrane Handbook for Systematic Reviews of Interventions 4.2.5. The Cochrane Library. 2005, Chichester, UK: John Wiley & Sons, Ltd, 3
3.
go back to reference Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, et al: Improving the quality of reporting of meta-analysis of randomised controlled trials: the QUOROM statement. Onkologie. 2000, 23 (6): 597-602.PubMed Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, et al: Improving the quality of reporting of meta-analysis of randomised controlled trials: the QUOROM statement. Onkologie. 2000, 23 (6): 597-602.PubMed
5.
go back to reference Clarke L, Clarke M, Clarke T: How useful are Cochrane reviews in identifying research needs?. Journal of Health Services Research & Policy. 2007, 12: 101-103.CrossRef Clarke L, Clarke M, Clarke T: How useful are Cochrane reviews in identifying research needs?. Journal of Health Services Research & Policy. 2007, 12: 101-103.CrossRef
6.
go back to reference Olkin I: Statistical and theoretical considerations in meta-analysis. J Clin Epidemiol. 1995, 48: 133-146.CrossRefPubMed Olkin I: Statistical and theoretical considerations in meta-analysis. J Clin Epidemiol. 1995, 48: 133-146.CrossRefPubMed
7.
go back to reference Sutton AJ, Cooper NJ, Jones DR, Lambert PC, Thompson JR, et al: Evidence-based sample size calculations based upon updated meta-analysis. Stat Med. 2007, 26: 2479-2500.CrossRefPubMed Sutton AJ, Cooper NJ, Jones DR, Lambert PC, Thompson JR, et al: Evidence-based sample size calculations based upon updated meta-analysis. Stat Med. 2007, 26: 2479-2500.CrossRefPubMed
8.
go back to reference Shojania KG, Sampson M, Ansari MT, Jun J, Doucette S, et al: How quickly do systematic reviews go out of date? A survival analysis. Annals of Internal Medicine. 2007, 147: 224-233.CrossRefPubMed Shojania KG, Sampson M, Ansari MT, Jun J, Doucette S, et al: How quickly do systematic reviews go out of date? A survival analysis. Annals of Internal Medicine. 2007, 147: 224-233.CrossRefPubMed
9.
go back to reference Stewart LA, Clarke MJ: Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Stat Med. 1995, 14: 2057-2079.CrossRefPubMed Stewart LA, Clarke MJ: Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Stat Med. 1995, 14: 2057-2079.CrossRefPubMed
10.
go back to reference Lambert P, Sutton AJ, Abrams KR, Jones DR: A comparison of summary patient level covariates in meta-regression with individual patient data meta-analyses. Journal of Clinical Epidemiology. 2002, 55: 86-94.CrossRefPubMed Lambert P, Sutton AJ, Abrams KR, Jones DR: A comparison of summary patient level covariates in meta-regression with individual patient data meta-analyses. Journal of Clinical Epidemiology. 2002, 55: 86-94.CrossRefPubMed
11.
go back to reference Simmonds MC, Higgins JPT: Covariate heterogeneity in meta-analysis: criteria for deciding between meta-regression and individual patient data. Statistics in Medicine. 2007, 26: 2982-2999.CrossRefPubMed Simmonds MC, Higgins JPT: Covariate heterogeneity in meta-analysis: criteria for deciding between meta-regression and individual patient data. Statistics in Medicine. 2007, 26: 2982-2999.CrossRefPubMed
12.
go back to reference Sutton AJ, Kendrick D, Coupland CAC: Meta-analysis of individual- and aggregate-level data. Statistics in Medicine. 2008, 27: 651-669.CrossRefPubMed Sutton AJ, Kendrick D, Coupland CAC: Meta-analysis of individual- and aggregate-level data. Statistics in Medicine. 2008, 27: 651-669.CrossRefPubMed
13.
go back to reference Kendrick D, Coupland C, Mulvaney C, Simpson J, Smith SJ, et al: Home safety education and provision of safety equipment for injury prevention. Cochrane Database of Systematic Reviews. 2007, Wiley, Chichester, CD005014- Kendrick D, Coupland C, Mulvaney C, Simpson J, Smith SJ, et al: Home safety education and provision of safety equipment for injury prevention. Cochrane Database of Systematic Reviews. 2007, Wiley, Chichester, CD005014-
14.
go back to reference Counsell CE, Clarke MJ, Slattery J, Sandercock PAG: The miracle of DICE therapy for acute stroke: Fact or fictional product of subgroup analysis?. BMJ. 1994, 309 (6970): 1677-1681.CrossRefPubMedPubMedCentral Counsell CE, Clarke MJ, Slattery J, Sandercock PAG: The miracle of DICE therapy for acute stroke: Fact or fictional product of subgroup analysis?. BMJ. 1994, 309 (6970): 1677-1681.CrossRefPubMedPubMedCentral
15.
go back to reference Lumley T: Network meta-analysis for indirect treatment comparisons. Statistics in Medicine. 2002, 21: 2313-2324.CrossRefPubMed Lumley T: Network meta-analysis for indirect treatment comparisons. Statistics in Medicine. 2002, 21: 2313-2324.CrossRefPubMed
16.
go back to reference Lu G, Ades AE: Combination of direct and indirect evidence in mixed treatment comparisons. Statistics in Medicine. 2004, 23: 3105-3124.CrossRefPubMed Lu G, Ades AE: Combination of direct and indirect evidence in mixed treatment comparisons. Statistics in Medicine. 2004, 23: 3105-3124.CrossRefPubMed
17.
go back to reference Lu G, Ades AE: Assessing evidence inconsistency in mixed treatment comparisons. Journal of the American Statistical Association. 2006, 101: 447-459.CrossRef Lu G, Ades AE: Assessing evidence inconsistency in mixed treatment comparisons. Journal of the American Statistical Association. 2006, 101: 447-459.CrossRef
18.
19.
go back to reference Cooper NJ, Sutton AJ, Lu G, Khunti K: Mixed comparison of stroke prevention treatments in individuals with non-rheumatic atrial fibrillation. Archives of Internal Medicine. 2006, 166: 1369-1275.CrossRef Cooper NJ, Sutton AJ, Lu G, Khunti K: Mixed comparison of stroke prevention treatments in individuals with non-rheumatic atrial fibrillation. Archives of Internal Medicine. 2006, 166: 1369-1275.CrossRef
20.
go back to reference Ades A, Sutton AJ: Multiple parameter evidence synthesis in epidemiology and medical decision-making: current approaches. Journal of the Royal Statistical Society Series A. 2006, 169: 5-35.CrossRef Ades A, Sutton AJ: Multiple parameter evidence synthesis in epidemiology and medical decision-making: current approaches. Journal of the Royal Statistical Society Series A. 2006, 169: 5-35.CrossRef
21.
go back to reference Buyse M, Molenberghs G, Burzykowski T, Renard D, Geys H: The validation of surrogate endpoints in meta-analyses of randomized experiments. Biostatistics. 2000, 1: 49-67.CrossRefPubMed Buyse M, Molenberghs G, Burzykowski T, Renard D, Geys H: The validation of surrogate endpoints in meta-analyses of randomized experiments. Biostatistics. 2000, 1: 49-67.CrossRefPubMed
22.
go back to reference Ades AE: A chain of evidence with mixed comparisons: models for multi-parameter synthesis and consistency of evidence. Statistics in Medicine. 2003, 22: 2995-3016.CrossRefPubMed Ades AE: A chain of evidence with mixed comparisons: models for multi-parameter synthesis and consistency of evidence. Statistics in Medicine. 2003, 22: 2995-3016.CrossRefPubMed
23.
go back to reference Clarke M, Alderson P, Chalmers I: Discussion sections in reports of controlled trials published in general medical journals. JAMA. 2002, 287: 2799-2801.CrossRefPubMed Clarke M, Alderson P, Chalmers I: Discussion sections in reports of controlled trials published in general medical journals. JAMA. 2002, 287: 2799-2801.CrossRefPubMed
24.
25.
go back to reference Cooper NJ, Jones DR, Sutton AJ: The use of systematic reviews when designing new studies. Clinical Trials. 2005, 2: 260-264.CrossRefPubMed Cooper NJ, Jones DR, Sutton AJ: The use of systematic reviews when designing new studies. Clinical Trials. 2005, 2: 260-264.CrossRefPubMed
26.
go back to reference Sterne JAC, Davey Smith G, Cox DR: Sifting the evidence – what's wrong with significance tests? Another comment on the role of statistical methods. BMJ. 2001, 322: 226-231.CrossRefPubMedPubMedCentral Sterne JAC, Davey Smith G, Cox DR: Sifting the evidence – what's wrong with significance tests? Another comment on the role of statistical methods. BMJ. 2001, 322: 226-231.CrossRefPubMedPubMedCentral
28.
go back to reference Turner RM, Thompson SG, Spiegelhalter DJ: Prior distributions for the intracluster correlation coefficient, based on multiple previous estimates, and their application in cluster randomized trials. Clinical Trials. 2005, 2: 108-118.CrossRefPubMed Turner RM, Thompson SG, Spiegelhalter DJ: Prior distributions for the intracluster correlation coefficient, based on multiple previous estimates, and their application in cluster randomized trials. Clinical Trials. 2005, 2: 108-118.CrossRefPubMed
29.
go back to reference Schulz KF, Grimes DA: Sample size calculations in randomised trials: mandatory and mystical. Lancet. 2005, 365 (9467): 1348-1353.CrossRefPubMed Schulz KF, Grimes DA: Sample size calculations in randomised trials: mandatory and mystical. Lancet. 2005, 365 (9467): 1348-1353.CrossRefPubMed
30.
go back to reference Pogue JM, Yousuf S: Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Controlled Clinical Trials. 1997, 18: 580-593.CrossRefPubMed Pogue JM, Yousuf S: Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Controlled Clinical Trials. 1997, 18: 580-593.CrossRefPubMed
31.
go back to reference Wetterslev J, Thorlund K, Brok J, Gluud C: Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Journal of Clinical Epidemiology. 2008, 61: 64-75.CrossRefPubMed Wetterslev J, Thorlund K, Brok J, Gluud C: Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Journal of Clinical Epidemiology. 2008, 61: 64-75.CrossRefPubMed
32.
go back to reference Muller HH, Schafer H: Adaptive group sequential designs for clinical trials: combining the advantages of adaptive and classical group sequential approaches. Biometrics. 2001, 57: 886-891.CrossRefPubMed Muller HH, Schafer H: Adaptive group sequential designs for clinical trials: combining the advantages of adaptive and classical group sequential approaches. Biometrics. 2001, 57: 886-891.CrossRefPubMed
33.
go back to reference Chow SC, Shao J, Wang H: Sample size calculations in clinical research. Chapman & Hall/CRC Biostatistics. 2003, Boca Raton, Florida, Second Chow SC, Shao J, Wang H: Sample size calculations in clinical research. Chapman & Hall/CRC Biostatistics. 2003, Boca Raton, Florida, Second
35.
go back to reference Thompson SG, Sharp SJ: Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999, 18: 2693-2708.CrossRefPubMed Thompson SG, Sharp SJ: Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999, 18: 2693-2708.CrossRefPubMed
36.
go back to reference Higgins JPT, Spiegelhalter DJ: Being sceptical about meta-analyses: a Bayesian perspective on magnesium trials in myocardial infarction. International Journal of Epidemiology. 2002, 31: 96-104.CrossRefPubMed Higgins JPT, Spiegelhalter DJ: Being sceptical about meta-analyses: a Bayesian perspective on magnesium trials in myocardial infarction. International Journal of Epidemiology. 2002, 31: 96-104.CrossRefPubMed
37.
go back to reference Margitic SE, Morgan TM, Sager MA, Furberg CD: Lessons learned from a prospective meta-analysis. J Am Geriatr Soc. 1995, 43: 435-439.CrossRefPubMed Margitic SE, Morgan TM, Sager MA, Furberg CD: Lessons learned from a prospective meta-analysis. J Am Geriatr Soc. 1995, 43: 435-439.CrossRefPubMed
38.
go back to reference Simes RJ: Prospective meta-analysis of cholesterol-lowering studies: The Prospective Pravastatin Pooling project (PPP) and the Cholesterol Treatment Trialists (CTT) collaboration. Am J Cardiol. 1995, 76 (9): 122C-126C.CrossRefPubMed Simes RJ: Prospective meta-analysis of cholesterol-lowering studies: The Prospective Pravastatin Pooling project (PPP) and the Cholesterol Treatment Trialists (CTT) collaboration. Am J Cardiol. 1995, 76 (9): 122C-126C.CrossRefPubMed
39.
go back to reference Berlin JA, Ghersi D: Preventing publication bias: registries and prospective meta-analysis. Publication bias in meta-analysis: prevention, assessment and adjustments. Edited by: Rothstein HR, Sutton AJ, Borenstein M. 2005, Chichester: Wiley Berlin JA, Ghersi D: Preventing publication bias: registries and prospective meta-analysis. Publication bias in meta-analysis: prevention, assessment and adjustments. Edited by: Rothstein HR, Sutton AJ, Borenstein M. 2005, Chichester: Wiley
40.
go back to reference Schmid CH, Cappelleri JC, Lau J: Bayesian methods to improve sample size approximations. Numerical Computer Methods Part D. Edited by: Johnson ML, Brand L. 2004, NY: Elsevier, 383: 406-427.CrossRef Schmid CH, Cappelleri JC, Lau J: Bayesian methods to improve sample size approximations. Numerical Computer Methods Part D. Edited by: Johnson ML, Brand L. 2004, NY: Elsevier, 383: 406-427.CrossRef
41.
go back to reference Sutton AJ, Song F, Gilbody SM, Abrams KR: Modelling publication bias in meta-analysis: a review. Statistical methods in medical research. 2000, 9: 421-445.CrossRefPubMed Sutton AJ, Song F, Gilbody SM, Abrams KR: Modelling publication bias in meta-analysis: a review. Statistical methods in medical research. 2000, 9: 421-445.CrossRefPubMed
42.
go back to reference Wolpert RL, Mengersen K: Adjusted likelihoods for synthesizing empirical evidence from studies that differ in quality and design: effects of environmental tobacco smoke. Statistical Science. 2004, 19: 450-471.CrossRef Wolpert RL, Mengersen K: Adjusted likelihoods for synthesizing empirical evidence from studies that differ in quality and design: effects of environmental tobacco smoke. Statistical Science. 2004, 19: 450-471.CrossRef
43.
go back to reference Ioannidis JPA, Contopoulos-Ioannidis DG, Lau J: Recursive cumulative meta-analysis: a diagnostic for the evolution of total randomized evidence from group and individual patient data. J Clin Epidemiol. 1999, 52: 281-291.CrossRefPubMed Ioannidis JPA, Contopoulos-Ioannidis DG, Lau J: Recursive cumulative meta-analysis: a diagnostic for the evolution of total randomized evidence from group and individual patient data. J Clin Epidemiol. 1999, 52: 281-291.CrossRefPubMed
44.
go back to reference Ades AE, Lu G, Higgins JPT: The interpretation of random-effects meta-analysis in decision models. Medical Decision Making. 2005, 25: 646-654.CrossRefPubMed Ades AE, Lu G, Higgins JPT: The interpretation of random-effects meta-analysis in decision models. Medical Decision Making. 2005, 25: 646-654.CrossRefPubMed
45.
go back to reference Study CAFA: Double-blinded RCT of warfarin vs. placebo in patients with nonrheumatic atrial fibrillation without prior embolic event. Ann Intern Med. 1991, 115: 818-822.CrossRef Study CAFA: Double-blinded RCT of warfarin vs. placebo in patients with nonrheumatic atrial fibrillation without prior embolic event. Ann Intern Med. 1991, 115: 818-822.CrossRef
46.
go back to reference Henderson WG, Moritz T, Goldman S, Copeland J, Sethi G: Use of cumulative meta-analysis in the design, monitoring, and final analysis of a clinical trial: A case study. Controlled Clin Trials. 1995, 16: 331-341.CrossRefPubMed Henderson WG, Moritz T, Goldman S, Copeland J, Sethi G: Use of cumulative meta-analysis in the design, monitoring, and final analysis of a clinical trial: A case study. Controlled Clin Trials. 1995, 16: 331-341.CrossRefPubMed
47.
go back to reference Prevost TC, Abrams KR, Jones DR: Hierarchical models in generalised synthesis of evidence: an example based on studies of breast cancer screening. Statistics in Medicine. 2000, 19: 3359-3376.CrossRefPubMed Prevost TC, Abrams KR, Jones DR: Hierarchical models in generalised synthesis of evidence: an example based on studies of breast cancer screening. Statistics in Medicine. 2000, 19: 3359-3376.CrossRefPubMed
48.
49.
go back to reference Sutton AJ, Cooper NJ, Abrams KR, Lambert PC, Jones DR: A Bayesian approach to evaluating net clinical beneft allowed for parameter uncertainty. Journal of Clinical Epidemiology. 2005, 58: 26-40.CrossRefPubMed Sutton AJ, Cooper NJ, Abrams KR, Lambert PC, Jones DR: A Bayesian approach to evaluating net clinical beneft allowed for parameter uncertainty. Journal of Clinical Epidemiology. 2005, 58: 26-40.CrossRefPubMed
50.
go back to reference Minelli C, Abrams KR, Sutton AJ, Cooper NJ: Benefits and harms associated with hormone replacement therapy: clinical decision analysis. BMJ. 2004, 328: 371-375.CrossRefPubMedPubMedCentral Minelli C, Abrams KR, Sutton AJ, Cooper NJ: Benefits and harms associated with hormone replacement therapy: clinical decision analysis. BMJ. 2004, 328: 371-375.CrossRefPubMedPubMedCentral
51.
go back to reference Cooper NJ, Sutton AJ, Abrams KR, Turner D, Wailoo A: Comprehensive decision analytical modelling in economic evaluation: A Bayesian approach. Health Economics. 2004, 13: 203-226.CrossRefPubMed Cooper NJ, Sutton AJ, Abrams KR, Turner D, Wailoo A: Comprehensive decision analytical modelling in economic evaluation: A Bayesian approach. Health Economics. 2004, 13: 203-226.CrossRefPubMed
52.
go back to reference Claxton K, Sculpher M, Drummond M: A rational framework for decision making by the National Institute For Clinical Excellence (NICE). Lancet. 2002, 360 (9334): 711-715.CrossRefPubMed Claxton K, Sculpher M, Drummond M: A rational framework for decision making by the National Institute For Clinical Excellence (NICE). Lancet. 2002, 360 (9334): 711-715.CrossRefPubMed
53.
go back to reference Claxton K: Bayesian approaches to the value of information: implications for the regulation of new pharmaceuticals. Health Economics. 1999, 8: 269-274.CrossRefPubMed Claxton K: Bayesian approaches to the value of information: implications for the regulation of new pharmaceuticals. Health Economics. 1999, 8: 269-274.CrossRefPubMed
54.
go back to reference Ades AE, Lu G, Claxton K: Expected value of sample information calculations in medical decision modeling. Medical Decision Making. 2004, 24: 207-227.CrossRefPubMed Ades AE, Lu G, Claxton K: Expected value of sample information calculations in medical decision modeling. Medical Decision Making. 2004, 24: 207-227.CrossRefPubMed
Metadata
Title
Evidence synthesis as the key to more coherent and efficient research
Authors
Alexander J Sutton
Nicola J Cooper
David R Jones
Publication date
01-12-2009
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2009
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-9-29

Other articles of this Issue 1/2009

BMC Medical Research Methodology 1/2009 Go to the issue