Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Health Research Policy and Systems
Published in: Health Research Policy and Systems 1/2009

Open Access 01-12-2009 | Guide

SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review

Authors: Simon Lewin, Andrew D Oxman, John N Lavis, Atle Fretheim

Published in: Health Research Policy and Systems | Special Issue 1/2009

Login to get access

Abstract

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.
The reliability of systematic reviews of the effects of health interventions is variable. Consequently, policymakers and others need to assess how much confidence can be placed in such evidence. The use of systematic and transparent processes to determine such decisions can help to prevent the introduction of errors and bias in these judgements. In this article, we suggest five questions that can be considered when deciding how much confidence to place in the findings of a systematic review of the effects of an intervention. These are: 1. Did the review explicitly address an appropriate policy or management question? 2. Were appropriate criteria used when considering studies for the review? 3. Was the search for relevant studies detailed and reasonably comprehensive? 4. Were assessments of the studies' relevance to the review topic and of their risk of bias reproducible? 5. Were the results similar from study to study?
Appendix
Available only for authorised users
Literature
1.
Lavis JN, Oxman AD, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). Introduction. Health Res Policy Syst. 2009, 7 (Suppl 1): I1-10.1186/1478-4505-7-S1-I1.PubMedCentralCrossRefPubMed
2.
Lavis JN, Posada FB, Haines A, Osei E: Use of research to inform public policymaking. Lancet. 2004, 364: 1615-21. 10.1016/S0140-6736(04)17317-0.CrossRefPubMed
3.
Oxman AD, Lavis JN, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). 1. What is evidence-informed policymaking. Health Res Policy Syst. 2009, 7 (Suppl 1): S1-10.1186/1478-4505-7-S1-S1.PubMedCentralCrossRefPubMed
4.
Oxman AD, Fretheim A, Lavis JN, Lewin S: SUPPORT Tools for evidence-informed health Policymaking (STP). 12. Finding and using research evidence about resource use and costs. Health Res Policy Syst. 2009, 7 (Suppl 1): S12-10.1186/1478-4505-7-S1-S12.PubMedCentralCrossRefPubMed
5.
Renfrew MJ, Craig D, Dyson L, McCormick F, Rice S, King SE, Misso K, Stenhouse E, Williams AF: Breastfeeding promotion for infants in neonatal units: a systematic review and economic analysis. Health Technol Assess. 2009, 13: 1-iv.CrossRefPubMed
6.
Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8: iii-72.CrossRefPubMed
7.
Carlsen B, Glenton C, Pope C: Thou shalt versus thou shalt not: a meta-synthesis of GPs' attitudes to clinical practice guidelines. Br J Gen Pract. 2007, 57: 971-8. 10.3399/096016407782604820.PubMedCentralCrossRefPubMed
8.
Mays N, Pope C, Popay J: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005, 10 (Suppl 1): 6-20. 10.1258/1355819054308576.CrossRefPubMed
9.
Munro SA, Lewin SA, Smith HJ, Engel ME, Fretheim A, Volmink J: Patient adherence to tuberculosis treatment: a systematic review of qualitative research. PLoS Med. 2007, 4: e238-10.1371/journal.pmed.0040238.PubMedCentralCrossRefPubMed
10.
Pound P, Britten N, Morgan M, Yardley L, Pope C, Daker-White G, Campbell R: Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med. 2005, 61: 133-55. 10.1016/j.socscimed.2004.11.063.CrossRefPubMed
11.
Laurant M, Reeves D, Hermens R, Braspenning J, Grol R, Sibbald B: Substitution of doctors by nurses in primary care. Cochrane Database Syst Rev. 2005, 2: CD001271-PubMed
12.
Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009, 2: CD003030-PubMed
13.
14.
Oxman AD, Schunemann HJ, Fretheim A: Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Res Policy Syst. 2006, 4: 20-10.1186/1478-4505-4-20.PubMedCentralCrossRefPubMed
15.
Oxman AD, Glasziou P, Williams JW: What should clinicians do when faced with conflicting recommendations?. BMJ. 2008, 337: a2530-10.1136/bmj.a2530.CrossRefPubMed
16.
Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, Moher D: Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA. 1998, 280: 278-80. 10.1001/jama.280.3.278.CrossRefPubMed
17.
Jorgensen AW, Hilden J, Gotzsche PC: Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ. 2006, 333: 782-10.1136/bmj.38973.444699.0B.PubMedCentralCrossRefPubMed
18.
Dixon E, Hameed M, Sutherland F, Cook DJ, Doig C: Evaluating meta-analyses in the general surgical literature: a critical appraisal. Ann Surg. 2005, 241: 450-9. 10.1097/01.sla.0000154258.30305.df.PubMedCentralCrossRefPubMed
19.
20.
Jadad AR, Moher M, Browman GP, Booker L, Sigouin C, Fuentes M, Stevens R: Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ. 2000, 320: 537-40. 10.1136/bmj.320.7234.537.PubMedCentralCrossRefPubMed
21.
Linde K, Willich SN: How objective are systematic reviews? Differences between reviews on complementary medicine. J R Soc Med. 2003, 96: 17-22. 10.1258/jrsm.96.1.17.PubMedCentralCrossRefPubMed
22.
Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM: Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007, 7: 10-10.1186/1471-2288-7-10.PubMedCentralCrossRefPubMed
23.
24.
Oxman AD, Guyatt GH: Validation of an index of the quality of review articles. J Clin Epidemiol. 1991, 44: 1271-8. 10.1016/0895-4356(91)90160-B.CrossRefPubMed
25.
26.
27.
Higgins JPT, Altman DF: Chapter 8: Assessing risk of bias in included studies. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008). Edited by: Higgins JPT, Green S. 2008, The Cochrane Collaboration, [http://​www.​cochrane-handbook.​org/​]CrossRef
28.
Lavis JN, Oxman AD, Souza NM, Lewin S, Gruen RL, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). 9. Assessing the applicability of the findings of a systematic review. Health Res Policy Syst. 2009, 7 (Suppl 1): S9-10.1186/1478-4505-7-S1-S9.PubMedCentralCrossRefPubMed
29.
Counsell C: Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews. Ann Intern Med. 1997, 127: 380-7.CrossRefPubMed
30.
31.
Meremikwu MM, Donegan S, Esu E: Chemoprophylaxis and intermittent treatment for preventing malaria in children. Cochrane Database Syst Rev. 2008, 2: CD003756-PubMed
32.
33.
34.
Grobler LA, Marais BJ, Mabunda S, Marindi P, Reuter H, Volmink J: Interventions for increasing the proportion of health professionals practicing in underserved communities. Cochrane Database Syst Rev. 2009, 1: CD005314-PubMed
35.
Lefebvre C, Manheimer E, Glanville J, on behalf of the Cochrane Information Retrieval Methods Group: Searching for studies. Cochrane Handbook for systematic reviews of interventions. Version 5.0.1 [updated September 2008]. Edited by: Higgins JPT, Green S. 2008, The Cochrane Collaboration, [http://​www.​cochrane-handbook.​org/​]
36.
Dickersin K, Min YI: Publication bias: the problem that won't go away. Ann N Y Acad Sci. 1993, 703: 135-46. 10.1111/j.1749-6632.1993.tb26343.x.CrossRefPubMed
37.
Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K: Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009, 21 (1): MR000006-
38.
Hopewell S, Clarke M, Stewart L, Tierney J: Time to publication for results of clinical trials. Cochrane Database Syst Rev. 2007, 2: MR000011-PubMed
39.
Lavis JN, Wilson M, Oxman AD, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). 4. Using research evidence to clarify a problem. Health Res Policy Syst. 2009, 7 (Suppl 1): S4-10.1186/1478-4505-7-S1-S4.PubMedCentralCrossRefPubMed
40.
Lavis JN, Wilson MG, Oxman AD, Grimshaw J, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). 5. Using research evidence to frame options to address a problem. Health Res Policy Syst. 2009, 7 (Suppl 1): S5-10.1186/1478-4505-7-S1-S5.PubMedCentralCrossRefPubMed
41.
Katrak P, Bialocerkowski A, Massy-Westropp N, Kumar VS, Grimmer K: A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology. 2004, 4: 22-10.1186/1471-2288-4-22.PubMedCentralCrossRefPubMed
42.
Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S: Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995, 16: 62-73. 10.1016/0197-2456(94)00031-W.CrossRefPubMed
43.
Shepperd S, Doll H, Broad J, Gladman J, Iliffe S, Langhorne P, Richards S, Martin F, Harris R: Early discharge hospital at home. Cochrane Database Syst Rev. 2009, 1: CD000356-PubMed
44.
Lewin SA, Dick J, Pond P, Zwarenstein M, Aja G, van Wyk B, Bosch-Capblanch X, Patrick M: Lay health workers in primary and community health care. Cochrane Database Syst Rev. 2005, 1: CD004015-PubMed
45.
Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD: Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006, 2: CD000259-PubMed
46.
Arblaster L, Lambert M, Entwistle V, Forster M, Fullerton D, Sheldon T, Watt I: A systematic review of the effectiveness of health service interventions aimed at reducing inequalities in health. J Health Serv Res Policy. 1996, 1: 93-103.PubMed
47.
Moher D, Jadad AR, Klassen TP: Guides for reading and interpreting systematic reviews: III. How did the authors synthesize the data and make their conclusions?. Arch Pediatr Adolesc Med. 1998, 152 (9): 915-20.CrossRefPubMed
48.
Oxman AD, Cook DJ, Guyatt GH: Users' guides to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA. 1994, 272: 1367-71. 10.1001/jama.272.17.1367.CrossRefPubMed
49.
Oxman AD, Lavis JN, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). 10. Taking equity into consideration when assessing the findings of a systematic review. Health Res Policy Syst. 2009, 7 (Suppl 1): S10-10.1186/1478-4505-7-S1-S10.PubMedCentralCrossRefPubMed
50.
Oxman AD, Lavis JN, Fretheim A, Lewin S: SUPPORT Tools for evidence-informed health Policymaking (STP). 17. Dealing with insufficient research evidence. Health Res Policy Syst. 2009, 7 (Suppl 1): S17-10.1186/1478-4505-7-S1-S17.PubMedCentralCrossRefPubMed
51.
Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A: Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005, 10: 45-53. 10.1258/1355819052801804.CrossRefPubMed
52.
Noyes J, Popay J, Pearson A, Hannes K, Booth A: Chapter 20: Qualitative research and Cochrane reviews. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008). Edited by: Higgins JPT, Green S. 2008, The Cochrane Collaboration, [http://​www.​cochrane-handbook.​org/​]
53.
Lavis JN: Supporting the Use of Systematic Reviews in Policymaking. PLoS Med. 6 (11): e1000141-10.1371/journal.pmed.1000141.
54.
Jefferson T, Demicheli V, Vale L: Quality of systematic reviews of economic evaluations in health care. JAMA. 2002, 287: 2809-12. 10.1001/jama.287.21.2809.CrossRefPubMed
55.
Siegfried N, Muller M, Deeks JJ, Volmink J: Male circumcision for prevention of heterosexual acquisition of HIV in men. Cochrane Database Syst Rev. 2009, 2: CD003362-PubMed
56.
McGuinness B, Craig D, Bullock R, Passmore P: Statins for the prevention of dementia. Cochrane Database Syst Rev. 2009, 2: CD003160-PubMed
Metadata
Title
SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review
Authors
Simon Lewin
Andrew D Oxman
John N Lavis
Atle Fretheim
Publication date
01-12-2009
Publisher
BioMed Central
Published in
Health Research Policy and Systems / Issue Special Issue 1/2009
Electronic ISSN: 1478-4505
DOI
https://doi.org/10.1186/1478-4505-7-S1-S8

Other articles of this Special Issue 1/2009

Health Research Policy and Systems 1/2009 Go to the issue

Introduction

SUPPORT Tools for evidence-informed health Policymaking (STP)

Guide

SUPPORT Tools for evidence-informed health Policymaking (STP) 12: Finding and using research evidence about resource use and costs

Guide

SUPPORT Tools for Evidence-informed Policymaking in health 18: Planning monitoring and evaluation of policies

Guide

SUPPORT Tools for evidence-informed health Policymaking (STP) 3: Setting priorities for supporting evidence-informed policymaking

Guide

SUPPORT Tools for evidence-informed health Policymaking (STP) 5: Using research evidence to frame options to address a problem

Guide

SUPPORT Tools for evidence-informed health Policymaking (STP) 7: Finding systematic reviews