Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2007

Open Access 01-12-2007 | Research article

Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews

Authors: Beverley J Shea, Jeremy M Grimshaw, George A Wells, Maarten Boers, Neil Andersson, Candyce Hamel, Ashley C Porter, Peter Tugwell, David Moher, Lex M Bouter

Published in: BMC Medical Research Methodology | Issue 1/2007

Login to get access

Abstract

Background

Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus.

Methods

A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity.

Results

The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity.

Conclusion

A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.
Appendix
Available only for authorised users
Literature
1.
Davidoff F, Haynes B, Sackett D, Smith R: Evidence-based medicine: a new journal to help doctors identify the information they need. BMJ. 1995, 310: 1085-6.CrossRefPubMedPubMedCentral
2.
Lau J, Ioannidis JPA, Schmid CH: Summing up evidence: one answer is not always enough. Lancet. 1998, 351: 123-127. 10.1016/S0140-6736(97)08468-7.CrossRefPubMed
3.
4.
Moher D, Soeken K, Sampson M, Campbell K, Ben Perot L, Berman B: Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine. BMC Pediatr. 2002, 2 (2):
5.
Jadad A, Moher M, Browman G, Booker L, Sigouin C, Fuentes M: Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ. 2000, 320: 537-540. 10.1136/bmj.320.7234.537.CrossRefPubMedPubMedCentral
6.
7.
Sacks H, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC: Meta-analyses of randomized controlled trials. N Engl J Med. 1987, 316 (8): 450-455.CrossRefPubMed
8.
Shea B, Dubé C, Moher D: Assessing the quality of reports of systematic reviews: the QUOROM statement compared to other tools. Systematic Reviews in Health Care: Meta-analysis in context. Edited by: Egger M, Smith GD, Altman DG. 2001, London: BMJ books, 122-139.CrossRef
9.
The Cochrane Library. 2004, Chichester, UK: John Wiley & Sons Ltd, 3:
10.
Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G: Language bias in randomised controlled trials published in English and German. Lancet. 1997, 350: 326-329. 10.1016/S0140-6736(97)02419-7.CrossRefPubMed
11.
Moher D, Pham B, Klassen T, Schulz K, Berlin J, Jadad A, Liberati A: What contributions do languages other than English make to the results of meta-analyses?. Journal of Clinical Epidemiology. 2000, 53: 964-972. 10.1016/S0895-4356(00)00188-8.CrossRefPubMed
12.
13.
Dickersin K: The existence of publication bias and risk factors for its occurrence. JAMA. 1990, 263: 1385-1389. 10.1001/jama.263.10.1385.CrossRefPubMed
14.
Dickersin K: How important is publication bias? A synthesis of available data. AIDS Educ Prev. 1997, 9: 15-21.PubMed
15.
Phillips C: Publication bias in situ. BMC Med Res Method. 2004, 4: 20-10.1186/1471-2288-4-20.CrossRef
16.
Pham B, Platt R, McAuley L, Sampson M, Klassen T, Moher D: Detecting and minimizing publication bias. A systematic review of methods. Technical Report. 2000, Thomas C. Chalmers Centre for Systematic Reviews, Ottawa, Canada
17.
18.
Sterne JAC, Gavaghan D, Egger M: Publication and related bias in meta-analysis – power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology. 2000, 53: 1119-1129. 10.1016/S0895-4356(00)00242-0.CrossRefPubMed
19.
Sutton A, Duval S, Tweedie R, Abrams K, Jones D: Empirical assessment of effect of publication bias on meta-analyses. BMJ. 2000, 320: 1574-1577. 10.1136/bmj.320.7249.1574.CrossRefPubMedPubMedCentral
20.
Hopewell S, McDonald S, Clarke M, Egger M: Grey literature in meta-analyses of randomized trials of health care interventions. The Cochrane Library. 2004, Chichester, UK: John Wiley & Sons, Ltd, 1:
21.
McAuley L, Pham B, Tugwell P, Moher D: Does the inclusion of grey literature influence the estimates of intervention effectiveness reported in meta-analyses?. Lancet. 2000, 356: 1228-1231. 10.1016/S0140-6736(00)02786-0.CrossRefPubMed
22.
Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, Pham B, Klassen TP: Assessing the quality of randomized controlledtrials: implications for the conduct of meta-analyses. Health Technol Assess. 1999, 3 (12): i-iv. 1–98PubMed
23.
Shea B: Assessing the quality of reporting meta-analyses of randomized controlled trials. MSc thesis. 1999, University of Ottawa, Department of Epidemiology and Community Medicine
24.
Downs S, Black N: The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998, 52: 377-384.CrossRefPubMedPubMedCentral
25.
Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ: Assessing the quality of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996, 17: 1-12. 10.1016/0197-2456(95)00134-4.CrossRefPubMed
26.
Verhagen AP, de Vet HCW, de Bie RA, Kessels AGH, Boers M, Bouter LM, Knipschild PG: The Delphi list: a consensus list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epid. 1998, 51: 1235-41. 10.1016/S0895-4356(98)00131-0.CrossRef
27.
Cluzeau FA: Development and application of an appraisal instrument for assessing the methodological quality of clinical practice guidelines in the United Kingdom. 2001, London, England: St.George's Hospital Medical School, University of London
28.
Kline P: An Easy Guide to factor Analysis. 1994, London: Routledge
29.
Norman GR, Streiner DL: Biostatistics: The Bare Essentials. 2000, St. Louis: Mosby, 2
30.
Streiner DL, Norman GR: Health Measurement Scales: A Practical Guide to their Development and Use. 2003, UK: Oxford University Press, 3
31.
McDowell I, Jenkinson C: Development standards for health measures. Journal of health Services Research and Policy. 1996, 1 (4): 238-246.PubMed
32.
33.
34.
Rothstein HR, Sutton AJ, Borenstein M, eds: Publication bias in meta-analysis: prevention, assessment and adjustments. 2005, Sussex: John Wiley and Sons
Metadata
Title
Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews
Authors
Beverley J Shea
Jeremy M Grimshaw
George A Wells
Maarten Boers
Neil Andersson
Candyce Hamel
Ashley C Porter
Peter Tugwell
David Moher
Lex M Bouter
Publication date
01-12-2007
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2007
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-7-10

Other articles of this Issue 1/2007

BMC Medical Research Methodology 1/2007 Go to the issue

Correspondence

A systematic comparison of software dedicated to meta-analysis of causal studies

Software

Developing algorithms for healthcare insurers to systematically monitor surgical site infection rates

Research article

Predictors of locating women six to eight years after contact: internet resources at recruitment may help to improve response rates in longitudinal research

Debate

Pragmatic trials in primary care. Methodological challenges and solutions demonstrated by the DIAMOND-study

Research article

Questionnaire discrimination: (re)-introducing coefficient δ

Research article

Do intrauterine or genetic influences explain the foetal origins of chronic disease? A novel experimental method for disentangling effects