Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2018

Open Access 01-12-2018 | Research article

Quality assessment of systematic reviews on total hip or knee arthroplasty using mod-AMSTAR

Authors: Xinyu Wu, Huan Sun, Xiaoqin Zhou, Ji Wang, Jing Li

Published in: BMC Medical Research Methodology | Issue 1/2018

Login to get access

Abstract

Background

Increasing numbers of systematic reviews (SRs) on total knee arthroplasty (TKA) and total hip arthroplasty (THA) have been published in recent years, but their quality has been unclear. The purpose of this study is to evaluate the methodological quality of SRs on TKA and THA.

Methods

We searched Ovid-Medline, Ovid-Embase, Cochrane Databases (including HTA, DARE, and CDSR), CBM, CNKI, Wang Fang, and VIP, from January 2014 to December 2015 for THA and TKA. The quality of SRs was assessed using the modified 25-item “Assessment of Multiple Systematic Reviews” (mod-AMSTAR) tool, which was based on the AMSTAR scale. A T-test, nonparametric test, and linear regression were conducted to assess the relationship between bibliographical characteristics and methodological quality.

Results

Sixty-three SRs were included, from which the majority of SRs (50, 79.4%) were conducted in Asia. Only 4 reviews were rated as high quality, and most were weak in providing a priori design (6, 9.5%), not limiting the publication type (8, 13%), providing an excluded primary studies list (4, 6.3%) and reporting support for the included primary studies (1, 1.6%). Reviews published in English journals performed better than did Chinese journals in duplicate data extraction (81.3% vs 46.7%, p = 0.017; 70.8% vs 33.3%, p = 0.009) and providing source of support for the SR (87.5% vs 33.3%, P < 0.001). Reviews published in journals with a higher impact factor were associated with a higher mod-AMSTAR score (regression coefficient: 0.38, 95%CI: 0.11–0.65; P = 0.006).

Conclusion

The methodological quality of the included SRs is far from satisfactory. Authors of SRs should conform to the recommendations outlined in the mod-AMSTAR items. Areas needing improvement were providing a priori design, not limiting the publication type, providing an excluded primary studies list, and reporting conflicts of interest.
Appendix
Available only for authorised users
Literature
1.
2.
Sally Green, Julian PT Higgins, Philip Alderson, etc. 1.2.2 what is a systematic review? In: The Cochrane handbook for systematic reviews of interventions. Version 5.1.0. 2011. http://​handbook-5-1.​cochrane.​org/​. Accessed 9 Aug 2017.
3.
Lau J, Ioannidis JPA, Schmid CH. Summing up evidence: one answer is not always enough. Lancet. 1998;351(9096):123–7.CrossRefPubMed
4.
Sequeira-Byron P, Fedorowicz Z, Jagannath VA, Sharif MO. An AMSTAR assessment of the methodological quality of systematic reviews of oral healthcare interventions published in the journal of applied oral science (JAOS). J Appl Oral Sci. 2011;19(5):440–7.CrossRefPubMedPubMedCentral
5.
Ho RS, Wu X, Yuan J, Liu S, Lai X, Wong SY, Chung VC. Methodological quality of meta-analyses on treatments for chronic obstructive pulmonary disease: a cross-sectional study using the AMSTAR (assessing the methodological quality of systematic reviews) tool. NPJ Primary Care Respiratory Medicine. 2015;25:14102.CrossRefPubMedPubMedCentral
6.
7.
Momeni A, Lee GK, Talley JR. The quality of systematic reviews in hand surgery: an analysis using AMSTAR. Plastic & Reconstructive Surgery. 2013;131(4):831–7.CrossRef
8.
Papageorgiou SN, Papadopoulos MA, Athanasiou AE. Evaluation of methodology and quality characteristics of systematic reviews in orthodontics. Orthod Craniofac Res. 2011;14(3):116–37.CrossRefPubMed
9.
Corbyons K, Han J, Neuberger MM, Dahm P. Methodological quality of systematic reviews published in the urological literature from 1998 to 2012. J Urol. 2015;194(5):1374–9.CrossRefPubMed
10.
Braga LH, Pemberton J, Demaria J, Lorenzo AJ. Methodological concerns and quality appraisal of contemporary systematic reviews and meta-analyses in pediatric urology. J Urol. 2011;186(1):266–71.CrossRefPubMed
11.
12.
13.
Carr AJ, Robertsson O, Graves S, Price AJ, Arden NK, Judge A, Beard DJ. Knee replacement. Lancet. 2012;379(9823):1331–40.CrossRefPubMed
14.
Ethgen O, Bruyere O, Richy F, Dardennes C, Reginster JY. Health-related quality of life in total hip and total knee arthroplasty. A qualitative and systematic review of the literature. Journal of Bone & Joint Surgery - American Volume. 2004;86-A(5):963–74.CrossRef
15.
16.
17.
Wen J, Ren Y, Wang L, Li Y, Liu Y, Zhou M, Liu P, Ye L, Li Y, Tian W. The reporting quality of meta-analyses improves: a random sampling study. J Clin Epidemiol. 2008;61(8):770–5.CrossRefPubMed
18.
Wu XY, Lam VC, Yu YF, Ho RS, Feng Y, Wong CH, Yip BH, Tsoi KK, Wong SY, Chung VC. Epidemiological characteristics and methodological quality of meta-analyses on diabetes mellitus treatment: a systematic review. Eur J Endocrinol. 2016;175(5):353–60.CrossRefPubMed
19.
Sharif MO, Janjua-Sharif FN, Ali H, Ahmed F. Systematic reviews explained: AMSTAR-how to tell the good from the bad and the ugly. Oral Health Dent Manag. 2013;12(1):9–16.PubMed
20.
Shea BJ, Bouter LM, Peterson J, Boers M, Andersson N, Ortiz Z, Ramsay T, Bai A, Shukla VK, Grimshaw JM. External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS One. 2007;2(12):e1350.CrossRefPubMedPubMedCentral
21.
Pollock A, Farmer SE, Brady MC, Langhorne P, Mead GE, Mehrholz J, van WF: Interventions for improving upper limb function after stroke. In: Cochrane Database of Systematic Reviews. John Wiley & Sons, Ltd; 2014.
22.
Kung J, Chiappelli F, Cajulis OO, Avezova R, Kossan G, Chew L, Maida CA. From systematic reviews to clinical recommendations for evidence-based health care: validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. Open Dent J. 2010;4:84–91.PubMedPubMedCentral
23.
Tang W, Hu J, Zhang H, Wu P, He H. Kappa coefficient: a popular measure of rater agreement. Shanghai archives of psychiatry. 2015;27(1):62–7.PubMedPubMedCentral
24.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. International journal of surgery (London, England). 2010;8(5):336–41.CrossRef
25.
Higgins BT, Barlow DR, Heagerty NE, Lin TJ. Anterior vs. posterior approach for total hip arthroplasty, a systematic review and meta-analysis. J Arthroplasty. 2015;30(3):419–34.CrossRefPubMed
26.
Verra WC, Van Den Boom LGH, Jacobs WCH, Schoones JW, Wymenga AB, Nelissen RGHH. Similar outcome after retention or sacrifice of the posterior cruciate ligament in total knee arthroplasty: a systematic review and meta-analysis. Acta Orthop. 2015;86(2):195–201.CrossRefPubMedPubMedCentral
27.
Berstock JR, Blom AW, Beswick AD. A systematic review and meta-analysis of complications following the posterior and lateral surgical approaches to total hip arthroplasty. Ann R Coll Surg Engl. 2015;97(1):11–6.CrossRefPubMedPubMedCentral
28.
Berstock JR, Blom AW, Beswick AD. A systematic review and meta-analysis of the standard versus mini-incision posterior approach to total hip arthroplasty. J Arthroplasty. 2014;29(10):1970–82.CrossRefPubMed
29.
Li N, Tan Y, Deng Y, Chen L. Posterior cruciate-retaining versus posterior stabilized total knee arthroplasty: a meta-analysis of randomized controlled trials. Knee Surg Sports Traumatol Arthrosc. 2014;22(3):556–64.CrossRefPubMed
30.
Wasiak J, Shen AY, Tan HB, Mahar R, Kan G, Khoo WR, Faggion CM Jr. Methodological quality assessment of paper-based systematic reviews published in oral health. Clin Oral Investig. 2016;20(3):399–431.CrossRefPubMed
31.
Wasiak J, Shen AY, Ware R, O'Donohoe TJ, Faggion CM, Jr.: Methodological quality and reporting of systematic reviews in hand and wrist pathology. J Hand Surg Eur Vol, 2017, 42(8):852–856.
32.
Tsertsvadze A, Grove A, Freeman K, Court R, Johnson S, Connock M, Clarke A, Sutcliffe P. Total hip replacement for the treatment of end stage arthritis of the hip: a systematic review and meta-analysis. PLoS ONE [Electronic Resource]. 2014;9(7):e99804.CrossRef
33.
Rebal BA, Babatunde OM, Lee JH, Geller JA, Patrick DA, Jr., Macaulay W: Imageless computer navigation in total knee arthroplasty provides superior short term functional outcomes: a meta-analysis. J Arthroplasty, 2014, 29(5):938–944.
34.
Han JL, Gandhi S, Bockoven CG, Narayan VM, Dahm P. The landscape of systematic reviews in urology (1998 to 2015): an assessment of methodological quality. BJU Int. 2017;119(4):638–49.CrossRefPubMed
35.
Green SHJ. Alderson P. Cochrane Handbook: Clarke M; 2008.
36.
Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, Stewart L. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Systematic Reviews. 2012;1(1)
37.
Peersman G, Stuyts B, Vandenlangenbergh T, Cartier P, Fennema P. Fixed- versus mobile-bearing UKA: a systematic review and meta-analysis. Knee Surg Sports Traumatol Arthrosc. 2015;23(11):3296–305.CrossRefPubMed
38.
Liu HW, Gu WD, Xu NW, Sun JY. Surgical approaches in total knee arthroplasty: a meta-analysis comparing the midvastus and subvastus to the medial peripatellar approach. J Arthroplasty. 2014;29(12):2298–304.CrossRefPubMed
39.
Li T, Zhou L, Zhuang Q, Weng X, Bian Y. Patellar denervation in total knee arthroplasty without patellar resurfacing and postoperative anterior knee pain: a meta-analysis of randomized controlled trials. J Arthroplasty. 2014;29(12):2309–13.CrossRefPubMed
40.
Tao L, Qianyu Z, Ke X, Lei Z, Xisheng W. Comparison of the clinical and radiological outcomes following midvastus and medial parapatellar approaches for total knee arthroplasty: a meta-analysis. Chin Med J. 2014;
41.
Cheng T. No clinical benefit of gender-specific total knee arthroplasty: a systematic review and meta-analysis of 6 randomized controlled trials. Author reply. Acta Orthop. 2015;86(2):274–5.CrossRefPubMed
42.
Bo ZD, Liao L, Zhao JM, Wei QJ, Ding XF, Yang B. Mobile bearing or fixed bearing? A meta-analysis of outcomes comparing mobile bearing and fixed bearing bilateral total knee replacements. Knee. 2014;21(2):374–81.CrossRefPubMed
43.
Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, Carpenter J, Rucker G, Harbord RM, Schmid CH, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ (Clinical research ed). 2011;d4002:343.
44.
Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. Publication and related biases. Health technology assessment (Winchester, England). 2000;4(10):1–115.
45.
McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet (London, England). 2000;356(9237):1228–31.CrossRef
46.
Wang H, Lou H, Zhang H, Jiang J, Liu K. Similar survival between uncemented and cemented fixation prostheses in total knee arthroplasty: a meta-analysis and systematic comparative analysis using registers. Knee Surg Sports Traumatol Arthrosc. 2014;22(12):3191–7.CrossRefPubMed
47.
Moskal JT, Capps SG. Rotating-platform TKA no different from fixed-bearing TKA regarding survivorship or performance: a meta-analysis. Clinical Orthopaedics & Related Research. 2014;472(7):2185–93.CrossRef
48.
Xie X, Lin L, Zhu B, Lu Y, Lin Z, Li Q. Will gender-specific total knee arthroplasty be a better choice for women? A systematic review and meta-analysis. European journal of orthopaedic surgery & traumatologie. 2014;24(8):1341–9.CrossRef
49.
Hu D, Yang X, Tan Y, Alaidaros M, Chen L. Ceramic-on-ceramic versus ceramic-on-polyethylene bearing surfaces in total hip arthroplasty. Orthopedics. 2015;38(4):e331-e338.CrossRef
50.
Campbell JM, Kavanagh S, Kurmis R, Munn Z. Systematic Reviews in Burns Care: Poor Quality and Getting Worse. Journal of burn care & research : official publication of the American Burn Association. 2017;38(2):e552–67.CrossRef
51.
Pang WK, Yeter KC, Torralba KD, Spencer HJ, Khan NA. Financial conflicts of interest and their association with outcome and quality of fibromyalgia drug therapy randomized controlled trials. Int J Rheum Dis. 2015;18(6):606–15.CrossRefPubMed
52.
Roseman M, Milette K, Bero LA, Coyne JC, Lexchin J, Turner EH, Thombs BD. Reporting of conflicts of interest in meta-analyses of trials of pharmacological treatments. JAMA. 2011;305(10):1008–17.CrossRefPubMed
53.
Sismondo S. How pharmaceutical industry funding affects trial outcomes: causal structures and responses. Social science & medicine (1982). 2008;66(9):1909–14.CrossRef
54.
Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA. 2003;290(7):921–8.CrossRefPubMed
55.
Oremus M, Oremus C, Hall GB, McKinnon MC. Inter-rater and test-retest reliability of quality assessments by novice student raters using the Jadad and Newcastle-Ottawa scales. BMJ Open. 2012;2(4)
56.
Hartling L, Milne A, Hamm MP, Vandermeer B, Ansari M, Tsertsvadze A, Dryden DM. Testing the Newcastle Ottawa scale showed low reliability between individual reviewers. J Clin Epidemiol. 2013;66(9):982–93.CrossRefPubMed
57.
Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol. 2010;25(9):603–5.CrossRefPubMed
58.
Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007;36(3):666–76.CrossRefPubMed
59.
Greenland S, O'Rourke K. On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions. Biostatistics. 2001;2(4):463–71.CrossRefPubMed
60.
Sterne JA, Hernan MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ (Clinical research ed). 2016;i4919:355.
61.
Fleming PS, Koletsi D, Seehra J, Pandis N. Systematic reviews published in higher impact clinical journals were of higher quality. J Clin Epidemiol. 2014;67(7):754–9.CrossRefPubMed
62.
Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed). 2017;j4008:358.
63.
Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10):1013–20.CrossRefPubMed
64.
Pieper D, Buechter RB, Li L, Prediger B, Eikermann M. Systematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties. J Clin Epidemiol. 2015;68(5):574–83.CrossRefPubMed
Metadata
Title
Quality assessment of systematic reviews on total hip or knee arthroplasty using mod-AMSTAR
Authors
Xinyu Wu
Huan Sun
Xiaoqin Zhou
Ji Wang
Jing Li
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2018
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-018-0488-8

Other articles of this Issue 1/2018

BMC Medical Research Methodology 1/2018 Go to the issue

Research article

Modelling memory decay after injuries using household survey data from Khartoum State, Sudan

Debate

CDSS-RM: a clinical decision support system reference model

Research article

The importance of including aliases in data linkage with vulnerable populations

Research article

Bias due to censoring of deaths when calculating extra length of stay for patients acquiring a hospital infection

Research article

Retention strategies in longitudinal cohort studies: a systematic review and meta-analysis

Study protocol

Barriers in general practitioners’ dementia diagnostics among people with a migration background in Germany (BaDeMi) - study protocol for a cross-sectional survey