Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Research article

Frequency of data extraction errors and methods to increase data extraction quality: a methodological review

Authors: Tim Mathes, Pauline Klaßen, Dawid Pieper

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

Our objective was to assess the frequency of data extraction errors and its potential impact on results in systematic reviews. Furthermore, we evaluated the effect of different extraction methods, reviewer characteristics and reviewer training on error rates and results.

Methods

We performed a systematic review of methodological literature in PubMed, Cochrane methodological registry, and by manual searches (12/2016). Studies were selected by two reviewers independently. Data were extracted in standardized tables by one reviewer and verified by a second.

Results

The analysis included six studies; four studies on extraction error frequency, one study comparing different reviewer extraction methods and two studies comparing different reviewer characteristics. We did not find a study on reviewer training. There was a high rate of extraction errors (up to 50%). Errors often had an influence on effect estimates. Different data extraction methods and reviewer characteristics had moderate effect on extraction error rates and effect estimates.

Conclusion

The evidence base for established standards of data extraction seems weak despite the high prevalence of extraction errors. More comparative studies are needed to get deeper insights into the influence of different extraction methods.
Appendix
Available only for authorised users
Literature
1.
Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. Available from www.​cochrane-handbook.​org.
2.
3.
Centre for reviews dissemination. CRD's guidance for undertaking reviews in health care. York Publishing Services Ltd. 2009;32:Jg.
4.
Joanna Briggs Institute. Reviewers’ manual: 2011 edition. Adelaide: JBI; 2014.
5.
Tricco AC, Tetzlaff J, Sampson M, Fergusson D, Cogo E, Horsley T, Moher D. Few systematic reviews exist documenting the extent of bias: a systematic review. J Clin Epidemiol. 2008;61(5):422–34.CrossRefPubMed
6.
Shemilt I, Khan N, Park S, Thomas J. Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews. Syst Rev. 2016;5(1):140.CrossRefPubMedPubMedCentral
7.
8.
Gøtzsche PC, Hróbjartsson A, Marić K, Tendal B. Data extraction errors in meta-analyses that use standardized mean differences. JAMA. 2007;298(4):430–7.CrossRefPubMed
9.
Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59:697–703.CrossRefPubMed
10.
11.
Peters MDJ, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6.CrossRefPubMed
12.
Zaza S, Wright De Aguero LK, Briss PA, Truman BI, Hopkins DP, Hennessy MH, Sosin DM, Anderson L, Carande Kulis VG, Teutsch SM, et al. Data collection instrument and procedure for systematic reviews in the guide to community preventive services. Task force on community preventive services. Am J Prev Med. 2000;18:44–74.CrossRefPubMed
13.
Haywood KL, Hargreaves J, White R, Lamb SE. Reviewing measures of outcome: reliability of data extraction. J Eval Clin Pract. 2004;10:329–37.CrossRefPubMed
14.
Horton J, Vandermeer B, Hartling L, Tjosvold L, Klassen TP, Buscemi N. Systematic review data extraction: cross-sectional study showed that experience did not increase accuracy. J Clin Epidemiol. 2010;63:289–98.CrossRefPubMed
15.
Tendal B, Higgins JP, Juni P, Hrobjartsson A, Trelle S, Nuesch E, Wandel S, Jorgensen AW, Gesser K, Ilsoe-Kristensen S et al: Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study. In: Bmj. vol. 339; 2009: b3128.
16.
Carroll C, Scope A, Kaltenthaler E: A case study of binary outcome data extraction across three systematic reviews of hip arthroplasty: errors and differences of selection. BMC research notes. 2013;6:539.
17.
Jones AP, Remmington T, Williamson PR, Ashby D, Smyth RL. High prevalence but low impact of data extraction and reporting errors were found in Cochrane systematic reviews. J Clin Epidemiol. 2005;58:741–2.CrossRefPubMed
18.
Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16(1):127.CrossRefPubMedPubMedCentral
19.
Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews. London: Cochrane; 2016.
20.
Morton S, Berg A, Levit L, Eden J. Finding what works in health care: standards for systematic reviews.National Academies Press; 2011.
21.
22.
Silagy CA, Middleton P, Hopewell S. Publishing protocols of systematic reviews: comparing what was done to what was planned. JAMA. 2002;287(21):2831–4.CrossRefPubMed
23.
Adie S, Harris IA, Naylor JM, Mittal R. CONSORT compliance in surgical randomized trials: are we there yet? A systematic review. Ann Surg. 2013;258(6):872–8.CrossRefPubMed
24.
Zheng SL, Chan FT, Maclean E, Jayakumar S, Nabeebaccus AA. Reporting trends of randomised controlled trials in heart failure with preserved ejection fraction: a systematic review. Open Heart. 2016;3(2):e000449.CrossRefPubMedPubMedCentral
25.
Halladay CW, Trikalinos TA, Schmid IT, Schmid CH, Dahabreh IJ. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol. 2015;68(9):1076–84.CrossRefPubMed
26.
Saldanha IJ, Schmid CH, Lau J, Dickersin K, Berlin JA, Jap J, Smith BT, Carini S, Chan W, De Bruijn B, et al. Evaluating data abstraction assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial. Syst Rev. 2016;5(1):196.CrossRefPubMedPubMedCentral
27.
28.
da Costa BR, Beckett B, Diaz A, Resta NM, Johnston BC, Egger M, Jüni P, Armijo-Olivo S. Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a prospective study. Syst Rev. 2017;6(1):44.CrossRefPubMedPubMedCentral
Metadata
Title
Frequency of data extraction errors and methods to increase data extraction quality: a methodological review
Authors
Tim Mathes
Pauline Klaßen
Dawid Pieper
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-017-0431-4

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue

Research article

Mobile electronic versus paper case report forms in clinical trials: a randomized controlled trial

Research article

The effect of postal questionnaire burden on response rate and answer patterns following admission to intensive care: a randomised controlled trial

Research article

Measurement properties of the Danish version of the Awareness and Beliefs about Cancer (ABC) measure

Research article

Centralising and optimising decentralised stroke care systems: a simulation study on short-term costs and effects

Research article

Incorporating nonlinearity into mediation analyses

Debate

Protocol adherence for continuously titrated interventions in randomized trials: an overview of the current methodology and case study