Skip to main content
Top
Published in: Systematic Reviews 1/2019

Open Access 01-12-2019 | Research

The Systematic Review Data Repository (SRDR): descriptive characteristics of publicly available data and opportunities for research

Authors: Ian J. Saldanha, Bryant T. Smith, Evangelia Ntzani, Jens Jap, Ethan M. Balk, Joseph Lau

Published in: Systematic Reviews | Issue 1/2019

Login to get access

Abstract

Background

Conducting systematic reviews (“reviews”) requires a great deal of effort and resources. Making data extracted during reviews available publicly could offer many benefits, including reducing unnecessary duplication of effort, standardizing data, supporting analyses to address secondary research questions, and facilitating methodologic research. Funded by the US Agency for Healthcare Research and Quality (AHRQ), the Systematic Review Data Repository (SRDR) is a free, web-based, open-source, data management and archival platform for reviews. Our specific objectives in this paper are to describe (1) the current extent of usage of SRDR and (2) the characteristics of all projects with publicly available data on the SRDR website.

Methods

We examined all projects with data made publicly available through SRDR as of November 12, 2019. We extracted information about the characteristics of these projects. Two investigators extracted and verified the data.

Results

SRDR has had 2552 individual user accounts belonging to users from 80 countries. Since SRDR’s launch in 2012, data have been made available publicly for 152 of the 735 projects in SRDR (21%), at a rate of 24.5 projects per year, on average. Most projects are in clinical fields (144/152 projects; 95%); most have evaluated interventions (therapeutic or preventive) (109/152; 72%). The most frequent health areas addressed are mental and behavioral disorders (31/152; 20%) and diseases of the eye and ocular adnexa (23/152; 15%). Two-thirds of the projects (104/152; 67%) were funded by AHRQ, and one-sixth (23/152; 15%) are Cochrane reviews. The 152 projects each address a median of 3 research questions (IQR 1–5) and include a median of 70 studies (IQR 20–130).

Conclusions

Until we arrive at a future in which the systematic review and broader research communities are comfortable with the accuracy of automated data extraction, re-use of data extracted by humans has the potential to help reduce redundancy and costs. The 152 projects with publicly available data through SRDR, and the more than 15,000 studies therein, are freely available to researchers and the general public who might be working on similar reviews or updates of reviews or who want access to the data for decision-making, meta-research, or other purposes.
Appendix
Available only for authorised users
Literature
1.
go back to reference Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington: The National Academies Presas; 2011. Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington: The National Academies Presas; 2011.
2.
go back to reference Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K. Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. J Clin Epidemiol. 2017;86:39–50.CrossRef Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K. Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. J Clin Epidemiol. 2017;86:39–50.CrossRef
3.
go back to reference Mayo-Wilson E, Li T, Fusco N, Dickersin K. Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study). Res Synth Methods. 2018;9(1):2–12.CrossRef Mayo-Wilson E, Li T, Fusco N, Dickersin K. Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study). Res Synth Methods. 2018;9(1):2–12.CrossRef
4.
go back to reference Ioannidis JP, Fanelli D, Dunne DD, Goodman SN. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 2015;13(10):e1002264.CrossRef Ioannidis JP, Fanelli D, Dunne DD, Goodman SN. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 2015;13(10):e1002264.CrossRef
5.
go back to reference Wolfenden L, Grimshaw J, Williams CM, Yoong SL. Time to consider sharing data extracted from trials included in systematic reviews. Syst Rev. 2016;5(1):185.CrossRef Wolfenden L, Grimshaw J, Williams CM, Yoong SL. Time to consider sharing data extracted from trials included in systematic reviews. Syst Rev. 2016;5(1):185.CrossRef
6.
go back to reference Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12.CrossRef Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12.CrossRef
7.
go back to reference Hannink G, Gooszen HG, Rovers MM. Comparison of registered and published primary outcomes in randomized clinical trials of surgical interventions. Ann Surg. 2013;257(5):818–23.CrossRef Hannink G, Gooszen HG, Rovers MM. Comparison of registered and published primary outcomes in randomized clinical trials of surgical interventions. Ann Surg. 2013;257(5):818–23.CrossRef
8.
go back to reference Ip S, Hadar N, Keefe S, et al. A web-based archive of systematic review data. Syst Rev. 2012;1:15.CrossRef Ip S, Hadar N, Keefe S, et al. A web-based archive of systematic review data. Syst Rev. 2012;1:15.CrossRef
9.
go back to reference Li T, Vedula SS, Hadar N, Parkin C, Lau J, Dickersin K. Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med. 2015;162(4):287–94.CrossRef Li T, Vedula SS, Hadar N, Parkin C, Lau J, Dickersin K. Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med. 2015;162(4):287–94.CrossRef
11.
go back to reference Page MJ, Shamseer L, Altman DG, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028.CrossRef Page MJ, Shamseer L, Altman DG, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028.CrossRef
12.
go back to reference Page MJ, Shamseer L, Tricco AC. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev. 2018;7(1):32.CrossRef Page MJ, Shamseer L, Tricco AC. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev. 2018;7(1):32.CrossRef
13.
go back to reference Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. An international registry of systematic-review protocols. Lancet (London, England). 2011;377(9760):108–9.CrossRef Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, Stewart L. An international registry of systematic-review protocols. Lancet (London, England). 2011;377(9760):108–9.CrossRef
14.
go back to reference Stewart L, Moher D, Shekelle P. Why prospective registration of systematic reviews makes sense. Syst Rev. 2012;1:7.CrossRef Stewart L, Moher D, Shekelle P. Why prospective registration of systematic reviews makes sense. Syst Rev. 2012;1:7.CrossRef
15.
go back to reference Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16:15.CrossRef Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16:15.CrossRef
16.
go back to reference Elliott JH, Synnot A, Turner T, et al. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol. 2017;91:23–30.CrossRef Elliott JH, Synnot A, Turner T, et al. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol. 2017;91:23–30.CrossRef
17.
go back to reference Abou-Setta AM, Jeyaraman MM, Attia A, et al. Methods for developing evidence reviews in short periods of time: a scoping review. PLoS One. 2016;11(12):e0165903.CrossRef Abou-Setta AM, Jeyaraman MM, Attia A, et al. Methods for developing evidence reviews in short periods of time: a scoping review. PLoS One. 2016;11(12):e0165903.CrossRef
18.
go back to reference Taichman DB, Backus J, Baethge C, et al. Sharing clinical trial data: a proposal from the International Committee of Medical Journal Editors. Rev Med Chil. 2016;144(1):11–3.CrossRef Taichman DB, Backus J, Baethge C, et al. Sharing clinical trial data: a proposal from the International Committee of Medical Journal Editors. Rev Med Chil. 2016;144(1):11–3.CrossRef
19.
go back to reference Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514.CrossRef Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514.CrossRef
20.
go back to reference Bashir R, Surian D, Dunn AG. Time-to-update of systematic reviews relative to the availability of new evidence. Syst Rev. 2018;7(1):195.CrossRef Bashir R, Surian D, Dunn AG. Time-to-update of systematic reviews relative to the availability of new evidence. Syst Rev. 2018;7(1):195.CrossRef
21.
go back to reference Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Syst Rev. 2015;4:78.CrossRef Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Syst Rev. 2015;4:78.CrossRef
22.
go back to reference O'Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS. Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of International Collaboration for the Automation of Systematic Reviews (ICASR). Sys Rev. 2018;7(1):3.CrossRef O'Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS. Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of International Collaboration for the Automation of Systematic Reviews (ICASR). Sys Rev. 2018;7(1):3.CrossRef
23.
go back to reference van Altena AJ, Spijker R, Olabarriaga SD. Usage of automation tools in systematic reviews. Res Synth Methods. 2019;10(1):72–82.CrossRef van Altena AJ, Spijker R, Olabarriaga SD. Usage of automation tools in systematic reviews. Res Synth Methods. 2019;10(1):72–82.CrossRef
24.
go back to reference Lau J. Editorial: systematic review automation thematic series. Syst Rev. 2019;8(1):70.CrossRef Lau J. Editorial: systematic review automation thematic series. Syst Rev. 2019;8(1):70.CrossRef
25.
go back to reference Tudur Smith C, Dwan K, Altman DG, Clarke M, Riley R, Williamson PR. Sharing individual participant data from clinical trials: an opinion survey regarding the establishment of a central repository. PLoS One. 2014;9(5):e97886.CrossRef Tudur Smith C, Dwan K, Altman DG, Clarke M, Riley R, Williamson PR. Sharing individual participant data from clinical trials: an opinion survey regarding the establishment of a central repository. PLoS One. 2014;9(5):e97886.CrossRef
26.
go back to reference Saldanha IJ, Schmid CH, Lau J, et al. Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial. Syst Rev. 2016;5(1):196.CrossRef Saldanha IJ, Schmid CH, Lau J, et al. Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial. Syst Rev. 2016;5(1):196.CrossRef
27.
go back to reference Jap J, Saldanha IJ, Smith BT, Lau J, Li T. Response to ‘Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms’. Syst Rev. 2018;7(1):18.CrossRef Jap J, Saldanha IJ, Smith BT, Lau J, Li T. Response to ‘Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms’. Syst Rev. 2018;7(1):18.CrossRef
28.
go back to reference Jap J, Saldanha IJ, Smith BT, Lau J, Schmid CH, Li T, Data Abstraction Assistant Investigators. Features and functioning of Data Abstraction Assistant, a software application for data abstraction during systematic reviews. Res Synth Methods. 2019;10(1):2–14 PubMed ID: 30325115. https://doi.org/10.1002/jrsm.1326.CrossRef Jap J, Saldanha IJ, Smith BT, Lau J, Schmid CH, Li T, Data Abstraction Assistant Investigators. Features and functioning of Data Abstraction Assistant, a software application for data abstraction during systematic reviews. Res Synth Methods. 2019;10(1):2–14 PubMed ID: 30325115. https://​doi.​org/​10.​1002/​jrsm.​1326.CrossRef
29.
go back to reference Dahabreh IJ, Trikalinos TA, Balk EM, Wong JB. AHRQ methods for effective health care guidance for the conduct and reporting of modeling and simulation studies in the context of health technology assessment. In: Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville: Agency for Healthcare Research and Quality (US); 2008. Dahabreh IJ, Trikalinos TA, Balk EM, Wong JB. AHRQ methods for effective health care guidance for the conduct and reporting of modeling and simulation studies in the context of health technology assessment. In: Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville: Agency for Healthcare Research and Quality (US); 2008.
30.
go back to reference Saldanha IJ, Li T, Yang C, Ugarte-Gil C, Rutherford GW, Dickersin K. Social network analysis identified central outcomes for core outcome sets using systematic reviews of HIV/AIDS. J Clin Epidemiol. 2016;70:164–75.CrossRef Saldanha IJ, Li T, Yang C, Ugarte-Gil C, Rutherford GW, Dickersin K. Social network analysis identified central outcomes for core outcome sets using systematic reviews of HIV/AIDS. J Clin Epidemiol. 2016;70:164–75.CrossRef
31.
go back to reference Saldanha IJ, Lindsley K, Do DV, et al. Comparison of clinical trial and systematic review outcomes for the 4 most prevalent eye diseases. JAMA Ophthalmol. 2017;135(9):933–40.CrossRef Saldanha IJ, Lindsley K, Do DV, et al. Comparison of clinical trial and systematic review outcomes for the 4 most prevalent eye diseases. JAMA Ophthalmol. 2017;135(9):933–40.CrossRef
32.
go back to reference Saldanha IJ, Lindsley KB, Lum F, Dickersin K, Li T. Reliability of the evidence addressing treatment of corneal diseases: a summary of systematic reviews. JAMA Ophthalmol. 2019;137(7):775–85.CrossRef Saldanha IJ, Lindsley KB, Lum F, Dickersin K, Li T. Reliability of the evidence addressing treatment of corneal diseases: a summary of systematic reviews. JAMA Ophthalmol. 2019;137(7):775–85.CrossRef
33.
go back to reference Saldanha IJ, Petris R, Han G, Dickersin K, Akpek EK. Research questions and outcomes prioritized by patients with dry eye. JAMA Ophthalmol. 2018;136(10):1170–9.CrossRef Saldanha IJ, Petris R, Han G, Dickersin K, Akpek EK. Research questions and outcomes prioritized by patients with dry eye. JAMA Ophthalmol. 2018;136(10):1170–9.CrossRef
34.
go back to reference Saldanha IJ, Lindsley K, Money S, Kimmel HJ, Smith BT, Dickersin K. Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study. (Submitted). Saldanha IJ, Lindsley K, Money S, Kimmel HJ, Smith BT, Dickersin K. Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study. (Submitted).
35.
go back to reference Li T, Saldanha IJ, Jap J, et al. A randomized trial provided new evidence on the accuracy and efficiency of traditional vs electronically annotated abstraction approaches in systematic reviews. J Clin Epidemiol. 2019;115:77–89.CrossRef Li T, Saldanha IJ, Jap J, et al. A randomized trial provided new evidence on the accuracy and efficiency of traditional vs electronically annotated abstraction approaches in systematic reviews. J Clin Epidemiol. 2019;115:77–89.CrossRef
36.
go back to reference Saldanha IJ, Senturk B, Smith BT, Robinson KA. Pilot to promote entry of structured data into the systematic review data repository: methods research report. Vol AHRQ publication no. 19-EH028-EF. Rockville: Agency for Healthcare Research and Quality; 2019.CrossRef Saldanha IJ, Senturk B, Smith BT, Robinson KA. Pilot to promote entry of structured data into the systematic review data repository: methods research report. Vol AHRQ publication no. 19-EH028-EF. Rockville: Agency for Healthcare Research and Quality; 2019.CrossRef
37.
go back to reference Tsujimoto Y, Tsutsumi Y, Kataoka Y, et al. Statistical significance did not affect time to publication in non-Cochrane systematic reviews: a metaepidemiological study. J Clin Epidemiol. 2019;115:25–34.CrossRef Tsujimoto Y, Tsutsumi Y, Kataoka Y, et al. Statistical significance did not affect time to publication in non-Cochrane systematic reviews: a metaepidemiological study. J Clin Epidemiol. 2019;115:25–34.CrossRef
Metadata
Title
The Systematic Review Data Repository (SRDR): descriptive characteristics of publicly available data and opportunities for research
Authors
Ian J. Saldanha
Bryant T. Smith
Evangelia Ntzani
Jens Jap
Ethan M. Balk
Joseph Lau
Publication date
01-12-2019
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2019
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/s13643-019-1250-y

Other articles of this Issue 1/2019

Systematic Reviews 1/2019 Go to the issue