Skip to main content
Top
Published in: Trials 1/2017

Open Access 01-12-2017 | Methodology

Implementation and results of an integrated data quality assurance protocol in a randomized controlled trial in Uttar Pradesh, India

Authors: Jonathon D. Gass Jr., Anamika Misra, Mahendra Nath Singh Yadav, Fatima Sana, Chetna Singh, Anup Mankar, Brandon J. Neal, Jennifer Fisher-Bowman, Jenny Maisonneuve, Megan Marx Delaney, Krishan Kumar, Vinay Pratap Singh, Narender Sharma, Atul Gawande, Katherine Semrau, Lisa R. Hirschhorn

Published in: Trials | Issue 1/2017

Login to get access

Abstract

Background

There are few published standards or methodological guidelines for integrating Data Quality Assurance (DQA) protocols into large-scale health systems research trials, especially in resource-limited settings. The BetterBirth Trial is a matched-pair, cluster-randomized controlled trial (RCT) of the BetterBirth Program, which seeks to improve quality of facility-based deliveries and reduce 7-day maternal and neonatal mortality and maternal morbidity in Uttar Pradesh, India. In the trial, over 6300 deliveries were observed and over 153,000 mother-baby pairs across 120 study sites were followed to assess health outcomes. We designed and implemented a robust and integrated DQA system to sustain high-quality data throughout the trial.

Methods

We designed the Data Quality Monitoring and Improvement System (DQMIS) to reinforce six dimensions of data quality: accuracy, reliability, timeliness, completeness, precision, and integrity. The DQMIS was comprised of five functional components: 1) a monitoring and evaluation team to support the system; 2) a DQA protocol, including data collection audits and targets, rapid data feedback, and supportive supervision; 3) training; 4) standard operating procedures for data collection; and 5) an electronic data collection and reporting system. Routine audits by supervisors included double data entry, simultaneous delivery observations, and review of recorded calls to patients. Data feedback reports identified errors automatically, facilitating supportive supervision through a continuous quality improvement model.

Results

The five functional components of the DQMIS successfully reinforced data reliability, timeliness, completeness, precision, and integrity. The DQMIS also resulted in 98.33% accuracy across all data collection activities in the trial. All data collection activities demonstrated improvement in accuracy throughout implementation. Data collectors demonstrated a statistically significant (p = 0.0004) increase in accuracy throughout consecutive audits. The DQMIS was successful, despite an increase from 20 to 130 data collectors.

Conclusions

In the absence of widely disseminated data quality methods and standards for large RCT interventions in limited-resource settings, we developed an integrated DQA system, combining auditing, rapid data feedback, and supportive supervision, which ensured high-quality data and could serve as a model for future health systems research trials. Future efforts should focus on standardization of DQA processes for health systems research.

Trial Registration

ClinicalTrials.gov identifier, NCT02148952. Registered on 13 February 2014.
Literature
1.
go back to reference CNS Summit Data Quality Monitoring Workgroup Core Members. Data quality monitoring in clinical trials: has it been worth it? An evaluation and prediction of the future by all stakeholders. Innov Clin Neurosci. 2016;13(1–2):27–33. CNS Summit Data Quality Monitoring Workgroup Core Members. Data quality monitoring in clinical trials: has it been worth it? An evaluation and prediction of the future by all stakeholders. Innov Clin Neurosci. 2016;13(1–2):27–33.
3.
4.
go back to reference Davis JRN, Vivian P, Woodcock J, Estabrook RW. Assuring data quality and validity in clinical trials for regulatory decision making: workshop report. Washington (DC): Institute of Medicine Roundtable on Research and Development of Drugs, Biologics, Medical Devices; 1999. Davis JRN, Vivian P, Woodcock J, Estabrook RW. Assuring data quality and validity in clinical trials for regulatory decision making: workshop report. Washington (DC): Institute of Medicine Roundtable on Research and Development of Drugs, Biologics, Medical Devices; 1999.
5.
go back to reference Nahm ML. Data quality in clinical research. In: Richesson RL, editor. Clinical research informatics. London: Springer-Verlag; 2012. p. 175–201.CrossRef Nahm ML. Data quality in clinical research. In: Richesson RL, editor. Clinical research informatics. London: Springer-Verlag; 2012. p. 175–201.CrossRef
6.
go back to reference Moher D, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352(9128):609–13.CrossRefPubMed Moher D, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352(9128):609–13.CrossRefPubMed
8.
go back to reference Goldhill DR, Sumner A. APACHE II, data accuracy and outcome prediction. Anaesthesia. 1998;53(10):937–43.CrossRefPubMed Goldhill DR, Sumner A. APACHE II, data accuracy and outcome prediction. Anaesthesia. 1998;53(10):937–43.CrossRefPubMed
9.
go back to reference Richesson RL, et al. Electronic health records based phenotyping in next-generation clinical trials: a perspective from the NIH Health Care Systems Collaboratory. J Am Med Inform Assoc. 2013;20(e2):e226–31.CrossRefPubMedPubMedCentral Richesson RL, et al. Electronic health records based phenotyping in next-generation clinical trials: a perspective from the NIH Health Care Systems Collaboratory. J Am Med Inform Assoc. 2013;20(e2):e226–31.CrossRefPubMedPubMedCentral
10.
go back to reference Society for Clinical Data Management. Good clinical data management practices. 2013. Society for Clinical Data Management. Good clinical data management practices. 2013.
11.
go back to reference Houston L, Probst Y, Humphries A. Measuring data quality through a source data verification audit in a clinical research setting. Stud Health Technol Inform. 2015;214:107–13.PubMed Houston L, Probst Y, Humphries A. Measuring data quality through a source data verification audit in a clinical research setting. Stud Health Technol Inform. 2015;214:107–13.PubMed
12.
go back to reference Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20(1):144–51.CrossRefPubMedPubMedCentral Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20(1):144–51.CrossRefPubMedPubMedCentral
13.
go back to reference Kahn MG, et al. A pragmatic framework for single-site and multisite data quality assessment in electronic health record-based clinical research. Med Care. 2012;50:S21–9.CrossRefPubMed Kahn MG, et al. A pragmatic framework for single-site and multisite data quality assessment in electronic health record-based clinical research. Med Care. 2012;50:S21–9.CrossRefPubMed
14.
go back to reference Chen H, et al. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014;11(5):5170–207.CrossRefPubMedPubMedCentral Chen H, et al. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014;11(5):5170–207.CrossRefPubMedPubMedCentral
16.
go back to reference van der Putten E, et al. A pilot study on the quality of data management in a cancer clinical trial. Control Clin Trials. 1987;8(2):96–100.CrossRefPubMed van der Putten E, et al. A pilot study on the quality of data management in a cancer clinical trial. Control Clin Trials. 1987;8(2):96–100.CrossRefPubMed
17.
go back to reference Horbar JD, Leahy KA. An assessment of data quality in the Vermont-Oxford Trials Network Database. Control Clin Trials. 1995;16(1):51–61.CrossRefPubMed Horbar JD, Leahy KA. An assessment of data quality in the Vermont-Oxford Trials Network Database. Control Clin Trials. 1995;16(1):51–61.CrossRefPubMed
19.
go back to reference Goldberg SI, Niemierko A, Turchin A. Analysis of data errors in clinical research databases. AMIA Annu Symp Proc. 2008;2008:242–6.PubMedCentral Goldberg SI, Niemierko A, Turchin A. Analysis of data errors in clinical research databases. AMIA Annu Symp Proc. 2008;2008:242–6.PubMedCentral
20.
go back to reference Hong MKH, Yao HHI, Pedersen JS, et al. Error rates in a clinical data repository: lessons from the transition to electronic data transfer—a descriptive study. BMJ Open. 2013;3:e002406. Hong MKH, Yao HHI, Pedersen JS, et al. Error rates in a clinical data repository: lessons from the transition to electronic data transfer—a descriptive study. BMJ Open. 2013;3:e002406.  
21.
go back to reference Macefield RC, et al. A systematic review of on-site monitoring methods for health-care randomised controlled trials. Clin Trials. 2013;10(1):104–24.CrossRefPubMed Macefield RC, et al. A systematic review of on-site monitoring methods for health-care randomised controlled trials. Clin Trials. 2013;10(1):104–24.CrossRefPubMed
22.
go back to reference Arts DGT, de Keizer NF, Scheffer G-J. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc. 2002;9(6):600–11.CrossRefPubMedPubMedCentral Arts DGT, de Keizer NF, Scheffer G-J. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc. 2002;9(6):600–11.CrossRefPubMedPubMedCentral
23.
go back to reference Wang RY, Strong DM. Beyond accuracy: what data quality means to data consumers. J Manage Inf Syst. 1996;12(4):5–33.CrossRef Wang RY, Strong DM. Beyond accuracy: what data quality means to data consumers. J Manage Inf Syst. 1996;12(4):5–33.CrossRef
24.
go back to reference Gassman JJ, et al. Data quality assurance, monitoring, and reporting. Control Clin Trials. 1995;16(2 Suppl):104s–36s.CrossRefPubMed Gassman JJ, et al. Data quality assurance, monitoring, and reporting. Control Clin Trials. 1995;16(2 Suppl):104s–36s.CrossRefPubMed
25.
go back to reference Richardson D, Chen S. Data quality assurance and quality control measures in large multicenter stroke trials: the African-American Antiplatelet Stroke Prevention Study experience. Trials. 2001;2(3):115.CrossRef Richardson D, Chen S. Data quality assurance and quality control measures in large multicenter stroke trials: the African-American Antiplatelet Stroke Prevention Study experience. Trials. 2001;2(3):115.CrossRef
27.
go back to reference Semrau K, Hirschhorn LR, Kodkany B, Spector J, Tuller DE, King G, Lisptiz S, Sharma N, Singh VP, Kumar B, Dhingra-Kumar N, Firestone R, Kumar V, Gawande A. Effectiveness of the WHO safe childbirth checklist program in reducing severe maternal, fetal, and newborn harm: study protocol for a matched-pair, cluster randomized controlled trial in Uttar Pradesh, India. Trials. 2016;17(576):1–10. Semrau K, Hirschhorn LR, Kodkany B, Spector J, Tuller DE, King G, Lisptiz S, Sharma N, Singh VP, Kumar B, Dhingra-Kumar N, Firestone R, Kumar V, Gawande A. Effectiveness of the WHO safe childbirth checklist program in reducing severe maternal, fetal, and newborn harm: study protocol for a matched-pair, cluster randomized controlled trial in Uttar Pradesh, India. Trials. 2016;17(576):1–10.
28.
go back to reference Brown W. Data quality assurance tool for program level indicators. MEASURE Evaluation, 2007. Brown W. Data quality assurance tool for program level indicators. MEASURE Evaluation, 2007.
30.
go back to reference Carter RE, Lipsitz SR, Tilley BC. Quasi-likelihood estimation for relative risk regression models. Biostatistics. 2005;6(1):39–44.CrossRefPubMed Carter RE, Lipsitz SR, Tilley BC. Quasi-likelihood estimation for relative risk regression models. Biostatistics. 2005;6(1):39–44.CrossRefPubMed
31.
go back to reference Needham DM, et al. Improving data quality control in quality improvement projects. International J Qual Health Care. 2009;21(2):145–50.CrossRef Needham DM, et al. Improving data quality control in quality improvement projects. International J Qual Health Care. 2009;21(2):145–50.CrossRef
32.
go back to reference Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff (Millwood). 2005;24(1):138–50.CrossRef Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff (Millwood). 2005;24(1):138–50.CrossRef
33.
go back to reference Hayes E, Kalmakis KA. From the sidelines: coaching as a nurse practitioner strategy for improving health outcomes. J Am Acad Nurse Pract. 2007;19(11):555–62.CrossRefPubMed Hayes E, Kalmakis KA. From the sidelines: coaching as a nurse practitioner strategy for improving health outcomes. J Am Acad Nurse Pract. 2007;19(11):555–62.CrossRefPubMed
34.
go back to reference Ivers NM, et al. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014;29(11):1534–41.CrossRefPubMedPubMedCentral Ivers NM, et al. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014;29(11):1534–41.CrossRefPubMedPubMedCentral
Metadata
Title
Implementation and results of an integrated data quality assurance protocol in a randomized controlled trial in Uttar Pradesh, India
Authors
Jonathon D. Gass Jr.
Anamika Misra
Mahendra Nath Singh Yadav
Fatima Sana
Chetna Singh
Anup Mankar
Brandon J. Neal
Jennifer Fisher-Bowman
Jenny Maisonneuve
Megan Marx Delaney
Krishan Kumar
Vinay Pratap Singh
Narender Sharma
Atul Gawande
Katherine Semrau
Lisa R. Hirschhorn
Publication date
01-12-2017
Publisher
BioMed Central
Published in
Trials / Issue 1/2017
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/s13063-017-2159-1

Other articles of this Issue 1/2017

Trials 1/2017 Go to the issue