Skip to main content
Top
Published in: BMC Public Health 1/2019

Open Access 01-12-2019 | Vaccination | Debate

Defining & assessing the quality, usability, and utilization of immunization data

Authors: Peter Bloland, Adam MacNeil

Published in: BMC Public Health | Issue 1/2019

Login to get access

Abstract

Background

High quality data are needed for decision-making at all levels of the public health system, from guiding public health activities at the local level, to informing national policy development, to monitoring the impact of global initiatives. Although a number of approaches have been developed to evaluate the underlying quality of routinely collected vaccination administrative data, there remains a lack of consensus around how data quality is best defined or measured.

Discussion

We present a definitional framework that is intended to disentangle many of the elements that have confused discussions of vaccination data quality to date. The framework describes immunization data in terms of three key characteristics: data quality, data usability, and data utilization. The framework also offers concrete suggestions for a specific set of indicators that could be used to better understand immunization those key characteristics, including Trueness, Concurrence, Relevancy, Efficiency, Completeness, Timeliness, Integrity, Consistency, and Utilization.

Conclusion

Being deliberate about the choice of indicators; being clear on their definitions, limitations, and methods of measurement; and describing how those indicators work together to give a more comprehensive and practical understanding of immunization data quality, usability, and use, should yield more informed, and therefore better, programmatic decision-making.
Literature
1.
go back to reference Chen H, Hailey D, Wang N, Yu P. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014;11(5):5170–207.CrossRef Chen H, Hailey D, Wang N, Yu P. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014;11(5):5170–207.CrossRef
2.
go back to reference Bos E, Batson A. Using immunization coverage rates for monitoring health sector performance. Washington, DC: Measurement and Interpretation Issues; 2000. Bos E, Batson A. Using immunization coverage rates for monitoring health sector performance. Washington, DC: Measurement and Interpretation Issues; 2000.
3.
go back to reference Murray CJL, Shengelia B, Gupta N, Moussavi S, Tandon A, Thieren M. Validity of reported vaccination coverage in 45 countries. Lancet. 2003;362:1022–7.CrossRef Murray CJL, Shengelia B, Gupta N, Moussavi S, Tandon A, Thieren M. Validity of reported vaccination coverage in 45 countries. Lancet. 2003;362:1022–7.CrossRef
4.
go back to reference Onta SR, Sabroe S, Hansen EH. The quality of immunization data from routine primary health care reports: a case from Nepal. Health Policy Plan. 1998;13(2):131–9.CrossRef Onta SR, Sabroe S, Hansen EH. The quality of immunization data from routine primary health care reports: a case from Nepal. Health Policy Plan. 1998;13(2):131–9.CrossRef
5.
go back to reference Bosch-Capblanch X, Ronveaux O, Doyle V, Remedios V, Bchir A. Accuracy and quality of immunization information systems in forty-one low income countries. Tropical Med Int Health : TM & IH. 2009;14(1):2–10. Bosch-Capblanch X, Ronveaux O, Doyle V, Remedios V, Bchir A. Accuracy and quality of immunization information systems in forty-one low income countries. Tropical Med Int Health : TM & IH. 2009;14(1):2–10.
6.
go back to reference Ronveaux O, Rickert D, Hadler S, Groom H, Lloyd J, Bchir A, et al. The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bull WHO. 2005;83:503–10.PubMed Ronveaux O, Rickert D, Hadler S, Groom H, Lloyd J, Bchir A, et al. The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bull WHO. 2005;83:503–10.PubMed
7.
go back to reference World Health Organization. The immunization data quality audit (DQA) procedure. Geneva: World Health Organization; 2003. World Health Organization. The immunization data quality audit (DQA) procedure. Geneva: World Health Organization; 2003.
8.
go back to reference World Health Organization, editor. The immunization data quality self-assessment (DQS) tool. Geneva: WHO; 2005. World Health Organization, editor. The immunization data quality self-assessment (DQS) tool. Geneva: WHO; 2005.
9.
go back to reference World Health Organization. Service availability and readiness assessment (SARA): an annual monitoring system for service delivery: reference manual. Geneva: World Health Organization; 2013. World Health Organization. Service availability and readiness assessment (SARA): an annual monitoring system for service delivery: reference manual. Geneva: World Health Organization; 2013.
10.
go back to reference Scott C, Clarke KEN, Grevendonk J, Dolan SB, Ahmed HO, Kamau P, et al. Country immunization information system assessments - Kenya, 2015 and Ghana, 2016. MMWR Morb Mortal Wkly Rep. 2017;66(44):1226–9.CrossRef Scott C, Clarke KEN, Grevendonk J, Dolan SB, Ahmed HO, Kamau P, et al. Country immunization information system assessments - Kenya, 2015 and Ghana, 2016. MMWR Morb Mortal Wkly Rep. 2017;66(44):1226–9.CrossRef
11.
go back to reference Dolan SB, MacNeil A. Comparison of inflation of third dose diphtheria tetanus pertussis (DTP3) administrative coverage to other vaccine antigens. Vaccine. 2017;35(27):3441–5.CrossRef Dolan SB, MacNeil A. Comparison of inflation of third dose diphtheria tetanus pertussis (DTP3) administrative coverage to other vaccine antigens. Vaccine. 2017;35(27):3441–5.CrossRef
12.
go back to reference Cutts FT, Izurieta HS, Rhoda DA. Measuring coverage in MNCH: design, implementation, and interpretation challenges associated with tracking vaccination coverage using household surveys. PLoS Med. 2013;10(5):e1001404.CrossRef Cutts FT, Izurieta HS, Rhoda DA. Measuring coverage in MNCH: design, implementation, and interpretation challenges associated with tracking vaccination coverage using household surveys. PLoS Med. 2013;10(5):e1001404.CrossRef
13.
go back to reference Miles M, Ryman TK, Dietz V, Zell E, Luman ET. Validity of vaccination cards and parental recall to estimate vaccination coverage: a systematic review of the literature. Vaccine. 2013;31(12):1560–8.CrossRef Miles M, Ryman TK, Dietz V, Zell E, Luman ET. Validity of vaccination cards and parental recall to estimate vaccination coverage: a systematic review of the literature. Vaccine. 2013;31(12):1560–8.CrossRef
15.
go back to reference Burton A, Kowalski R, Gacic-Dobo M, Karimov R, Brown D. A formal representation of the WHO and UNICEF estimates of National Immunization Coverage: a computational logic approach. PLoS One. 2012;7(10):e47806.CrossRef Burton A, Kowalski R, Gacic-Dobo M, Karimov R, Brown D. A formal representation of the WHO and UNICEF estimates of National Immunization Coverage: a computational logic approach. PLoS One. 2012;7(10):e47806.CrossRef
16.
go back to reference Lim SS, Stein DB, Charrow A, Murray CJL. Tracking progress towards universal childhood immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage. Lancet. 2008;372:2031–46.CrossRef Lim SS, Stein DB, Charrow A, Murray CJL. Tracking progress towards universal childhood immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage. Lancet. 2008;372:2031–46.CrossRef
17.
go back to reference International Organization for Standardization. Accuracy (trueness and precision) of measurement methods and results - part 1: general principles and definitions. 1994. International Organization for Standardization. Accuracy (trueness and precision) of measurement methods and results - part 1: general principles and definitions. 1994.
18.
go back to reference McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014;67(3):267–77.CrossRef McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014;67(3):267–77.CrossRef
19.
go back to reference Cardemil CV, Gilroy KE, Callaghan-Koru JA, Nsona H, Bryce J. Comparison of methods for assessing quality of care for community case management of sick children: an application with community health workers in Malawi. The American journal of tropical medicine and hygiene. 2012;87(5 Suppl):127–36.CrossRef Cardemil CV, Gilroy KE, Callaghan-Koru JA, Nsona H, Bryce J. Comparison of methods for assessing quality of care for community case management of sick children: an application with community health workers in Malawi. The American journal of tropical medicine and hygiene. 2012;87(5 Suppl):127–36.CrossRef
20.
go back to reference Ward K, Mugenyi K, Benke A, Luzze H, Kyozira C, Immaculate A, et al. Enhancing workforce capacity to improve vaccination data quality, Uganda. Emerg Infect Dis. 2017;23(13). Ward K, Mugenyi K, Benke A, Luzze H, Kyozira C, Immaculate A, et al. Enhancing workforce capacity to improve vaccination data quality, Uganda. Emerg Infect Dis. 2017;23(13).
21.
go back to reference Nicol E, Bradshaw D, Phillips T, Dudley L. Human factors affecting the quality of routinely collected data in South Africa. Studies in Health Technolology and Informformatics. 2013;192:788–92. Nicol E, Bradshaw D, Phillips T, Dudley L. Human factors affecting the quality of routinely collected data in South Africa. Studies in Health Technolology and Informformatics. 2013;192:788–92.
22.
go back to reference Foreit K, Moreland S, LaFond A. Data demand and information use in the health sector: conceptual framework. Chapel Hill: MEASURE Evaluation, Carolina Population Center; 2006. p. 1–17. Foreit K, Moreland S, LaFond A. Data demand and information use in the health sector: conceptual framework. Chapel Hill: MEASURE Evaluation, Carolina Population Center; 2006. p. 1–17.
23.
go back to reference The Oxford dictionary of statistical terms. Oxford: Oxford University Press; 2006 2006. The Oxford dictionary of statistical terms. Oxford: Oxford University Press; 2006 2006.
24.
go back to reference Organization for Economic Co-operation and Development. OECD glossary of statistical terms. Organization for Economic co-operation and Development; 2003. Organization for Economic Co-operation and Development. OECD glossary of statistical terms. Organization for Economic co-operation and Development; 2003.
25.
go back to reference Colton T. Statistics in medicine. Boston: Little, Brown and Company; 1974. Colton T. Statistics in medicine. Boston: Little, Brown and Company; 1974.
26.
go back to reference Kotz S, Johnson NL. Encyclopedia of statistical sciences. Wiley; 2006. Kotz S, Johnson NL. Encyclopedia of statistical sciences. Wiley; 2006.
27.
go back to reference McGraw-Hill dictionary of scientific and technical terms. 6th ed. New York: McGraw-Hill; 2003. McGraw-Hill dictionary of scientific and technical terms. 6th ed. New York: McGraw-Hill; 2003.
Metadata
Title
Defining & assessing the quality, usability, and utilization of immunization data
Authors
Peter Bloland
Adam MacNeil
Publication date
01-12-2019
Publisher
BioMed Central
Keyword
Vaccination
Published in
BMC Public Health / Issue 1/2019
Electronic ISSN: 1471-2458
DOI
https://doi.org/10.1186/s12889-019-6709-1

Other articles of this Issue 1/2019

BMC Public Health 1/2019 Go to the issue