Skip to main content
Top
Published in: Journal of Community Health 4/2015

01-08-2015 | Original Paper

Data for Program Management: An Accuracy Assessment of Data Collected in Household Registers by Community Health Workers in Southern Kayonza, Rwanda

Authors: Tisha Mitsunaga, Bethany L. Hedt-Gauthier, Elias Ngizwenayo, Didi Bertrand Farmer, Erick Gaju, Peter Drobac, Paulin Basinga, Lisa Hirschhorn, Michael L. Rich, Peter J. Winch, Fidele Ngabo, Cathy Mugeni

Published in: Journal of Community Health | Issue 4/2015

Login to get access

Abstract

Community health workers (CHWs) collect data for routine services, surveys and research in their communities. However, quality of these data is largely unknown. Utilizing poor quality data can result in inefficient resource use, misinformation about system gaps, and poor program management and effectiveness. This study aims to measure CHW data accuracy, defined as agreement between household registers compared to household member interview and client records in one district in Eastern province, Rwanda. We used cluster-lot quality assurance sampling to randomly sample six CHWs per cell and six households per CHW. We classified cells as having ‘poor’ or ‘good’ accuracy for household registers for five indicators, calculating point estimates of percent of households with accurate data by health center. We evaluated 204 CHW registers and 1,224 households for accuracy across 34 cells in southern Kayonza. Point estimates across health centers ranged from 79 to 100 % for individual indicators and 61 to 72 % for the composite indicator. Recording error appeared random for all but the widely under-reported number of women on modern family planning method. Overall, accuracy was largely ‘good’ across cells, with varying results by indicator. Program managers should identify optimum thresholds for ‘good’ data quality and interventions to reach them according to data use. Decreasing variability and improving quality will facilitate potential of these routinely-collected data to be more meaningful for community health program management. We encourage further studies assessing CHW data quality and the impact training, supervision and other strategies have on improving it.
Literature
1.
go back to reference World Health Organization (WHO). (2007). Everybody’s business: Strengthening health systems to improve health outcomes: WHO’s framework for action. Geneva: WHO. World Health Organization (WHO). (2007). Everybody’s business: Strengthening health systems to improve health outcomes: WHO’s framework for action. Geneva: WHO.
2.
go back to reference Forster, M., Bailey, C., Brinkhof, M., et al. (2008). Electronic medical record systems, data quality and loss to follow-up: Survey of antiretroviral therapy programmes in resource-limited settings. Bulletin of the World Health Organization, 86, 939–947.PubMedCentralPubMedCrossRef Forster, M., Bailey, C., Brinkhof, M., et al. (2008). Electronic medical record systems, data quality and loss to follow-up: Survey of antiretroviral therapy programmes in resource-limited settings. Bulletin of the World Health Organization, 86, 939–947.PubMedCentralPubMedCrossRef
3.
go back to reference Garrib, A., Stoops, N., McKenzie, A., et al. (2008). An evaluation of the district health information system in rural South Africa. South African Medical Journal, 98(7), 549–552.PubMed Garrib, A., Stoops, N., McKenzie, A., et al. (2008). An evaluation of the district health information system in rural South Africa. South African Medical Journal, 98(7), 549–552.PubMed
4.
go back to reference Makombe, S. D., Hochgesang, M., Jahn, A., et al. (2008). Assessing the quality of data aggregated by antiretroviral treatment clinics in Malawi. Bulletin of the World Health Organization, 86(4), 310–314.PubMedCentralPubMedCrossRef Makombe, S. D., Hochgesang, M., Jahn, A., et al. (2008). Assessing the quality of data aggregated by antiretroviral treatment clinics in Malawi. Bulletin of the World Health Organization, 86(4), 310–314.PubMedCentralPubMedCrossRef
5.
go back to reference Maokola, W., Willey, B., Shirima, K., et al. (2011). Enhancing the routine health information system in rural southern Tanzania: Successes, challenges and lessons learned. Tropical Medicine & International Health, 16(6), 721–730.CrossRef Maokola, W., Willey, B., Shirima, K., et al. (2011). Enhancing the routine health information system in rural southern Tanzania: Successes, challenges and lessons learned. Tropical Medicine & International Health, 16(6), 721–730.CrossRef
6.
go back to reference Mate, K. S., Bennett, B., Mphatswe, W., Barker, P., & Rollins, N. (2009). Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS One, 4(5), e5483.PubMedCentralPubMedCrossRef Mate, K. S., Bennett, B., Mphatswe, W., Barker, P., & Rollins, N. (2009). Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS One, 4(5), e5483.PubMedCentralPubMedCrossRef
7.
go back to reference Mavimbe, J., Braa, J., & Bjune, G. (2005). Assessing immunization data quality from routine reports in Mozambique. BMC Public Health, 5(108), 1–8. Mavimbe, J., Braa, J., & Bjune, G. (2005). Assessing immunization data quality from routine reports in Mozambique. BMC Public Health, 5(108), 1–8.
8.
go back to reference Ndira, S. P., Rosenberger, K. D., & Wetter, T. (2008). Assessment of data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda): A qualitative and quantitative comparative study. Methods of Information in Medicine, 47(6), 489–498.PubMed Ndira, S. P., Rosenberger, K. D., & Wetter, T. (2008). Assessment of data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda): A qualitative and quantitative comparative study. Methods of Information in Medicine, 47(6), 489–498.PubMed
9.
go back to reference Rowe, A. K., Kachur, S. P., Yoon, S. S., Lynch, M., Slutsker, L., & Steketee, R. W. (2009). Caution is required when using health facility-based data to evaluate the health impact of malaria control efforts in Africa. Malaria Journal, 8(209), 1–3. Rowe, A. K., Kachur, S. P., Yoon, S. S., Lynch, M., Slutsker, L., & Steketee, R. W. (2009). Caution is required when using health facility-based data to evaluate the health impact of malaria control efforts in Africa. Malaria Journal, 8(209), 1–3.
10.
go back to reference Bhutta, Z. A., Darmstadt, G. L., Hasan, B. S., & Haws, R. A. (2005). Community-based interventions for improving perinatal and neonatal health outcomes in developing countries: A review of the evidence. Pediatrics, 115(2 Suppl.), 519–617.PubMed Bhutta, Z. A., Darmstadt, G. L., Hasan, B. S., & Haws, R. A. (2005). Community-based interventions for improving perinatal and neonatal health outcomes in developing countries: A review of the evidence. Pediatrics, 115(2 Suppl.), 519–617.PubMed
11.
go back to reference Bhutta, Z. A., Chopra, M., Axelson, H., et al. (2010). Countdown to 2015 decade report (2000–2010): Taking stock of maternal, newborn, and child survival. Lancet, 375(9730), 2032–2044.PubMedCrossRef Bhutta, Z. A., Chopra, M., Axelson, H., et al. (2010). Countdown to 2015 decade report (2000–2010): Taking stock of maternal, newborn, and child survival. Lancet, 375(9730), 2032–2044.PubMedCrossRef
12.
go back to reference Hafeez, A., Mohamud, B. K., Shiekh, M. R., Shah, S. A., & Jooma, R. (2011). Lady health workers programme in Pakistan: Challenges, achievements and the way forward. Journal of the Pakistan Medical Association, 61(3), 210–215.PubMed Hafeez, A., Mohamud, B. K., Shiekh, M. R., Shah, S. A., & Jooma, R. (2011). Lady health workers programme in Pakistan: Challenges, achievements and the way forward. Journal of the Pakistan Medical Association, 61(3), 210–215.PubMed
13.
go back to reference Haines, A., Sanders, D., Lehmann, U., et al. (2007). Achieving child survival goals: Potential contribution of community health workers. Lancet, 369(9579), 2121–2131.PubMedCrossRef Haines, A., Sanders, D., Lehmann, U., et al. (2007). Achieving child survival goals: Potential contribution of community health workers. Lancet, 369(9579), 2121–2131.PubMedCrossRef
14.
go back to reference Lehmann, U., & Sanders, D. (2007). Community health workers: What do we know about them? The state of the evidence on programmes, activities, costs and impact on health outcomes of using community health workers. Geneva: World Health Organization. Lehmann, U., & Sanders, D. (2007). Community health workers: What do we know about them? The state of the evidence on programmes, activities, costs and impact on health outcomes of using community health workers. Geneva: World Health Organization.
15.
go back to reference Prata, N., Quaiyum, M. A., Passano, P., et al. (2012). Training traditional birth attendants to use misoprostol and an absorbent delivery mat in home births. Social Science and Medicine, 75(11), 2021–2027.PubMedCrossRef Prata, N., Quaiyum, M. A., Passano, P., et al. (2012). Training traditional birth attendants to use misoprostol and an absorbent delivery mat in home births. Social Science and Medicine, 75(11), 2021–2027.PubMedCrossRef
16.
go back to reference United Nations Statistics Division. (2012). Millennium Development Goals Indicators: The official United Nations site for the MDG indicators: About the millennium development goals indicators. New York: United Nations. United Nations Statistics Division. (2012). Millennium Development Goals Indicators: The official United Nations site for the MDG indicators: About the millennium development goals indicators. New York: United Nations.
17.
go back to reference Young, M., Wolfheim, C., Marsh, D. R., & Hammamy, D. (2012). World Health Organization/United Nations Children’s Fund joint statement on integrated community case management: An equity-focused strategy to improve access to essential treatment services for children. American Journal of Tropical Medicine and Hygeine, 87(5 Suppl.), 6–10.CrossRef Young, M., Wolfheim, C., Marsh, D. R., & Hammamy, D. (2012). World Health Organization/United Nations Children’s Fund joint statement on integrated community case management: An equity-focused strategy to improve access to essential treatment services for children. American Journal of Tropical Medicine and Hygeine, 87(5 Suppl.), 6–10.CrossRef
18.
go back to reference Otieno, C. F., Kaseje, D., Ochieng, B. M., & Githae, M. N. (2011). Reliability of community health worker collected data for planning and policy in a peri-urban area of Kisumu, Kenya. Journal of Community Health, 37(1), 48–53.PubMedCentralCrossRef Otieno, C. F., Kaseje, D., Ochieng, B. M., & Githae, M. N. (2011). Reliability of community health worker collected data for planning and policy in a peri-urban area of Kisumu, Kenya. Journal of Community Health, 37(1), 48–53.PubMedCentralCrossRef
19.
go back to reference Admon, A. J., Bazile, J., Makungwa, H., et al. (2013). Assessing and improving data quality from community health workers: A successful intervention in Neno, Malawi. Public Health Action, 3(1), 56–59.PubMedCentralPubMedCrossRef Admon, A. J., Bazile, J., Makungwa, H., et al. (2013). Assessing and improving data quality from community health workers: A successful intervention in Neno, Malawi. Public Health Action, 3(1), 56–59.PubMedCentralPubMedCrossRef
20.
go back to reference Mahmood, S., & Ayub, M. (2010). Accuracy of primary health care statistics reported by community based lady health workers in district Lahore. Journal of the Pakistan Medical Association, 60(8), 649–653.PubMed Mahmood, S., & Ayub, M. (2010). Accuracy of primary health care statistics reported by community based lady health workers in district Lahore. Journal of the Pakistan Medical Association, 60(8), 649–653.PubMed
21.
go back to reference Helleringer, S. (2010). Heilbrunn department of population and family health, Columbia University. Personal Communication. Helleringer, S. (2010). Heilbrunn department of population and family health, Columbia University. Personal Communication.
22.
go back to reference MoH [Rwanda]. (2008). National community health policy. Rwanda: MoH. MoH [Rwanda]. (2008). National community health policy. Rwanda: MoH.
23.
go back to reference Mugeni, C. (2012). Community health desk, Rwanda Ministry of Health. Personal Communication. Mugeni, C. (2012). Community health desk, Rwanda Ministry of Health. Personal Communication.
24.
go back to reference Mitsunaga, T., Hedt-Gauthier, B., Ngizwenayo, E., Farmer, D. B., et al. (2013). Utilizing community health worker data for program management and evaluation: Systems for data quality assessments and baseline results from Rwanda. Social Science and Medicine, 85, 87–92.PubMedCrossRef Mitsunaga, T., Hedt-Gauthier, B., Ngizwenayo, E., Farmer, D. B., et al. (2013). Utilizing community health worker data for program management and evaluation: Systems for data quality assessments and baseline results from Rwanda. Social Science and Medicine, 85, 87–92.PubMedCrossRef
25.
go back to reference Mugeni, C., Levine, A. C., Munyaneza, R. M., et al. (2014). Nationwide implementation of integrated community case management of childhood illness in Rwanda. Global Health, Science and Practice, 2(3), 328–341.PubMedCentralPubMedCrossRef Mugeni, C., Levine, A. C., Munyaneza, R. M., et al. (2014). Nationwide implementation of integrated community case management of childhood illness in Rwanda. Global Health, Science and Practice, 2(3), 328–341.PubMedCentralPubMedCrossRef
26.
go back to reference MoH [Rwanda]. (2011). Trainer’s guide: “Integrated management of child illness” community case management. Rwanda: MoH. MoH [Rwanda]. (2011). Trainer’s guide: “Integrated management of child illness” community case management. Rwanda: MoH.
27.
go back to reference Mugeni, C. (2011). Community health program in Rwanda overview: Presentation at workshop: Evidence to policy: Using impact evaluation and data for informed decision making. Kigali, Rwanda. Mugeni, C. (2011). Community health program in Rwanda overview: Presentation at workshop: Evidence to policy: Using impact evaluation and data for informed decision making. Kigali, Rwanda.
28.
go back to reference MoH [Rwanda]. (2008). Procedures manual for the Rwanda community health information system (SISCom): Section I: Data recording and reporting; version 1.1. Rwanda: MoH. MoH [Rwanda]. (2008). Procedures manual for the Rwanda community health information system (SISCom): Section I: Data recording and reporting; version 1.1. Rwanda: MoH.
29.
go back to reference MoH [Rwanda] Community Health Desk. (2013). Community PBF [performance-based financing] in Rwanda. Presentation. MoH [Rwanda] Community Health Desk. (2013). Community PBF [performance-based financing] in Rwanda. Presentation.
30.
go back to reference Lwanga, S. K., Lemeshow, S., & World Health Organization. (1991). Sample size determination in health studies: A practical manual. Geneva: World Health Organization. Lwanga, S. K., Lemeshow, S., & World Health Organization. (1991). Sample size determination in health studies: A practical manual. Geneva: World Health Organization.
31.
go back to reference Pagano, M., & Valadez, J. J. (2010). Commentary: Understanding practical lot quality assurance sampling. International Journal of Epidemiology, 39(1), 69–71.PubMedCentralPubMedCrossRef Pagano, M., & Valadez, J. J. (2010). Commentary: Understanding practical lot quality assurance sampling. International Journal of Epidemiology, 39(1), 69–71.PubMedCentralPubMedCrossRef
32.
go back to reference Robertson, S. E., & Valadez, J. J. (2006). Global review of health care surveys using lot quality assurance sampling (LQAS), 1984–2004. Social Science and Medicine, 63, 1648–1660.PubMedCrossRef Robertson, S. E., & Valadez, J. J. (2006). Global review of health care surveys using lot quality assurance sampling (LQAS), 1984–2004. Social Science and Medicine, 63, 1648–1660.PubMedCrossRef
33.
go back to reference Valadez, J. J. (1991). Assessing child survival programs in developing countries: Testing lot quality assurance sampling. Boston, MA: Harvard School of Public Health; Distributed by Harvard University Press. Dept. of Population and International Health. Valadez, J. J. (1991). Assessing child survival programs in developing countries: Testing lot quality assurance sampling. Boston, MA: Harvard School of Public Health; Distributed by Harvard University Press. Dept. of Population and International Health.
34.
go back to reference Hedt-Gauthier, B. L., Mitsunaga, T., Hund, L., Olives, C., & Pagano, P. (2013). The effect of clustering on lot quality assurance sampling: A probabilistic model to correctly calculate sample sizes with an illustration of community health worker data quality assessments. Emerging Themes in Epidemiology, 10(11), 1–11. Hedt-Gauthier, B. L., Mitsunaga, T., Hund, L., Olives, C., & Pagano, P. (2013). The effect of clustering on lot quality assurance sampling: A probabilistic model to correctly calculate sample sizes with an illustration of community health worker data quality assessments. Emerging Themes in Epidemiology, 10(11), 1–11.
35.
go back to reference Gilroy, K., Winch, P. J., Diawara, A., et al. (2004). Impact of IMCI training and language used by provider on quality of counseling provided to parents of sick children in Bougouni District, Mali. Patient Education and Counseling, 54(1), 35–44.PubMedCrossRef Gilroy, K., Winch, P. J., Diawara, A., et al. (2004). Impact of IMCI training and language used by provider on quality of counseling provided to parents of sick children in Bougouni District, Mali. Patient Education and Counseling, 54(1), 35–44.PubMedCrossRef
36.
go back to reference Rowe, A. K., Lama, M., Onikpo, F., & Deming, M. S. (2002). Design effects and intraclass correlation coefficients from a health facility cluster survey in Benin. International Journal for Quality in Health Care, 14(6), 521–523.PubMedCrossRef Rowe, A. K., Lama, M., Onikpo, F., & Deming, M. S. (2002). Design effects and intraclass correlation coefficients from a health facility cluster survey in Benin. International Journal for Quality in Health Care, 14(6), 521–523.PubMedCrossRef
37.
go back to reference The Global Fund. (2008). Data quality audit tool: Guidelines for implementation. Geneva: The Global Fund. The Global Fund. (2008). Data quality audit tool: Guidelines for implementation. Geneva: The Global Fund.
38.
go back to reference WHO. (2008). Assessing the national health information system: An assessment tool: Version 4.00. Geneva: WHO. WHO. (2008). Assessing the national health information system: An assessment tool: Version 4.00. Geneva: WHO.
39.
go back to reference Mphatswe, W., Mate, K. S., Bennett, B., et al. (2012). Improving public health information: A data quality intervention in KwaZulu-Natal, South Africa. Bulletin of the World Health Organization, 90, 176–182.PubMedCentralPubMedCrossRef Mphatswe, W., Mate, K. S., Bennett, B., et al. (2012). Improving public health information: A data quality intervention in KwaZulu-Natal, South Africa. Bulletin of the World Health Organization, 90, 176–182.PubMedCentralPubMedCrossRef
40.
go back to reference Neupane, S., Odendaal, W., Friedman, I., Jassat, W., Schneider, H., & Doherty, T. (2014). Comparing a paper based monitoring and evaluation system to a mHealth system to support the national community health worker programme, South Africa: An evaluation. BMC Medical Informatics and Decision Making, 14(1), 69.PubMedCentralPubMedCrossRef Neupane, S., Odendaal, W., Friedman, I., Jassat, W., Schneider, H., & Doherty, T. (2014). Comparing a paper based monitoring and evaluation system to a mHealth system to support the national community health worker programme, South Africa: An evaluation. BMC Medical Informatics and Decision Making, 14(1), 69.PubMedCentralPubMedCrossRef
41.
go back to reference Marshall, A., & Fehringer, J. (2013). Supportive supervision in monitoring and evaluation with community-based health staff in HIV programs: A case study from Haiti. Chapel Hill, NC: MEASURE Evaluation. Marshall, A., & Fehringer, J. (2013). Supportive supervision in monitoring and evaluation with community-based health staff in HIV programs: A case study from Haiti. Chapel Hill, NC: MEASURE Evaluation.
42.
go back to reference Marshall, A., & Fehringer, J. (2014). Supportive supervision in monitoring and evaluation with community-based health staff in HIV programs: A case study from Ethiopia. Chapel Hill, NC: MEASURE Evaluation. Marshall, A., & Fehringer, J. (2014). Supportive supervision in monitoring and evaluation with community-based health staff in HIV programs: A case study from Ethiopia. Chapel Hill, NC: MEASURE Evaluation.
43.
go back to reference Aqil, A., Lippeveld, T., & Hozumi, D. (2009). PRISM framework: A paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy and Planning, 24(3), 217–228.PubMedCentralPubMedCrossRef Aqil, A., Lippeveld, T., & Hozumi, D. (2009). PRISM framework: A paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy and Planning, 24(3), 217–228.PubMedCentralPubMedCrossRef
44.
go back to reference MEASURE Evaluation. (2012). Data demand and information use in the health sector: Case study series. Chapel Hill: MEASURE Evaluation. MEASURE Evaluation. (2012). Data demand and information use in the health sector: Case study series. Chapel Hill: MEASURE Evaluation.
45.
go back to reference Nutley, T. (2012). Improving data use in decision making: An intervention to strengthen health systems. Chapel Hill: MEASURE Evaluation. Nutley, T. (2012). Improving data use in decision making: An intervention to strengthen health systems. Chapel Hill: MEASURE Evaluation.
46.
go back to reference Biddlecom, A. E., & Fapohunda, B. M. (1998). Covert contraceptive use: Prevalence, motivations, and consequences. Studies in Family Planning, 29(4), 360–372.PubMedCrossRef Biddlecom, A. E., & Fapohunda, B. M. (1998). Covert contraceptive use: Prevalence, motivations, and consequences. Studies in Family Planning, 29(4), 360–372.PubMedCrossRef
47.
go back to reference Castle, S., Konate, M. K., Ulin, P. R., & Martin, S. (1999). A qualitative study of clandestine contraceptive use in urban Mali. Studies in Family Planning, 30(3), 231–248.PubMedCrossRef Castle, S., Konate, M. K., Ulin, P. R., & Martin, S. (1999). A qualitative study of clandestine contraceptive use in urban Mali. Studies in Family Planning, 30(3), 231–248.PubMedCrossRef
48.
go back to reference Mitsunaga, T., Hedt-Gauthier, B., Ngizwenayo, E., Bertrand Farmer, D., et al. (2012). Improving child health data quality by community health workers: A routine system and baseline results. In: Annual international child health conference, Kigali, Rwanda. Mitsunaga, T., Hedt-Gauthier, B., Ngizwenayo, E., Bertrand Farmer, D., et al. (2012). Improving child health data quality by community health workers: A routine system and baseline results. In: Annual international child health conference, Kigali, Rwanda.
Metadata
Title
Data for Program Management: An Accuracy Assessment of Data Collected in Household Registers by Community Health Workers in Southern Kayonza, Rwanda
Authors
Tisha Mitsunaga
Bethany L. Hedt-Gauthier
Elias Ngizwenayo
Didi Bertrand Farmer
Erick Gaju
Peter Drobac
Paulin Basinga
Lisa Hirschhorn
Michael L. Rich
Peter J. Winch
Fidele Ngabo
Cathy Mugeni
Publication date
01-08-2015
Publisher
Springer US
Published in
Journal of Community Health / Issue 4/2015
Print ISSN: 0094-5145
Electronic ISSN: 1573-3610
DOI
https://doi.org/10.1007/s10900-014-9977-9

Other articles of this Issue 4/2015

Journal of Community Health 4/2015 Go to the issue