Skip to main content
Top
Published in: BMC Public Health 1/2013

Open Access 01-12-2013 | Research article

Can we trust measures of healthcare utilization from household surveys?

Authors: Evelyn Korkor Ansah, Timothy Powell-Jackson

Published in: BMC Public Health | Issue 1/2013

Login to get access

Abstract

Background

The research community relies heavily on measures of healthcare utilization from household surveys to understand health seeking choices and to evaluate interventions in developing countries. Such measures are known to suffer from recall problems but there is limited evidence of whether the method of data collection affects evaluation findings. We compared the results of a randomized trial of free healthcare using utilization data from two sources.

Methods

Data are from a study in Ghana, in which 2,194 households containing 2,592 children under 5 y old were randomized into a prepayment scheme providing free primary and some referral care, or to a control group whose families paid user fees for healthcare. Data on morbidity and health seeking behaviour were collected using a standard household survey administered at endline and a pictorial diary given to households over a six month period, collected at monthly intervals.

Results

Self-reported measures of morbidity and healthcare utilization were substantially lower in the household survey than the pictorial diary when the recall period was over a month. Introducing free healthcare had a positive effect on primary care visits based on the pictorial diary and a non-significant negative effect according to the household survey. Using any clinic visit in the past month as the outcome, the difference in the effect of free care between the two data collection methods was 3.6 percentage points (p = 0.078).

Conclusions

The findings raise methodological concerns about measures of healthcare utilization from household surveys, particularly in the evaluation of health financing interventions.
Appendix
Available only for authorised users
Literature
1.
go back to reference King G, Gakidou E, Imai K, Lakin J, Moore RT, Nall C, Ravishankar N, Vargas M, Tellez-Rojo MM, Avila JE, Avila MH, Llamas HH: Public policy for the poor? A randomised assessment of the Mexican universal health insurance programme. Lancet. 2009, 373: 1447-1454. 10.1016/S0140-6736(09)60239-7.CrossRefPubMed King G, Gakidou E, Imai K, Lakin J, Moore RT, Nall C, Ravishankar N, Vargas M, Tellez-Rojo MM, Avila JE, Avila MH, Llamas HH: Public policy for the poor? A randomised assessment of the Mexican universal health insurance programme. Lancet. 2009, 373: 1447-1454. 10.1016/S0140-6736(09)60239-7.CrossRefPubMed
2.
go back to reference Babiarz KS, Miller G, Yi H, Zhang L, Rozelle S: New evidence on the impact of China's New Rural Cooperative Medical Scheme and its implications for rural primary healthcare: multivariate difference-in-difference analysis. BMJ. 2010, 341: c5617-10.1136/bmj.c5617.CrossRefPubMed Babiarz KS, Miller G, Yi H, Zhang L, Rozelle S: New evidence on the impact of China's New Rural Cooperative Medical Scheme and its implications for rural primary healthcare: multivariate difference-in-difference analysis. BMJ. 2010, 341: c5617-10.1136/bmj.c5617.CrossRefPubMed
3.
go back to reference Wagstaff A, Lindelow M, Jun G, Ling X, Juncheng Q: Extending health insurance to the rural population: an impact evaluation of China's new cooperative medical scheme. J Health Econ. 2009, 28: 1-19. 10.1016/j.jhealeco.2008.10.007.CrossRefPubMed Wagstaff A, Lindelow M, Jun G, Ling X, Juncheng Q: Extending health insurance to the rural population: an impact evaluation of China's new cooperative medical scheme. J Health Econ. 2009, 28: 1-19. 10.1016/j.jhealeco.2008.10.007.CrossRefPubMed
4.
go back to reference Mills A, Ataguba JE, Akazili J, Borghi J, Garshong B, Makawia S, Mtei G, Harris B, Macha J, Meheus F, McIntyre D: Equity in financing and use of health care in Ghana, South Africa, and Tanzania: implications for paths to universal coverage. Lancet. 2012, 380: 126-133. 10.1016/S0140-6736(12)60357-2.CrossRefPubMed Mills A, Ataguba JE, Akazili J, Borghi J, Garshong B, Makawia S, Mtei G, Harris B, Macha J, Meheus F, McIntyre D: Equity in financing and use of health care in Ghana, South Africa, and Tanzania: implications for paths to universal coverage. Lancet. 2012, 380: 126-133. 10.1016/S0140-6736(12)60357-2.CrossRefPubMed
5.
go back to reference O'Donnell O, van Doorslaer E, Wagstaff A, Lindelow M: Analysing health equity using household survey data: a guide to techniques and their implementation. 2008, Washington DC: World Bank O'Donnell O, van Doorslaer E, Wagstaff A, Lindelow M: Analysing health equity using household survey data: a guide to techniques and their implementation. 2008, Washington DC: World Bank
7.
go back to reference Das J, Hammer J, Sanchez-Paramo C: The impact of recall periods on reported morbidity and health seeking behavior. J Dev Econ. 2012, 98: 76-88. 10.1016/j.jdeveco.2011.07.001.CrossRef Das J, Hammer J, Sanchez-Paramo C: The impact of recall periods on reported morbidity and health seeking behavior. J Dev Econ. 2012, 98: 76-88. 10.1016/j.jdeveco.2011.07.001.CrossRef
8.
go back to reference Arnold BF, Galiani S, Ram PK, Hubbard AE, Briceno B, Gertler PJ, Colford JM: Optimal recall period for caregiver-reported illness in risk factor and intervention studies: a multicountry study. Am J Epidemiol. 2013, 177: 361-370. 10.1093/aje/kws281.CrossRefPubMed Arnold BF, Galiani S, Ram PK, Hubbard AE, Briceno B, Gertler PJ, Colford JM: Optimal recall period for caregiver-reported illness in risk factor and intervention studies: a multicountry study. Am J Epidemiol. 2013, 177: 361-370. 10.1093/aje/kws281.CrossRefPubMed
9.
go back to reference Higgins J, Altman D, Sterne J: Assessing risk of bias in included studies. Cochrane Handbook for Systematic Reviews of Interventions Version 510. Edited by: Higgins J, Green S. 2011, Available from http://www.cochrane-handbook.org. The Cochrane Collaboration Higgins J, Altman D, Sterne J: Assessing risk of bias in included studies. Cochrane Handbook for Systematic Reviews of Interventions Version 510. Edited by: Higgins J, Green S. 2011, Available from http://​www.​cochrane-handbook.​org. The Cochrane Collaboration
10.
go back to reference Ansah EK, Narh-Bana S, Asiamah S, Dzordzordzi V, Biantey K, Dickson K, Gyapong JO, Koram KA, Greenwood BM, Mills A, Whitty CJ: Effect of removing direct payment for health care on utilisation and health outcomes in Ghanaian children: a randomised controlled trial. PLoS Med. 2009, 6 (1): 48-58.CrossRef Ansah EK, Narh-Bana S, Asiamah S, Dzordzordzi V, Biantey K, Dickson K, Gyapong JO, Koram KA, Greenwood BM, Mills A, Whitty CJ: Effect of removing direct payment for health care on utilisation and health outcomes in Ghanaian children: a randomised controlled trial. PLoS Med. 2009, 6 (1): 48-58.CrossRef
11.
go back to reference Powell-Jackson T, Hanson K, Whitty CJ, Ansah EK: Who Benefits from Removing User Fees for Health Care? Evidence from a Randomised Experiment in Ghana. 2012 Powell-Jackson T, Hanson K, Whitty CJ, Ansah EK: Who Benefits from Removing User Fees for Health Care? Evidence from a Randomised Experiment in Ghana. 2012
12.
go back to reference Wiseman V, Conteh L, Matovu F: Using diaries to collect data in resource-poor settings: questions on design and implementation. Health Policy Plan. 2005, 20: 394-404. 10.1093/heapol/czi042.CrossRefPubMed Wiseman V, Conteh L, Matovu F: Using diaries to collect data in resource-poor settings: questions on design and implementation. Health Policy Plan. 2005, 20: 394-404. 10.1093/heapol/czi042.CrossRefPubMed
13.
go back to reference Witschi C: Short-term dietary recall and recording methods. Nutritional Epidemiology. Edited by: Willett W. 1990, New York: Oxford University Press Witschi C: Short-term dietary recall and recording methods. Nutritional Epidemiology. Edited by: Willett W. 1990, New York: Oxford University Press
14.
go back to reference Coxon A: Diaries and sexual behavior: The use of sexual diaries as method and substance in researching gay men’s response to HIV/AIDS. Challenge and innovation and methodological advances in social research on HIV/AIDS: Social aspects of AIDS. Edited by: Boulton M. 1994, London: Taylor & Francis Coxon A: Diaries and sexual behavior: The use of sexual diaries as method and substance in researching gay men’s response to HIV/AIDS. Challenge and innovation and methodological advances in social research on HIV/AIDS: Social aspects of AIDS. Edited by: Boulton M. 1994, London: Taylor & Francis
15.
go back to reference Gillmore M, Gaylord J, Hartway J, Hoppe MJ, Morrison DM, Leigh BC, Rainey DT: Daily data collection of sexual and other health-related behaviors. J Sex Res. 2001, 38: 35-42. 10.1080/00224490109552068.CrossRef Gillmore M, Gaylord J, Hartway J, Hoppe MJ, Morrison DM, Leigh BC, Rainey DT: Daily data collection of sexual and other health-related behaviors. J Sex Res. 2001, 38: 35-42. 10.1080/00224490109552068.CrossRef
16.
go back to reference Hornsby PP, Wilcox AJ: Validity of questionnaire information on frequency of coitus. Am J Epidemiol. 1989, 130: 94-99.PubMed Hornsby PP, Wilcox AJ: Validity of questionnaire information on frequency of coitus. Am J Epidemiol. 1989, 130: 94-99.PubMed
17.
go back to reference McAuliffe TL, DiFranceisco W, Reed BR: Effects of question format and collection mode on the accuracy of retrospective surveys of health risk behavior: a comparison with daily sexual activity diaries. Health Psychol. 2007, 26: 60-67.CrossRefPubMed McAuliffe TL, DiFranceisco W, Reed BR: Effects of question format and collection mode on the accuracy of retrospective surveys of health risk behavior: a comparison with daily sexual activity diaries. Health Psychol. 2007, 26: 60-67.CrossRefPubMed
18.
go back to reference Leigh BC, Gillmore MR, Morrison DM: Comparison of diary and retrospective measures for recording alcohol consumption and sexual activity. J Clin Epidemiol. 1998, 51: 119-127. 10.1016/S0895-4356(97)00262-X.CrossRefPubMed Leigh BC, Gillmore MR, Morrison DM: Comparison of diary and retrospective measures for recording alcohol consumption and sexual activity. J Clin Epidemiol. 1998, 51: 119-127. 10.1016/S0895-4356(97)00262-X.CrossRefPubMed
19.
go back to reference Poikolainen K, Karkkainen P: Diary gives more accurate information about alcohol consumption than questionnaire. Drug Alcohol Depend. 1993, 11: 209-216.CrossRef Poikolainen K, Karkkainen P: Diary gives more accurate information about alcohol consumption than questionnaire. Drug Alcohol Depend. 1993, 11: 209-216.CrossRef
20.
go back to reference Coxon AP, Coxon NH, Weatherburn P, Hunt AJ, Hickson F, Davies PM, McManus TJ: Sex role separation in sexual diaries of homosexual men. AIDS. 1993, 7: 877-882. 10.1097/00002030-199306000-00018.CrossRefPubMed Coxon AP, Coxon NH, Weatherburn P, Hunt AJ, Hickson F, Davies PM, McManus TJ: Sex role separation in sexual diaries of homosexual men. AIDS. 1993, 7: 877-882. 10.1097/00002030-199306000-00018.CrossRefPubMed
22.
go back to reference Whitty C, Jones RJ: A comparison of prospective and retrospective diary methods of assessing alcohol use among university undergraduates. J Public Health Med. 1992, 14: 264-270.PubMed Whitty C, Jones RJ: A comparison of prospective and retrospective diary methods of assessing alcohol use among university undergraduates. J Public Health Med. 1992, 14: 264-270.PubMed
23.
go back to reference Wheeler L, Reis H: Self-recording of everyday life events: origins, types and uses. J Pers. 1991, 59: 339-354. 10.1111/j.1467-6494.1991.tb00252.x.CrossRef Wheeler L, Reis H: Self-recording of everyday life events: origins, types and uses. J Pers. 1991, 59: 339-354. 10.1111/j.1467-6494.1991.tb00252.x.CrossRef
24.
go back to reference Stone AA, Kessler RC, Haythornthwaite JA: Measuring daily events and experiences: decisions for the researcher. J Pers. 1991, 59: 575-607. 10.1111/j.1467-6494.1991.tb00260.x.CrossRefPubMed Stone AA, Kessler RC, Haythornthwaite JA: Measuring daily events and experiences: decisions for the researcher. J Pers. 1991, 59: 575-607. 10.1111/j.1467-6494.1991.tb00260.x.CrossRefPubMed
25.
go back to reference Cannell C, Marguiz K, Laurent A: A Summary of Studies of Interviewing Methodology. 1977, Washington, D.C.: Government Printing OfficeCrossRef Cannell C, Marguiz K, Laurent A: A Summary of Studies of Interviewing Methodology. 1977, Washington, D.C.: Government Printing OfficeCrossRef
26.
go back to reference OECD: Private health expenditure - report on the conclusions of the project: 11th meeting of health accounts experts and correspondents for health expenditure data. 2009, Paris: Organisation for Economic Co-operation and Development OECD: Private health expenditure - report on the conclusions of the project: 11th meeting of health accounts experts and correspondents for health expenditure data. 2009, Paris: Organisation for Economic Co-operation and Development
27.
go back to reference Ross DA, Vaughan JP: Health interview surveys in developing countries: a methodological review. Stud Fam Plann. 1986, 17: 78-94. 10.2307/1967068.CrossRefPubMed Ross DA, Vaughan JP: Health interview surveys in developing countries: a methodological review. Stud Fam Plann. 1986, 17: 78-94. 10.2307/1967068.CrossRefPubMed
Metadata
Title
Can we trust measures of healthcare utilization from household surveys?
Authors
Evelyn Korkor Ansah
Timothy Powell-Jackson
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Public Health / Issue 1/2013
Electronic ISSN: 1471-2458
DOI
https://doi.org/10.1186/1471-2458-13-853

Other articles of this Issue 1/2013

BMC Public Health 1/2013 Go to the issue