Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2016 | Research article

Bias of health estimates obtained from chronic disease and risk factor surveillance systems using telephone population surveys in Australia: results from a representative face-to-face survey in Australia from 2010 to 2013

Authors: Eleonora Dal Grande, Catherine R. Chittleborough, Stefano Campostrini, Anne W. Taylor

Published in: BMC Medical Research Methodology | Issue 1/2016

Abstract

Background

Emerging communication technologies have had an impact on population-based telephone surveys worldwide. Our objective was to examine the potential biases of health estimates in South Australia, a state of Australia, obtained via current landline telephone survey methodologies and to report on the impact of mobile-only household on household surveys.

Methods

Data from an annual multi-stage, systematic, clustered area, face-to-face population survey, Health Omnibus Survey (approximately 3000 interviews annually), included questions about telephone ownership to assess the population that were non-contactable by current telephone sampling methods (2006 to 2013). Univariable analyses (2010 to 2013) and trend analyses were conducted for sociodemographic and health indicator variables in relation to telephone status. Relative coverage biases (RCB) of two hypothetical telephone samples was undertaken by examining the prevalence estimates of health status and health risk behaviours (2010 to 2013): directory-listed numbers, consisting mainly of landline telephone numbers and a small proportion of mobile telephone numbers; and a random digit dialling (RDD) sample of landline telephone numbers which excludes mobile-only households.

Results

Telephone (landline and mobile) coverage in South Australia is very high (97 %). Mobile telephone ownership increased slightly (7.4 %), rising from 89.7 % in 2006 to 96.3 % in 2013; mobile-only households increased by 431 % over the eight year period from 5.2 % in 2006 to 27.6 % in 2013. Only half of the households have either a mobile or landline number listed in the telephone directory. There were small differences in the prevalence estimates for current asthma, arthritis, diabetes and obesity between the hypothetical telephone samples and the overall sample. However, prevalence estimate for diabetes was slightly underestimated (RCB value of −0.077) in 2013. Mixed RCB results were found for having a mental health condition for both telephone samples. Current smoking prevalence was lower for both hypothetical telephone samples in absolute differences and RCB values: −0.136 to −0.191 for RDD landline samples and −0.129 to −0.313 for directory-listed samples.

Conclusion

These findings suggest landline-based sampling frames used in Australia, when appropriately weighted, produce reliable representative estimates for some health indicators but not for all. Researchers need to be aware of their limitations and potential biased estimates.

Available only for authorised users

McQueen DV, Puska P. Global behavioral risk factor surveillance. Berlin: Springer; 2003.CrossRef

Taylor AW, Dal Grande E. Chronic disease and risk factor surveillance using the SA Monitoring and Surveillance System (SAMSS)–history, results and future challenges. Public Health Bulletin. 2008;5(3):17–21.

NSW Population Health Survey Methods - 2012 update [http://www.health.nsw.gov.au/surveys/other/Documents/population-health-surveys-methods-2012.pdf] Accessed 24 Aug 2014.

The WA Health and Wellbeing Surveillance System (WAHWSS), Design and Methodology, Technical Paper No 1 Version 2 [http://ww2.health.wa.gov.au/~/media/Files/Corporate/Reports%20and%20publications/Population%20surveys/2003-Technical-paperno1-Design-and-Methodology.ashx] 15 Apr 2016.

Behavioral Risk Factor Surveillance System. 2010 Summary Data Quality Report. In: BRFSS 2010 Survey Data and Documentation. Atlanta: Centers for Disease of Control and Prevention; 2011.

Bingle CL, Holowaty PH, Koren IE, Picard L, Steward PJ, Feltis SL. An evaluation of the Ontario Rapid Risk Factor Surveillance System. Can J Public Health. 2005;96(2):145–50.PubMed

Baldissera S, Campostrini S, Binkin N, Minardi V, Minelli G, Ferrante G, Salmaso S. Features and initial assessment of the Italian Behavioral Risk Factor Surveillance System (PASSI), 2007–2008. Prev Chronic Dis. 2011;8(1):A24.PubMedPubMedCentral

Campostrini S, McQueen D, Taylor A, Daly A. World Alliance for Risk Factor Surveillance White Paper on Surveillance and Health Promotion. AIMS Public Health. 2015;2(1):10–26.CrossRef

Choi BCK, Pak AWP, Ottoson JM. Understanding the basic concepts of public health surveillance [Speaker's Corner]. J Epidemiol Community Health. 2002;56(6):402.CrossRefPubMedPubMedCentral

10.

Taylor AW, Campostrini S, Gill TK, Carter P, Dal Grande E, Herriot M. The use of chronic disease risk factor surveillance systems for evidence-based decision-making: physical activity and nutrition as examples. Int J Public Health. 2010;55(4):243–9.CrossRefPubMed

11.

Lepkowski JM, Tucker C, Brick JM, Leeuw EDD, Japec L, Lavrakas PJ, Link MW, Sangster RL. Advances in Telephone Survey Methodology. Hoboken: Wiley; 2008.

12.

Mokdad AH. The Behavioral Risk Factors Surveillance System: Past, Present, and Future. Annu Rev Public Health. 2009;30(1):43–54.CrossRefPubMed

13.

Australian Communications and Media Authority. Communications Report 2009–10 Series. Report 2 - Take-up and use of voice services by Australian consumers. In: Corporate Publications. Melbourne: Commonwealth of Australia; 2010.

14.

Australian Communications and Media Authority. Communications report 2011–12. In: Corporate Publications. Melbourne: Commonwealth of Australia; 2013.

15.

Häder M, Häder S, Kühne M. Telephone Surveys in Europe. Research and Practice. Heidelberg: Springer; 2012.CrossRef

16.

Brick JM. The Future of Survey Sampling. Public Opin Q. 2011;75(5):872–88.CrossRef

17.

Kempf AM, Remington PL. New Challenges for Telephone Survey Research in the Twenty-First Century. Annu Rev Public Health. 2007;28(1):113–26.CrossRefPubMed

18.

Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opin Q. 2006;70(5):646–75.CrossRef

19.

Peytchev A. Consequences of Survey Nonresponse. ANNALS Am Acad Pol Soc Sci. 2013;645(1):88–111.CrossRef

20.

Dal Grande E, Taylor AW, Wilson DH. Is there a difference in health estimates between people with listed and unlisted telephone numbers? Aust N Z J Public Health. 2005;29(5):448–56.CrossRefPubMed

21.

Taylor AW, Wilson DH, Wakefield M. Differences in health estimates using telephone and door-to-door survey methods--a hypothetical exercise. Aust N Z J Public Health. 1998;22(2):223–6.CrossRefPubMed

22.

Wilson DH, Starr GJ, Taylor AW, Dal Grande E. Random digit dialling and Electronic White Pages samples compared: demographic profiles and health estimates. Aust N Z J Public Health. 1999;23(6):627–33.CrossRefPubMed

23.

Australian Bureau of Statistics. Household Telephone Connections, Australia, August 1991. In: Catalogue of Publications, vol. 4110.0. Canberra: Commonwealth of Australia; 1991.

24.

Steel D, Vella J, Harrington P. Quality issues in telephone surveys: coverage, non-response and quota sampling. Aust J Stat. 1996;38(1):15–34.CrossRef

25.

Guterbock TM, Diop A, Ellis JM, Holmes JL, Le KT. Who needs RDD? Combining directory listings with cell phone exchanges for an alternative telephone sampling frame. Soc Sci Res. 2011;40(3):860–72.CrossRef

26.

Dal Grande E, Taylor AW. Sampling and coverage issues of telephone surveys used for collecting health information in Australia: results from a face-to-face survey from 1999 to 2008. BMC Med Res Methodol. 2010;10:77.CrossRefPubMed

27.

Liu B, Brotherton J, Shellard D, Donovan B, Saville M, Kaldor J. Mobile phones are a viable option for surveying young Australian women: a comparison of two telephone survey methods. BMC Med Res Methodol. 2011;11(1):159.CrossRefPubMedPubMedCentral

28.

Blumberg SJ, Luke JV. Coverage Bias in Traditional Telephone Surveys of Low-Income and Young Adults. Public Opin Q. 2007;71(5):734–49.CrossRef

29.

Dal Grande E, Taylor AW, Fullerton SP, Chittleborough CR, Campostrini S. Can telephone surveys be used for collecting health information in Australia? Adelaide: Population Health Congress; 2012.

30.

Blumberg SJ, Luke JV. Wireless substitution: early release of estimates from the National Health Interview Survey, January-June 2012. In: The National Health Interview Survey Early Release Program. Atlanta: National Center for Health Statistics; 2012. p. 2012.

31.

Tucker C, Brick JM, Meekins B. Household Telephone Service and Usage Patterns in the United states in 2004: Implications for Telephone Samples. Public Opin Q. 2007;71(1):3–22.CrossRef

32.

Mohorko A, de Leeuw E, Hox J. Coverage bias in European telephone surveys: developments of landline and mobile phone coverage across countries and over time. In: Survey Methods: Insights from the Field. 2013.

33.

Kuusela V, Simpanen M. Spotlights on selected European countries, Finland. In: Häder M, Häder S, Kühne M, editors. Telephone Surveys in Europe: Research and Practice. Heidelberg: Springer; 2012. p. 37–45.

34.

Barr ML, van Ritten J, Steel D, Thackway S. Inclusion of mobile phone numbers into an ongoing population health survey in New South Wales, Australia: design, methods, call outcomes, costs and sample representativeness. BMC Med Res Methodol. 2012;12(1):177.CrossRefPubMedPubMedCentral

35.

Hu SS, Balluz L, Battaglia MP, Frankel MR. Improving Public Health Surveillance Using a Dual-Frame Survey of Landline and Cell Phone Numbers. Am J Epidemiol. 2011;173(6):703–11.CrossRefPubMed

36.

Australian Communications and Media Authority. Six emeriging trends in media and communications. Occasional paper. In: Corporate Publication. Commonwealth of Australia: Melbourne; 2014.

37.

Taylor AW, Dal Grande E, Wilson DH. The South Australian Health Omnibus Survey 15 years on: has public health benefited? Public Health Bulletin. 2006;3(1):30–2.

38.

Wilson DH, Wakefield M, Taylor AW. The South Australian Health Omnibus Survey. Health Promot J Austr. 1992;2:47–9.

39.

Australian Bureau of Statistics. Socio-Economic Indexes for Areas (SEIFA), 2011. In: Technical Paper. Canberra: Commonwealth of Australia; 2013.

40.

World Health Organization (WHO). Obesity: Preventing and managing the global epidemic. Geneva: WHO; 2000.

41.

Fuchs M, Busse B. The coverage bias of mobile web surveys across European countries. Int J Internet Sci. 2009;4(1):21–33.

42.

Keeter S, Kennedy C, Clark A, Tompson T, Mokrzycki M. What's Missing from National Landline RDD Surveys?: The Impact of the Growing Cell-Only Population. Public Opin Q. 2007;71(5):772–92.CrossRef

43.

Kuusela V, Callegaro M, Vehovar V. The influence of mobile telephones on telephone surveys. In: Lepkowski JM, Tucker C, Brick JM, de Leeuw ED, Japec L, Lavrakas PJ, Link MW, Sangster RL, editors. Advances in Telephone Survey Methodology. Hoboken: Wiley; 2008. p. 87–112.

44.

Australian Bureau of Statistics. 2011, Census, TableBuilder Pro. Canberra: Commonwealth of Australia; 2011.

45.

Holborn AT, Reavley NJ, Jorm AF. Differences between landline and mobile-only respondents in a dual-frame mental health literacy survey. Aust N Z J Public Health. 2012;36(2):192–3.CrossRefPubMed

46.

Livingston M, Dietze P, Ferris J, Pennay D, Hayes L, Lenton S. Surveying alcohol and other drug use through telephone sampling: a comparison of landline and mobile phone samples. BMC Med Res Methodol. 2013;13(1):41.CrossRefPubMedPubMedCentral

47.

Barr ML, Ferguson RA, Steel DG. Inclusion of mobile telephone numbers into an ongoing population health survey in New South Wales, Australia, using an overlapping dual-frame design: impact on the time series. BMC Res Notes. 2014;7:517.CrossRefPubMedPubMedCentral

48.

Link MW, Battaglia MP, Frankel MR, Osborn L, Mokdad AH. A Comparison of Address-Based Sampling (ABS) Versus Random-Digit Dialing (RDD) for General Population Surveys. Public Opin Q. 2008;72(1):6–27.CrossRef

49.

Pennay D, Vickers N. Dual-frame Omnibus Survey. Technical and methodological summary report. Melbourne: The Social Research Centre; 2012.

50.

Dillman DA, Tarnai J. Administrative issues in mixed mode surveys. In: Groves RM, Biemer PP, Lyberg LE, Massey JT, Nicholls WL, Waksberg J, editors. Telephone Survey Methodology. Hoboken: Wiley; 1988. p. 509–28.

51.

Brick JM, Lepkowski JM. Multiple mode and frame telephone surveys. In: Lepkowski JM, Tucker C, Brick JM, de Leeuw ED, Japec L, Lavrakas PJ, Link MW, Sangster RL, editors. Advances in Telephone Survey Methodology. Hoboken: Wiley; 2007. p. 149–69.CrossRef

52.

Dillman DA. Mail and telephone surveys: the total design method. New York: Wiley; 1978.

53.

Couper MP. The Future of Modes of Data Collection. Public Opin Q. 2011;75(5):889–908.CrossRef

54.

Pierannunzi C, Town M, Garvin W, Shaw FE, Balluz L. Methodologic Changes in the Behavioral Risk Factor Surveillance System in 2011 and Potential Effects on Prevalence Estimates. MMWR Morb Mortal Wkly Rep. 2012;61(22):410–3.

55.

Battaglia MP, Frankel MR, Link MW. Improving standard poststratification techniques for random-digit-dialing telephone surveys. Survey Res Methods. 2008;2(1):11–9.

56.

Frankel MR, Srinath KP, Hoaglin DC, Battaglia MP, Smith PJ, Wright RA, Khare M. Adjustments for non-telephone bias in random-digit-dialling surveys. Stat Med. 2003;22(9):1611–26.CrossRefPubMed

Title: Bias of health estimates obtained from chronic disease and risk factor surveillance systems using telephone population surveys in Australia: results from a representative face-to-face survey in Australia from 2010 to 2013
Authors: Eleonora Dal Grande
Catherine R. Chittleborough
Stefano Campostrini
Anne W. Taylor
Publication date: 01-12-2016
Publisher: BioMed Central
Published in: BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-016-0145-z

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Bias of health estimates obtained from chronic disease and risk factor surveillance systems using telephone population surveys in Australia: results from a representative face-to-face survey in Australia from 2010 to 2013

Abstract

Background

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2016

Exploring how individuals complete the choice tasks in a discrete choice experiment: an interview study

Evaluation of a multi-arm multi-stage Bayesian design for phase II drug selection trials – an example in hemato-oncology

No rationale for 1 variable per 10 events criterion for binary logistic regression analysis

Can the buck always be passed to the highest level of clustering?

Feasibility and validity of a statistical adjustment to reduce self-report bias of height and weight in wave 1 of the Add Health study

Eliciting parental support for the use of newborn blood spots for pediatric research