Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Research article

The feasibility of web surveys for obtaining patient-reported outcomes from cancer survivors: a randomized experiment comparing survey modes and brochure enclosures

Authors: Morgan M. Millar, Joanne W. Elena, Lisa Gallicchio, Sandra L. Edwards, Marjorie E. Carter, Kimberly A. Herget, Carol Sweeney

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

Central cancer registries are often used to survey population-based samples of cancer survivors. These surveys are typically administered via paper or telephone. In most populations, web surveys obtain much lower response rates than paper surveys. This study assessed the feasibility of web surveys for collecting patient-reported outcomes via a central cancer registry.

Methods

Potential participants were sampled from Utah Cancer Registry records. Sample members were randomly assigned to receive a web or paper survey, and then randomized to either receive or not receive an informative brochure describing the cancer registry. We calculated adjusted risk ratios with 95% confidence intervals to compare response likelihood and the demographic profile of respondents across study arms.

Results

The web survey response rate (43.2%) was lower than the paper survey (50.4%), but this difference was not statistically significant (adjusted risk ratio = 0.88, 95% confidence interval = 0.72, 1.07). The brochure also did not significantly influence the proportion responding (adjusted risk ratio = 1.03, 95% confidence interval = 0.85, 1.25). There were few differences in the demographic profiles of respondents across the survey modes. Older age increased likelihood of response to a paper questionnaire but not a web questionnaire.

Conclusions

Web surveys of cancer survivors are feasible without significantly influencing response rates, but providing a paper response option may be advisable particularly when surveying older individuals. Further examination of the varying effects of brochure enclosures across different survey modes is warranted.
Appendix
Available only for authorised users
Literature
1.
2.
3.
White MC, Babcock F, Hayes NS, Mariotto AB, Wong FL, Kohler BA, Weir HK. The history and use of cancer registry data by public health cancer control programs in the United States. Cancer. 2017;123(Suppl 24):4969–76.PubMedCrossRef
4.
Newcomb PA, Love RR, Phillips JL, Buckmaster BJ. Using a population-based cancer registry for recruitment in a pilot cancer control study. Prev Med. 1990;19(1):61–5.PubMedCrossRef
5.
Pakilit AT, Kahn BA, Petersen L, Abraham LS, Greendale GA, Ganz PA. Making effective use of tumor registries for cancer survivorship research. Cancer. 2001;92(5):1305–14.PubMedCrossRef
6.
Carpentier MY, Tiro JA, Savas LS, Bartholomew LK, Melhado TV, Coan SP, Argenbright KE, Vernon SW. Are cancer registries a viable tool for cancer survivor outreach? A feasibility study. J Cancer Surviv. 2013;7(1):155–63.PubMedCrossRef
7.
Kelly BJ, Fraze TK, Hornik RC. Response rates to a mailed survey of a representative sample of cancer patients randomly drawn from the Pennsylvania cancer registry: a randomized trial of incentive and length effects. BMC Med Res Methodol. 2010;10(1):65.PubMedPubMedCentralCrossRef
8.
Sweeney C, Edwards S, Baumgartner KB, Herrick JS, Palmer L, Murtaugh MA, Stroup A, Slattery ML. Recruiting Hispanic women for a population-based study: validity of surname search, and characteristics of non-participants. Am J Epidemiol. 2007;166(10):1210–9.PubMedCrossRef
9.
Ramirez AG, Miller AR, Gallion K, San Miguel de Majors S, Chalela P, García Arámburo S. Testing three different cancer genetics registry recruitment methods with Hispanic cancer patients and their family members previously registered in local cancer registries in Texas. Community Genet. 2008;11(4):215–23.PubMedCrossRef
10.
Pal T, Rocchio E, Garcia A, Rivers D, Vadaparampil S. Recruitment of black women for a study of inherited breast cancer using a cancer registry-based approach. Genet Test Mol Biomarkers. 2011;15(1–2):69–77.PubMedCrossRef
11.
Clinton-McHarg T, Carey M, Sanson-Fisher R, Tracey E. Recruitment of representative samples for low incidence cancer populations: do registries deliver? BMC Med Res Methodol. 2011;11(1):5.PubMedPubMedCentralCrossRef
12.
Wingo PA, Jamison PM, Hiatt RA, Weir HK, Gargiullo PM, Hutton M, Lee NC, Hall HI. Building the infrastructure for nationwide cancer surveillance and control--a comparison between the national program of cancer registries (npcr) and the surveillance, epidemiology, and end results (SEER) program (United States). Cancer Causes Control. 2003;14(2):175–93.PubMedCrossRef
13.
Tucker TC, Howe HL. Measuring the quality of population-based cancer registries: the naaccr perspective. J Registry Manag. 2001;28(1):41–4.
14.
Weir HK, Johnson CJ, Mariotto AB, Turner D, Wilson RJ, Nishri D, Ward KC. Evaluation of north American association of central cancer registries' (naaccr) data for use in population-based cancer survival studies. J Natl Cancer Inst Monogr. 2014;2014(49):198–209.PubMedPubMedCentralCrossRef
15.
Tucker TC, Durbin EB, JK MD, Huang B. Unlocking the potential of population-based cancer registries. Cancer. 2019;125(21):3729–37.PubMedCrossRef
16.
Blanchard CM, Courneya KS, Stein K. Cancer survivors’ adherence to lifestyle behavior recommendations and associations with health-related quality of life: results from the American cancer society's scs-ii. J Clin Oncol Off J Am Soc Clin Oncol. 2008;26(13):2198–204.CrossRef
17.
Arora NK, Hamilton AS, Potosky AL, Rowland JH, Aziz NM, Bellizzi KM, Klabunde CN, McLaughlin W, Stevens J. Population-based survivorship research using cancer registries: a study of non-hodgkin's lymphoma survivors. J Cancer Surviv. 2007;1(1):49–63.PubMedCrossRef
18.
Camp NJ, Parry M, Knight S, Abo R, Elliott G, Rigas SH, Balasubramanian SP, Reed MWR, McBurney H, Latif A, et al. Fine-mapping casp8 risk variants in breast cancer. Cancer Epidemiol Biomarkers Prev. 2012;21(1):176–81.PubMedCrossRef
19.
Resnick MJ, Koyama T, Fan K-H, Albertsen PC, Goodman M, Hamilton AS, Hoffman RM, Potosky AL, Stanford JL, Stroup AM, et al. Long-term functional outcomes after treatment for localized prostate cancer. N Engl J Med. 2013;368(5):436–45.PubMedPubMedCentralCrossRef
20.
Kinney AY, Boonyasiriwat W, Walters ST, Pappas LM, Stroup AM, Schwartz MD, Edwards SL, Rogers A, Kohlmann WK, Boucher KM, et al. Telehealth personalized cancer risk communication to motivate colonoscopy in relatives of patients with colorectal cancer: the family care randomized controlled trial. J Clin Oncol. 2014;32(7):654–62.PubMedPubMedCentralCrossRef
21.
Hall AE, Sanson-Fisher RW, Lynagh MC, Threlfall T, D'Este CA. Format and readability of an enhanced invitation letter did not affect participation rates in a cancer registry-based study: a randomized controlled trial. J Clin Epidemiol. 2013;66(1):85–94.PubMedCrossRef
22.
Oral E, Simonsen N, Brennan C, Berken J, Su LJ, Mohler JL, Bensen JT, Fontham ETH. Unit nonresponse in a population-based study of prostate cancer. PLoS One. 2016;11(12):e0168364.PubMedPubMedCentralCrossRef
23.
Curtin R, Presser S, Singer E. Changes in telephone survey non-response over the past quarter century. Public Opin Q. 2005;69(1):87–98.CrossRef
24.
25.
Brick JM, Williams D. Explaining rising nonresponse rates in cross-sectional surveys. Ann Am Acad Pol Soc Sci. 2012;645(1):36–59.CrossRef
26.
Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17(9):643–53.PubMedCrossRef
27.
Morton LM, Cahill J, Hartge P. Reporting participation in epidemiologic studies: a survey of practice. Am J Epidemiol. 2006;163(3):197–203.PubMedCrossRef
28.
Tolonen H, Helakorpi S, Talala K, Helasoja V, Martelin T, Prattala R. 25-year trends and socio-demographic differences in response rates: Finnish adult health behaviour survey. Eur J Epidemiol. 2006;21(6):409–15.PubMedCrossRef
29.
Brick JM, Tourangeau R. Responsive survey designs for reducing nonresponse bias. J Official Stat. 2017;33(3):735.CrossRef
30.
Wakefield CE, Fardell JE, Doolan EL, Aaronson NK, Jacobsen PB, Cohn RJ, King M. Participation in psychosocial oncology and quality-of-life research: A systematic review. Lancet Oncol. 18(3):e153–e65.PubMedCrossRef
31.
Drivsholm T, Eplov LF, Davidsen M, Jorgensen T, Ibsen H, Hollnagel H, Borch-Johnsen K. Representativeness in population-based studies: a detailed description of non-response in a Danish cohort study. Scand J Public Health. 2006;34(6):623–31.PubMedCrossRef
32.
Millar MM, Kinney AY, Camp NJ, Cannon-Albright LA, Hashibe M, Penson DF, Kirchhoff AC, Neklason DW, Gilsenan AW, Dieck GS, et al. Predictors of response outcomes for research recruitment through a central cancer registry: evidence from 17 recruitment efforts for population-based studies. Am J Epidemiol. 2019;188(5):928–39.PubMedCrossRefPubMedCentral
33.
Rogers PA, Haddow L, Thomson AK, Fritschi L, Girschik J, Boyle T, El Zaemey S, Heyworth JS. Including questionnaires with the invitation package appeared to increase the response fraction among women. J Clin Epidemiol. 2012;65(6):696–9.PubMedCrossRef
34.
Guo Y, Kopec JA, Cibere J, Li LC, Goldsmith CH. Population survey features and response rates: a randomized experiment. Am J Public Health. 2016;106(8):1422–6.PubMedPubMedCentralCrossRef
35.
Kongsved SM, Basnov M, Holm-Christensen K, Hjollund NH. Response rate and completeness of questionnaires: a randomized study of internet versus paper-and-pencil versions. J Med Internet Res. 2007;9(3):e25.PubMedPubMedCentralCrossRef
36.
Pit SW, Vo T, Pyakurel S. The effectiveness of recruitment strategies on general practitioner's survey response rates - a systematic review. BMC Med Res Methodol. 2014;14:76.PubMedPubMedCentralCrossRef
37.
Ebert JF, Huibers L, Christensen B, Christensen MB. Paper- or web-based questionnaire invitations as a method for data collection: cross-sectional comparative study of differences in response rate, completeness of data, and financial cost. J Med Internet Res. 2018;20(1):e24.PubMedPubMedCentralCrossRef
38.
Fowler FJ Jr, Cosenza C, Cripps LA, Edgman-Levitan S, Cleary PD. The effect of administration mode on CAHPS survey response rates and results: a comparison of mail and web-based approaches. Health Serv Res. 2019;54(3):714–21.PubMedCrossRefPubMedCentral
39.
40.
Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys the tailored design method. 4th ed. Hoboken: Wiley; 2014.
41.
42.
Weinberg BA, Marshall JL, Salem ME. The growing challenge of young adults with colorectal cancer. Oncology (Williston Park). 2017;31(5):381–9.
43.
Blum-Barnett E, Madrid S, Burnett-Hartman A, Mueller SR, McMullen CK, Dwyer A, Feigelson HS. Financial burden and quality of life among early-onset colorectal cancer survivors: a qualitative analysis. Health Expect. 2019.
44.
Armenian SH, Gibson CJ, Rockne RC, Ness KK. Premature aging in young cancer survivors. J Natl Cancer Inst. 2019;111(3):226–32.PubMedCrossRef
45.
Lu L, Deane J, Sharp L. Understanding survivors' needs and outcomes: the role of routinely collected data. Curr Opin Support Palliat Care. 2018;12(3):254–60.PubMedCrossRef
46.
Kaul S, Avila JC, Mutambudzi M, Russell H, Kirchhoff AC, Schwartz CL. Mental distress and health care use among survivors of adolescent and young adult cancer: a cross-sectional analysis of the national health interview survey. Cancer. 2017;123(5):869–78.PubMedCrossRef
47.
Warner EL, Nam GE, Zhang Y, McFadden M, Wright J, Spraker-Perlman H, Kinney AY, Oeffinger KC, Kirchhoff AC. Health behaviors, quality of life, and psychosocial health among survivors of adolescent and young adult cancers. J Cancer Surviv. 2016;10(2):280–90.PubMedCrossRef
48.
Nathan PC, TO H, Kirchhoff AC, Park ER, Yabroff KR. Financial hardship and the economic effect of childhood cancer survivorship. J Clin Oncol. 2018;36(21):2198–205.PubMedCrossRef
49.
50.
Qualtrics Online Survey Software. Qualtrics. Provo, UT. 2016.
51.
Dillman DA, Smyth JD, Christian LM. Internet, mail, and mixed-mode surveys : the tailored design method. 3rd ed. Hoboken: Wiley & Sons; 2009.
52.
53.
Zou G. A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol. 2004;159(7):702–6.CrossRefPubMed
54.
Cummings P. Methods for estimating adjusted risk ratios. Stata J. 2009;9(2):175–96.CrossRef
55.
Stata Statistical Software: Release 13. College Station: StataCorp; 2013.
56.
Shih T-H, Fan X. Comparing response rates from web and mail surveys: a meta-analysis. Field Method. 2008;20(3):249–71.CrossRef
57.
Whitehead L. Methodological issues in internet-mediated research: A randomized comparison of internet versus mailed questionnaires. J Med Internet Res. 2011;13(4):e109–e.PubMedPubMedCentralCrossRef
58.
Akl EA, Maroun N, Klocke RA, Montori V, Schunemann HJ. Electronic mail was not better than postal mail for surveying residents and faculty. J Clin Epidemiol. 2005;58(4):425–9.PubMedCrossRef
59.
Weaver L, Beebe TJ, Rockwood T. The impact of survey mode on the response rate in a survey of the factors that influence Minnesota physicians’ disclosure practices. BMC Med Res Methodol. 2019;19(1):73.PubMedPubMedCentralCrossRef
60.
61.
Link MW, Mokdad A. Can web and mail survey modes improve participation in an RDD-based national health surveillance? J Off Stat. 2006;22(2):293–312.
62.
Kaplowitz MD, Hadlock TD, Levine R. A comparison of web and mail survey response rates. Public Opin Q. 2004;68(1):94–101.CrossRef
63.
Messer BL, Dillman DA. Surveying the general public over the internet using address-based sampling and mail contact procedures. Public Opin Q. 2011;75(3):429–57.CrossRef
64.
Smyth JD, Dillman DA, Christian LM, O'Neill AC. Using the internet to survey small towns and communities: limitations and possibilities in the early 21st century. Am Behav Sci. 2010;53(9):1423–48.CrossRef
65.
Hagan TL, Belcher SM, Donovan HS. Mind the mode: differences in paper vs. web-based survey modes among women with cancer. J Pain Symptom Manag. 2017;54(3):368–75.CrossRef
66.
Smith AB, King M, Butow P, Olver I. A comparison of data quality and practicality of online versus postal questionnaires in a sample of testicular cancer survivors. Psychooncology. 2013;22(1):233–7.PubMedCrossRef
67.
68.
McMaster HS, LeardMann CA, Speigle S, Dillman DA, Millennium Cohort Family Study Team. An experimental comparison of web-push vs. Paper-only survey procedures for conducting an in-depth health survey of military spouses. BMC Med Res Methodol. 2017;17(1):73.PubMedPubMedCentralCrossRef
69.
Kilsdonk E, van Dulmen-den Broeder E, van der Pal HJ, Hollema N, Kremer LC, van den Heuvel-Eibrink MM, van Leeuwen FE, Jaspers MW, van den Berg MH. Effect of web-based versus paper-based questionnaires and follow-up strategies on participation rates of Dutch childhood cancer survivors: a randomized controlled trial. JMIR Cancer. 2015;1(2):e11.PubMedPubMedCentralCrossRef
70.
Nakash RA, Hutton JL, Jørstad-Stein EC, Gates S, Lamb SE. Maximising response to postal questionnaires – a systematic review of randomised trials in health research. BMC Med Res Methodol. 2006;6(1):5.PubMedPubMedCentralCrossRef
71.
Parkes R, Kreiger N, James B, Johnson KC. Effects on subject response of information brochures and small cash incentives in a mail-based case-control study. Ann Epidemiol. 2000;10(2):117–24.PubMedCrossRef
72.
Youl PH, Janda M, Lowe JB, Aitken JF. Does the type of promotional material influence men's attendance at skin screening clinics? Health Promot J Austr. 2005;16(3):229–32.PubMedCrossRef
73.
Smith T, Stein KD, Mehta CC, Kaw C, Kepner J, Stafford J, Baker F. The rationale, design and implementation of the American cancer society's studies of cancer survivors. Cancer. 2007;109(1):1–12.PubMedCrossRef
74.
Simmons RG, Lee Y-CA, Stroup AM, Edwards SL, Rogers A, Johnson C, Wiggins CL, Hill DA, Cress RD, Lowery J, et al. Examining the challenges of family recruitment to behavioral intervention trials: factors associated with participation and enrollment in a multi-state colonoscopy intervention trial. Trials. 2013;14:116.PubMedPubMedCentralCrossRef
75.
Moorman PG, Newman B, Millikan RC, Tse CK, Sandler DP. Participation rates in a case-control study: the impact of age, race, and race of interviewer. Ann Epidemiol. 1999;9(3):188–95.PubMedCrossRef
76.
Girgis A, Boyes A, Sanson-Fisher RW, Burrows S. Perceived needs of women diagnosed with breast cancer: rural versus urban location. Aust N Z J Public Health. 2000;24(2):166–73.PubMedCrossRef
77.
Mols F, Oerlemans S, Denollet J, Roukema JA, van de Poll-Franse LV. Type D personality is associated with increased comorbidity burden and health care utilization among 3080 cancer survivors. Gen Hosp Psychiatry. 2012;34(4):352–9.PubMedCrossRef
78.
Midkiff KD, Andrews EB, Gilsenan AW, Deapen DM, Harris DH, Schymura MJ, Hornicek FJ. The experience of accommodating privacy restrictions during implementation of a large-scale surveillance study of an osteoporosis medication. Pharmacoepidemiol Drug Saf. 2016;25(8):960–8.PubMedPubMedCentralCrossRef
Metadata
Title
The feasibility of web surveys for obtaining patient-reported outcomes from cancer survivors: a randomized experiment comparing survey modes and brochure enclosures
Authors
Morgan M. Millar
Joanne W. Elena
Lisa Gallicchio
Sandra L. Edwards
Marjorie E. Carter
Kimberly A. Herget
Carol Sweeney
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0859-9

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue

Research Article

Analysing detection of chronic diseases with prolonged sub-clinical periods: modelling and application to hypertension in the U.S.

Research article

The power of the group: comparison of interviews and group concept mapping for identifying patient-important outcomes of care

Research article

The Danish health and morbidity surveys: study design and participant characteristics

Research article

Explainable statistical learning in public health for policy development: the case of real-world suicide data

Research article

Estimating the loss of lifetime function using flexible parametric relative survival models

Research article

Evidence-based sizing of non-inferiority trials using decision models