Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2019 | Research article

The impact of survey mode on the response rate in a survey of the factors that influence Minnesota physicians’ disclosure practices

Authors: Lesley Weaver, Timothy J. Beebe, Todd Rockwood

Published in: BMC Medical Research Methodology | Issue 1/2019

Abstract

Background

There is evidence that the physician response rate is declining. In response to this, methods for increasing the physician response rate are currently being explored. This paper examines the response rate and extent of non-response bias in a mixed-mode study of Minnesota physicians.

Methods

This mode experiment was embedded in a survey study on the factors that influence physicians’ willingness to disclose medical errors and adverse events to patients and their families. Physicians were randomly selected from a list of licensed physicians obtained from the Minnesota Board of Medical Practice. Afterwards, they were randomly assigned to either a single-mode (mail-only or web-only) or mixed-mode (web-mail or mail-web) design. Differences in response rate and nonresponse bias were assessed using Fischer’s Exact Test.

Results

The overall response rate was 18.60%. There were no statistically significant differences in the response rate across modes (p – value = 0.410). The non-response analysis indicates that responders and non-responders did not differ with respect to speciality or practice location.

Conclusions

The mode of administration did not affect the physician response rate.

Cull WL, O’Connor KG, Sharp S, Tang SS. Response rates and response bias for 50 surveys of pediatricians. Health Serv Res 2005;40(1):213–226. Available from https://doi.org/10.1111/j.1475-6773.2005.00350.x

Cook JV, Dickinson HO, Eccles MP. Response rates in postal surveys of healthcare professionals between 1996 and 2005: an observational study. BMC HealthServ Res 2009;9(160). Available from https://doi.org/https://doi.org/10.1186/1472-6963-9-160.

Cho YI, Johnson TP, VanGeest JB. Enhancing surveys of health care professionals: a meta-analysis of techniques to improve response. Eval Health Prof 2013;36(3):382–407. Available from https://doi.org/10.1177/0163278713496425.

McLeod CC, Klabunde CN, Willis GB, Stark D. Health care provider surveys in the United States, 2000-2010: a review. Eval Health Prof 2013;36(1):106–126. Available from https://doi.org/10.1177/0163278712474001

Kellerman SE, Herold J. Physicians response to surveys. A review of the literature. Am J Prev Med 2001;20(1):61–67. Available from https://doi.org/10.1016/S0749-3797(00)00258-0

Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA-J Am Med Assoc. 2012;307(17). https://doi.org/10.1001/jama.2012.3532.

Pit SW, Vo T, Pyakurel S. The effectiveness of recruitment strategies on general practitioner’s survey response rates—a systematic review. BMC Med Res Methodol 2014;14(76). Available from https://doi.org/https://doi.org/10.1186/1471-2288-14-76.

Jepson C, Asch DA, Hershey JC, Ubel PA. In a mailed physician survey, questionnaire length had a threshold effect on response rate. J Clin Epidemiol 2005;58(1):103–105. Available from https://doi.org/https://doi.org/10.1016/j.jclinepi.2004.06.004

Barclay S, Todd C, Finlay I, Grande G, Wyatt P. Not another questionnaire! Maximizing the response rate, predicting non-response and assessing non-response bias in postal questionnaire studies of GPs. Fam Pract 2002;19(1):105–111. Available from https://doi.org/https://doi.org/10.1093/fampra/19.1.105

10.

Bjertnaes OA, Garrett A, Botten G. Nonresponse bias and cost-effectiveness in a Norwegian survey of family physicians. Eval Health Prof 2008;31(1):65–80. Available from https://doi.org/https://doi.org/10.1177/0163278707311874.

11.

VanGeest JB, Johnson TP, Welch VL. Methodologies for improving response rates in surveys of physicians: a systematic review. Eval Health Prof 2007;30(4):303–321. Available from https://doi.org/https://doi.org/10.1177/0163278707307899

12.

Flanigan TS, McFarlene E, Cook S. Conducting survey research among physicians and other medical professionals – a review of the current literature. New Orleans, LA: American Statistical Association Proceedings of the Survey Research Methods Section 2008;4136–4147.

13.

Ziegenfuss JY, Niederhauser BD, Kallmes D, Beebe TJ. An assessment of incentive versus survey length trade-offs in a web survey of radiologists. J Med Internet Res. 2013;15(3):e49. https://doi.org/10.2196/jmir.2322.CrossRefPubMedPubMedCentral

14.

Leeuw E, Berzelak N. Survey mode or survey modes? In: Wolf C, Joye D, Smith TW, Fu Y, editors. The SAGE handbook of survey methodology. London: SAGE Publications Ltd; 2016. p. 142–56.CrossRef

15.

Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P, et al. Internet versus mailed questionnaires: a controlled comparison. J Med Internet Res. 2004;6(4):e39. https://doi.org/10.2196/jmir.6.4.e39.CrossRefPubMedPubMedCentral

16.

Scott A, Jeon S, Joyce CM, Humphreys JS, Kalb G, Witt J, et al. A randomized trial and economic evaluation of the effect of response mode on response rate, response bias, and item non-response in a survey of doctors. BMC Med Res Methodol 2011;11(126). Available from https://doi.org/https://doi.org/10.1186/1471-2288-11-126.

17.

Dykema J, Jones NR, Piche T, Stevenson J. Surveying clinicians by web: current issues in design and administration. Eval Health Prof 2013;36(3):352–381. Available from https://doi.org/https://doi.org/10.1177/0163278713496630.

18.

Partin MR, Powell AA, Burgess DJ, Haggstrom DA, Gravely AA, Halek K, et al. Adding postal follow-up to a web-based survey of primary care and gastroenterology clinic physician chiefs improved response rates but not response quality or representativeness. Eval Health Prof 2015;38(3):382–403. Available from https://doi.org/https://doi.org/10.1177/0163278713513586.

19.

Beebe TJ, Jacobson RM, Jenkins SM, Lackore KA, Finney-Rutten LJ. Testing the impact of mixed-mode designs (mail and web) and multiple contact attempts within mode (mail or web) on clinician survey response. Health Serv Res 2018;53(S1):3070–3083. Available from https://doi.org/https://doi.org/10.1111/1475-6773.12827

20.

Medway RL, Fulton J. When more gets you less: a meta-analysis of the effect of concurrent options on mail survey response rates. Public Opin Quart 2012;76(4):733–746. Available from https://doi.org/https://doi.org/10.1093/poq/nfs047.

21.

Beebe TJ, Locke III GR, Barnes SA, Davern ME, Anderson KJ. Mixing web and mail methods in a survey of physicians. Health Serv Res 2007;42(3 Pt 1):1219–1234. Available from https://doi.org/https://doi.org/10.1111/j.1475-6773.2006.00652.x

22.

Puleo E, Zapka J, White MJ, Mouchawar J, Somkin C, Taplin S. Caffeine, cajoling, and other strategies to maximize clinician survey response rates. Eval Health Prof 2002;25(2):169–184. Available from https://doi.org/https://doi.org/10.1177/016327870202500203.

23.

McMahon SR, Iwamoto M, Massoudi MS, Yusuf HR, Stevenson JM, David F, et al. Comparison of e-mail, fax, and postal surveys of pediatricians. Pediatrics. 2003;111(4):e299–303 Available from http://pediatrics.aappublications.org/content/142/6?current-issue=y.CrossRef

24.

Ash JS, Gorman PN, Seshadri V, Hersh WR. Computerized physician order entry in U.S. hospitals: results of a 2002 survey. J Am Med Inform Assoc 2004;11(2):95–99. Available from https://doi.org/https://doi.org/10.1197/jamia.M1427

25.

Shah TU, Voils CI, McNeil R, Wu R, Fisher DA. Understanding gastroenterologist adherence to polyp surveillance guidelines. Am J Gastroenterol. 2012;107(9):1283–7. https://doi.org/10.1038/ajg.2012.59.CrossRefPubMedPubMedCentral

26.

American Association for Public Opinion Research. Standard definitions: final dispositions of case codes and outcome rates for surveys. 9th ed. Oakbrook Terrace: AAPOR; 2016.

27.

Washington, Wyoming, Alaska, Montana, and Idaho (WWAMI) Rural Health Research Center. RUCA Data [Data set]. Available from http://depts.washington.edu/uwruca/ruca-download.php [Accessed 28^th June 2018].

28.

StataCorp. Stata Statistical Software: Release 15. College Station: StataCorp LLC; 2017.

29.

Seguin R, Godwin M, MacDonald S, McCall M. E-mail or snail mail? Randomized controlled trial on which works better for surveys. Can Fam Physician. 2004;50(3):414–9 Available from https://www.cfpc.ca/Home/. Accessed 16 Oct 2018.PubMedPubMedCentral

30.

Cunningham CT, Quan H, Hemmelgarn B, Noseworthy T, Beck CA, Dixon E, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol. 2015;15(32) Available from https://doi.org/10.1186/s12874-015-0016-z.

31.

Groves RM, Presser S, Dipko S. The role of topic interest in survey participation decisions. Public Opin Quart 2004;68(1):2–31. Available from https://doi.org/https://doi.org/10.1093/poq/nfh002.

32.

Groves RM, Singer E, Corning A. Leverage-saliency theory of survey participation: description and an illustration. Public Opin Quart. 2000;64(3):299–308 Available from https://www.jstor.org/stable/3078721. Accessed 12 Oct 2018.CrossRef

33.

Wu AW, Folkman S, McPhee AJ, Lo B. Do house officers learn from their mistakes? BMJ Qual Saf 2003;12(3):221–226. Available from http://dx.doi.org/https://doi.org/10.1136/qhc.12.3.221

34.

Hobgood C, Hevia A, Tamayo-Sarver JH, Weiner B, Riviello R. The influence of the causes and contexts of medical errors on emergency medicine residents’ responses to their errors: an exploration. Acad Med. 2005;80(8):758–64 Available from https://journals.lww.com/academicmedicine/pages/default.aspx. Accessed 28 Mar 2018.CrossRef

35.

Waterman AD, Garbutt J, Hazel E, Dunagan WC, Levinson W, Fraser VJ, et al. The emotional impact of medical errors on practicing physicians in the United States and Canada. Jt Comm J Qual Patient Saf 2007;33(8):467–476. Available from https://doi.org/https://doi.org/10.1016/S1553-7250(07)33050-X.

36.

Schwappach DLB, Boluarte TA. The emotional impact of medical error involvement on physicians: a call for leadership and organizational accountability. Swiss Med Wkly. 2009;139:9–15 Available from https://smw.ch/. Accessed 29 June 2018.PubMed

37.

James KM, Ziegenfuss JY, Tilburt JC, Harris AM, Beebe TJ. Getting physicians to respond: the impact of incentive type and timing on physician survey response rates. Health Serv Res 2011;46(1 Pt 1):232–242. Available from https://doi.org/https://doi.org/10.1111/j.1475-6773.2010.01181.x.

Title: The impact of survey mode on the response rate in a survey of the factors that influence Minnesota physicians’ disclosure practices
Authors: Lesley Weaver
Timothy J. Beebe
Todd Rockwood
Publication date: 01-12-2019
Publisher: BioMed Central
Published in: BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-019-0719-7

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

The impact of survey mode on the response rate in a survey of the factors that influence Minnesota physicians’ disclosure practices

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2019

Aims, design and preliminary findings of the Hellenic National Nutrition and Health Survey (HNNHS)

Tests of trend between disease outcomes and ordinal covariates discretized from underlying continuous variables: simulation studies and applications to NHANES 2007–2008

Recommendations to conduct and report systematic reviews in medical literature: a scoping review

Crowding in the emergency department in the absence of boarding – a transition regression model to predict departures and waiting time

Goal attainment scaling as an outcome measure in rare disease trials: a conceptual proposal for validation

Determining a cutoff score for the family burden interview schedule using three statistical methods