Top

Published in:

01-12-2024 | Original Article

A survey of surveys: an evaluation of the quality of published surveys in neurosurgery

Authors: Subeikshanan Venkatesan, Sukumar Kalvapudi, Varun Muppidi, Karthik Ajith, Akshat Dutt, Venkatesh Shankar Madhugiri

Published in: Acta Neurochirurgica | Issue 1/2024

Abstract

Purpose

Surveys generate valuable data in epidemiologic and qualitative clinical research. The quality of a survey depends on its design, the number of responses it receives, and the reporting of the results. In this study, we aimed to assess the quality of surveys in neurosurgery.

Methods

Neurosurgical surveys published between 2000 and 2020 (inclusive) were identified from PubMed. Various datapoints regarding the surveys were collated. The number of citations received by the papers was determined from Google Scholar. A 6-dimensional quality assessment tool was applied to the surveys. Parameters from this tool were combined with the number of responses received to create the survey quality score (SQS).

Results

A total of 618 surveys were included for analysis. The target sample size correlated with the number of responses received. The response rate correlated positively with the target sample size and the number of reminders sent and negatively with the number of questions in the survey. The median number of authors on neurosurgery survey papers was 6. The number of authors correlated with the SQS and the number of citations received by published survey papers. The median normalized SQS for neurosurgical surveys was 65%. The nSQS independently predicted the citations received per year by surveys.

Conclusions

The modifiable factors that correlated with improvements in survey design were optimizing the number of questions, maximizing the target sample size, and incorporating reminders in the survey design. Increasing the number of contributing authors led to improvements in survey quality. The SQS was validated and correlated well with the citations received by surveys.

Aerny-Perreten N, Dominguez-Berjon MF, Esteban-Vasallo MD, Garcia-Riolobos C (2015) Participation and factors associated with late or non-response to an online survey in primary care. J Eval Clin Pract 21:688–693. https://doi.org/10.1111/jep.12367CrossRefPubMed

Ahmed A, Adam M, Ghafar NA, Muhammad M, Ebrahim NA (2016) Impact of article page count and number of authors on citations in disability related fields: a systematic review article. Iran J Public Health 45:1118–1125PubMedPubMedCentral

Archer T (2017) Characteristics associated with increasing the response rates of web-based surveys. Practical Assessment, Research, and Evaluation 12. https://doi.org/10.7275/ec5p-zg50

Artino AR Jr, Phillips AW, Utrankar A, Ta AQ, Durning SJ (2018) “The questions shape the answers”: assessing the quality of published survey instruments in health professions education research. Acad Med 93:456–463. https://doi.org/10.1097/ACM.0000000000002002CrossRefPubMed

Attenello FJ, Buchanan IA, Wen T, Donoho DA, McCartney S, Cen SY, Khalessi AA, Cohen-Gadol AA, Cheng JS, Mack WJ, Schirmer CM, Swartz KR, Prall JA, Stroink AR, Giannotta SL, Klimo P (2018) Factors associated with burnout among US neurosurgery residents: a nationwide survey. J Neurosurg 129:1349–1363. https://doi.org/10.3171/2017.9.JNS17996CrossRefPubMed

Badger F, Werrett J (2005) Room for improvement? Reporting response rates and recruitment in nursing research in the past decade. J Adv Nurs 51:502–510CrossRefPubMed

Bennett C, Khangura S, Brehaut JC, Graham ID, Moher D, Potter BK, Grimshaw JM (2010) Reporting guidelines for survey research: an analysis of published guidance and reporting practices. PLoS Med 8:e1001069. https://doi.org/10.1371/journal.pmed.1001069CrossRefPubMed

Bjertnaes OA, Garratt A, Botten G (2008) Nonresponse bias and cost-effectiveness in a Norwegian survey of family physicians. Eval Health Prof 31:65–80. https://doi.org/10.1177/0163278707311874CrossRefPubMed

Bramstedt KA (2020) The carnage of substandard research during the COVID-19 pandemic: a call for quality. J Med Ethics 46:803–807. https://doi.org/10.1136/medethics-2020-106494CrossRefPubMed

10.

Burns KEA, Kho ME (2015) How to assess a survey report: a guide for readers and peer reviewers. CMAJ 187:E198–E205. https://doi.org/10.1503/cmaj.140545CrossRefPubMedPubMedCentral

11.

Cheung KL, Ten Klooster PM, Smit C, de Vries H, Pieterse ME (2017) The impact of non-response bias due to sampling in public health studies: a comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health. BMC Public Health 17:276. https://doi.org/10.1186/s12889-017-4189-8CrossRefPubMedPubMedCentral

12.

Chirico F, Teixeira da Silva JA, Magnavita N (2020) “Questionable” peer review in the publishing pandemic during the time of COVID-19: implications for policy makers and stakeholders. Croat Med J 61:300–301. https://doi.org/10.3325/cmj.2020.61.300CrossRefPubMedPubMedCentral

13.

Cohen-Gadol AA, Piepgras DG, Krishnamurthy S, Fessler RD (2005) Resident duty hours reform: results of a national survey of the program directors and residents in neurosurgery training programs. Neurosurgery 56:398–403; discussion 398–403. https://doi.org/10.1227/01.neu.0000147999.64356.57

14.

Compton J, Glass N, Fowler T (2019) Evidence of selection bias and non-response bias in patient satisfaction surveys. Iowa Orthop J 39:195–201PubMedPubMedCentral

15.

Cook JV, Dickinson HO, Eccles MP (2009) Response rates in postal surveys of healthcare professionals between 1996 and 2005: an observational study. BMC Health Serv Res 9:160. https://doi.org/10.1186/1472-6963-9-160CrossRefPubMedPubMedCentral

16.

Cuschieri S (2019) The STROBE guidelines. Saudi J Anaesth 13:S31–S34. https://doi.org/10.4103/sja.SJA_543_18CrossRefPubMedPubMedCentral

17.

de Koning R, Egiz A, Kotecha J, Ciuculete AC, Ooi SZY, Bankole NDA, Erhabor J, Higginbotham G, Khan M, Dalle DU, Sichimba D, Bandyopadhyay S, Kanmounye US (2021) Survey fatigue during the COVID-19 pandemic: an analysis of neurosurgery survey response rates. Front Surg 8:690680. https://doi.org/10.3389/fsurg.2021.690680

18.

Duffett M, Burns KE, Adhikari NK, Arnold DM, Lauzier F, Kho ME, Meade MO, Hayani O, Koo K, Choong K (2012) Quality of reporting of surveys in critical care journals: a methodologic review. Crit Care Med 40:441–449CrossRefPubMed

19.

Eysenbach G (2004) Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). vol 6. Gunther Eysenbach Centre for Global eHealth Innovation, Toronto, Canada,

20.

Fincham JE (2008) Response rates and responsiveness for surveys, standards, and the Journal. American journal of pharmaceutical education 72

21.

Geyer ED, Miller R, Kim SS, Tobias JD, Nafiu OO, Tumin D (2020) Quality and impact of survey research among anesthesiologists: a systematic review. Adv Med Educ Pract 11:587–599. https://doi.org/10.2147/AMEP.S259908CrossRefPubMedPubMedCentral

22.

Greenberg JK, Jeffe DB, Carpenter CR, Yan Y, Pineda JA, Lumba-Brown A, Keller MS, Berger D, Bollo RJ, Ravindra VM, Naftel RP, Dewan MC, Shah MN, Burns EC, O’Neill BR, Hankinson TC, Whitehead WE, Adelson PD, Tamber MS, McDonald PJ, Ahn ES, Titsworth W, West AN, Brownson RC, Limbrick DD (2018) North American survey on the post-neuroimaging management of children with mild head injuries. J Neurosurg Pediatr 23:227–235. https://doi.org/10.3171/2018.7.PEDS18263CrossRefPubMedPubMedCentral

23.

Grimshaw J (2014) Surge (the survey reporting guideline). Guidelines for reporting health research: a user's manual 206–213

24.

Hlatshwako TG, Shah SJ, Kosana P, Adebayo E, Hendriks J, Larsson EC, Hensel DJ, Erausquin JT, Marks M, Michielsen K, Saltis H, Francis JM, Wouters E, Tucker JD (2021) Online health survey research during COVID-19. Lancet Digit Health 3:e76–e77. https://doi.org/10.1016/S2589-7500(21)00002-9CrossRefPubMedPubMedCentral

25.

Jaykaran, (2011) How to increase response rate to a questionnaire study? Indian J Pharmacol 43:93–94. https://doi.org/10.4103/0253-7613.75687CrossRefPubMed

26.

Jones D, Story D, Clavisi O, Jones R, Peyton P (2006) An introductory guide to survey research in anaesthesia. Anaesth Intensive Care 34:245–253. https://doi.org/10.1177/0310057X0603400219CrossRefPubMed

27.

Kelley K, Clark B, Brown V, Sitzia J (2003) Good practice in the conduct and reporting of survey research. Int J Qual Health Care 15:261–266. https://doi.org/10.1093/intqhc/mzg031CrossRefPubMed

28.

Khan NR, Derstine PL, Gienapp AJ, Klimo P, Barbaro NM (2020) A survey of neurological surgery residency program mentorship practices compared to accreditation council for graduate medical education resident outcome data. Neurosurgery 87:E566–E572. https://doi.org/10.1093/neuros/nyz479CrossRefPubMed

29.

Langbecker D, Caffery LJ, Gillespie N, Smith AC (2017) Using survey methods in telehealth research: a practical guide. J Telemed Telecare 23:770–779. https://doi.org/10.1177/1357633X17721814CrossRefPubMed

30.

Leidman E, Mwirigi LM, Maina-Gathigi L, Wamae A, Imbwaga AA, Bilukha OO (2018) Assessment of anthropometric data following investments to ensure quality: Kenya demographic health surveys case study, 2008 to 2009 and 2014. Food Nutr Bull 39:406–419. https://doi.org/10.1177/0379572118783181CrossRefPubMedPubMedCentral

31.

Leung L (2015) Validity, reliability, and generalizability in qualitative research. Journal of family medicine and primary care 4:324CrossRefPubMedPubMedCentral

32.

Li AH-T, Thomas SM, Farag A, Duffett M, Garg AX, Naylor KL (2014) Quality of survey reporting in nephrology journals: a methodologic review. Clin J Am Soc Nephrol 9:2089CrossRefPubMedPubMedCentral

33.

Madhugiri VS, Moiyadi A, Nagella AB, Singh V, Shetty P (2021) A questionnaire-based survey of clinical neuro-oncological practice in India. Neurol India 69:659–664. https://doi.org/10.4103/0028-3886.319199CrossRefPubMed

34.

McFarlane E, Olmsted MG, Murphy J, Hill CA (2007) Nonresponse bias in a mail survey of physicians. Eval Health Prof 30:170–185. https://doi.org/10.1177/0163278707300632CrossRefPubMed

35.

McGhan WF, Al M, Doshi JA, Kamae I, Marx SE, Rindress D (2009) The ISPOR good practices for quality improvement of cost-effectiveness research task force report. Value Health 12:1086–1099. https://doi.org/10.1111/j.1524-4733.2009.00605.xCrossRefPubMed

36.

McGovern ME, Canning D, Barnighausen T (2018) Accounting for non-response bias using participation incentives and survey design: an application using gift vouchers. Econ Lett 171:239–244. https://doi.org/10.1016/j.econlet.2018.07.040CrossRefPubMedPubMedCentral

37.

Meng-Jia Wu KZ, Francisca Fils-Aime (2022) Response rates of online surveys in published research: a meta-analysis. Computers in Human Behavior Reports 7

38.

Miller C, Lundy P, Woodrow S (2020) International electives in neurological surgery training: a survey of program directors from Accreditation Council for Graduate Medical Education-approved neurological surgery programs. J Neurosurg 134:1967–1973. https://doi.org/10.3171/2020.4.JNS20618CrossRefPubMed

39.

Mukuria AG (2003) Using national health and nutrition surveys for policy and programs: experiences from the demographic and health surveys. Forum Nutr 56:207–208PubMed

40.

Nagella AB, Ravishankar M, Hemanth Kumar VR (2016) Anaesthesia practice and reproductive outcomes: facts unveiled. Indian J Anaesth 60:225. https://doi.org/10.4103/0019-5049.177883CrossRefPubMedPubMedCentral

41.

Pagano MB, Dunbar NM, Tinmouth A, Apelseth TO, Lozano M, Cohn CS, Stanworth SJ, Collaborative BEfST, (2018) A methodological review of the quality of reporting of surveys in transfusion medicine. Transfusion 58:2720–2727CrossRefPubMed

42.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg 88:105906CrossRefPubMed

43.

Patel SS, Webster RK, Greenberg N, Weston D, Brooks SK (2020) Research fatigue in COVID-19 pandemic and post-disaster research: causes, consequences and recommendations. Disaster Prev Manag 29(4):445–455. https://doi.org/10.1108/DPM-05-2020-0164CrossRefPubMedPubMedCentral

44.

Pit SW, Vo T, Pyakurel S (2014) The effectiveness of recruitment strategies on general practitioner’s survey response rates - a systematic review. BMC Med Res Methodol 14:76. https://doi.org/10.1186/1471-2288-14-76CrossRefPubMedPubMedCentral

45.

Rathi K, Kamboj P, Bansal PG, Toteja GS (2018) A review of selected nutrition & health surveys in India. Indian J Med Res 148:596–611. https://doi.org/10.4103/ijmr.IJMR_1808_18CrossRefPubMedPubMedCentral

46.

Rosen T, Olsen J (2006) Invited commentary: the art of making questionnaires better. Am J Epidemiol 164:1145–1149CrossRefPubMed

47.

Rybakov KN, Beckett R, Dilley I, Sheehan AH (2020) Reporting quality of survey research articles published in the pharmacy literature. Res Social Adm Pharm 16:1354–1358. https://doi.org/10.1016/j.sapharm.2020.01.005CrossRefPubMed

48.

Sammut R, Griscti O, Norman IJ (2021) Strategies to improve response rates to web surveys: a literature review. Int J Nurs Stud 123:104058. https://doi.org/10.1016/j.ijnurstu.2021.104058CrossRefPubMed

49.

Schilling LM, Kozak K, Lundahl K, Dellavalle RP (2006) Inaccessible novel questionnaires in published medical research: hidden methods, hidden costs. Am J Epidemiol 164:1141–1144CrossRefPubMed

50.

Schulz KF, Altman DG, Moher D (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. J Pharmacol Pharmacother 1:100–107CrossRefPubMedPubMedCentral

51.

Sharma A, Minh Duc NT, Luu Lam Thang T, Nam NH, Ng SJ, Abbas KS, Huy NT, Marušić A, Paul CL, Kwok J (2021) A consensus-based checklist for reporting of survey studies (CROSS). J Gen Intern Med 36:3179–3187CrossRefPubMedPubMedCentral

52.

Singh S, Sagar R (2021) A critical look at online survey or questionnaire-based research studies during COVID-19. Asian J Psychiatr 65:102850. https://doi.org/10.1016/j.ajp.2021.102850CrossRefPubMedPubMedCentral

53.

So R, Shinohara K, Aoki T, Tsujimoto Y, Suganuma AM, Furukawa TA (2018) Effect of recruitment methods on response rate in a web-based study for primary care physicians: factorial randomized controlled trial. J Med Internet Res 20:e28. https://doi.org/10.2196/jmir.8561CrossRefPubMedPubMedCentral

54.

Story DA, Gin V, na Ranong V, Poustie S, Jones D; ANZCA Trials Group (2011) Inconsistent survey reporting in anesthesia journals. Anesth Analg 113(3):591–595. https://doi.org/10.1213/ANE.0b013e3182264aafCrossRef

55.

Koch GG, Gillings DB, Stokes ME (1980) Biostatistical implications of design, sampling, and measurement to health science data analysis. Annu Rev Public Health 1:163–225. https://doi.org/10.1146/annurev.pu.01.050180.001115CrossRefPubMed

56.

Teixeira da Silva JA, Bornemann-Cimenti H, Tsigaris P (2021) Optimizing peer review to minimize the risk of retracting COVID-19-related literature. Med Health Care Philos 24:21–26. https://doi.org/10.1007/s11019-020-09990-zCrossRefPubMed

57.

Tran EM, Tran MM, Clark MA, Scott IU, Margo CE, Cosenza C, Johnson TP, Greenberg PB (2020) Assessing the quality of published surveys in ophthalmology. Ophthalmic Epidemiol 27:339–343. https://doi.org/10.1080/09286586.2020.1746359CrossRefPubMed

58.

Turgeon AF, Lauzier F, Burns KE, Meade MO, Scales DC, Zarychanski R, Moore L, Zygun DA, McIntyre LA, Kanji S, Hebert PC, Murat V, Pagliarello G, Fergusson DA, Canadian Critical Care Trials G (2013) Determination of neurologic prognosis and clinical decision making in adult patients with severe traumatic brain injury: a survey of Canadian intensivists, neurosurgeons, and neurologists. Crit Care Med 41:1086–1093. https://doi.org/10.1097/CCM.0b013e318275d046

59.

Tyrer S, Heyman B (2016) Sampling in epidemiological research: issues, hazards and pitfalls. BJPsych Bull 40:57–60. https://doi.org/10.1192/pb.bp.114.050203CrossRefPubMedPubMedCentral

60.

Van Otterloo J, Richards JL, Seib K, Weiss P, Omer SB (2011) Gift card incentives and non-response bias in a survey of vaccine providers: the role of geographic and demographic factors. PLoS ONE 6:e28108. https://doi.org/10.1371/journal.pone.0028108CrossRefPubMedPubMedCentral

61.

Vardhana Rao MV, Sahu D, Nair S, Sharma RK, Gulati BK, Acharya R, Mahapatra B, Ramesh S, Khan N, Chaudhuri T, Sandal K, Deepani V, Dey S, Saggurti N (2022) National guidelines for data quality in surveys: an overview. Indian J Med Res 156:715–720. https://doi.org/10.4103/ijmr.ijmr_1261_22CrossRefPubMed

62.

Zeleke AA, Worku AG, Demissie A, Otto-Sobotka F, Wilken M, Lipprandt M, Tilahun B, Rohrig R (2019) Evaluation of electronic and paper-pen data capturing tools for data quality in a public health survey in a health and demographic surveillance site, Ethiopia: randomized controlled crossover health care information technology evaluation. JMIR Mhealth Uhealth 7:e10995. https://doi.org/10.2196/10995CrossRefPubMedPubMedCentral

Title: A survey of surveys: an evaluation of the quality of published surveys in neurosurgery
Authors: Subeikshanan Venkatesan
Sukumar Kalvapudi
Varun Muppidi
Karthik Ajith
Akshat Dutt
Venkatesh Shankar Madhugiri
Publication date: 01-12-2024
Publisher: Springer Vienna
Published in: Acta Neurochirurgica / Issue 1/2024
Print ISSN: 0001-6268
Electronic ISSN: 0942-0940
DOI: https://doi.org/10.1007/s00701-024-06042-w

Keynote webinar | Spotlight on medication adherence

Springer Medicine

A survey of surveys: an evaluation of the quality of published surveys in neurosurgery

Abstract

Purpose

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2024

Clinical impact of craniectomy on shunt-dependent hydrocephalus after intracerebral hemorrhage: A propensity score-matched analysis

Visual outcomes and optimal timing for repeat surgery in cases of postoperative hematoma following transsphenoidal surgery for pituitary neuroendocrine tumors: A retrospective cohort study

Customised pre-operative cranioplasty to achieve maximal surgical resection of tumours with osseous involvement—a case series

Location-based clinical and angiographic profile of brain arteriovenous malformations – a single-center observational study

Development and validation of a 23-gene expression signature for molecular subtyping of medulloblastoma in a long-term Chinese cohort

Incidence of surgically treated chronic subdural hematoma after head injury with normal initial computed tomography