Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2021 | Technical advance

Data quality assessment and subsampling strategies to correct distributional bias in prevalence studies

Authors: A. D’Ambrosio, J. Garlasco, F. Quattrocolo, C. Vicentini, C. M. Zotti

Published in: BMC Medical Research Methodology | Issue 1/2021

Abstract

Background

Healthcare-associated infections (HAIs) represent a major Public Health issue. Hospital-based prevalence studies are a common tool of HAI surveillance, but data quality problems and non-representativeness can undermine their reliability.

Methods

This study proposes three algorithms that, given a convenience sample and variables relevant for the outcome of the study, select a subsample with specific distributional characteristics, boosting either representativeness (Probability and Distance procedures) or risk factors’ balance (Uniformity procedure). A “Quality Score” (QS) was also developed to grade sampled units according to data completeness and reliability.

The methodologies were evaluated through bootstrapping on a convenience sample of 135 hospitals collected during the 2016 Italian Point Prevalence Survey (PPS) on HAIs.

Results

The QS highlighted wide variations in data quality among hospitals (median QS 52.9 points, range 7.98–628, lower meaning better quality), with most problems ascribable to ward and hospital-related data reporting. Both Distance and Probability procedures produced subsamples with lower distributional bias (Log-likelihood score increased from 7.3 to 29 points). The Uniformity procedure increased the homogeneity of the sample characteristics (e.g., − 58.4% in geographical variability).

The procedures selected hospitals with higher data quality, especially the Probability procedure (lower QS in 100% of bootstrap simulations). The Distance procedure produced lower HAI prevalence estimates (6.98% compared to 7.44% in the convenience sample), more in line with the European median.

Conclusions

The QS and the subsampling procedures proposed in this study could represent effective tools to improve the quality of prevalence studies, decreasing the biases that can arise due to non-probabilistic sample collection.

Available only for authorised users

Suetens C, et al. Prevalence of healthcare-associated infections, estimated incidence and composite antimicrobial resistance index in acute care hospitals and long-term care facilities: results from two European point prevalence surveys, 2016 to 2017. Eurosurveillance. 2018;23(46):1800516. https://doi.org/10.2807/1560-7917.ES.2018.23.46.1800516.CrossRefPubMedCentral

Bianco A, Capano MS, Mascaro V, Pileggi C, Pavia M. Prospective surveillance of healthcare-associated infections and patterns of antimicrobial resistance of pathogens in an Italian intensive care unit. Antimicrob Resist Infect Control. 2018;7(1). https://doi.org/10.1186/s13756-018-0337-x.

Hopmans TEM, et al. Trends in prevalence of healthcare-associated infections and antimicrobial use in hospitals in the Netherlands: 10 years of national point-prevalence surveys. J Hosp Infect. 2019. https://doi.org/10.1016/j.jhin.2019.10.005.

ECDC. Surveillance of antimicrobial resistance in Europe: Stockholm; 2017. https://doi.org/10.2900/230516.

Global antimicrobial resistance surveillance system (GLASS) report: early implementation 2016-2017. Geneva: World Health Organization; 2017. Licence: CC BY-NC-SA 3.0 IGO.

ECDC. European Centre for Disease Prevention and Control. Technical document. Point prevalence survey of healthcare-associated infections and antimicrobial use in European acute care hospitals. Protocol version 5.3. Stockholm: ECDC; 2016. https://doi.org/10.2900/374985.CrossRef

L. D. H. Carl Suetens, Susan Hopkins, Jana Kolman, “European Centre for Disease Prevention and Control. Point prevalence survey of healthcareassociated infections and antimicrobial use in European acute care hospitals,” Stockholm, 2013. doi: https://doi.org/10.2900/86011.

Bornstein MH, Jager J, Putnick DL. Sampling in developmental science: situations, shortcomings, solutions, and standards. Dev Rev. 2013;33(4):357–70. https://doi.org/10.1016/j.dr.2013.08.003.CrossRefPubMedPubMedCentral

Dipartimento Scienze della Salute Pubblica e Pediatriche - Università di Torino, “Secondo studio di prevalenza italiano sulle infezioni correlate all’assistenza e sull’uso di antibiotici negli ospedali per acuti – Protocollo ECDC,” 2018. [Online]. Available: http://www.salute.gov.it/imgs/C_17_pubblicazioni_2791_allegato.pdf. Accessed Sept 2019.

10.

“European Centre for Disease Prevention and Control HelicsWin.Net 2.3 − user manual.” ECDC, Stockholm, 2016.

11.

“Open Data - Dati - Posti letto per struttura ospedaliera.” http://www.dati.salute.gov.it/dati/dettaglioDataset.jsp?menu=dati&idPag=18. Accessed Sept 2019.

12.

Vicentini C, et al. Point prevalence data on antimicrobial usage in Italian acute-care hospitals: evaluation and comparison of results from two national surveys (2011–2016). Infect Control Hosp Epidemiol. 2020. https://doi.org/10.1017/ice.2020.18.

13.

Hilbe JM. “Can binary logistic models be overdispersed?” Jet Propulsion Laboratory, California Institute of Technology and Arizona State University; 2013.

14.

Efron B, Tibshirani RJ. An introduction to bootstrapping. 1st ed. New York: Chapman & Hall; 1993.CrossRef

15.

Gelman A, Jakulin A, Pittau MG, Su YS. A weakly informative default prior distribution for logistic and other regression models. Ann Appl Stat. 2008;2(4):1360–83. https://doi.org/10.1214/08-AOAS191.CrossRef

16.

Kruschke J. Doing Bayesian data analysis: a tutorial introduction with R JAGS, and Stan; 2015.

17.

C. T. R. R, “A language and environment for statistical computing. Vienna, Austria. https://www.R- project.org:,” R Found. Stat. Comput. 2017., 2017.

18.

Lyons R. Statistical correctness. Libr Inf Sci Res. 2011;33(1):92–5. https://doi.org/10.1016/j.lisr.2010.07.016.CrossRef

19.

J. Jager, D. L. Putnick, and M. H. Bornstein, “Ii. More THAN JUST convenient: the scientific merits of homogeneous convenience samples,” Monogr Soc Res Child Dev., 2017, doi: https://doi.org/10.1111/mono.12296.

20.

Tyrer S, Heyman B. Sampling in epidemiological research: issues, hazards and pitfalls. BJPsych Bull. 2016;40(2):57–60. https://doi.org/10.1192/pb.bp.114.050203.CrossRefPubMedPubMedCentral

21.

Setia M. Methodology series module 5: Sampling strategies. Indian J Dermatol. 2016;61(5):505–1. https://doi.org/10.4103/0019-5154.190118.CrossRefPubMedPubMedCentral

22.

Fielding N, Lee RM, Blank G. The SAGE handbook of online research methods. London: SAGE Publications Ltd.; 2008.CrossRef

23.

A. Agodi et al., “Epidemiology of intensive care unit-acquired sepsis in Italy: results of the SPIN-UTI network,” Ann di Ig., 2018, doi: https://doi.org/10.7416/ai.2018.2247.

24.

M. L. Moro, M. Marchi, R. Buttazzi, and S. Nascetti, “Progress in infection prevention and control in Italy: a nationwide survey,” J Hosp Infect., 2011, doi: https://doi.org/10.1016/j.jhin.2010.08.009.

25.

Donders ART, van der Heijden GJMG, Stijnen T, Moons KGM. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59(10):1087–91. https://doi.org/10.1016/j.jclinepi.2006.01.014.CrossRefPubMed

26.

Little RJA, Rubin DB. Statistical Analysis with Missing Data. 3rd ed. Hoboken: Wiley; 2020.

27.

Mason AJ. Bayesian methods for modelling non-random missing data mechanisms in longitudinal studies. In: Technical report. London: Imperial College; 2010.

28.

Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581–92. https://doi.org/10.1186/1471-2105-12-432.CrossRef

29.

Zhang Z. Missing data imputation: Focusing on single imputation. Ann Transl Med. 2016;4(1). https://doi.org/10.3978/j.issn.2305-5839.2015.12.38.

30.

Kang H. The prevention and handling of the missing data. Korean J Anesthesiol. 2013;64(5):402–6. https://doi.org/10.4097/kjae.2013.64.5.402.CrossRefPubMedPubMedCentral

31.

Sterne JAC, et al. Multiple imputation for missing data in epidemiological and clinical research: Potential and pitfalls. BMJ (Online). 2009. https://doi.org/10.1136/bmj.b2393.

32.

T. J. Bird et al., “Statistical solutions for error and bias in global citizen science datasets,” Biol Conserv. 2014, doi: https://doi.org/10.1016/j.biocon.2013.07.037.

33.

Brakenhoff TB, Mitroiu M, Keogh RH, Moons KGM, Groenwold RHH, van Smeden M. Measurement error is often neglected in medical literature: a systematic review. J Clin Epidemiol. 2018;98:89–97. https://doi.org/10.1016/j.jclinepi.2018.02.023.CrossRefPubMed

34.

Armstrong BG. Effect of measurement error on epidemiological studies of environmental and occupational exposures. Occup Environ Med. 1998. https://doi.org/10.1136/oem.55.10.651.

35.

L. D. H. Carl Suetens, Susan Hopkins, Jana Kolman, “European Centre for Disease Prevention and Control. Point prevalence survey of healthcare-associated infections and antimicrobial use in European acute care hospitals,” Stockholm, 2013. doi: https://doi.org/10.2900/86011.

36.

Castilla J, et al. Seroprevalence of HIV and HTLV in a representative sample of the Spanish population. Epidemiol Infect. 2000;125(1):159–62. https://doi.org/10.1017/S0950268899004203.CrossRefPubMedPubMedCentral

37.

Bonander C, Nilsson A, Björk J, Bergström GML, Strömberg U. Participation weighting based on sociodemographic register data improved external validity in a population-based cohort study. J Clin Epidemiol. 2019;108:54–63. https://doi.org/10.1016/j.jclinepi.2018.12.011.CrossRefPubMed

38.

Gonzzlez JMPPS, Regglez-Castillo M, Vidal-Melii C. A procedure for selecting representative subsamples of a population from a simple random sample. SSRN Electron J. 2015. https://doi.org/10.2139/ssrn.2655972.

39.

Li F, Zhou M, Zou Z, Li W, Huang C, He Z. A risk prediction model for invasive fungal disease in critically ill patients in the intensive care unit. Asian Nurs Res. 2018;12(4):299–303. https://doi.org/10.1016/j.anr.2018.11.004.CrossRef

40.

C. M. Pirkle, Y. Y. Wu, M. V. Zunzunegui, and J. F. Gómez, “Model-based recursive partitioning to identify risk clusters for metabolic syndrome and its components: findings from the international mobility in aging study,” BMJ Open, 2018, doi: https://doi.org/10.1136/bmjopen-2017-018680.

41.

Morris CN, Lysy M. Shrinkage estimation in multilevel normal models. Stat Sci. 2012;27(1):115–34. https://doi.org/10.1214/11-STS363.CrossRef

42.

Y. M. Park and Y. Kim, “A spatially filtered multilevel model to account for spatial dependency: application to self-rated health status in South Korea,” Int J Health Geogr., 2014, doi: https://doi.org/10.1186/1476-072X-13-6.

43.

S. Greenland, “Ecologic versus individual-level sources of bias in ecologicestimates of contextual health effects,” Int J Epidemiol., 2001, doi: https://doi.org/10.1093/ije/30.6.1343.

44.

Tyser AR, Abtahi AM, McFadden M, Presson AP. Evidence of non-response bias in the Press-Ganey patient satisfaction survey. BMC Health Serv Res. 2016;16(1). https://doi.org/10.1186/s12913-016-1595-z.

45.

Taylor BV, et al. Assessing possible selection bias in a national voluntary MS longitudinal study in Australia. Mult Scler J. 2013;19(12):1627–31. https://doi.org/10.1177/1352458513481511.CrossRef

46.

Knoll L, Felten MK, Ackermann D, Kraus T. Non-response bias in a surveillance program for asbestos-related lung cancer. J Occup Health. 2011;53(1):16–22. https://doi.org/10.1539/joh.L10061.CrossRefPubMed

47.

Al-Azazi S, Singer A, Rabbani R, Lix LM. Combining population-based administrative health records and electronic medical records for disease surveillance. BMC Med Inform Decis Mak. 2019;19(1). https://doi.org/10.1186/s12911-019-0845-5.

48.

Bull C, Byrnes J, Hettiarachchi R, Downes M. A systematic review of the validity and reliability of patient-reported experience measures. Health Serv Res. 2019;54(5):1023–35. https://doi.org/10.1111/1475-6773.13187.CrossRefPubMedPubMedCentral

49.

Cawley J, Maclean JC, Hammer M, Wintfeld N. Reporting error in weight and its implications for bias in economic models. Econ Hum Biol. 2015;19:27–44. https://doi.org/10.1016/j.ehb.2015.07.001.CrossRefPubMed

Title: Data quality assessment and subsampling strategies to correct distributional bias in prevalence studies
Authors: A. D’Ambrosio
J. Garlasco
F. Quattrocolo
C. Vicentini
C. M. Zotti
Publication date: 01-12-2021
Publisher: BioMed Central
Published in: BMC Medical Research Methodology / Issue 1/2021
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-021-01277-y

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Data quality assessment and subsampling strategies to correct distributional bias in prevalence studies

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2021

Modular literature review: a novel systematic search and review method to support priority setting in health policy and practice

Quantifying parent engagement in the randomized Fuel for Fun impact study identified design considerations and BMI relationships

Shifting research culture to address the mismatch between where trials recruit and where populations with the most disease live: a qualitative study

Visualizing adverse events in clinical trials using correspondence analysis with R-package visae

Employing multiple synchronous outcome samples per subject to improve study efficiency

Intra-database validation of case-identifying algorithms using reconstituted electronic health records from healthcare claims data