Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2014

Open Access 01-12-2014 | Research article

Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research

Authors: Alexander Rusanov, Nicole G Weiskopf, Shuang Wang, Chunhua Weng

Published in: BMC Medical Informatics and Decision Making | Issue 1/2014

Login to get access

Abstract

Background

To demonstrate that subject selection based on sufficient laboratory results and medication orders in electronic health records can be biased towards sick patients.

Methods

Using electronic health record data from 10,000 patients who received anesthetic services at a major metropolitan tertiary care academic medical center, an affiliated hospital for women and children, and an affiliated urban primary care hospital, the correlation between patient health status and counts of days with laboratory results or medication orders, as indicated by the American Society of Anesthesiologists Physical Status Classification (ASA Class), was assessed with a Negative Binomial Regression model.

Results

Higher ASA Class was associated with more points of data: compared to ASA Class 1 patients, ASA Class 4 patients had 5.05 times the number of days with laboratory results and 6.85 times the number of days with medication orders, controlling for age, sex, emergency status, admission type, primary diagnosis, and procedure.

Conclusions

Imposing data sufficiency requirements for subject selection allows researchers to minimize missing data when reusing electronic health records for research, but introduces a bias towards the selection of sicker patients. We demonstrated the relationship between patient health and quantity of data, which may result in a systematic bias towards the selection of sicker patients for research studies and limit the external validity of research conducted using electronic health record data. Additionally, we discovered other variables (i.e., admission status, age, emergency classification, procedure, and diagnosis) that independently affect data sufficiency.
Appendix
Available only for authorised users
Literature
1.
go back to reference Blumenthal D: Stimulating the adoption of health information technology. N Engl J Med. 2009, 360 (15): 1477-1479.CrossRefPubMed Blumenthal D: Stimulating the adoption of health information technology. N Engl J Med. 2009, 360 (15): 1477-1479.CrossRefPubMed
2.
go back to reference Blumenthal D, Tavenner M: The “meaningful use” regulation for electronic health records. N Engl J Med. 2010, 363 (6): 501-504.CrossRefPubMed Blumenthal D, Tavenner M: The “meaningful use” regulation for electronic health records. N Engl J Med. 2010, 363 (6): 501-504.CrossRefPubMed
3.
go back to reference Charles D, King J, Patel V, Furukawa M: The Office of the National Coordinator for Health Information Technology. ONC Data Brief No. 9: Adoption of Electronic Health record Systems among U.S. Non-federal Acute Care Hospitals: 2008-20012. 2013 Charles D, King J, Patel V, Furukawa M: The Office of the National Coordinator for Health Information Technology. ONC Data Brief No. 9: Adoption of Electronic Health record Systems among U.S. Non-federal Acute Care Hospitals: 2008-20012. 2013
4.
go back to reference Hsiao C, Hing E: National Center for Health Statistics. NCHS Data Brief No. 111: Use and characteristics of electronic health record systems among office-based physician practice: United States, 2001-2012. 2012 Hsiao C, Hing E: National Center for Health Statistics. NCHS Data Brief No. 111: Use and characteristics of electronic health record systems among office-based physician practice: United States, 2001-2012. 2012
5.
go back to reference Bloomrosen M, Detmer DE: Advancing the Framework: Use of Health Data—A Report of a Working Conference of the American Medical Informatics Association. J Am Med Inform Assoc. 2008, 15 (6): 715-722.CrossRefPubMedPubMedCentral Bloomrosen M, Detmer DE: Advancing the Framework: Use of Health Data—A Report of a Working Conference of the American Medical Informatics Association. J Am Med Inform Assoc. 2008, 15 (6): 715-722.CrossRefPubMedPubMedCentral
6.
go back to reference Hersh WR: Adding value to the electronic health record through secondary use of data for quality assurance, research, and surveillance. Am J Manage Care. 2007, 13 (6 Part 1): 277-278. Hersh WR: Adding value to the electronic health record through secondary use of data for quality assurance, research, and surveillance. Am J Manage Care. 2007, 13 (6 Part 1): 277-278.
7.
go back to reference Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang PC, Detmer DE, Expert P: Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper. J Am Med Inform Assoc. 2007, 14 (1): 1-9.CrossRefPubMedPubMedCentral Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang PC, Detmer DE, Expert P: Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper. J Am Med Inform Assoc. 2007, 14 (1): 1-9.CrossRefPubMedPubMedCentral
8.
go back to reference Morgan S, Grootendorst P, Lexchin J, Cunningham C, Greyson D: The cost of drug development: a systematic review. Health Policy. 2011, 100 (1): 4-17.CrossRefPubMed Morgan S, Grootendorst P, Lexchin J, Cunningham C, Greyson D: The cost of drug development: a systematic review. Health Policy. 2011, 100 (1): 4-17.CrossRefPubMed
9.
go back to reference Sung NS, Crowley WF, Genel M, Salber P, Sandy L, Sherwood LM, Johnson SB, Catanese V, Tilson H, Getz K, Larson EL, Scheinberg D, Reece EA, Slavkin H, Dobs A, Grebb J, Martinez RA, Korn A, Rimoin D: Central challenges facing the national clinical research enterprise. JAMA. 2003, 289 (10): 1278-1287.CrossRefPubMed Sung NS, Crowley WF, Genel M, Salber P, Sandy L, Sherwood LM, Johnson SB, Catanese V, Tilson H, Getz K, Larson EL, Scheinberg D, Reece EA, Slavkin H, Dobs A, Grebb J, Martinez RA, Korn A, Rimoin D: Central challenges facing the national clinical research enterprise. JAMA. 2003, 289 (10): 1278-1287.CrossRefPubMed
10.
go back to reference Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PR, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Cimino JJ, Saltz JH: Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care. 2013, 51 (8 Suppl 3): S30-S37.CrossRefPubMedPubMedCentral Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PR, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Cimino JJ, Saltz JH: Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care. 2013, 51 (8 Suppl 3): S30-S37.CrossRefPubMedPubMedCentral
11.
go back to reference Chan KS, Fowles JB, Weiner JP: Review: electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev. 2010, 67 (5): 503-527.CrossRefPubMed Chan KS, Fowles JB, Weiner JP: Review: electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev. 2010, 67 (5): 503-527.CrossRefPubMed
13.
14.
go back to reference Weiskopf NG, Hripcsak G, Swaminathan S, Weng C: Defining and measuring completeness of electronic health records for secondary use. J Biomed Inform. 2013, 46 (5): 830-836.CrossRefPubMed Weiskopf NG, Hripcsak G, Swaminathan S, Weng C: Defining and measuring completeness of electronic health records for secondary use. J Biomed Inform. 2013, 46 (5): 830-836.CrossRefPubMed
15.
go back to reference Barr MS: The clinical record: a 200-year-old 21st-century challenge. Ann Intern Med. 2010, 153 (10): 682-683.CrossRefPubMed Barr MS: The clinical record: a 200-year-old 21st-century challenge. Ann Intern Med. 2010, 153 (10): 682-683.CrossRefPubMed
16.
go back to reference Rothwell PM: External validity of randomised controlled trials: “to whom do the results of this trial apply?”. Lancet. 2005, 365 (9453): 82-93.CrossRefPubMed Rothwell PM: External validity of randomised controlled trials: “to whom do the results of this trial apply?”. Lancet. 2005, 365 (9453): 82-93.CrossRefPubMed
17.
go back to reference Woratanarat P, Thaveeratitharm C, Woratanarat T, Angsanuntsukh C, Attia J, Thakkinstian A: Meta-analysis of hypercoagulability genetic polymorphisms in Perthes disease. J Orthop Res. 2014, 32 (1): 1-7.CrossRefPubMed Woratanarat P, Thaveeratitharm C, Woratanarat T, Angsanuntsukh C, Attia J, Thakkinstian A: Meta-analysis of hypercoagulability genetic polymorphisms in Perthes disease. J Orthop Res. 2014, 32 (1): 1-7.CrossRefPubMed
18.
go back to reference Hudson DL, Cohen ME: Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Conference 2013. Merging medical informatics and automated diagnostic methods. 2013, 4783-4786. Hudson DL, Cohen ME: Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Conference 2013. Merging medical informatics and automated diagnostic methods. 2013, 4783-4786.
19.
go back to reference Green BB, Anderson ML, Cook AJ, Catz S, Fishman PA, McClure JB, Reid R: Using body mass index data in the electronic health record to calculate cardiovascular risk. Am J Prev Med. 2012, 42 (4): 342-347.CrossRefPubMedPubMedCentral Green BB, Anderson ML, Cook AJ, Catz S, Fishman PA, McClure JB, Reid R: Using body mass index data in the electronic health record to calculate cardiovascular risk. Am J Prev Med. 2012, 42 (4): 342-347.CrossRefPubMedPubMedCentral
20.
go back to reference Wang RY, Strong DM: Beyond accuracy: what data quality means to data consumers. J Manage Inf Syst. 1996, 12 (4): 5-34.CrossRef Wang RY, Strong DM: Beyond accuracy: what data quality means to data consumers. J Manage Inf Syst. 1996, 12 (4): 5-34.CrossRef
21.
22.
go back to reference Weiskopf NG, Rusanov A, Weng C: Sick patients have more data: the non-random completeness of electronic health records. AMIA Annu Symp Proc. 2013, 2013: 1472-1477.PubMedPubMedCentral Weiskopf NG, Rusanov A, Weng C: Sick patients have more data: the non-random completeness of electronic health records. AMIA Annu Symp Proc. 2013, 2013: 1472-1477.PubMedPubMedCentral
23.
go back to reference Lee ML, Yano EM, Wang M, Simon BF, Rubenstein LV: What patient population does visit-based sampling in primary care settings represent?. Med Care. 2002, 40 (9): 761-770.CrossRefPubMed Lee ML, Yano EM, Wang M, Simon BF, Rubenstein LV: What patient population does visit-based sampling in primary care settings represent?. Med Care. 2002, 40 (9): 761-770.CrossRefPubMed
24.
go back to reference de Groot V, Beckerman H, Lankhorst GJ, Bouter LM: How to measure comorbidity. a critical review of available methods. J Clin Epidemiol. 2003, 56 (3): 221-229.CrossRefPubMed de Groot V, Beckerman H, Lankhorst GJ, Bouter LM: How to measure comorbidity. a critical review of available methods. J Clin Epidemiol. 2003, 56 (3): 221-229.CrossRefPubMed
25.
go back to reference Wennberg JE, Staiger DO, Sharp SM, Gottlieb DJ, Bevan G, McPherson K, Welch HG: Observational intensity bias associated with illness adjustment: cross sectional analysis of insurance claims. BMJ. 2013, 346: f549-CrossRefPubMedPubMedCentral Wennberg JE, Staiger DO, Sharp SM, Gottlieb DJ, Bevan G, McPherson K, Welch HG: Observational intensity bias associated with illness adjustment: cross sectional analysis of insurance claims. BMJ. 2013, 346: f549-CrossRefPubMedPubMedCentral
26.
go back to reference Saklad M: Grading of patients for surgical procedures. Anesthesiology. 1941, 2 (3): 281-284.CrossRef Saklad M: Grading of patients for surgical procedures. Anesthesiology. 1941, 2 (3): 281-284.CrossRef
28.
go back to reference Davenport DL, Bowe EA, Henderson WG, Khuri SF, Mentzer RM: National Surgical Quality Improvement Program (NSQIP) risk factors can be used to validate American Society of Anesthesiologists Physical Status Classification (ASA PS) levels. Ann Surg. 2006, 243 (5): 636-641. discussion 641-634CrossRefPubMedPubMedCentral Davenport DL, Bowe EA, Henderson WG, Khuri SF, Mentzer RM: National Surgical Quality Improvement Program (NSQIP) risk factors can be used to validate American Society of Anesthesiologists Physical Status Classification (ASA PS) levels. Ann Surg. 2006, 243 (5): 636-641. discussion 641-634CrossRefPubMedPubMedCentral
29.
go back to reference Dripps RD, Lamont A, Eckenhoff JE: The role of anesthesia in surgical mortality. JAMA. 1961, 178: 261-266.CrossRefPubMed Dripps RD, Lamont A, Eckenhoff JE: The role of anesthesia in surgical mortality. JAMA. 1961, 178: 261-266.CrossRefPubMed
30.
go back to reference Vacanti CJ, VanHouten RJ, Hill RC: A statistical analysis of the relationship of physical status to postoperative mortality in 68,388 cases. Anesth Analg. 1970, 49 (4): 564-566.CrossRefPubMed Vacanti CJ, VanHouten RJ, Hill RC: A statistical analysis of the relationship of physical status to postoperative mortality in 68,388 cases. Anesth Analg. 1970, 49 (4): 564-566.CrossRefPubMed
31.
go back to reference Wolters U, Wolf T, Stutzer H, Schroder T: ASA classification and perioperative variables as predictors of postoperative outcome. Br J Anaesth. 1996, 77 (2): 217-222.CrossRefPubMed Wolters U, Wolf T, Stutzer H, Schroder T: ASA classification and perioperative variables as predictors of postoperative outcome. Br J Anaesth. 1996, 77 (2): 217-222.CrossRefPubMed
35.
go back to reference Hilbe JM: Negative Binomial Regression. 2011, New York: Cambridge University Press, 2CrossRef Hilbe JM: Negative Binomial Regression. 2011, New York: Cambridge University Press, 2CrossRef
36.
go back to reference Collins SA, Cato K, Albers D, Scott K, Stetson PD, Bakken S, Vawdrey DK: Relationship between nursing documentation and patients’ mortality. Am J Crit Care. 2013, 22 (4): 306-313.CrossRefPubMedPubMedCentral Collins SA, Cato K, Albers D, Scott K, Stetson PD, Bakken S, Vawdrey DK: Relationship between nursing documentation and patients’ mortality. Am J Crit Care. 2013, 22 (4): 306-313.CrossRefPubMedPubMedCentral
37.
go back to reference Collins SA, Vawdrey DK: “Reading between the lines” of flow sheet data: nurses’ optional documentation associated with cardiac arrest outcomes. Appl Nurs Res. 2012, 25 (4): 251-257.CrossRefPubMed Collins SA, Vawdrey DK: “Reading between the lines” of flow sheet data: nurses’ optional documentation associated with cardiac arrest outcomes. Appl Nurs Res. 2012, 25 (4): 251-257.CrossRefPubMed
38.
Metadata
Title
Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research
Authors
Alexander Rusanov
Nicole G Weiskopf
Shuang Wang
Chunhua Weng
Publication date
01-12-2014
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2014
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/1472-6947-14-51

Other articles of this Issue 1/2014

BMC Medical Informatics and Decision Making 1/2014 Go to the issue