Skip to main content
Top
Published in: Critical Care 1/2015

Open Access 01-12-2015 | Research

Validity of administrative data in recording sepsis: a systematic review

Authors: Rachel J Jolley, Keri Jo Sawka, Dean W Yergens, Hude Quan, Nathalie Jetté, Christopher J Doig

Published in: Critical Care | Issue 1/2015

Login to get access

Abstract

Introduction

Administrative health data have been used to study sepsis in large population-based studies. The validity of these study findings depends largely on the quality of the administrative data source and the validity of the case definition used. We systematically reviewed the literature to assess the validity of case definitions of sepsis used with administrative data.

Methods

Embase and MEDLINE were searched for published articles with International Classification of Diseases (ICD) coded data used to define sepsis. Abstracts and full-text articles were reviewed in duplicate. Data were abstracted from all eligible full-text articles, including ICD-9- and/or ICD-10-based case definitions, sensitivity (Sn), specificity (Sp), positive predictive value (PPV) and negative predictive value (NPV).

Results

Of 2,317 individual studies identified, 12 full-text articles met all eligibility criteria. A total of 38 sepsis case definitions were tested, which included over 130 different ICD codes. The most common ICD-9 codes were 038.x, 790.7 and 995.92, and the most common ICD-10 codes were A40.x and A41.x. The PPV was reported in ten studies and ranged from 5.6% to 100%, with a median of 50%. Other tests of diagnostic accuracy were reported only in some studies. Sn ranged from 5.9% to 82.3%; Sp ranged from 78.3% to 100%; and NPV ranged from 62.1% to 99.7%.

Conclusions

The validity of administrative data in recording sepsis varied substantially across individual studies and ICD definitions. Our work may serve as a reference point for consensus towards an improved and harmonized ICD-coded definition of sepsis.
Appendix
Available only for authorised users
Literature
1.
go back to reference Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29:1303–10.CrossRef Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29:1303–10.CrossRef
2.
go back to reference Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348:1546–54.CrossRef Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348:1546–54.CrossRef
3.
go back to reference Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long-term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304:1787–94.CrossRef Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long-term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304:1787–94.CrossRef
4.
go back to reference Lynn LA. The diagnosis of sepsis revisited – a challenge for young medical scientists in the 21st century. Patient Saf Surg. 2014;8:1.CrossRef Lynn LA. The diagnosis of sepsis revisited – a challenge for young medical scientists in the 21st century. Patient Saf Surg. 2014;8:1.CrossRef
5.
go back to reference Quach S, Hennessy DA, Faris P, Fong A, Quan H, Doig C. A comparison between the APACHE II and Charlson Index Score for predicting hospital mortality in critically ill patients. BMC Health Serv Res. 2009;9:129.CrossRef Quach S, Hennessy DA, Faris P, Fong A, Quan H, Doig C. A comparison between the APACHE II and Charlson Index Score for predicting hospital mortality in critically ill patients. BMC Health Serv Res. 2009;9:129.CrossRef
6.
go back to reference Zhan C, Miller MR. Administrative data based patient safety research: a critical review. Qual Saf Health Care. 2003;12:ii58–63.CrossRef Zhan C, Miller MR. Administrative data based patient safety research: a critical review. Qual Saf Health Care. 2003;12:ii58–63.CrossRef
8.
go back to reference Jetté N, Quan H, Hemmelgarn B, Drosler S, Maass C, Moskal L, et al. The development, evolution, and modifications of ICD-10: challenges to the international comparability of mortality data. Med Care. 2010;48:1105–10.CrossRef Jetté N, Quan H, Hemmelgarn B, Drosler S, Maass C, Moskal L, et al. The development, evolution, and modifications of ICD-10: challenges to the international comparability of mortality data. Med Care. 2010;48:1105–10.CrossRef
9.
go back to reference Odeh M. Sepsis, septicaemia, sepsis syndrome, and septic shock: the correct definition and use. Postgrad Med J. 1996;72:66.CrossRef Odeh M. Sepsis, septicaemia, sepsis syndrome, and septic shock: the correct definition and use. Postgrad Med J. 1996;72:66.CrossRef
10.
go back to reference Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101:1644–55.CrossRef Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101:1644–55.CrossRef
11.
go back to reference Sands KE, Bates DW, Lanken PN, Graman PS, Hibberd PL, Kahn KL, et al. Epidemiology of sepsis syndrome in 8 academic medical centers. JAMA. 1997;278:234–40.CrossRef Sands KE, Bates DW, Lanken PN, Graman PS, Hibberd PL, Kahn KL, et al. Epidemiology of sepsis syndrome in 8 academic medical centers. JAMA. 1997;278:234–40.CrossRef
12.
go back to reference Iwashyna TJ, Odden A, Rohde J, Bonham C, Kuhn L, Malani P, et al. Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis. Med Care. 2014;526:e39–43.CrossRef Iwashyna TJ, Odden A, Rohde J, Bonham C, Kuhn L, Malani P, et al. Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis. Med Care. 2014;526:e39–43.CrossRef
13.
go back to reference Quach S, Blais C, Quan H. Administrative data have high variation in validity for recording heart failure. Can J Cardiol. 2010;26:306–12.CrossRef Quach S, Blais C, Quan H. Administrative data have high variation in validity for recording heart failure. Can J Cardiol. 2010;26:306–12.CrossRef
14.
go back to reference St Germaine-Smith CS, Metcalfe A, Pringsheim T, Roberts JI, Beck CA, Hemmelgarn BR, et al. Recommendations for optimal ICD codes to study neurologic conditions a systematic review. Neurology. 2012;79:1049–55.CrossRef St Germaine-Smith CS, Metcalfe A, Pringsheim T, Roberts JI, Beck CA, Hemmelgarn BR, et al. Recommendations for optimal ICD codes to study neurologic conditions a systematic review. Neurology. 2012;79:1049–55.CrossRef
16.
go back to reference Yergens DW, Dutton DJ, Patten SB. An overview of the statistical methods reported by studies using the canadian community health survey. BMC Med Res Methodol. 2014;14:15.CrossRef Yergens DW, Dutton DJ, Patten SB. An overview of the statistical methods reported by studies using the canadian community health survey. BMC Med Res Methodol. 2014;14:15.CrossRef
17.
go back to reference Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64:821–9.CrossRef Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64:821–9.CrossRef
18.
go back to reference Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRef Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRef
19.
go back to reference Cevasco M, Borzecki AM, Chen Q, Zrelak PA, Shin M, Romano PS, et al. Positive predictive value of the AHRQ patient safety indicator “postoperative sepsis”: implications for practice and policy. J Am Coll Surg. 2011;212:954–61.CrossRef Cevasco M, Borzecki AM, Chen Q, Zrelak PA, Shin M, Romano PS, et al. Positive predictive value of the AHRQ patient safety indicator “postoperative sepsis”: implications for practice and policy. J Am Coll Surg. 2011;212:954–61.CrossRef
20.
go back to reference Gedeborg R, Furebring M, Michaëlsson K. Diagnosis-dependent misclassification of infections using administrative data variably affected the incidence and mortality estimates of ICU patients. J Clin Epidemiol. 2007;60:155–62.CrossRef Gedeborg R, Furebring M, Michaëlsson K. Diagnosis-dependent misclassification of infections using administrative data variably affected the incidence and mortality estimates of ICU patients. J Clin Epidemiol. 2007;60:155–62.CrossRef
21.
go back to reference Grijalva CG, Chung CP, Stein CM, Gideon PS, Dyer SM, Mitchel EF, et al. Computerized definitions showed high positive predictive values for identifying hospitalizations for congestive heart failure and selected infections in Medicaid enrollees with rheumatoid arthritis. Pharmacoepidemiol Drug Saf. 2008;17:890–5.CrossRef Grijalva CG, Chung CP, Stein CM, Gideon PS, Dyer SM, Mitchel EF, et al. Computerized definitions showed high positive predictive values for identifying hospitalizations for congestive heart failure and selected infections in Medicaid enrollees with rheumatoid arthritis. Pharmacoepidemiol Drug Saf. 2008;17:890–5.CrossRef
22.
go back to reference Ibrahim I, Jacobs IG, Webb SAR, Finn J. Accuracy of International Classification of Diseases, 10th Revision codes for identifying severe sepsis in patients admitted from the emergency department. Crit Care Resusc. 2012;14:112–8.PubMed Ibrahim I, Jacobs IG, Webb SAR, Finn J. Accuracy of International Classification of Diseases, 10th Revision codes for identifying severe sepsis in patients admitted from the emergency department. Crit Care Resusc. 2012;14:112–8.PubMed
23.
go back to reference Lawson EH, Louie R, Zingmond DS, Brook RH, Hall BL, Han L, et al. A comparison of clinical registry versus administrative claims data for reporting of 30-day surgical complications. Ann Surg. 2012;256:973–81.CrossRef Lawson EH, Louie R, Zingmond DS, Brook RH, Hall BL, Han L, et al. A comparison of clinical registry versus administrative claims data for reporting of 30-day surgical complications. Ann Surg. 2012;256:973–81.CrossRef
24.
go back to reference Madsen KM, Schonheyder HC, Kristensen B, Nielsen GL, Sorensen HT. Can hospital discharge diagnosis be used for surveillance of bacteremia? A data quality study of a Danish hospital discharge registry. Infect Control Hosp Epidemiol. 1998;19:175–80.CrossRef Madsen KM, Schonheyder HC, Kristensen B, Nielsen GL, Sorensen HT. Can hospital discharge diagnosis be used for surveillance of bacteremia? A data quality study of a Danish hospital discharge registry. Infect Control Hosp Epidemiol. 1998;19:175–80.CrossRef
25.
go back to reference Ollendorf DA, Fendrick AM, Massey K, Williams GR, Oster G. Is sepsis accurately coded on hospital bills? Value Health. 2002;5:79–81.CrossRef Ollendorf DA, Fendrick AM, Massey K, Williams GR, Oster G. Is sepsis accurately coded on hospital bills? Value Health. 2002;5:79–81.CrossRef
26.
go back to reference Quan H, Eastwood C, Cunningham CT, Liu M, Flemons W, De Coster C, et al. Validity of AHRQ patient safety indicators derived from ICD-10 hospital discharge abstract data (chart review study). BMJ Open. 2013;3:e003716.CrossRef Quan H, Eastwood C, Cunningham CT, Liu M, Flemons W, De Coster C, et al. Validity of AHRQ patient safety indicators derived from ICD-10 hospital discharge abstract data (chart review study). BMJ Open. 2013;3:e003716.CrossRef
27.
go back to reference Ramanathan R, Leavell P, Stockslager G, Mays C, Harvey D, Duane TM. Validity of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) screening for sepsis in surgical mortalities. Surg Infect (Larchmt). 2014;15:513–6.CrossRef Ramanathan R, Leavell P, Stockslager G, Mays C, Harvey D, Duane TM. Validity of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) screening for sepsis in surgical mortalities. Surg Infect (Larchmt). 2014;15:513–6.CrossRef
28.
go back to reference Schneeweiss S, Robicsek A, Scranton R, Zuckerman D, Soloman DH. Veteran’s Affairs hospital discharge database coded serious bacterial infections accurately. J Clin Epidemiol. 2007;60:397–409.CrossRef Schneeweiss S, Robicsek A, Scranton R, Zuckerman D, Soloman DH. Veteran’s Affairs hospital discharge database coded serious bacterial infections accurately. J Clin Epidemiol. 2007;60:397–409.CrossRef
29.
go back to reference Whittaker SA, Mikkelsen ME, Gaieski DF, Koshy S, Kean C, Fuchs BD. Severe sepsis cohorts derived from claims-based strategies appear to be biased toward a more severely ill patient population. Crit Care Med. 2013;41:945–53.CrossRef Whittaker SA, Mikkelsen ME, Gaieski DF, Koshy S, Kean C, Fuchs BD. Severe sepsis cohorts derived from claims-based strategies appear to be biased toward a more severely ill patient population. Crit Care Med. 2013;41:945–53.CrossRef
30.
go back to reference Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al. International Sepsis Definitions Conference. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Intensive Care Med. 2003;29:530–8.CrossRef Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al. International Sepsis Definitions Conference. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Intensive Care Med. 2003;29:530–8.CrossRef
31.
go back to reference Rohde JM, Odden AJ, Bonham C, Kuhn L, Malani PN, Chen LM, et al. The epidemiology of acute organ system dysfunction from severe sepsis outside of the intensive care unit. J Hosp Med. 2013;8:243–7.CrossRef Rohde JM, Odden AJ, Bonham C, Kuhn L, Malani PN, Chen LM, et al. The epidemiology of acute organ system dysfunction from severe sepsis outside of the intensive care unit. J Hosp Med. 2013;8:243–7.CrossRef
32.
go back to reference O’Malley KJ, Cook KF, Price MD, Wildes KR, Hurdle JF, Ashton CM. Measuring diagnoses: ICD code accuracy. Health Serv Res. 2005;40:1620–39.CrossRef O’Malley KJ, Cook KF, Price MD, Wildes KR, Hurdle JF, Ashton CM. Measuring diagnoses: ICD code accuracy. Health Serv Res. 2005;40:1620–39.CrossRef
33.
go back to reference Poeze M, Ramsay G, Gerlach H, Rubulotta F, Levy M. An international sepsis survey: a study of doctors’ knowledge and perception about sepsis. Crit Care. 2004;8:R409–13.CrossRef Poeze M, Ramsay G, Gerlach H, Rubulotta F, Levy M. An international sepsis survey: a study of doctors’ knowledge and perception about sepsis. Crit Care. 2004;8:R409–13.CrossRef
34.
go back to reference Assunção M, Akamine N, Cardoso GS, Mello PV, Teles JM, Nunes AL, et al. Survey on physician’s knowledge of sepsis: do they recognize it promptly? J Crit Care. 2010;25:545–52.CrossRef Assunção M, Akamine N, Cardoso GS, Mello PV, Teles JM, Nunes AL, et al. Survey on physician’s knowledge of sepsis: do they recognize it promptly? J Crit Care. 2010;25:545–52.CrossRef
35.
go back to reference Bone RC. Why new definitions of sepsis and organ failure are needed. Am J Med. 1993;95:348–50.CrossRef Bone RC. Why new definitions of sepsis and organ failure are needed. Am J Med. 1993;95:348–50.CrossRef
36.
go back to reference Trzeciak S, Zanotti-Cavazzoni S, Parrillo JE, Dellinger RP. Inclusion criteria for clinical trials in sepsis: did the American College of Chest Physicians/Society of Critical Care Medicine consensus conference definitions of sepsis have an impact? Chest. 2005;127:242–5.CrossRef Trzeciak S, Zanotti-Cavazzoni S, Parrillo JE, Dellinger RP. Inclusion criteria for clinical trials in sepsis: did the American College of Chest Physicians/Society of Critical Care Medicine consensus conference definitions of sepsis have an impact? Chest. 2005;127:242–5.CrossRef
37.
go back to reference Goto M, Ohl ME, Schweizer ML, Perencevich EN. Accuracy of administrative code data for the surveillance of healthcare-associated infections: a systematic review and metaanalysis. Clin Infect Dis. 2014;58:688–96.CrossRef Goto M, Ohl ME, Schweizer ML, Perencevich EN. Accuracy of administrative code data for the surveillance of healthcare-associated infections: a systematic review and metaanalysis. Clin Infect Dis. 2014;58:688–96.CrossRef
38.
go back to reference Quan H, Khan N, Hemmelgarn BR, Tu K, Chen G, Campbell N, et al. Validation of a case definition to define hypertension using administrative data. Hypertension. 2009;54:1423–8.CrossRef Quan H, Khan N, Hemmelgarn BR, Tu K, Chen G, Campbell N, et al. Validation of a case definition to define hypertension using administrative data. Hypertension. 2009;54:1423–8.CrossRef
39.
go back to reference Southern DA, Roberts B, Edwards A, Dean S, Norton P, Svenson LW, et al. Validity of administrative data claim-based methods for identifying individuals with diabetes at a population level. Can J Public Health. 2010;101:61–4.CrossRef Southern DA, Roberts B, Edwards A, Dean S, Norton P, Svenson LW, et al. Validity of administrative data claim-based methods for identifying individuals with diabetes at a population level. Can J Public Health. 2010;101:61–4.CrossRef
40.
go back to reference Drösler SE, Romano PS, Sundararajan V, Burnand B, Colin C, Pincus H, et al. How many diagnosis fields are needed to capture safety events in administrative data? Findings and recommendations from the WHO ICD-11 Topic Advisory Group on Quality and Safety. Int J Qual Health Care. 2014;26:16–25.CrossRef Drösler SE, Romano PS, Sundararajan V, Burnand B, Colin C, Pincus H, et al. How many diagnosis fields are needed to capture safety events in administrative data? Findings and recommendations from the WHO ICD-11 Topic Advisory Group on Quality and Safety. Int J Qual Health Care. 2014;26:16–25.CrossRef
Metadata
Title
Validity of administrative data in recording sepsis: a systematic review
Authors
Rachel J Jolley
Keri Jo Sawka
Dean W Yergens
Hude Quan
Nathalie Jetté
Christopher J Doig
Publication date
01-12-2015
Publisher
BioMed Central
Published in
Critical Care / Issue 1/2015
Electronic ISSN: 1364-8535
DOI
https://doi.org/10.1186/s13054-015-0847-3

Other articles of this Issue 1/2015

Critical Care 1/2015 Go to the issue