Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2016

Open Access 01-12-2016 | Research article

Diagnostic accuracy of administrative data algorithms in the diagnosis of osteoarthritis: a systematic review

Authors: Swastina Shrestha, Amish J. Dave, Elena Losina, Jeffrey N. Katz

Published in: BMC Medical Informatics and Decision Making | Issue 1/2016

Login to get access

Abstract

Background

Administrative health care data are frequently used to study disease burden and treatment outcomes in many conditions including osteoarthritis (OA). OA is a chronic condition with significant disease burden affecting over 27 million adults in the US. There are few studies examining the performance of administrative data algorithms to diagnose OA. The purpose of this study is to perform a systematic review of administrative data algorithms for OA diagnosis; and, to evaluate the diagnostic characteristics of algorithms based on restrictiveness and reference standards.

Methods

Two reviewers independently screened English-language articles published in Medline, Embase, PubMed, and Cochrane databases that used administrative data to identify OA cases. Each algorithm was classified as restrictive or less restrictive based on number and type of administrative codes required to satisfy the case definition. We recorded sensitivity and specificity of algorithms and calculated positive likelihood ratio (LR+) and positive predictive value (PPV) based on assumed OA prevalence of 0.1, 0.25, and 0.50.

Results

The search identified 7 studies that used 13 algorithms. Of these 13 algorithms, 5 were classified as restrictive and 8 as less restrictive. Restrictive algorithms had lower median sensitivity and higher median specificity compared to less restrictive algorithms when reference standards were self-report and American college of Rheumatology (ACR) criteria. The algorithms compared to reference standard of physician diagnosis had higher sensitivity and specificity than those compared to self-reported diagnosis or ACR criteria.

Conclusions

Restrictive algorithms are more specific for OA diagnosis and can be used to identify cases when false positives have higher costs e.g. interventional studies. Less restrictive algorithms are more sensitive and suited for studies that attempt to identify all cases e.g. screening programs.
Appendix
Available only for authorised users
Literature
1.
go back to reference Iezzoni LI. Assessing quality using administrative data. Ann Intern Med. 1997;127(8):666–74. Epub 1998/02/12.CrossRefPubMed Iezzoni LI. Assessing quality using administrative data. Ann Intern Med. 1997;127(8):666–74. Epub 1998/02/12.CrossRefPubMed
3.
4.
go back to reference Bernatsky S, Lix L, O’Donnell S, Lacaille D, Network C. Consensus statements for the use of administrative health data in rheumatic disease research and surveillance. J Rheumatol. 2013;40(1):66–73. doi:10.3899/jrheum.120835. Epub 2012/11/03.CrossRefPubMed Bernatsky S, Lix L, O’Donnell S, Lacaille D, Network C. Consensus statements for the use of administrative health data in rheumatic disease research and surveillance. J Rheumatol. 2013;40(1):66–73. doi:10.​3899/​jrheum.​120835. Epub 2012/11/03.CrossRefPubMed
5.
go back to reference Saczynski JS, Andrade SE, Harrold LR, Tjia J, Cutrona SL, Dodd KS, et al. A systematic review of validated methods for identifying heart failure using administrative data. Pharmacoepidemiol Drug Saf. 2012;21 Suppl 1:129–40. doi:10.1002/pds.2313. PubMed PMID: 22262599, PubMed Central PMCID: PMC3808171, Epub 2012/01/25.CrossRefPubMed Saczynski JS, Andrade SE, Harrold LR, Tjia J, Cutrona SL, Dodd KS, et al. A systematic review of validated methods for identifying heart failure using administrative data. Pharmacoepidemiol Drug Saf. 2012;21 Suppl 1:129–40. doi:10.​1002/​pds.​2313. PubMed PMID: 22262599, PubMed Central PMCID: PMC3808171, Epub 2012/01/25.CrossRefPubMed
6.
go back to reference De Coster C, Quan H, Finlayson A, Gao M, Halfon P, Humphries KH, et al. Identifying priorities in methodological research using ICD-9-CM and ICD-10 administrative data: report from an international consortium. BMC Health Serv Res. 2006;6:77. doi:10.1186/1472-6963-6-77. Epub 2006/06/17. PubMed PMID: 16776836; PubMed Central PMCID: PMC1513221.CrossRefPubMedPubMedCentral De Coster C, Quan H, Finlayson A, Gao M, Halfon P, Humphries KH, et al. Identifying priorities in methodological research using ICD-9-CM and ICD-10 administrative data: report from an international consortium. BMC Health Serv Res. 2006;6:77. doi:10.​1186/​1472-6963-6-77. Epub 2006/06/17. PubMed PMID: 16776836; PubMed Central PMCID: PMC1513221.CrossRefPubMedPubMedCentral
7.
go back to reference Carnahan RM, Moores KG. Mini-Sentinel’s systematic reviews of validated methods for identifying health outcomes using administrative and claims data: methods and lessons learned. Pharmacoepidemiol Drug Saf. 2012;21 Suppl 1:82–9. doi:10.1002/pds.2321. Epub 2012/01/25.CrossRefPubMed Carnahan RM, Moores KG. Mini-Sentinel’s systematic reviews of validated methods for identifying health outcomes using administrative and claims data: methods and lessons learned. Pharmacoepidemiol Drug Saf. 2012;21 Suppl 1:82–9. doi:10.​1002/​pds.​2321. Epub 2012/01/25.CrossRefPubMed
8.
go back to reference Nguyen M, Ball R, Midthun K, Lieu TA. The Food and Drug Administration’s post-licensure rapid immunization safety monitoring program: strengthening the federal vaccine safety enterprise. Pharmacoepidemiol Drug Saf. 2012;21 Suppl 1:291–7. doi:10.1002/pds.2323. Epub 2012/01/25.CrossRefPubMed Nguyen M, Ball R, Midthun K, Lieu TA. The Food and Drug Administration’s post-licensure rapid immunization safety monitoring program: strengthening the federal vaccine safety enterprise. Pharmacoepidemiol Drug Saf. 2012;21 Suppl 1:291–7. doi:10.​1002/​pds.​2323. Epub 2012/01/25.CrossRefPubMed
9.
go back to reference Lawrence RC, Felson DT, Helmick CG, Arnold LM, Choi H, Deyo RA, et al. Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part II Arthritis Rheum. 2008;58(1):26–35. doi:10.1002/art.23176. PubMed PMID: 18163497, PubMed Central PMCID: PMC3266664, Epub 2008/01/01.CrossRefPubMed Lawrence RC, Felson DT, Helmick CG, Arnold LM, Choi H, Deyo RA, et al. Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part II Arthritis Rheum. 2008;58(1):26–35. doi:10.​1002/​art.​23176. PubMed PMID: 18163497, PubMed Central PMCID: PMC3266664, Epub 2008/01/01.CrossRefPubMed
10.
go back to reference World Health Organization. The global burden of disease: 2004 update. Geneva: WHO Press; 2008. World Health Organization. The global burden of disease: 2004 update. Geneva: WHO Press; 2008.
11.
go back to reference Solomon DH, Avorn J, Wang PS, Vaillant G, Cabral D, Mogun H, et al. Prescription opioid use among older adults with arthritis or low back pain. Arthritis Rheum. 2006;55(1):35–41. doi:10.1002/art.21697. Epub 2006/02/08.CrossRefPubMed Solomon DH, Avorn J, Wang PS, Vaillant G, Cabral D, Mogun H, et al. Prescription opioid use among older adults with arthritis or low back pain. Arthritis Rheum. 2006;55(1):35–41. doi:10.​1002/​art.​21697. Epub 2006/02/08.CrossRefPubMed
12.
go back to reference Harrold LR, Yood RA, Straus W, Andrade SE, Reed JI, Cernieux J, et al. Challenges of estimating health service utilization for osteoarthritis patients on a population level. J Rheumatol. 2002;29(9):1931–6. Epub 2002/09/18.PubMed Harrold LR, Yood RA, Straus W, Andrade SE, Reed JI, Cernieux J, et al. Challenges of estimating health service utilization for osteoarthritis patients on a population level. J Rheumatol. 2002;29(9):1931–6. Epub 2002/09/18.PubMed
13.
go back to reference Kopec JA, Rahman MM, Sayre EC, Cibere J, Flanagan WM, Aghajanian J, et al. Trends in physician-diagnosed osteoarthritis incidence in an administrative database in British Columbia, Canada, 1996–1997 through 2003–2004. Arthritis Rheum. 2008;59(7):929–34. doi:10.1002/art.23827. Epub 2008/06/26.CrossRefPubMed Kopec JA, Rahman MM, Sayre EC, Cibere J, Flanagan WM, Aghajanian J, et al. Trends in physician-diagnosed osteoarthritis incidence in an administrative database in British Columbia, Canada, 1996–1997 through 2003–2004. Arthritis Rheum. 2008;59(7):929–34. doi:10.​1002/​art.​23827. Epub 2008/06/26.CrossRefPubMed
14.
go back to reference Katz JN, Barrett J, Mahomed NN, Baron JA, Wright RJ, Losina E. Association between hospital and surgeon procedure volume and the outcomes of total knee replacement. J Bone Joint Surg Am. 2004;86-A(9):1909–16. Epub 2004/09/03.PubMed Katz JN, Barrett J, Mahomed NN, Baron JA, Wright RJ, Losina E. Association between hospital and surgeon procedure volume and the outcomes of total knee replacement. J Bone Joint Surg Am. 2004;86-A(9):1909–16. Epub 2004/09/03.PubMed
15.
go back to reference Katz JN, Losina E, Barrett J, Phillips CB, Mahomed NN, Lew RA, et al. Association between hospital and surgeon procedure volume and outcomes of total hip replacement in the United States medicare population. J Bone Joint Surg Am. 2001;83-A(11):1622–9. Epub 2001/11/10.PubMed Katz JN, Losina E, Barrett J, Phillips CB, Mahomed NN, Lew RA, et al. Association between hospital and surgeon procedure volume and outcomes of total hip replacement in the United States medicare population. J Bone Joint Surg Am. 2001;83-A(11):1622–9. Epub 2001/11/10.PubMed
16.
go back to reference Widdifield J, Labrecque J, Lix L, Paterson JM, Bernatsky S, Tu K, et al. Systematic review and critical appraisal of validation studies to identify rheumatic diseases in health administrative databases. Arthritis Care Res (Hoboken). 2013;65(9):1490–503. doi:10.1002/acr.21993. Epub 2013/02/26.CrossRef Widdifield J, Labrecque J, Lix L, Paterson JM, Bernatsky S, Tu K, et al. Systematic review and critical appraisal of validation studies to identify rheumatic diseases in health administrative databases. Arthritis Care Res (Hoboken). 2013;65(9):1490–503. doi:10.​1002/​acr.​21993. Epub 2013/02/26.CrossRef
18.
go back to reference Leong A, Dasgupta K, Bernatsky S, Lacaille D, Avina-Zubieta A, Rahme E. Systematic review and meta-analysis of validation studies on a diabetes case definition from health administrative records. PLoS One. 2013;8(10):e75256. doi:10.1371/journal.pone.0075256. PubMed PMID: 24130696, PubMed Central PMCID: PMC3793995, Epub 2013/10/17.CrossRefPubMedPubMedCentral Leong A, Dasgupta K, Bernatsky S, Lacaille D, Avina-Zubieta A, Rahme E. Systematic review and meta-analysis of validation studies on a diabetes case definition from health administrative records. PLoS One. 2013;8(10):e75256. doi:10.​1371/​journal.​pone.​0075256. PubMed PMID: 24130696, PubMed Central PMCID: PMC3793995, Epub 2013/10/17.CrossRefPubMedPubMedCentral
19.
go back to reference Altman DG, Bland JM, Diagnostic tests. 1: Sensitivity and specificity. BMJ. 1994;308(6943):1552. Epub 1994/06/11. PubMed PMID: 8019315; PubMed Central PMCID: PMC2540489.CrossRefPubMedPubMedCentral Altman DG, Bland JM, Diagnostic tests. 1: Sensitivity and specificity. BMJ. 1994;308(6943):1552. Epub 1994/06/11. PubMed PMID: 8019315; PubMed Central PMCID: PMC2540489.CrossRefPubMedPubMedCentral
20.
go back to reference Simel DL, Samsa GP, Matchar DB. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol. 1991;44(8):763–70. Epub 1991/01/01.CrossRefPubMed Simel DL, Samsa GP, Matchar DB. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol. 1991;44(8):763–70. Epub 1991/01/01.CrossRefPubMed
21.
go back to reference Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997;16(9):981–91. Epub 1997/05/15.CrossRefPubMed Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997;16(9):981–91. Epub 1997/05/15.CrossRefPubMed
22.
go back to reference Mikkelsen WM, Dodge HJ, Duff IF, Kato H. Estimates of the prevalence of rheumatic diseases in the population of Tecumseh, Michigan, 1959–60. J Chronic Dis. 1967;20(6):351–69. Epub 1967/06/01.CrossRefPubMed Mikkelsen WM, Dodge HJ, Duff IF, Kato H. Estimates of the prevalence of rheumatic diseases in the population of Tecumseh, Michigan, 1959–60. J Chronic Dis. 1967;20(6):351–69. Epub 1967/06/01.CrossRefPubMed
23.
go back to reference Fowles JB, Lawthers AG, Weiner JP, Garnick DW, Petrie DS, Palmer RH. Agreement between physicians’ office records and Medicare Part B claims data. Health Care Financ Rev. 1995;16(4):189–99.PubMedPubMedCentral Fowles JB, Lawthers AG, Weiner JP, Garnick DW, Petrie DS, Palmer RH. Agreement between physicians’ office records and Medicare Part B claims data. Health Care Financ Rev. 1995;16(4):189–99.PubMedPubMedCentral
24.
go back to reference Kadhim-Saleh A, Green M, Williamson T, Hunter D, Birtwhistle R. Validation of the diagnostic algorithms for 5 chronic conditions in the Canadian Primary Care Sentinel Surveillance Network (CPCSSN): a Kingston Practice-based Research Network (PBRN) report. J Am Board Fam Med. 2013;26(2):159–67. doi:10.3122/jabfm.2013.02.120183. Epub 2013/03/09.CrossRefPubMed Kadhim-Saleh A, Green M, Williamson T, Hunter D, Birtwhistle R. Validation of the diagnostic algorithms for 5 chronic conditions in the Canadian Primary Care Sentinel Surveillance Network (CPCSSN): a Kingston Practice-based Research Network (PBRN) report. J Am Board Fam Med. 2013;26(2):159–67. doi:10.​3122/​jabfm.​2013.​02.​120183. Epub 2013/03/09.CrossRefPubMed
25.
go back to reference Harrold LR, Yood RA, Andrade SE, Reed JI, Cernieux J, Straus W, et al. Evaluating the predictive value of osteoarthritis diagnoses in an administrative database. Arthritis Rheum. 2000;43(8):1881–5.CrossRefPubMed Harrold LR, Yood RA, Andrade SE, Reed JI, Cernieux J, Straus W, et al. Evaluating the predictive value of osteoarthritis diagnoses in an administrative database. Arthritis Rheum. 2000;43(8):1881–5.CrossRefPubMed
26.
go back to reference Williamson T, Green ME, Birtwhistle R, Khan S, Garies S, Wong ST, et al. Validating the 8 CPCSSN case definitions for chronic disease surveillance in a primary care database of electronic health records. Ann Fam Med. 2014;12(4):367–72. doi:10.1370/afm.1644. PubMed PMID: 25024246, PubMed Central PMCID: PMC4096475, Epub 2014/07/16.CrossRefPubMedPubMedCentral Williamson T, Green ME, Birtwhistle R, Khan S, Garies S, Wong ST, et al. Validating the 8 CPCSSN case definitions for chronic disease surveillance in a primary care database of electronic health records. Ann Fam Med. 2014;12(4):367–72. doi:10.​1370/​afm.​1644. PubMed PMID: 25024246, PubMed Central PMCID: PMC4096475, Epub 2014/07/16.CrossRefPubMedPubMedCentral
27.
go back to reference Coleman N, Halas G, Peeler W, Casaclang N, Williamson T, Katz A. From patient care to research: a validation study examining the factors contributing to data quality in a primary care electronic medical record database. BMC Fam Pract. 2015;16:11. doi:10.1186/s12875-015-0223-z. PubMed PMID: 25649201, PubMed Central PMCID: PMC4324413, Epub 2015/02/05.CrossRefPubMedPubMedCentral Coleman N, Halas G, Peeler W, Casaclang N, Williamson T, Katz A. From patient care to research: a validation study examining the factors contributing to data quality in a primary care electronic medical record database. BMC Fam Pract. 2015;16:11. doi:10.​1186/​s12875-015-0223-z. PubMed PMID: 25649201, PubMed Central PMCID: PMC4324413, Epub 2015/02/05.CrossRefPubMedPubMedCentral
28.
go back to reference Lix L, Yogendran M, Burchill C, Metge C, McKeen N, Moore D, et al. Defining and Validating Chronic Diseases: An Administrative Data Approach. Winnipeg: Manitoba Centre for Health Policy; 2006. Lix L, Yogendran M, Burchill C, Metge C, McKeen N, Moore D, et al. Defining and Validating Chronic Diseases: An Administrative Data Approach. Winnipeg: Manitoba Centre for Health Policy; 2006.
29.
go back to reference Rahman JA M, Kopec JA, Cibere J. Abstract no. 342 The Validation of Administrative Osteoarthritis Diagnosis using a Clinical and Radiological Population-Based Cohort from British Columbia, Canada. Osteoarthritis Cartilage. 2014;14(Supplement 4):S150. Rahman JA M, Kopec JA, Cibere J. Abstract no. 342 The Validation of Administrative Osteoarthritis Diagnosis using a Clinical and Radiological Population-Based Cohort from British Columbia, Canada. Osteoarthritis Cartilage. 2014;14(Supplement 4):S150.
30.
go back to reference Birtwhistle R, Keshavjee K, Lambert-Lanning A, Godwin M, Greiver M, Manca D, et al. Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. J Am Board Fam Med. 2009;22(4):412–22. doi:10.3122/jabfm.2009.04.090081. Epub 2009/07/10.CrossRefPubMed Birtwhistle R, Keshavjee K, Lambert-Lanning A, Godwin M, Greiver M, Manca D, et al. Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. J Am Board Fam Med. 2009;22(4):412–22. doi:10.​3122/​jabfm.​2009.​04.​090081. Epub 2009/07/10.CrossRefPubMed
32.
go back to reference Jencks SF, Williams DK, Kay TL. Assessing hospital-associated deaths from discharge data. The role of length of stay and comorbidities. JAMA. 1988;260(15):2240–6. Epub 1988/10/21.CrossRefPubMed Jencks SF, Williams DK, Kay TL. Assessing hospital-associated deaths from discharge data. The role of length of stay and comorbidities. JAMA. 1988;260(15):2240–6. Epub 1988/10/21.CrossRefPubMed
33.
go back to reference Kim C, Nevitt MC, Niu J, Clancy MM, Lane NE, Link TM, et al. Association of hip pain with radiographic evidence of hip osteoarthritis: diagnostic test study. BMJ. 2015;351:h5983. doi:10.1136/bmj.h5983. Epub 2015/12/04. PubMed PMID: 26631296; PubMed Central PMCID: PMC4667842.CrossRefPubMedPubMedCentral Kim C, Nevitt MC, Niu J, Clancy MM, Lane NE, Link TM, et al. Association of hip pain with radiographic evidence of hip osteoarthritis: diagnostic test study. BMJ. 2015;351:h5983. doi:10.​1136/​bmj.​h5983. Epub 2015/12/04. PubMed PMID: 26631296; PubMed Central PMCID: PMC4667842.CrossRefPubMedPubMedCentral
34.
go back to reference Hannan MT, Felson DT, Pincus T. Analysis of the discordance between radiographic changes and knee pain in osteoarthritis of the knee. J Rheumatol. 2000;27(6):1513–7. Epub 2000/06/14.PubMed Hannan MT, Felson DT, Pincus T. Analysis of the discordance between radiographic changes and knee pain in osteoarthritis of the knee. J Rheumatol. 2000;27(6):1513–7. Epub 2000/06/14.PubMed
35.
go back to reference Bedson J, Croft PR. The discordance between clinical and radiographic knee osteoarthritis: a systematic search and summary of the literature. BMC Musculoskelet Disord. 2008;9:116. doi:10.1186/1471-2474-9-116. PubMed PMID: 18764949, PubMed Central PMCID: PMC2542996, Epub 2008/09/04.CrossRefPubMedPubMedCentral Bedson J, Croft PR. The discordance between clinical and radiographic knee osteoarthritis: a systematic search and summary of the literature. BMC Musculoskelet Disord. 2008;9:116. doi:10.​1186/​1471-2474-9-116. PubMed PMID: 18764949, PubMed Central PMCID: PMC2542996, Epub 2008/09/04.CrossRefPubMedPubMedCentral
Metadata
Title
Diagnostic accuracy of administrative data algorithms in the diagnosis of osteoarthritis: a systematic review
Authors
Swastina Shrestha
Amish J. Dave
Elena Losina
Jeffrey N. Katz
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2016
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-016-0319-y

Other articles of this Issue 1/2016

BMC Medical Informatics and Decision Making 1/2016 Go to the issue