Top

BMC Cancer

Published in:

Open Access 01-12-2021 | Breast Cancer | Research article

Evaluation of algorithms using administrative health and structured electronic medical record data to determine breast and colorectal cancer recurrence in a Canadian province

Using algorithms to determine breast and colorectal cancer recurrence

Authors: Pascal Lambert, Marshall Pitz, Harminder Singh, Kathleen Decker

Published in: BMC Cancer | Issue 1/2021

Abstract

Background

Algorithms that use administrative health and electronic medical record (EMR) data to determine cancer recurrence have the potential to replace chart reviews. This study evaluated algorithms to determine breast and colorectal cancer recurrence in a Canadian province with a universal health care system.

Methods

Individuals diagnosed with stage I-III breast or colorectal cancer diagnosed from 2004 to 2012 in Manitoba, Canada were included. Pre-specified and conditional inference tree algorithms using administrative health and structured EMR data were developed. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) correct classification, and scaled Brier scores were measured.

Results

The weighted pre-specified variable algorithm for the breast cancer validation cohort (N = 1181, 167 recurrences) demonstrated 81.1% sensitivity, 93.2% specificity, 61.4% PPV, 97.4% NPV, 91.8% correct classification, and scaled Brier score of 0.21. The weighted conditional inference tree algorithm demonstrated 68.5% sensitivity, 97.0% specificity, 75.4% PPV, 95.8% NPV, 93.6% correct classification, and scaled Brier score of 0.39. The weighted pre-specified variable algorithm for the colorectal validation cohort (N = 693, 136 recurrences) demonstrated 77.7% sensitivity, 92.8% specificity, 70.7% PPV, 94.9% NPV, 90.1% correct classification, and scaled Brier score of 0.33. The conditional inference tree algorithm demonstrated 62.6% sensitivity, 97.8% specificity, 86.4% PPV, 92.2% NPV, 91.4% correct classification, and scaled Brier score of 0.42.

Conclusions

Algorithms developed in this study using administrative health and structured EMR data to determine breast and colorectal cancer recurrence had moderate sensitivity and PPV, high specificity, NPV, and correct classification, but low accuracy. The accuracy is similar to other algorithms developed to classify recurrence only (i.e., distinguished from second primary) and inferior to algorithms that do not make this distinction. The accuracy of algorithms for determining cancer recurrence only must improve before replacing chart reviews.

Yu X. In: Feuerstein M, Ganz P, editors. Epidemiology of Cancer recurrence, second primary Cancer, and comorbidity among Cancer survivors. New York: Springer; 2011.CrossRef

North American Association of Central Cancer Registries. APPENDIX C - Data Quality Indicators by Year and Registry. In: Hotes Ellison J, Wu XC, McLaughlin C, Lake A, Firth R, et al., editors. Cancer In North America: 1999–2003 Volume One: Incidence. Springfield: North American Association of Cancer Registries Inc.; 2006. p. II-325.

Lamont EB, Hernon JE, Weeks JC, Henderson C, Earle CR, Schilsky RL, et al. Measuring disease-free survival and cancer relapse using medicare claims from CALGB breast cancer trial participants (Companion to 9344). J Natl Cancer Inst. 2006;98(18)1335-8.

Livaudais-Toman J, Franco R, Prasad-Hayes M, Howell EA, Wisnivesky J, Bickell NA. A validation of administrative claims data to measure ovarian cancer recurrence and secondary debluking surgery. EGEMS. 2016;4(1):1208.

Chubak J, Yu O, Pocobelli G, Lamerato L, Webster J, Prout MN, et al. Administrative data algorithms to identify second breast cancer events following early-stage invasive cancer. J Natl Cancer Inst. 2012;104(12):931–40. https://doi.org/10.1093/jnci/djs233.CrossRefPubMedPubMedCentral

Rasmussen LA, Jensen H, Flytkjaer Virgilsen L, Beck Jellesmark Thorsen L, Vrou Offersen B, Vedsted P. A validated algorithm for register-based identification of patients with recurrence of breast cancer - based on Danish Breast Cancer Group (DBCG) data. Cancer Epidemiol. 2019;59:129–34.CrossRef

Hassett MJ, Ritzwoller DP, Taback N, Carroll N, Cronin AM, Ting GV, et al. Validating billing/encounter codes as indicators of lung, colorectal, breast, and prostate cancer recurrence using 2 large contemporary cohorts. Med Care. 2014;52(10):e65–73. https://doi.org/10.1097/MLR.0b013e318277eb6f.CrossRefPubMedPubMedCentral

Hassett MJ, Uno H, Cronin AM, Hornbrook MC, Ritzwoller DP. Detecting lung and colorectal cancer recurrence using structured clinical/administrative data to enable outcomes research and population health management. Med Care. 2017;55(12):e88–98. https://doi.org/10.1097/MLR.0000000000000404.CrossRefPubMedPubMedCentral

Xu Y, Kong S, Cheung WY, Bouchard-Fortier A, Dort JC, Quan H, et al. Development and validation of case-finding algorithms for recurrence of breast cancer using routinely collected administrative data. BMC Cancer. 2019;19(210):210.

10.

Cairncross ZF, Nelson G, Shack L, Metcalfe A. Validation in Alberta of an administrative data algorithm to identify cancer recurrence. Curr Oncol. 2020;27(3):e343–e6. https://doi.org/10.3747/co.27.5861.CrossRefPubMedPubMedCentral

11.

Manitoba Health, Seniors and Active Living. Population Report, June 1, 2019. Winnipeg: Manitoba Health, Seniors and Active Living; 2019.

12.

Robinson JR, Young TK, Roos LL, Gelskey DE. Estimating the burden of disease. Comparing administrative data and self-reports. Med Care. 1997;35(9):932–47. https://doi.org/10.1097/00005650-199709000-00006.CrossRefPubMed

13.

Roos LL, Mustard CA, Nicol JP, McLarran DF, Malenka DJ, et al. Registries and administrative data: organization and accuracy. Med Care. 1993;31(3):201–12. https://doi.org/10.1097/00005650-199303000-00002.CrossRefPubMed

14.

Roos LL, Walld R, Uhanova J, Bond R. Physician visits, hospitalizations, and socioeconomic status: ambulatory case sensitive condintions in a Canadian setting. Health Serv Res. 2005;40(4):1167–85. https://doi.org/10.1111/j.1475-6773.2005.00407.x.CrossRefPubMedPubMedCentral

15.

Arvold ND, Taghaian AG, Niemierko A, Abi Raad RF, Srreedhara M, Nguyen PL, et al. Age, breast cancer subtype approximation, and local recurrence after breast-conserving therapy. J Clin Oncol. 2011;29(29):3885–91. https://doi.org/10.1200/JCO.2011.36.1105.CrossRefPubMedPubMedCentral

16.

Siegel R, Miller K, Fedewa S, Ahnen D, Meester R, Barzi A, et al. Colorectal cancer statistics, 2017. CA Cancer J Clin. 2017;67(3):177–93. https://doi.org/10.3322/caac.21395.CrossRefPubMed

17.

Chatfield C. Model uncertainty, data mining and statistical inference. J R Stat Soc A. 1995;158(3):419–66. https://doi.org/10.2307/2983440.CrossRef

18.

Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc. 1983;78(382):316–31. https://doi.org/10.1080/01621459.1983.10477973.CrossRef

19.

Harrell FE, Lee K, Mark D. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing erros. Stat Med. 1996;15(4):361–87. https://doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4.

20.

Hothorn T, Hornik K, Zeileis A. Unbiased recursive partitioning: a conditional inference framework. J Comput Graph Stat. 2006;15(3):651–74. https://doi.org/10.1198/106186006X133933.CrossRef

21.

Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-38.

22.

Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and udating. New York: Springer-Verlag; 2009. https://doi.org/10.1007/978-0-387-77244-8.CrossRef

23.

Albrechtsen S, Rasmussen M, Thoresen S, Irgens LM, Iversen OE. Pregnancy outcomes in women before and after cervical conisation: population based cohort study. BMJ. 2008;337(sep18 1):a1343. https://doi.org/10.1136/bmj.a1343.CrossRefPubMedPubMedCentral

24.

Banerjee I, Bozkurt S, Caswell-Jin JL, Kurian AW, Rubin DL. Natural language processing approaches to detect the timeline of metastatic recurrence of breast cancer. JCO Cllin Cancer Inform. 2019:1–12.

25.

Carrell DS, Halgrim S, Tran D-T, Buist DS, Chubak J, Chapman WW, et al. Using natural language processing to improve efficiency of manual chart abstraction in research: the case of breast cancer recurrence. Am J Epidemiol. 2013;179(6):749–58.CrossRef

26.

Ling AY, Kurian AW, Caswell-Jin JL, Sledge GW Jr, Shah NH, Tamang SR. Using natural language processing to construct a metastatic breast cancer cohort from linked cancer registry and electronic medical records data. JAMIA Open. 2019;2(4):528–37. https://doi.org/10.1093/jamiaopen/ooz040.CrossRefPubMedPubMedCentral

27.

Zeng Z, Espino S, Roy A, Li X, Khan SA, Clare SE, et al. Using natural language processing and machine learning to identify breast cancer local recurrence. BMC Bioinformatics. 2018;19(S17):498–508. https://doi.org/10.1186/s12859-018-2466-x.CrossRefPubMedPubMedCentral

28.

Roos LL, Traverse D, Turner D. Delivering prevention: the role of public programs in delivering care to high-risk populations. Med Care. 1999;37(6):JS264–JS78.PubMed

29.

McPhee SJ, Nguyen TT, Shema SJ, Nguyen B, Somkin C, Vo P, et al. Validation of recall of breast and cervical cancer screening by women in an ethcinally diverse population. Prev Med. 2002;35(5):463–73. https://doi.org/10.1006/pmed.2002.1096.CrossRefPubMed

30.

McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276–82.CrossRef

Title: Evaluation of algorithms using administrative health and structured electronic medical record data to determine breast and colorectal cancer recurrence in a Canadian province
Using algorithms to determine breast and colorectal cancer recurrence
Authors: Pascal Lambert
Marshall Pitz
Harminder Singh
Kathleen Decker
Publication date: 01-12-2021
Publisher: BioMed Central
Keywords: Breast Cancer
Breast Cancer
Colorectal Cancer
Colorectal Cancer
Published in: BMC Cancer / Issue 1/2021
Electronic ISSN: 1471-2407
DOI: https://doi.org/10.1186/s12885-021-08526-9

Webinar | 19-02-2024 | 17:30 (CET)

Keynote webinar | Spotlight on antibody–drug conjugates in cancer

Watch now

Antibody–drug conjugates (ADCs) are novel agents that have shown promise across multiple tumor types. Explore the current landscape of ADCs in breast and lung cancer with our experts, and gain insights into the mechanism of action, key clinical trials data, existing challenges, and future directions.

Dr. Véronique Diéras

Prof. Fabrice Barlesi

Developed by: Springer Medicine

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2021

Diagnostic codes of cancer in Skåne healthcare register: a validation study using individual-level data in southern Sweden

High expression of eIF4E is associated with tumor macrophage infiltration and leads to poor prognosis in breast cancer

Novel meso-substituted porphyrin derivatives and its potential use in photodynamic therapy of cancer

Efficacy and safety of rucaparib in previously treated, locally advanced or metastatic urothelial carcinoma from a phase 2, open-label trial (ATLAS)

Detection of tumor-derived extracellular vesicles in plasma from patients with solid cancer

Adjuvant albumin-bound paclitaxel combined with S-1 vs. oxaliplatin combined with capecitabine after D2 gastrectomy in patients with stage III gastric adenocarcinoma: a phase III multicenter, open-label, randomized controlled clinical trial protocol

Keynote webinar | Spotlight on antibody–drug conjugates in cancer