Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2013

Open Access 01-12-2013 | Research article

Validation of a registry-derived risk algorithm based on treatment protocol as a proxy for disease risk in childhood acute lymphoblastic leukemia

Authors: Sumit Gupta, Jason D Pole, Astrid Guttmann, Lillian Sung

Published in: BMC Medical Research Methodology | Issue 1/2013

Login to get access

Abstract

Background

Administrative databases and cancer registries are frequently used to conduct population-based research, but often lack clinical data necessary for risk stratification. Our objective was to determine the criterion validity of a risk-stratification algorithm based on treatment characteristics available from a pediatric cancer registry as a proxy for disease risk, by comparing it to traditional biology-based risk classifications.

Methods

We identified all children with acute lymphoblastic leukemia diagnosed at a single institution between January 2000 and June 2011, and linked them to a population-based cancer registry. Several risk algorithms were then constructed using disease risk variables collected through chart review by a pediatric oncologist, and compared to a risk algorithm based on treatment protocol name and age, available from the registry.

Results

Of 596 patients identified, 579 (97.1%) met inclusion criteria and were successfully linked. The registry-based algorithm showed almost perfect agreement with a biology-based algorithm based on age, initial white blood cell count, immunophenotype and cytogenetics (kappa=0.85, 95th confidence interval 0.81-0.90). Discrepant cases were often due to the presence of unusual high risk features not captured by standard disease-risk variables but reflected in clinicians’ choices of higher intensity treatment protocols.

Conclusions

Protocol name represents a valid proxy of disease risk, allowing for risk stratification while conducting comparative effectiveness research using cancer registries and health services data. Future studies should examine the validity of treatment-based risk algorithms in other malignancies and using other treatment characteristics commonly found in health services data, such as the receipt of specific chemotherapeutic agents.
Literature
1.
go back to reference Ellison LF, Pogany L, Mery LS: Childhood and adolescent cancer survival: a period analysis of data from the Canadian Cancer Registry. Eur J Cancer. 2007, 43: 1967-1975. 10.1016/j.ejca.2007.05.014.CrossRefPubMed Ellison LF, Pogany L, Mery LS: Childhood and adolescent cancer survival: a period analysis of data from the Canadian Cancer Registry. Eur J Cancer. 2007, 43: 1967-1975. 10.1016/j.ejca.2007.05.014.CrossRefPubMed
2.
go back to reference Pui C-H, Pei D, Pappo AS, Howard SC, Cheng C, Sandlund JT, Furman WL, Ribeiro RC, Spunt SL, Rubnitz JE, et al: Treatment outcomes in black and white children with cancer: results from the SEER database and St. Jude Children's Research Hospital, 1992 through 2007. J Clin Oncol. 2012, 30: 2005-2012. 10.1200/JCO.2011.40.8617.CrossRefPubMedPubMedCentral Pui C-H, Pei D, Pappo AS, Howard SC, Cheng C, Sandlund JT, Furman WL, Ribeiro RC, Spunt SL, Rubnitz JE, et al: Treatment outcomes in black and white children with cancer: results from the SEER database and St. Jude Children's Research Hospital, 1992 through 2007. J Clin Oncol. 2012, 30: 2005-2012. 10.1200/JCO.2011.40.8617.CrossRefPubMedPubMedCentral
3.
go back to reference Smith MA, Seibel NL, Altekruse SF, Ries LAG, Melbert DL, O'Leary M, Smith FO, Reaman GH: Outcomes for children and adolescents with cancer: Challenges for the twenty-first century. J Clin Oncol. 2010, 28: 2625-2634. 10.1200/JCO.2009.27.0421.CrossRefPubMedPubMedCentral Smith MA, Seibel NL, Altekruse SF, Ries LAG, Melbert DL, O'Leary M, Smith FO, Reaman GH: Outcomes for children and adolescents with cancer: Challenges for the twenty-first century. J Clin Oncol. 2010, 28: 2625-2634. 10.1200/JCO.2009.27.0421.CrossRefPubMedPubMedCentral
4.
go back to reference Hunger SP, Lu X, Devidas M, Camitta BM, Gaynon PS, Winick NJ, Reaman GH, Carroll WL: Improved survivial for children and adolescents with acute lymphoblastic leukemia between 1990 and 2005: a report from the Children's Oncology Group. J Clin Oncol. 2012, 30: 1663-1669. 10.1200/JCO.2011.37.8018.CrossRefPubMedPubMedCentral Hunger SP, Lu X, Devidas M, Camitta BM, Gaynon PS, Winick NJ, Reaman GH, Carroll WL: Improved survivial for children and adolescents with acute lymphoblastic leukemia between 1990 and 2005: a report from the Children's Oncology Group. J Clin Oncol. 2012, 30: 1663-1669. 10.1200/JCO.2011.37.8018.CrossRefPubMedPubMedCentral
5.
go back to reference Aplenc R, Fisher BT, Huang YS, Li Y, Alonzo TA, Gerbing RB, Hall M, Bertoch D, Keren R, Seif AE, et al: Merging of the national cancer institute-funded cooperative oncology group data with an administrative data source to develop a more effective platform for clinical trial analysis and comparitive effectiveness research: a report from the Children's oncology group. Pharmacoepidemiol Drug Saf. 2012, 21 (Suppl 2): 37-43.CrossRefPubMedPubMedCentral Aplenc R, Fisher BT, Huang YS, Li Y, Alonzo TA, Gerbing RB, Hall M, Bertoch D, Keren R, Seif AE, et al: Merging of the national cancer institute-funded cooperative oncology group data with an administrative data source to develop a more effective platform for clinical trial analysis and comparitive effectiveness research: a report from the Children's oncology group. Pharmacoepidemiol Drug Saf. 2012, 21 (Suppl 2): 37-43.CrossRefPubMedPubMedCentral
6.
go back to reference Walker DM, Fisher BT, Seif AE, Huang YS, Torp K, Li Y, Aplenc R: Dexrazoxane use in pediatric patients with acute lymphoblastic or myeloid leukemia from 1999 and 2009: analysis of a national cohort of patients in the pediatric health information systems database. Pediatr Blood Cancer. 2012, 60 (4): 616-620.CrossRefPubMedPubMedCentral Walker DM, Fisher BT, Seif AE, Huang YS, Torp K, Li Y, Aplenc R: Dexrazoxane use in pediatric patients with acute lymphoblastic or myeloid leukemia from 1999 and 2009: analysis of a national cohort of patients in the pediatric health information systems database. Pediatr Blood Cancer. 2012, 60 (4): 616-620.CrossRefPubMedPubMedCentral
7.
go back to reference Moller TR, Garwicz S, Barlow L, Winther JF, Glattre E, Olafsdottir G, Olsen JH, Perfekt R, Ritvanen A, Sankila R, et al: Decreasing late mortality among five-year survivors of cancer in childhood and adolescence: a population-based study in the Nordic Countries. J Clin Oncol. 2001, 19 (13): 3173-3181.PubMed Moller TR, Garwicz S, Barlow L, Winther JF, Glattre E, Olafsdottir G, Olsen JH, Perfekt R, Ritvanen A, Sankila R, et al: Decreasing late mortality among five-year survivors of cancer in childhood and adolescence: a population-based study in the Nordic Countries. J Clin Oncol. 2001, 19 (13): 3173-3181.PubMed
8.
go back to reference Fisher BT, Zaoutis TE, Leckerman KH, Localio R, Aplenc R: Risk factors for renal failure in pediatric patients with acute myeloid leukemia: a retrospective cohort study. Pediatr Blood Cancer. 2010, 55 (4): 655-661. 10.1002/pbc.22601.CrossRefPubMedPubMedCentral Fisher BT, Zaoutis TE, Leckerman KH, Localio R, Aplenc R: Risk factors for renal failure in pediatric patients with acute myeloid leukemia: a retrospective cohort study. Pediatr Blood Cancer. 2010, 55 (4): 655-661. 10.1002/pbc.22601.CrossRefPubMedPubMedCentral
9.
go back to reference Koch SV, Kejs AMT, Engholm G, Johansen C, Schmiegelow K: Educational attainment among survivors of childhood cancer: A population-based cohort study in Denmark. Br J Cancer. 2004, 91 (5): 923-928.PubMedPubMedCentral Koch SV, Kejs AMT, Engholm G, Johansen C, Schmiegelow K: Educational attainment among survivors of childhood cancer: A population-based cohort study in Denmark. Br J Cancer. 2004, 91 (5): 923-928.PubMedPubMedCentral
10.
go back to reference Pui CH, Campana D, Pei D, Bowman WP, Sandlund JT, Kaste SC, Ribeiro RC, Rubnitz JE, Raimondi SC, Onciu M, et al: Treating childhood acute lymphoblastic leukemia without cranial irradiation. N Engl J Med. 2009, 360: 2730-2741. 10.1056/NEJMoa0900386.CrossRefPubMedPubMedCentral Pui CH, Campana D, Pei D, Bowman WP, Sandlund JT, Kaste SC, Ribeiro RC, Rubnitz JE, Raimondi SC, Onciu M, et al: Treating childhood acute lymphoblastic leukemia without cranial irradiation. N Engl J Med. 2009, 360: 2730-2741. 10.1056/NEJMoa0900386.CrossRefPubMedPubMedCentral
12.
go back to reference Meyer AM, Carpenter WR, Abernethy AP, Sturmer T, Kosorok MR: Data for cancer comparative effectiveness research: Past, present, and future potential. Cancer. 2012, 118: 5186-5197. 10.1002/cncr.27552.CrossRefPubMedPubMedCentral Meyer AM, Carpenter WR, Abernethy AP, Sturmer T, Kosorok MR: Data for cancer comparative effectiveness research: Past, present, and future potential. Cancer. 2012, 118: 5186-5197. 10.1002/cncr.27552.CrossRefPubMedPubMedCentral
13.
go back to reference Rubnitz JE, Inaba H, Dahl G, Ribeiro RC, Bowman WP, Taub J, Pounds S, Razzouk BI, Lacayo NJ, Cao X, et al: Minimal residual disease-directed therapy for childhood acute myeloid leukemia: Results of the AML02 multicenter trial. Lancet Oncol. 2010, 11 (6): 543-552. 10.1016/S1470-2045(10)70090-5.CrossRefPubMedPubMedCentral Rubnitz JE, Inaba H, Dahl G, Ribeiro RC, Bowman WP, Taub J, Pounds S, Razzouk BI, Lacayo NJ, Cao X, et al: Minimal residual disease-directed therapy for childhood acute myeloid leukemia: Results of the AML02 multicenter trial. Lancet Oncol. 2010, 11 (6): 543-552. 10.1016/S1470-2045(10)70090-5.CrossRefPubMedPubMedCentral
14.
go back to reference Greenberg ML, Barr RD, DiMonte B, McLaughlin E, Greenberg C: Childhood cancer registries in Ontario, Canada: lessons learned from a comparison of two registries. Int J Cancer. 2003, 105 (1): 88-91. 10.1002/ijc.11004.CrossRefPubMed Greenberg ML, Barr RD, DiMonte B, McLaughlin E, Greenberg C: Childhood cancer registries in Ontario, Canada: lessons learned from a comparison of two registries. Int J Cancer. 2003, 105 (1): 88-91. 10.1002/ijc.11004.CrossRefPubMed
15.
go back to reference Arico M, Schrappe M, Hunger SP, Carroll WL, Conter V, Galimberti S, Manabe A, Saha V, Baruchel A, Vettenranta K, et al: Clinical outcome of children with newly diagnosed Philadelphia chromosome-positive acute lymphoblastic leukemia treated between 1995 and 2005. J Clin Oncol. 2010, 28 (31): 4755-4761. 10.1200/JCO.2010.30.1325.CrossRefPubMedPubMedCentral Arico M, Schrappe M, Hunger SP, Carroll WL, Conter V, Galimberti S, Manabe A, Saha V, Baruchel A, Vettenranta K, et al: Clinical outcome of children with newly diagnosed Philadelphia chromosome-positive acute lymphoblastic leukemia treated between 1995 and 2005. J Clin Oncol. 2010, 28 (31): 4755-4761. 10.1200/JCO.2010.30.1325.CrossRefPubMedPubMedCentral
16.
go back to reference Heerema NA, Nachman JB, Sather HN, Sensel MG, Lee MK, Hutchinson R, Lange BJ, Steinherz PG, Bostrom B, Gaynon PS, et al: Hypodiploidy with less than 45 chromosomes confers adverse risk in childhood acute lymphoblastic leukemia: a report from the children's cancer group. Blood. 1999, 94 (12): 4036-4045.PubMed Heerema NA, Nachman JB, Sather HN, Sensel MG, Lee MK, Hutchinson R, Lange BJ, Steinherz PG, Bostrom B, Gaynon PS, et al: Hypodiploidy with less than 45 chromosomes confers adverse risk in childhood acute lymphoblastic leukemia: a report from the children's cancer group. Blood. 1999, 94 (12): 4036-4045.PubMed
17.
go back to reference Borowitz MJ, Devidas M, Hunger SP, Bowman WP, Carroll A, Carroll WL, Linda S, Martin PL, Pullen J, Viswanatha D, et al: Clinical significance of minimal residual disease in childhood acute lymphoblastic leukemia and its relationship to other prognostic factors: a children's oncology group study. Blood. 2008, 111: 5477-5485. 10.1182/blood-2008-01-132837.CrossRefPubMedPubMedCentral Borowitz MJ, Devidas M, Hunger SP, Bowman WP, Carroll A, Carroll WL, Linda S, Martin PL, Pullen J, Viswanatha D, et al: Clinical significance of minimal residual disease in childhood acute lymphoblastic leukemia and its relationship to other prognostic factors: a children's oncology group study. Blood. 2008, 111: 5477-5485. 10.1182/blood-2008-01-132837.CrossRefPubMedPubMedCentral
18.
go back to reference Schrappe M, Camitta B, Pui CH: Long-term results of large prospective trials in childhood acute lymphoblastic leukemia. Leukemia. 2000, 14 (12): 2193-2194. 10.1038/sj.leu.2401977.CrossRefPubMed Schrappe M, Camitta B, Pui CH: Long-term results of large prospective trials in childhood acute lymphoblastic leukemia. Leukemia. 2000, 14 (12): 2193-2194. 10.1038/sj.leu.2401977.CrossRefPubMed
19.
go back to reference Landis R, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.CrossRefPubMed Landis R, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.CrossRefPubMed
20.
go back to reference Fleiss JL: Statistical methods for rates and proportions. 1981, New York: John Wiley, 2 Fleiss JL: Statistical methods for rates and proportions. 1981, New York: John Wiley, 2
21.
go back to reference Campana D: Minimal residual disease in acute lymphoblastic leukemia. Hematology. 2010, 2010 (1): 7-12. 10.1182/asheducation-2010.1.7.CrossRefPubMed Campana D: Minimal residual disease in acute lymphoblastic leukemia. Hematology. 2010, 2010 (1): 7-12. 10.1182/asheducation-2010.1.7.CrossRefPubMed
22.
go back to reference Chen I-M, Harvey RC, Mullighan CG, Gastier-Foster F, Wharton W, Kang H, Borowitz MJ, Camitta BM, Carroll AJ, Devidas M, et al: Outcome modeling with CRLF2, IKZF1, JAK, and minimal residual disease in pediatric acute lymphoblastic leukemia: a children's oncology group. Blood. 2012, 119 (15): 3512-3522. 10.1182/blood-2011-11-394221.CrossRefPubMedPubMedCentral Chen I-M, Harvey RC, Mullighan CG, Gastier-Foster F, Wharton W, Kang H, Borowitz MJ, Camitta BM, Carroll AJ, Devidas M, et al: Outcome modeling with CRLF2, IKZF1, JAK, and minimal residual disease in pediatric acute lymphoblastic leukemia: a children's oncology group. Blood. 2012, 119 (15): 3512-3522. 10.1182/blood-2011-11-394221.CrossRefPubMedPubMedCentral
23.
go back to reference Moorman AV, Richards SM, Robinson HM, Strefford JC, Gibson BES, Kinsey SE, Eden TOB, Vora AJ, Mitchell CD, Harrison CJ: Prognosis of children with acute lymphoblastic leukemia (ALL) and intrachromosomal amplification of chromosome 21 (iAMP21). Blood. 2007, 109 (6): 2327-2330. 10.1182/blood-2006-08-040436.CrossRefPubMed Moorman AV, Richards SM, Robinson HM, Strefford JC, Gibson BES, Kinsey SE, Eden TOB, Vora AJ, Mitchell CD, Harrison CJ: Prognosis of children with acute lymphoblastic leukemia (ALL) and intrachromosomal amplification of chromosome 21 (iAMP21). Blood. 2007, 109 (6): 2327-2330. 10.1182/blood-2006-08-040436.CrossRefPubMed
24.
go back to reference Revesz T, Kardos G, Kajtar P, Schuler D: The adverse effect of prolonged prednisolone pretreatment in children with acute lymphoblastic leukemia. Cancer. 1985, 55 (8): 1637-1640. 10.1002/1097-0142(19850415)55:8<1637::AID-CNCR2820550804>3.0.CO;2-H.CrossRefPubMed Revesz T, Kardos G, Kajtar P, Schuler D: The adverse effect of prolonged prednisolone pretreatment in children with acute lymphoblastic leukemia. Cancer. 1985, 55 (8): 1637-1640. 10.1002/1097-0142(19850415)55:8<1637::AID-CNCR2820550804>3.0.CO;2-H.CrossRefPubMed
25.
go back to reference Bonilla M, Gupta S, Vasquez R, Fuentes SL, DeReyes G, Ribeiro KCB, Sung L: Predictors of outcome and methodological issues in children with acute lymphoblastic leukemia in El Salvador. Eur J Cancer. 2010, 46 (18): 3280-3286. 10.1016/j.ejca.2010.07.001.CrossRefPubMed Bonilla M, Gupta S, Vasquez R, Fuentes SL, DeReyes G, Ribeiro KCB, Sung L: Predictors of outcome and methodological issues in children with acute lymphoblastic leukemia in El Salvador. Eur J Cancer. 2010, 46 (18): 3280-3286. 10.1016/j.ejca.2010.07.001.CrossRefPubMed
26.
go back to reference Arndt CA, Rose PS, Folpe AL, Laack NN: Common musculoskeletal tumors of childhood and adolescence. Mayo Clin Proc. 2012, 87 (5): 475-487. 10.1016/j.mayocp.2012.01.015.CrossRefPubMedPubMedCentral Arndt CA, Rose PS, Folpe AL, Laack NN: Common musculoskeletal tumors of childhood and adolescence. Mayo Clin Proc. 2012, 87 (5): 475-487. 10.1016/j.mayocp.2012.01.015.CrossRefPubMedPubMedCentral
27.
go back to reference Fisher BT, Harris T, Torp K, Seif AE, Shah A, Huang YS, Bailey LC, Kersun LS, Reilly AF, Rheingold SR, et al: Establishment of an 11-year cohort of 8733 pediatric patients hospitalized at United States free-standing children's hospitals with de novo acute lymphoblastic leukemia from health care administrative data. Med Care. 2012, [Epub ahead of print] Fisher BT, Harris T, Torp K, Seif AE, Shah A, Huang YS, Bailey LC, Kersun LS, Reilly AF, Rheingold SR, et al: Establishment of an 11-year cohort of 8733 pediatric patients hospitalized at United States free-standing children's hospitals with de novo acute lymphoblastic leukemia from health care administrative data. Med Care. 2012, [Epub ahead of print]
28.
go back to reference Bleyer A, Budd T, Montello M: Adolescents and young adults with cancer: the scope of the problem and criticality of clinical trials. Cancer. 2006, 7 (Suppl): 1645-1655.CrossRef Bleyer A, Budd T, Montello M: Adolescents and young adults with cancer: the scope of the problem and criticality of clinical trials. Cancer. 2006, 7 (Suppl): 1645-1655.CrossRef
29.
go back to reference Bleyer A: The adolescent gap in cancer treatment. J Regist Manage. 1996, 23: 114-115. Bleyer A: The adolescent gap in cancer treatment. J Regist Manage. 1996, 23: 114-115.
Metadata
Title
Validation of a registry-derived risk algorithm based on treatment protocol as a proxy for disease risk in childhood acute lymphoblastic leukemia
Authors
Sumit Gupta
Jason D Pole
Astrid Guttmann
Lillian Sung
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2013
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-13-68

Other articles of this Issue 1/2013

BMC Medical Research Methodology 1/2013 Go to the issue