Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Informatics and Decision Making
Published in: BMC Medical Informatics and Decision Making 1/2021

01-12-2021 | Rituximab | Research article

Applying probability calibration to ensemble methods to predict 2-year mortality in patients with DLBCL

Authors: Shuanglong Fan, Zhiqiang Zhao, Hongmei Yu, Lei Wang, Chuchu Zheng, Xueqian Huang, Zhenhuan Yang, Meng Xing, Qing Lu, Yanhong Luo

Published in: BMC Medical Informatics and Decision Making | Issue 1/2021

Login to get access

Abstract

Background

Under the influences of chemotherapy regimens, clinical staging, immunologic expressions and other factors, the survival rates of patients with diffuse large B-cell lymphoma (DLBCL) are different. The accurate prediction of mortality hazards is key to precision medicine, which can help clinicians make optimal therapeutic decisions to extend the survival times of individual patients with DLBCL. Thus, we have developed a predictive model to predict the mortality hazard of DLBCL patients within 2 years of treatment.

Methods

We evaluated 406 patients with DLBCL and collected 17 variables from each patient. The predictive variables were selected by the Cox model, the logistic model and the random forest algorithm. Five classifiers were chosen as the base models for ensemble learning: the naïve Bayes, logistic regression, random forest, support vector machine and feedforward neural network models. We first calibrated the biased outputs from the five base models by using probability calibration methods (including shape-restricted polynomial regression, Platt scaling and isotonic regression). Then, we aggregated the outputs from the various base models to predict the 2-year mortality of DLBCL patients by using three strategies (stacking, simple averaging and weighted averaging). Finally, we assessed model performance over 300 hold-out tests.

Results

Gender, stage, IPI, KPS and rituximab were significant factors for predicting the deaths of DLBCL patients within 2 years of treatment. The stacking model that first calibrated the base model by shape-restricted polynomial regression performed best (AUC = 0.820, ECE = 8.983, MCE = 21.265) in all methods. In contrast, the performance of the stacking model without undergoing probability calibration is inferior (AUC = 0.806, ECE = 9.866, MCE = 24.850). In the simple averaging model and weighted averaging model, the prediction error of the ensemble model also decreased with probability calibration.

Conclusions

Among all the methods compared, the proposed model has the lowest prediction error when predicting the 2-year mortality of DLBCL patients. These promising results may indicate that our modeling strategy of applying probability calibration to ensemble learning is successful.
Literature
1.
Jemal A, Siegel R, Xu JQ, et al. Cancer statistics. CA Cancer J Clin. 2013;52(5):1–24.
2.
Roschewski M, Staudt LM, Wilson WH. Diffuse large B-cell lymphoma—treatment approaches in the molecular era. Nat Rev Clin Oncol. 2014;11(1):12–23.PubMedCrossRef
3.
4.
Martelli M, Ferreri AJM, Agostinelli C, et al. Diffuse large B-cell lymphoma. Crit Rev Oncol Hematol. 2013;87(2):146–71.PubMedCrossRef
5.
Tilly H, Vitolo U, Walewski J, et al. Diffuse large B-cell lymphoma (DLBCL): ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2012;23(Suppl 7):vii78–82.PubMedCrossRef
6.
Horn H, Ziepert M, Wartenberg M, et al. Different biological risk factors in young poor-prognosis and elderly patients with diffuse large B-cell lymphoma. Leukemia. 2015;29(7):1564–70.PubMedCrossRef
7.
Morrison VA, Hamlin P, Soubeyran P, et al. Diffuse large B-cell lymphoma in the elderly: impact of prognosis, comorbidities, geriatric assessment, and supportive care on clinical practice. An International Society of Geriatric Oncology (SIOG) expert position paper. J Geriatr Oncol. 2015;6(2):141–52.PubMedCrossRef
8.
Jameson JL, Longo DL. Precision medicine — personalized, problematic, and promising. N Engl J Med. 2015;372(23):2229–34.PubMedCrossRef
9.
10.
11.
Zhou ZH. Ensemble learning. In: Maching learning. Beijing: Tsinghua University press; 2016. p. 171–96.
12.
Ren Y, Zhang L, Suganthan PN. Ensemble classification and regression-recent developments, applications and future directions. IEEE Comput Intell Mag. 2016;11(1):41–53.CrossRef
13.
14.
Kohavi R, Wolpert DH. Bias plus variance decomposition for zero-one loss functions. In: ICML'96: Proceedings of the thirteenth international conference on international conference on machine learning. 1996. p. 275–283.
15.
Wang Y, Pan Z, Zheng J, et al. A hybrid ensemble method for pulsar candidate classification. Astrophysics Space Sci. 2019;364(8):139.CrossRef
16.
Qiu X, Zhang L, Ren Y, et al. Ensemble deep learning for regression and time series forecasting. Computational intelligence in ensemble learning, 2015.
17.
Ortiz A, Munilla J, Górriz JM, et al. Ensembles of deep learning architectures for the early diagnosis of the Alzheimer’s disease. Int J Neural Syst. 2016;26(7):1650025.PubMedCrossRef
18.
Boström H. Calibrating random forests. Seventh Int Conf Mach Learn Appl. 2008;2008:121–6.
19.
Platt J. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classifiers. 1999;10(3):61–74.
20.
21.
Zadrozny B, Elkan C. Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2002. p. 694–9.CrossRef
22.
Zadrozny B, Elkan C. Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers. ICML. 2001;1:609–16.
23.
Niculescu-Mizil A, Caruana R. Predicting good probabilities with supervised learning. Bonn: Association for Computing Machinery; 2005. p. 625–32.
24.
Wang Y, Li L, Dang C. Calibrating classification probabilities with shape-restricted polynomial regression. IEEE Trans Pattern Anal Mach Intell. 2019;41(8):1813–27.PubMedCrossRef
25.
Stone GW, Maehara A, Lansky AJ, et al. A prospective natural-history study of coronary atherosclerosis. N Engl J Med. 2011;364(3):226–35.PubMedCrossRef
26.
Peduzzi P, Concato J, Feinstein AR, et al. Importance of events per independent variable in proportional hazards regression analysis II. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995;48(12):1503–10.PubMedCrossRef
27.
He X, Chen Z, Fu T, et al. Ki-67 is a valuable prognostic predictor of lymphoma but its utility varies in lymphoma subtypes: evidence from a systematic meta-analysis. BMC Cancer. 2014;14(1):153.PubMedPubMedCentralCrossRef
28.
Song MK, Chung JS, Lee JJ, et al. High Ki-67 expression in involved bone marrow predicts worse clinical outcome in diffuse large B cell lymphoma patients treated with R-CHOP therapy. Int J Hematol. 2015;101(2):140–7.PubMedCrossRef
29.
Weigend A. On overfitting and the effective number of hidden units. Proc Connectionist Models Summer School. 1993;4(4):381–91.
30.
Caruana R, Lawrence S, Giles CL. Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In: Advances in neural information processing systems. 2000. p. 402–408.
31.
Lawrence S, Giles CL, Tsoi AC. Lessons in neural network training: overfitting may be harder than expected. In: National conference on artificial intelligence. 1997. p. 540–545.
32.
Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural Netw. 1989;2(5):359–66.CrossRef
33.
Zhou ZH. Neural networks. In: Maching learning. Beijing: Tsinghua University press; 2016. p. 97–120.
34.
Ayer M, Brunk HD, Ewing GM, et al. An empirical distribution function for sampling with incomplete information. Ann Math Stat. 1955;26(4):641–7.CrossRef
35.
36.
Alba AC, Agoritsas T, Walsh M, et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. 2017;318(14):1377–84.PubMedCrossRef
37.
Hosmer DW, Hosmer T, Le Cessie S, et al. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16(9):965–80.PubMedCrossRef
38.
Naeini MP, Cooper GF, Hauskrecht M. Binary classifier calibration using a Bayesian non-parametric approach. In: Proceedings of the 2015 SIAM International Conference on Data Mining; 2015. p. 208–16.
39.
Diamond S, Boyd S. CVXPY: a Python-embedded modeling language for convex optimization. J Mach Learn Res. 2016;17(1):2909–13.
40.
Agrawal A, Verschueren R, Diamond S, et al. A rewriting system for convex optimization problems. J Control Decis. 2018;5(1):42–60.CrossRef
41.
Coiffier B, Lepage E, Brière J, et al. CHOP chemotherapy plus rituximab compared with CHOP alone in elderly patients with diffuse large-B-cell lymphoma. N Engl J Med. 2002;346(4):235–42.PubMedCrossRef
42.
Pfreundschuh M, Trümper L, Osterborg A, et al. CHOP-like chemotherapy plus rituximab versus CHOP-like chemotherapy alone in young patients with good-prognosis diffuse large-B-cell lymphoma: a randomised controlled trial by the MabThera international trial (MInT) group. Lancet Oncol. 2006;7:379–91.PubMedCrossRef
43.
Coiffier B, Thieblemont C, Van Den Neste E, et al. Long-term outcome of patients in the LNH-98.5 trial, the first randomized study comparing rituximab-CHOP to standard CHOP chemotherapy in DLBCL patients : a study by the Groupe d'Etudes des Lymphomes de l'Adulte. Blood. 2010;116(12):2040–5.PubMedPubMedCentralCrossRef
44.
Fu K, Weisenburger DD, Choi WWL, et al. Addition of rituximab to standard chemotherapy improves the survival of both the germinal center B-cell-like and non-germinal center B-cell-like subtypes of diffuse large B-cell lymphoma. J Clin Oncol. 2008;26(28):4587–94.PubMedCrossRef
45.
Chinese Society of Hematology. Guidelines for the diagnosis and treatment of diffuse large B-cell lymphoma in China (2013 edition). Chin J Hematol. 2013;34(9):816–9.
46.
Zhang A, Ohshima K, Sato K, et al. Prognostic clinicopathologic factors, including immunologic expression in diffuse large B-cell lymphomas. Pathol Int. 2010;49(12):1043–52.CrossRef
47.
Zelenetz A, Gordon L, Abramson J, et al. NCCN clinical practice guidelines in oncology: B-cell lymphomas, Version 3.2019. J Natl Compr Canc Netw. 2019;17(6):650–61.PubMedCrossRef
Metadata
Title
Applying probability calibration to ensemble methods to predict 2-year mortality in patients with DLBCL
Authors
Shuanglong Fan
Zhiqiang Zhao
Hongmei Yu
Lei Wang
Chuchu Zheng
Xueqian Huang
Zhenhuan Yang
Meng Xing
Qing Lu
Yanhong Luo
Publication date
01-12-2021
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2021
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-020-01354-0

Other articles of this Issue 1/2021

BMC Medical Informatics and Decision Making 1/2021 Go to the issue

Research article

The prediction of asymptomatic carotid atherosclerosis with electronic health records: a comparative study of six machine learning models

Technical advance

ReportFlow: an application for EEG visualization and reporting using cloud platform

Research article

Factors affecting the mature use of electronic medical records by primary care physicians: a systematic review

Research article

Predicting cardiovascular health trajectories in time-series electronic health records with LSTM models

Research article

Record linkage under suboptimal conditions for data-intensive evaluation of primary care in Rio de Janeiro, Brazil

Research article

A method to determine a personalized set of online exercises for improving the positive mental health of a caregiver of a chronically ill patient