Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2015

Open Access 01-12-2015 | Research Article

The extension of total gain (TG) statistic in survival models: properties and applications

Authors: Babak Choodari-Oskooei, Patrick Royston, Mahesh K.B. Parmar

Published in: BMC Medical Research Methodology | Issue 1/2015

Login to get access

Abstract

Background

The results of multivariable regression models are usually summarized in the form of parameter estimates for the covariates, goodness-of-fit statistics, and the relevant p-values. These statistics do not inform us about whether covariate information will lead to any substantial improvement in prediction. Predictive ability measures can be used for this purpose since they provide important information about the practical significance of prognostic factors. R2-type indices are the most familiar forms of such measures in survival models, but they all have limitations and none is widely used.

Methods

In this paper, we extend the total gain (TG) measure, proposed for a logistic regression model, to survival models and explore its properties using simulations and real data. TG is based on the binary regression quantile plot, otherwise known as the predictiveness curve. Standardised TG ranges from 0 (no explanatory power) to 1 (‘perfect’ explanatory power).

Results

The results of our simulations show that unlike many of the other R2-type predictive ability measures, TG is independent of random censoring. It increases as the effect of a covariate increases and can be applied to different types of survival models, including models with time-dependent covariate effects. We also apply TG to quantify the predictive ability of multivariable prognostic models developed in several disease areas.

Conclusions

Overall, TG performs well in our simulation studies and can be recommended as a measure to quantify the predictive ability in survival models.
Appendix
Available only for authorised users
Literature
1.
Choodari-Oskooei B, Royston P, Parmar MKB. A simulation study of predictive ability measures in a survival model I: Explained variation measures. Stat Med. 2012; 31(23):2627–43.CrossRefPubMed
2.
Demaris A. Explained variance in logistic regression a Monte Carlo study of proposed measures. Sociol Methods Res. 2002; 77:329–42.
3.
Choodari-Oskooei B, Royston P, Parmar MKB. A simulation study of predictive ability measures in a survival model II: Explained randomness and predictive accuracy measures. Stat Med. 2012; 31(23):2644–59.CrossRefPubMed
4.
Spiegelhalter DJ. Probabilistic prediction in patient management and clinical trials. Stat Med. 1986; 5:421–33.CrossRefPubMed
5.
Nam BH, D’Agostino RB. Discrimination index, the area under the ROC curve. Goodness-of-fit tests and model validity. 2002; 1:267–280.CrossRef
6.
Steyerberg EW, Vickers AJ, Cook RN, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010; 21(1):128–138.CrossRefPubMedPubMedCentral
7.
Harrell F E, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA. 1982; 247(18):2543–6.CrossRefPubMed
8.
Copas J. The effectiveness of risk scores: the logit rank plot. Appl Stat. 1999; 48(2):165–83.
9.
Bura E, Gastwirth JL. The binary regression quantile plot: Assessing the importance of predictors in binary regression visually. Biometrical J. 2001; 43(1):5–21.CrossRef
10.
11.
Choodari-Oskooei B. Summarising Predictive Ability of a Survival Model and Applications in Medical Research. PhD thesis, University College London, 2008.
12.
Kent J, O’Quigley J. Measures of dependence for censored survival data. Biometrika. 1988; 75(3):525–34.CrossRef
13.
14.
Graf E, Schmoor C, Sauerbrei W, Schumacher M. Assessment and comparison of prognostic classification schemes for survival data. Stat Med. 1999; 18:2529–45.CrossRefPubMed
15.
Royston P, Sauerbrei W. A new measure of prognostic separation in survival data. Stat Med. 2004; 23:723–48.CrossRefPubMed
16.
Schmid M, Potapov S. A comparison of estimators to evaluate the discriminatory power of time-to-event models. Stat Med. 2012; 31(23):2588–609.CrossRefPubMed
17.
18.
Pepe MS, Feng Z, Huang Y, Longton G, Prentice R, Thompson IM, Zheng Y. Integrating the predictiveness of a marker with its performance as a classifier. Am J Epidemiol. 2008; 167:362–368.CrossRefPubMed
19.
Mittlbock M, Schemper M. Explained variation for logistic regression - small sample adjustments confidence intervals and predictive precision. Biometrical J. 2002; 44(3):263–72.CrossRef
20.
Sauerbrei W, Royston P. Building multivariable prognostic and diagnostic models: transformation of the predictors by using fractional polynomials. J R Stat Soc (Series A). 1999; 162:71–94. Corrigendum: J R Stat Soc. (Series A) 2002; 165:399–400.CrossRef
21.
Schumacher M, Bastert G, Bojar H, Hubner K, Olshewski M, Sauerbrei W, Schmoor C, Beyerle C, Newmann RLA, Rauschecker HF. Randomised 2*2 trial evaluating hormonal treatment and the duration of chemotherapy in node-positive breast cancer patients. J Clinical Oncol. 1994; 12:2086–2093.
22.
Burton A, Altman DG, Royston P, Holder RL. The design of simulation studies in medical statistics. Stat Med. 2006; 25:4279–92.CrossRefPubMed
23.
Fleishman AI. A method for simulating non-normal distributions. Psychometrika. 1978; 43:521–31.CrossRef
24.
Mok TS, Wu YL, Thongprasert S, Yang CH, Chu DT, Saijo N, Sunpaweravong P, Han B, Margono B, Ichinose Y, Nishiwak Y, Ohe Y, Yang J-J, Chewaskulyong B, Jiang H, Duffield EL, Watkins C L, Armour AA, Fukuoka M. Gefitinib or carboplatin?paclitaxel in pulmonary adenocarcinoma. N Engl J Med. 2009; 361:947–57.CrossRefPubMed
25.
Royston P, Lambert PC. Flexible Parametric Survival Analysis Using Stata: beyond the Cox model. Texas: Stata Press; 2011.
26.
Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006; 25(1):127–41.CrossRefPubMed
27.
Rosenwald A, Wright G, Chan WC, et al. The use of molecular profiling to predict survival after chemotherapy for diffuse large-b-cell lymphoma. N Engl J Med. 2002; 346:1937–47.CrossRefPubMed
28.
Fleming TR, Harrington DP. Counting processes and survival analysis. New York: Wiley and Sons; 1991.
29.
Ritchie R, Griffiths G, Parmar MKB. Interferon-alfa and survival in metastatic renal carcinoma: early results of a randomised controlled trial. Lancet. 1999; 353(9146):14–17.CrossRef
30.
Byar DP, Green SB. The choice of treatment for cancer patients based on covariate information - application to prostate cancer. Bulletin Du Cancer. 1980; 67(4):477–90.PubMed
31.
Royston P, Parmar MKB. Flexible parametric models for censored survival data with application to prognostic modelling and estimation of treatment effects. Stat Med. 2002; 21:2175–97.CrossRefPubMed
Metadata
Title
The extension of total gain (TG) statistic in survival models: properties and applications
Authors
Babak Choodari-Oskooei
Patrick Royston
Mahesh K.B. Parmar
Publication date
01-12-2015
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2015
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-015-0042-x

Other articles of this Issue 1/2015

BMC Medical Research Methodology 1/2015 Go to the issue

Research article

Moderate agreement between self-reported stroke and hospital-recorded stroke in two cohorts of Australian women: a validation study

Research article

Aiming for a representative sample: Simulating random versus purposive strategies for hospital selection

Research article

Development, inter-rater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider-based prevention and treatment programs targeting children and youth

Correspondence

Do personalised e-mail invitations increase the response rates of breast cancer survivors invited to participate in a web-based behaviour change intervention? A quasi-randomised 2-arm controlled trial

Research article

The impact of standardizing the definition of visits on the consistency of multi-database observational health research

Research article

Subgroup identification for treatment selection in biomarker adaptive design