Skip to main content
Top
Published in: European Journal of Epidemiology 1/2013

01-01-2013 | COMMENTARY

Net reclassification improvement: a link between statistics and clinical practice

Authors: Maarten J. G. Leening, Nancy R. Cook

Published in: European Journal of Epidemiology | Issue 1/2013

Login to get access

Excerpt

A little over a decade ago, European [1] and US guidelines [2] on cardiovascular disease prevention first recommended the use of various global risk assessment models. These guidelines directly aid clinicians in making decisions on consultation of a healthy lifestyle and initiation of drug treatment. The recommendations to use risk scoring algorithms invigorated researchers to improve on these existing functions and thereby heralded the current upswing in risk prediction research. Ever since a plethora of additional risk factors have been proposed, ranging from simple questions on familial predisposition and laboratory measures, to state-of-the-art vascular imaging. In order to provide guidance the American Heart Association issued a comprehensive statement on the stepwise evaluation of the value of novel markers [3]. The expert panel suggested that a new marker should be prospectively associated with the outcome, it should add predictive information over established risk factors, the addition of the marker should have the potential to modify an individual’s risk sufficiently to change treatment recommendations, and finally whether this improves clinical outcomes in a cost-effective manner. …
Literature
1.
go back to reference De Backer G, Ambrosioni E, Borch-Johnsen K, Brotons C, Cifkova R, Dallongeville J, et al. European guidelines on cardiovascular disease prevention in clinical practice. Third joint task force of european and other societies on cardiovascular disease prevention in clinical practice. Eur Heart J. 2003;24(17):1601–10.PubMedCrossRef De Backer G, Ambrosioni E, Borch-Johnsen K, Brotons C, Cifkova R, Dallongeville J, et al. European guidelines on cardiovascular disease prevention in clinical practice. Third joint task force of european and other societies on cardiovascular disease prevention in clinical practice. Eur Heart J. 2003;24(17):1601–10.PubMedCrossRef
2.
go back to reference Executive summary of the third report of the national cholesterol education program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III). JAMA. 2001;285(19):2486–97. Executive summary of the third report of the national cholesterol education program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III). JAMA. 2001;285(19):2486–97.
3.
4.
go back to reference Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.PubMed Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.PubMed
5.
go back to reference Harrell FE Jr, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA. 1982;247(18):2543–6.PubMedCrossRef Harrell FE Jr, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA. 1982;247(18):2543–6.PubMedCrossRef
6.
go back to reference Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol. 2004;159(9):882–90.PubMedCrossRef Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol. 2004;159(9):882–90.PubMedCrossRef
7.
9.
go back to reference Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72. doi:10.1002/sim.2929.PubMedCrossRef Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72. doi:10.​1002/​sim.​2929.PubMedCrossRef
10.
go back to reference Muhlenbruch K, Heraclides A, Steyerberg EW, Joost HG, Boeing H, Schulze MB. Assessing improvement in disease prediction using net reclassification improvement: impact of risk cut-offs and number of risk categories. Eur J Epidemiol. 2012. doi:10.1007/s10654-012-9744-0.PubMed Muhlenbruch K, Heraclides A, Steyerberg EW, Joost HG, Boeing H, Schulze MB. Assessing improvement in disease prediction using net reclassification improvement: impact of risk cut-offs and number of risk categories. Eur J Epidemiol. 2012. doi:10.​1007/​s10654-012-9744-0.PubMed
11.
go back to reference Mealiffe ME, Stokowski RP, Rhees BK, Prentice RL, Pettinger M, Hinds DA. Assessment of clinical validity of a breast cancer risk model combining genetic and clinical information. J Natl Cancer Inst. 2010;102(21):1618–27. doi:10.1093/jnci/djq388.PubMedCrossRef Mealiffe ME, Stokowski RP, Rhees BK, Prentice RL, Pettinger M, Hinds DA. Assessment of clinical validity of a breast cancer risk model combining genetic and clinical information. J Natl Cancer Inst. 2010;102(21):1618–27. doi:10.​1093/​jnci/​djq388.PubMedCrossRef
12.
go back to reference Mihaescu R, van Zitteren M, van Hoek M, Sijbrands EJ, Uitterlinden AG, Witteman JC, et al. Improvement of risk prediction by genomic profiling: reclassification measures versus the area under the receiver operating characteristic curve. Am J Epidemiol. 2010;172(3):353–61. doi:10.1093/aje/kwq122.PubMedCrossRef Mihaescu R, van Zitteren M, van Hoek M, Sijbrands EJ, Uitterlinden AG, Witteman JC, et al. Improvement of risk prediction by genomic profiling: reclassification measures versus the area under the receiver operating characteristic curve. Am J Epidemiol. 2010;172(3):353–61. doi:10.​1093/​aje/​kwq122.PubMedCrossRef
13.
go back to reference Cook NR, Paynter NP. Comments on ‘Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers’ by M. J. Pencina, R. B. D’Agostino, Sr. and E. W. Steyerberg. Stat Med. 2012;31(1):93–5. doi:10.1002/sim.4209.PubMedCrossRef Cook NR, Paynter NP. Comments on ‘Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers’ by M. J. Pencina, R. B. D’Agostino, Sr. and E. W. Steyerberg. Stat Med. 2012;31(1):93–5. doi:10.​1002/​sim.​4209.PubMedCrossRef
14.
go back to reference Pencina MJ, D’Agostino RB Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21. doi:10.1002/sim.4085.PubMedCrossRef Pencina MJ, D’Agostino RB Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21. doi:10.​1002/​sim.​4085.PubMedCrossRef
15.
go back to reference Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150(11):795–802.PubMed Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150(11):795–802.PubMed
16.
go back to reference Cook NR. Clinically relevant measures of fit? A note of caution. Am J Epidemiol. 2012;176(6):488–91.PubMedCrossRef Cook NR. Clinically relevant measures of fit? A note of caution. Am J Epidemiol. 2012;176(6):488–91.PubMedCrossRef
17.
go back to reference Hosmer DW, Lemeshow S. Goodness of fit tests for the multiple logistic regression model. Commun Stat-Theory Methods. 1980;9(10):1043–69.CrossRef Hosmer DW, Lemeshow S. Goodness of fit tests for the multiple logistic regression model. Commun Stat-Theory Methods. 1980;9(10):1043–69.CrossRef
22.
go back to reference Steyerberg EW. Clinical prediction models: A practical approach to development, validation, and updating. New York: Springer; 2009. Steyerberg EW. Clinical prediction models: A practical approach to development, validation, and updating. New York: Springer; 2009.
23.
go back to reference Yeboah J, McClelland RL, Polonsky TS, Burke GL, Sibley CT, O’Leary D, et al. Comparison of novel risk markers for improvement in cardiovascular risk assessment in intermediate-risk individuals. JAMA. 2012;308(8):788–95. doi:10.1001/jama.2012.9624.PubMedCrossRef Yeboah J, McClelland RL, Polonsky TS, Burke GL, Sibley CT, O’Leary D, et al. Comparison of novel risk markers for improvement in cardiovascular risk assessment in intermediate-risk individuals. JAMA. 2012;308(8):788–95. doi:10.​1001/​jama.​2012.​9624.PubMedCrossRef
24.
go back to reference Cook NR. Comments on ‘Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929). Stat Med. 2008;27(2):191–5. doi:10.1002/sim.2987.PubMedCrossRef Cook NR. Comments on ‘Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929). Stat Med. 2008;27(2):191–5. doi:10.​1002/​sim.​2987.PubMedCrossRef
27.
go back to reference Visvanathan K, Chlebowski RT, Hurley P, Col NF, Ropka M, Collyar D, et al. American society of clinical oncology clinical practice guideline update on the use of pharmacologic interventions including tamoxifen, raloxifene, and aromatase inhibition for breast cancer risk reduction. J Clin Oncol. 2009;27(19):3235–58. doi:10.1200/JCO.2008.20.5179.PubMedCrossRef Visvanathan K, Chlebowski RT, Hurley P, Col NF, Ropka M, Collyar D, et al. American society of clinical oncology clinical practice guideline update on the use of pharmacologic interventions including tamoxifen, raloxifene, and aromatase inhibition for breast cancer risk reduction. J Clin Oncol. 2009;27(19):3235–58. doi:10.​1200/​JCO.​2008.​20.​5179.PubMedCrossRef
28.
go back to reference Munshi NC, Anderson KC, Bergsagel PL, Shaughnessy J, Palumbo A, Durie B, et al. Consensus recommendations for risk stratification in multiple myeloma: report of the International Myeloma Workshop Consensus Panel 2. Blood. 2011;117(18):4696–700. doi:10.1182/blood-2010-10-300970.PubMedCrossRef Munshi NC, Anderson KC, Bergsagel PL, Shaughnessy J, Palumbo A, Durie B, et al. Consensus recommendations for risk stratification in multiple myeloma: report of the International Myeloma Workshop Consensus Panel 2. Blood. 2011;117(18):4696–700. doi:10.​1182/​blood-2010-10-300970.PubMedCrossRef
29.
go back to reference Worth LJ, Worth LJ, Lingaratnam S, Taylor A, Hayward AM, Morrissey S, Cooney J, et al. Use of risk stratification to guide ambulatory management of neutropenic fever. Australian Consensus Guidelines 2011 Steering Committee. Intern Med J. 2011;41(1b):82–9. doi:10.1111/j.1445-5994.2010.02339.x.PubMedCrossRef Worth LJ, Worth LJ, Lingaratnam S, Taylor A, Hayward AM, Morrissey S, Cooney J, et al. Use of risk stratification to guide ambulatory management of neutropenic fever. Australian Consensus Guidelines 2011 Steering Committee. Intern Med J. 2011;41(1b):82–9. doi:10.​1111/​j.​1445-5994.​2010.​02339.​x.PubMedCrossRef
Metadata
Title
Net reclassification improvement: a link between statistics and clinical practice
Authors
Maarten J. G. Leening
Nancy R. Cook
Publication date
01-01-2013
Publisher
Springer Netherlands
Published in
European Journal of Epidemiology / Issue 1/2013
Print ISSN: 0393-2990
Electronic ISSN: 1573-7284
DOI
https://doi.org/10.1007/s10654-012-9759-6

Other articles of this Issue 1/2013

European Journal of Epidemiology 1/2013 Go to the issue