Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Research article

Statistical evaluation of adding multiple risk factors improves Framingham stroke risk score

Authors: Xiao-Hua Zhou, Xiaonan Wang, Ashlee Duncan, Guizhou Hu, Jiayin Zheng

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

Framingham Stroke Risk Score (FSRS) is the most well-regarded risk appraisal tools for evaluating an individual’s absolute risk on stroke onset. However, several widely accepted risk factors for stroke were not included in the original Framingham model. This study proposed a new model which combines an existing risk models with new risk factors using synthesis analysis, and applied it to the longitudinal Atherosclerosis Risk in Communities (ARIC) data set.

Methods

Risk factors in original prediction models and new risk factors in proposed model had been discussed. Three measures, like discrimination, calibration and reclassification, were used to evaluate the performance of the original Framingham model and new risk prediction model.

Results

Modified C-statistics, Hosmer-Lemeshow Test and classless NRI, class NRI were the statistical indices which, respectively, denoted the performance of discrimination, calibration and reclassification for evaluating the newly developed risk prediction model on stroke onset. It showed that the NEW-STROKE (new stroke risk score prediction model) model had higher modified C-statistics, smaller Hosmer-Lemeshow chi-square values after recalibration than original FSRS model, and the classless NRI and class NRI of the NEW-STROKE model over the original FSRS model were all significantly positive in overall group.

Conclusion

The NEW-STROKE integrated with seven literature-derived risk factors outperformed the original FSRS model in predicting the risk score of stroke. It illustrated that seven literature-derived risk factors contributed significantly to stroke risk prediction.
Appendix
Available only for authorised users
Literature
1.
go back to reference Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ. Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet (London, England). 2006;367:1747–57.CrossRef Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ. Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet (London, England). 2006;367:1747–57.CrossRef
2.
go back to reference Kochanek KD, Murphy SL, Xu J, Arias E. Mortality in the United States, 2013. NCHS Data Brief. 2014;178:1–8. Kochanek KD, Murphy SL, Xu J, Arias E. Mortality in the United States, 2013. NCHS Data Brief. 2014;178:1–8.
3.
go back to reference Mozaffarian D, Benjamin EJ, Go AS, et al. Heart disease and stroke statistics--2015 update: a report from the American Heart Association. Circulation. 2015;131:229–322.CrossRef Mozaffarian D, Benjamin EJ, Go AS, et al. Heart disease and stroke statistics--2015 update: a report from the American Heart Association. Circulation. 2015;131:229–322.CrossRef
4.
go back to reference Naghavi M, Wang H, Lozano R, et al. Global, regional, and national age-sex specific all-cause and cause-specific mortality for 240 causes of death, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet (London, England). 2015;385:117–71.CrossRef Naghavi M, Wang H, Lozano R, et al. Global, regional, and national age-sex specific all-cause and cause-specific mortality for 240 causes of death, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet (London, England). 2015;385:117–71.CrossRef
6.
go back to reference Howard G, Anderson R, Sorlie P, Andrews V, Backlund E, Burke GL. Ethnic differences in stroke mortality between non-Hispanic whites, Hispanic whites, and blacks. The National Longitudinal Mortality Study. Stroke. 1994;25:2120–5.CrossRefPubMed Howard G, Anderson R, Sorlie P, Andrews V, Backlund E, Burke GL. Ethnic differences in stroke mortality between non-Hispanic whites, Hispanic whites, and blacks. The National Longitudinal Mortality Study. Stroke. 1994;25:2120–5.CrossRefPubMed
7.
go back to reference Manolio TA, Kronmal RA, Burke GL, O’Leary DH, Price TR. Short-term predictors of incident stroke in older adults. The Cardiovascular Health Study. Stroke. 1996;27:1479–86.CrossRefPubMed Manolio TA, Kronmal RA, Burke GL, O’Leary DH, Price TR. Short-term predictors of incident stroke in older adults. The Cardiovascular Health Study. Stroke. 1996;27:1479–86.CrossRefPubMed
8.
go back to reference Whisnant JP. Modeling of risk factors for ischemic stroke.The Willis Lecture. Stroke. 1997;28:1840–4.CrossRefPubMed Whisnant JP. Modeling of risk factors for ischemic stroke.The Willis Lecture. Stroke. 1997;28:1840–4.CrossRefPubMed
9.
go back to reference Berger K, Schulte H, Stogbauer F, Assmann G. Incidence and risk factors for stroke in an occupational cohort: the PROCAM Study. Prospective Cardiovascular Muenster Study. Stroke. 1998;29:1562–6.CrossRefPubMed Berger K, Schulte H, Stogbauer F, Assmann G. Incidence and risk factors for stroke in an occupational cohort: the PROCAM Study. Prospective Cardiovascular Muenster Study. Stroke. 1998;29:1562–6.CrossRefPubMed
10.
go back to reference Morgenstern LB, Smith MA, Lisabeth LD, et al. Excess stroke in Mexican Americans compared with non-Hispanic Whites: the Brain Attack Surveillance in Corpus Christi Project. Am J Epidemiol. 2004;160:376–83.CrossRefPubMedPubMedCentral Morgenstern LB, Smith MA, Lisabeth LD, et al. Excess stroke in Mexican Americans compared with non-Hispanic Whites: the Brain Attack Surveillance in Corpus Christi Project. Am J Epidemiol. 2004;160:376–83.CrossRefPubMedPubMedCentral
11.
go back to reference Harmsen P, Lappas G, Rosengren A, Wilhelmsen L. Long-term risk factors for stroke: twenty-eight years of follow-up of 7457 middle-aged men in Goteborg, Sweden. Stroke. 2006;37:1663–7.CrossRefPubMed Harmsen P, Lappas G, Rosengren A, Wilhelmsen L. Long-term risk factors for stroke: twenty-eight years of follow-up of 7457 middle-aged men in Goteborg, Sweden. Stroke. 2006;37:1663–7.CrossRefPubMed
12.
go back to reference O’Donnell MJ, Xavier D, Liu L, et al. Risk factors for ischaemic and intracerebral haemorrhagic stroke in 22 countries (the INTERSTROKE study): a case–control study. Lancet (London, England). 2010;376:112–23.CrossRef O’Donnell MJ, Xavier D, Liu L, et al. Risk factors for ischaemic and intracerebral haemorrhagic stroke in 22 countries (the INTERSTROKE study): a case–control study. Lancet (London, England). 2010;376:112–23.CrossRef
14.
go back to reference Anderson KM, Odell PM, Wilson PWF, Kannel WB. Cardiovascular disease risk profiles. Am Heart J. 1990;121:293–8.CrossRef Anderson KM, Odell PM, Wilson PWF, Kannel WB. Cardiovascular disease risk profiles. Am Heart J. 1990;121:293–8.CrossRef
15.
go back to reference Wolf PA, D’Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: A risk profile from the Framingham study. Stroke. 1991;22:312–8.CrossRefPubMed Wolf PA, D’Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: A risk profile from the Framingham study. Stroke. 1991;22:312–8.CrossRefPubMed
16.
go back to reference D’Agostino RB, Wolf PA, Belanger AJ, Kannel WB. Stroke risk profile: adjustment for antihypertensive medication. Stroke. 1994;25:40–3.CrossRefPubMed D’Agostino RB, Wolf PA, Belanger AJ, Kannel WB. Stroke risk profile: adjustment for antihypertensive medication. Stroke. 1994;25:40–3.CrossRefPubMed
17.
go back to reference Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007;335:136.CrossRefPubMedPubMedCentral Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007;335:136.CrossRefPubMedPubMedCentral
18.
go back to reference Ridker PM, Buring JE, Rifai N, Cook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA. 2007;297:611–9.CrossRefPubMed Ridker PM, Buring JE, Rifai N, Cook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA. 2007;297:611–9.CrossRefPubMed
19.
go back to reference Jee SH, Park JW, Lee SY, et al. Stroke risk prediction model: a risk profile from the Korean study. Atherosclerosis. 2008;197:318–25.CrossRefPubMed Jee SH, Park JW, Lee SY, et al. Stroke risk prediction model: a risk profile from the Korean study. Atherosclerosis. 2008;197:318–25.CrossRefPubMed
20.
go back to reference Hippisley-Cox J, Coupland C, Brindle P. Derivation and validation of QStroke score for predicting risk of ischaemic stroke in primary care and comparison with other risk scores: a prospective open cohort study. BMJ. 2013;346:f2573.CrossRefPubMedPubMedCentral Hippisley-Cox J, Coupland C, Brindle P. Derivation and validation of QStroke score for predicting risk of ischaemic stroke in primary care and comparison with other risk scores: a prospective open cohort study. BMJ. 2013;346:f2573.CrossRefPubMedPubMedCentral
21.
go back to reference Wilson PWF, D’Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.CrossRefPubMed Wilson PWF, D’Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.CrossRefPubMed
22.
go back to reference Kurth T, Gaziano JM, Berger K, et al. Body mass index and the risk of stroke in men. Arch Intern Med. 2002;162:2557–62.CrossRefPubMed Kurth T, Gaziano JM, Berger K, et al. Body mass index and the risk of stroke in men. Arch Intern Med. 2002;162:2557–62.CrossRefPubMed
23.
go back to reference Kurth T, Gaziano JM, Rexrode KM, et al. Prospective study of body mass index and risk of stroke in apparently healthy women. Circulation. 2005;111:1992–8.CrossRefPubMed Kurth T, Gaziano JM, Rexrode KM, et al. Prospective study of body mass index and risk of stroke in apparently healthy women. Circulation. 2005;111:1992–8.CrossRefPubMed
24.
go back to reference Lu M, Ye W, Adami HO, Weiderpass E. Prospective study of body size and risk for stroke amongst women below age 60. J Intern Med. 2006;260:442–50.CrossRefPubMed Lu M, Ye W, Adami HO, Weiderpass E. Prospective study of body size and risk for stroke amongst women below age 60. J Intern Med. 2006;260:442–50.CrossRefPubMed
25.
go back to reference Hu G, Tuomilehto J, Silventoinen K, Sarti C, Mannisto S, Jousilahti P. Body mass index, waist circumference, and waist-hip ratio on the risk of total and type-specific stroke. Arch Intern Med. 2007;167:1420–7.CrossRefPubMed Hu G, Tuomilehto J, Silventoinen K, Sarti C, Mannisto S, Jousilahti P. Body mass index, waist circumference, and waist-hip ratio on the risk of total and type-specific stroke. Arch Intern Med. 2007;167:1420–7.CrossRefPubMed
26.
go back to reference Park JW, Lee SY, Kim SY, Choe H, Jee SH. BMI and stroke risk in Korean women. Obesity (Silver Spring). 2008;16:396–401.CrossRef Park JW, Lee SY, Kim SY, Choe H, Jee SH. BMI and stroke risk in Korean women. Obesity (Silver Spring). 2008;16:396–401.CrossRef
27.
go back to reference Khaw KT, Barrett-Connor E. Family history of stroke as an independent predictor of ischemic heart disease in men and stroke in women. Am J Epidemiol. 1986;123:59–66.CrossRefPubMed Khaw KT, Barrett-Connor E. Family history of stroke as an independent predictor of ischemic heart disease in men and stroke in women. Am J Epidemiol. 1986;123:59–66.CrossRefPubMed
28.
go back to reference Kiely DK, Wolf PA, Cupples LA, Beiser AS, Myers RH. Familial aggregation of stroke.The Framingham Study. Stroke. 1993;24:1366–71.CrossRefPubMed Kiely DK, Wolf PA, Cupples LA, Beiser AS, Myers RH. Familial aggregation of stroke.The Framingham Study. Stroke. 1993;24:1366–71.CrossRefPubMed
29.
go back to reference Flossmann E, Schulz UG, Rothwell PM. Systematic review of methods and results of studies of the genetic epidemiology of ischemic stroke. Stroke. 2004;35:212–27.CrossRefPubMed Flossmann E, Schulz UG, Rothwell PM. Systematic review of methods and results of studies of the genetic epidemiology of ischemic stroke. Stroke. 2004;35:212–27.CrossRefPubMed
30.
go back to reference Touze E, Rothwell PM. Sex differences in heritability of ischemic stroke: a systematic review and meta-analysis. Stroke. 2008;39:16–23.CrossRefPubMed Touze E, Rothwell PM. Sex differences in heritability of ischemic stroke: a systematic review and meta-analysis. Stroke. 2008;39:16–23.CrossRefPubMed
31.
go back to reference Hu G, Root M. Building prediction models for coronary heart disease by synthesizing multiple longitudinal research findings. Eur J Cardiovasc Prev Rehabil. 2005;12:459–64.CrossRefPubMed Hu G, Root M. Building prediction models for coronary heart disease by synthesizing multiple longitudinal research findings. Eur J Cardiovasc Prev Rehabil. 2005;12:459–64.CrossRefPubMed
33.
go back to reference Expert Panel on Detection E, and Treatment of High Blood Cholesterol in Adults. Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, And Treatment of High Blood Cholesterol In Adults (Adult Treatment Panel III). JAMA. 2001;285:2486–97.CrossRef Expert Panel on Detection E, and Treatment of High Blood Cholesterol in Adults. Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, And Treatment of High Blood Cholesterol In Adults (Adult Treatment Panel III). JAMA. 2001;285:2486–97.CrossRef
34.
go back to reference Kannel WB, D’Agostino RB, Silbershatz H, Belanger AJ, Wilson PW, Levy D. Profile for estimating risk of heart failure. Arch Intern Med. 1999;159:1197–204.CrossRefPubMed Kannel WB, D’Agostino RB, Silbershatz H, Belanger AJ, Wilson PW, Levy D. Profile for estimating risk of heart failure. Arch Intern Med. 1999;159:1197–204.CrossRefPubMed
35.
go back to reference Parikh NI, Pencina MJ, Wang TJ, et al. A risk score for predicting near-term incidence of hypertension: the Framingham Heart Study. Ann Intern Med. 2008;148:102–10.CrossRefPubMed Parikh NI, Pencina MJ, Wang TJ, et al. A risk score for predicting near-term incidence of hypertension: the Framingham Heart Study. Ann Intern Med. 2008;148:102–10.CrossRefPubMed
36.
go back to reference Schnabel RB, Sullivan LM, Levy D, et al. Development of a risk score for atrial fibrillation (Framingham Heart Study): a community-based cohort study. Lancet (London, England). 2009;373:739–45.CrossRef Schnabel RB, Sullivan LM, Levy D, et al. Development of a risk score for atrial fibrillation (Framingham Heart Study): a community-based cohort study. Lancet (London, England). 2009;373:739–45.CrossRef
37.
go back to reference Wilson PW, Meigs JB, Sullivan L, Fox CS, Nathan DM, D’Agostino Sr RB. Prediction of incident diabetes mellitus in middle-aged adults: the Framingham Offspring Study. Arch Intern Med. 2007;167:1068–74.CrossRefPubMed Wilson PW, Meigs JB, Sullivan L, Fox CS, Nathan DM, D’Agostino Sr RB. Prediction of incident diabetes mellitus in middle-aged adults: the Framingham Offspring Study. Arch Intern Med. 2007;167:1068–74.CrossRefPubMed
38.
go back to reference Hu G, Root M, Duncan AW. Adding multiple risk factors improves Framingham coronary heart disease risk scores. Vasc Health Risk Manag. 2014;10:557–62.PubMedPubMedCentral Hu G, Root M, Duncan AW. Adding multiple risk factors improves Framingham coronary heart disease risk scores. Vasc Health Risk Manag. 2014;10:557–62.PubMedPubMedCentral
39.
go back to reference Sheng E, Zhou XH, Chen H, Hu G, Duncan A. A new synthesis analysis method for building logistic regression prediction models. Stat Med. 2014;33:2567–76.CrossRefPubMed Sheng E, Zhou XH, Chen H, Hu G, Duncan A. A new synthesis analysis method for building logistic regression prediction models. Stat Med. 2014;33:2567–76.CrossRefPubMed
40.
go back to reference Samsa G, Hu G, Root M. Combining information from multiple data sources to create multivariable risk models: Illustration and preliminary assessment of a new method. J Biomed Biotechnol. 2005;2:113–23.CrossRef Samsa G, Hu G, Root M. Combining information from multiple data sources to create multivariable risk models: Illustration and preliminary assessment of a new method. J Biomed Biotechnol. 2005;2:113–23.CrossRef
41.
go back to reference Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA. 1982;247:2543–46.CrossRefPubMed Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA. 1982;247:2543–46.CrossRefPubMed
42.
go back to reference Harrell FE, Lee KL, Califf RM, Pryor DB, Lee KL, Rosati RA. Regression modeling strategies for improved prognostic prediction. Stat Med. 1984;3:143–52.CrossRefPubMed Harrell FE, Lee KL, Califf RM, Pryor DB, Lee KL, Rosati RA. Regression modeling strategies for improved prognostic prediction. Stat Med. 1984;3:143–52.CrossRefPubMed
43.
go back to reference Hajime U, Tianxi C, Pencima MJ, D’Agostino RB, Wei LJ. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med. 2011;30:1105–17. Hajime U, Tianxi C, Pencima MJ, D’Agostino RB, Wei LJ. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med. 2011;30:1105–17.
44.
go back to reference Harrell Jr FE, Lee KL, Mark DB. Tutorial in biostatistics multivariable prognostic models:issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.CrossRef Harrell Jr FE, Lee KL, Mark DB. Tutorial in biostatistics multivariable prognostic models:issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.CrossRef
45.
go back to reference Pencina MJ, D’Agostino Sr RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30:11–21.CrossRefPubMed Pencina MJ, D’Agostino Sr RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30:11–21.CrossRefPubMed
46.
go back to reference The ARIC investigators. The Atherosclerosis Risk in Communities(ARIC) Study: design and objectives. Am J Epidemiol. 1989;129:687–702.CrossRef The ARIC investigators. The Atherosclerosis Risk in Communities(ARIC) Study: design and objectives. Am J Epidemiol. 1989;129:687–702.CrossRef
47.
go back to reference Rosamond WD, Folsom AR, Chambless LE, Wang CH, McGovern PG, Howard G, Copper LS, Shahar E. Stroke incidence and Survival Among Middle-Aged Adults 9-Year Follow-Up of the Atherosclerosis Risk in Communities(ARIC) cohort. Stroke. 1999;30:736–43.CrossRefPubMed Rosamond WD, Folsom AR, Chambless LE, Wang CH, McGovern PG, Howard G, Copper LS, Shahar E. Stroke incidence and Survival Among Middle-Aged Adults 9-Year Follow-Up of the Atherosclerosis Risk in Communities(ARIC) cohort. Stroke. 1999;30:736–43.CrossRefPubMed
48.
go back to reference Tzoulaki I, Liberopoulos G, Ioannidis JP. Assessment of claims of improved prediction beyond the Framingham risk score. J Am Med Assoc. 2009;302:2345–52.CrossRef Tzoulaki I, Liberopoulos G, Ioannidis JP. Assessment of claims of improved prediction beyond the Framingham risk score. J Am Med Assoc. 2009;302:2345–52.CrossRef
Metadata
Title
Statistical evaluation of adding multiple risk factors improves Framingham stroke risk score
Authors
Xiao-Hua Zhou
Xiaonan Wang
Ashlee Duncan
Guizhou Hu
Jiayin Zheng
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-017-0330-8

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue