Top

Irish Journal of Medical Science (1971 -)

Published in:

01-08-2018 | Review Article

Progress of statistical analysis in biomedical research through the historical review of the development of the Framingham score

Authors: Aleksandra Ignjatović, Miodrag Stojanović, Zoran Milošević, Marija Anđelković Apostolović

Published in: Irish Journal of Medical Science (1971 -) | Issue 3/2018

Abstract

Background

The interest in developing risk models in medicine not only is appealing, but also associated with many obstacles in different aspects of predictive model development. Initially, the association of biomarkers or the association of more markers with the specific outcome was proven by statistical significance, but novel and demanding questions required the development of new and more complex statistical techniques.

Methods

Progress of statistical analysis in biomedical research can be observed the best through the history of the Framingham study and development of the Framingham score.

Results

Evaluation of predictive models comes from a combination of the facts which are results of several metrics. Using logistic regression and Cox proportional hazards regression analysis, the calibration test, and the ROC curve analysis should be mandatory and eliminatory, and the central place should be taken by some new statistical techniques. In order to obtain complete information related to the new marker in the model, recently, there is a recommendation to use the reclassification tables by calculating the net reclassification index and the integrated discrimination improvement. Decision curve analysis is a novel method for evaluating the clinical usefulness of a predictive model. It may be noted that customizing and fine-tuning of the Framingham risk score initiated the development of statistical analysis.

Conclusion

Clinically applicable predictive model should be a trade-off between all abovementioned statistical metrics, a trade-off between calibration and discrimination, accuracy and decision-making, costs and benefits, and quality and quantity of patient’s life.

Kannel WB, Dawber TR, Kagan A, Revotskie N, Stokes J, Iii (1961) Factors of risk in the development of coronary heart disease—six-year follow-up experience: the Framingham study. Ann Intern Med 55 (1):33–50. https://doi.org/10.7326/0003-4819-55-1-33

Dawber TR, Meadors GF, Moore FE (1951) Epidemiological approaches to heart disease: the Framingham study. Am J Public Health Nations Health 41(3):279–286. https://doi.org/10.2105/AJPH.41.3.279 CrossRefPubMedPubMedCentral

Kannel WB, McGee D, Gordon T (1976) A general cardiovascular risk profile: the Framingham study. Am J Cardiol 38(1):46–51. https://doi.org/10.1016/0002-9149(76)90061-8 CrossRefPubMed

Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, Mulvihill JJ (1989) Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. JNCI: J Natl Cancer Inst 81(24):1879–1886. https://doi.org/10.1093/jnci/81.24.1879 CrossRefPubMed

Ridker PM, Rifai N, Rose L, Buring JE, Cook NR (2002) Comparison of C-reactive protein and low-density lipoprotein cholesterol levels in the prediction of first cardiovascular events. N Engl J Med 347(20):1557–1565. https://doi.org/10.1056/NEJMoa021993 CrossRefPubMed

Koenig W, Lowel H, Baumert J, Meisinger C (2004) C-reactive protein modulates risk prediction based on the Framingham score: implications for future risk assessment: results from a large cohort study in southern Germany. Circulation 109(11):1349–1353. https://doi.org/10.1161/01.CIR.0000120707.98922.E3 CrossRefPubMed

Grover SA (1995) Identifying adults at increased risk of coronary disease. How well do the current cholesterol guidelines work? JAMA 274(10):801–806. https://doi.org/10.1001/jama.274.10.801 CrossRefPubMed

Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, Mulvihill JJ (1989) Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst 81(24):1879–1886. https://doi.org/10.1093/jnci/81.24.1879 CrossRefPubMed

Kannel WB, Feinleib M, McNamara PM, Garrison RJ, Castelli WP (1979) An investigation of coronary heart disease in families. The Framingham offspring study. Am J Epidemiol 110(3):281–290. https://doi.org/10.1093/oxfordjournals.aje.a112813 CrossRefPubMed

10.

Wilson PW, D’Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB (1998) Prediction of coronary heart disease using risk factor categories. Circulation 97(18):1837–1847. https://doi.org/10.1161/01.CIR.97.18.1837 CrossRefPubMed

11.

Anderson KM, Wilson P, Odell PM, Kannel WB (1991) An updated coronary risk profile. A statement for health professionals. Circulation 83(1):356–362. https://doi.org/10.1161/01.CIR.83.1.356 CrossRefPubMed

12.

Wolf PA, D'agostino RB, Belanger AJ, Kannel WB (1991) Probability of stroke: a risk profile from the Framingham study. Stroke 22(3):312–318. https://doi.org/10.1161/01.STR.22.3.312 CrossRefPubMed

13.

Murabito JM, D'agostino RB, Silbershatz H (1998) Intermittent claudication: a risk profile from the Framingham heart study. J Vasc Interv Radiol 9(1):172

14.

Kannel WB, D'agostino RB, Silbershatz H, Belanger AJ, Wilson PW, Levy D (1999) Profile for estimating risk of heart failure. Arch Intern Med 159(11):1197–1204. https://doi.org/10.1001/archinte.159.11.1197 CrossRefPubMed

15.

D’Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, Kannel WB (2008) General cardiovascular risk profile for use in primary care. Circulation 117(6):743–753. https://doi.org/10.1161/CIRCULATIONAHA.107.699579 CrossRefPubMed

16.

Zebrack JS, Anderson JL, Maycock CA, Horne BD, Bair TL, Muhlestein JB, Intermountain Heart Collaborative Study G (2002) Usefulness of high-sensitivity C-reactive protein in predicting long-term risk of death or acute myocardial infarction in patients with unstable or stable angina pectoris or acute myocardial infarction. Am J Cardiol 89(2):145–149CrossRefPubMed

17.

Veeranna V, Zalawadiya SK, Niraj A, Pradhan J, Ference B, Burack RC, Jacob S, Afonso L (2011) Homocysteine and reclassification of cardiovascular disease risk. J Am Coll Cardiol 58(10):1025–1033. https://doi.org/10.1016/j.jacc.2011.05.028 CrossRefPubMed

18.

Meisinger C, Loewel H, Mraz W, Koenig W (2005) Prognostic value of apolipoprotein B and A-I in the prediction of myocardial infarction in middle-aged men and women: results from the MONICA/KORA Augsburg cohort study. Eur Heart J 26(3):271–278. https://doi.org/10.1093/eurheartj/ehi003 CrossRefPubMed

19.

Kotur-Stevuljevic J, Memon L, Stefanovic A, Spasic S, Spasojevic-Kalimanovska V, Bogavac-Stanojevic N, Kalimanovska-Ostric D, Jelic-Ivanovic Z, Zunic G (2007) Correlation of oxidative stress parameters and inflammatory markers in coronary artery disease patients. Clin Biochem 40(3–4):181–187. https://doi.org/10.1016/j.clinbiochem.2006.09.007 CrossRefPubMed

20.

Brand RJ, Rosenman RH, Sholtz RI, Friedman M (1976) Multivariate prediction of coronary heart disease in the Western Collaborative Group Study compared to the findings of the Framingham study. Circulation 53(2):348–355. https://doi.org/10.1161/01.CIR.53.2.348 CrossRefPubMed

21.

Gordon T, Castelli WP, Hjortland MC, Kannel WB, Dawber TR (1977) Predicting coronary heart disease in middle-aged and older persons. The Framington study. JAMA 238(6):497–499. https://doi.org/10.1001/jama.1977.03280060041018 CrossRefPubMed

22.

Pepe MS (2004) Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 159(9):882–890. https://doi.org/10.1093/aje/kwh101 CrossRefPubMed

23.

Harrell FE Jr, Lee KL, Mark DB (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15(4):361–387. https://doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4 CrossRefPubMed

24.

D'Agostino RB Sr, Pencina MJ, Massaro JM, Coady S (2013) Cardiovascular disease risk assessment: insights from Framingham. Glob Heart 8(1):11–23. https://doi.org/10.1016/j.gheart.2013.01.001 CrossRefPubMed

25.

Gronnesby JK, Borgan O (1996) A method for checking regression models in survival analysis based on the risk score. Lifetime Data Anal 2(4):315–328. https://doi.org/10.1007/BF00127305 CrossRefPubMed

26.

Demler OV, Paynter NP, Cook NR (2015) Tests of calibration and goodness-of-fit in the survival setting. Stat Med 34(10):1659–1680. https://doi.org/10.1002/sim.6428 CrossRefPubMedPubMedCentral

27.

Parzen M, Lipsitz SR (1999) A global goodness-of-fit statistic for Cox regression models. Biometrics 55(2):580–584. https://doi.org/10.1111/j.0006-341X.1999.00580.x CrossRefPubMed

28.

Antolini L, Nam B-H, D'Agostino RB (2004) Inference on correlated discrimination measures in survival analysis: a nonparametric approach. Commun Stat Theory Methods 33(9):2117–2135. https://doi.org/10.1081/STA-200026579 CrossRef

29.

DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics:837–845

30.

Cook NR (2008) Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem 54(1):17–23. https://doi.org/10.1373/clinchem.2007.096529 CrossRefPubMed

31.

Zweig MH, Campbell G (1993) Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem 39(4):561–577PubMed

32.

Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS (2008) Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 27(2):157–172; discussion 207-112. https://doi.org/10.1002/sim.2929 CrossRefPubMed

33.

Pencina MJ, D'Agostino RB Sr, Demler OV (2012) Novel metrics for evaluating improvement in discrimination: net reclassification and integrated discrimination improvement for normal variables and nested models. Stat Med 31(2):101–113. https://doi.org/10.1002/sim.4348 CrossRefPubMed

34.

Pepe MS, Feng Z, Gu JW (2008) Comments on ‘Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., statistics in medicine. Stat Med 27(2):173–181. https://doi.org/10.1002/sim.2991 CrossRefPubMed

35.

Steyerberg EW, Van Calster B, Pencina MJ (2011) Performance measures for prediction models and markers: evaluation of predictions and classifications. Rev Esp Cardiol 64(9):788–794. https://doi.org/10.1016/j.recesp.2011.04.017 CrossRefPubMed

36.

Whittemore AS (2010) Evaluating health risk models. Stat Med 29(23):2438–2452. https://doi.org/10.1002/sim.3991 PubMedPubMedCentralCrossRef

37.

Chambless LE, Cummiskey CP, Cui G (2011) Several methods to assess improvement in risk prediction models: extension to survival analysis. Stat Med 30(1):22–38. https://doi.org/10.1002/sim.4026 CrossRefPubMed

38.

D'Agostino RB, Sr., Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, Kannel WB (2008) General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 117 (6):743–753. https://doi.org/10.1161/circulationaha.107.699579

39.

Pencina MJ, D'Agostino RB, Sr., Steyerberg EW (2011) Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med 30 (1):11–21. https://doi.org/10.1002/sim.4085

40.

Polak JF, Pencina MJ, Pencina KM, O’donnell CJ, Wolf PA, D’Agostino RB Sr (2011) Carotid-wall intima–media thickness and cardiovascular events. N Engl J Med 365(3):213–221. https://doi.org/10.1056/NEJMoa1012592 CrossRefPubMedPubMedCentral

41.

Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19(4):453–473. https://doi.org/10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5 CrossRefPubMed

42.

Steyerberg EW, Harrell FE, Jr., Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD (2001) Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 54 (8):774–781, https://doi.org/10.1016/S0895-4356(01)00341-9

43.

Ritter AV, Shugars DA, Bader JD (2010) Root caries risk indicators: a systematic review of risk models. Community Dent Oral Epidemiol 38(5):383–397. https://doi.org/10.1111/j.1600-0528.2010.00551.x CrossRefPubMedPubMedCentral

44.

Begley CG, Ioannidis JP (2015) Reproducibility in science. Circ Res 116(1):116–126. https://doi.org/10.1161/CIRCRESAHA.114.303819 CrossRefPubMed

45.

D'Agostino RB Sr, Grundy S, Sullivan LM, Wilson P, Group CRP (2001) Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA 286(2):180–187. https://doi.org/10.1001/jama.286.2.180 CrossRefPubMed

46.

Vickers AJ, Elkin EB (2006) Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak Int J Soc Med Decis Mak 26(6):565–574. https://doi.org/10.1177/0272989X06295361 CrossRef

47.

Vickers AJ (2008) Decision analysis for the evaluation of diagnostic tests, prediction models and molecular markers. Am Stat 62(4):314–320. https://doi.org/10.1198/000313008X370302 CrossRefPubMedPubMedCentral

48.

Vickers AJ, Cronin AM, Elkin EB, Gonen M (2008) Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak 8(1):53. https://doi.org/10.1186/1472-6947-8-53 CrossRefPubMedPubMedCentral

49.

Tsalatsanis A, Hozo I, Vickers A, Djulbegovic B (2010) A regret theory approach to decision curve analysis: a novel method for eliciting decision makers’ preferences and decision-making. BMC Med Inform Decis Mak 10(1):51. https://doi.org/10.1186/1472-6947-10-51 CrossRefPubMedPubMedCentral

50.

Babyak MA (2004) What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med 66(3):411–421PubMed

51.

Collins GS, Omar O, Shanyinde M, LM Y (2013) A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. J Clin Epidemiol 66(3):268–277. https://doi.org/10.1016/j.jclinepi.2012.06.020 CrossRefPubMed

52.

Tzoulaki I, Liberopoulos G, Ioannidis JP (2009) Assessment of claims of improved prediction beyond the Framingham risk score. JAMA 302(21):2345–2352. https://doi.org/10.1001/jama.2009.1757 CrossRefPubMed

53.

Simpkin AL, Schwartzstein RM (2016) Tolerating uncertainty—the next medical revolution? N Engl J Med 375(18):1713–1715. https://doi.org/10.1056/NEJMp1606402 CrossRefPubMed

54.

Royston P, Altman DG, Sauerbrei W (2006) Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 25(1):127–141. https://doi.org/10.1002/sim.2331 CrossRefPubMed

55.

Lagakos SW (1988) Effects of mismodelling and mismeasuring explanatory variables on tests of their association with a response variable. Stat Med 7(1–2):257–274. https://doi.org/10.1002/sim.4780070126 CrossRefPubMed

56.

Maxwell SE, Delaney HD (1993) Bivariate median splits and spurious statistical significance. Psychol Bull 113(1):181–190. https://doi.org/10.1037/0033-2909.113.1.181 CrossRef

57.

Garrison RJ, Castelli WP, Feinleib M, Kannel WB, Havlik RJ, Padgett SJ, McNamara PM (1979) The association of total cholesterol, triglycerides and plasma lipoprotein cholesterol levels in first degree relatives and spouse pairs. Am J Epidemiol 110(3):313–321. https://doi.org/10.1093/oxfordjournals.aje.a112816 CrossRefPubMed

58.

D’Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, Kannel WB (2008) General cardiovascular risk profile for use in primary care. Framingham Heart Study 117(6):743–753. https://doi.org/10.1161/circulationaha.107.699579 CrossRef

59.

Peduzzi P, Concato J, Feinstein AR, Holford TR (1995) Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 48(12):1503–1510. https://doi.org/10.1016/0895-4356(95)00048-8 CrossRefPubMed

60.

Kerr KF, Meisner A, Thiessen-Philbrook H, Coca SG, Parikh CR (2015) RiGoR: reporting guidelines to address common sources of bias in risk model development. Biomarker Res 3(1):2. https://doi.org/10.1186/s40364-014-0027-7 CrossRef

Title: Progress of statistical analysis in biomedical research through the historical review of the development of the Framingham score
Authors: Aleksandra Ignjatović
Miodrag Stojanović
Zoran Milošević
Marija Anđelković Apostolović
Publication date: 01-08-2018
Publisher: Springer London
Published in: Irish Journal of Medical Science (1971 -) / Issue 3/2018
Print ISSN: 0021-1265
Electronic ISSN: 1863-4362
DOI: https://doi.org/10.1007/s11845-017-1718-5

ACC 2024 Congress Coverage

Heart Failure Video

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Progress of statistical analysis in biomedical research through the historical review of the development of the Framingham score

Abstract

Background

Methods

Results

Conclusion

ACC 2024 Congress Coverage

Year in Review: Heart failure and cardiomyopathies

Semaglutide benefits people with type 2 diabetes and obesity-related heart failure

Incentivised to exercise

New intervention for STEMI-related cardiogenic shock

» View more highlights on the ACC 2024 congress hub page

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 3/2018

Assessment of patient concerns: a review

A cadaveric study of aortic arch variation in an Irish population

A rare ocular complication of neisseria gonorrhoeae

p16, p21, and p53 proteins play an important role in development of pancreatic intraepithelial neoplastic

Assessment of patients’ attitudes towards weight loss in an Irish general practice setting

Correction to: Epidemiology of chronic hepatitis B virus in Ireland using routinely collected surveillance and administrative data, 2004–2014

ACC 2024 Congress Coverage

Year in Review: Heart failure and cardiomyopathies

Semaglutide benefits people with type 2 diabetes and obesity-related heart failure

Incentivised to exercise

New intervention for STEMI-related cardiogenic shock

» View more highlights on the ACC 2024 congress hub page