Skip to main content
Top
Published in: Journal of Nuclear Cardiology 4/2023

Open Access 04-01-2023 | ORIGINAL ARTICLE

External validation and update of the J-ACCESS model in an Italian cohort of patients undergoing stress myocardial perfusion imaging

Authors: Mario Petretta, MD, Rosario Megna, PhD, Roberta Assante, MD, PhD, Emilia Zampella, MD, PhD, Carmela Nappi, MD, PhD, Valeria Gaudieri, MD, PhD, Teresa Mannarino, MD, Roberta Green, PhD, Valeria Cantoni, PhD, Adriana D’Antonio, MD, Mariarosaria Panico, PhD, Wanda Acampa, MD, PhD, Alberto Cuocolo, MD

Published in: Journal of Nuclear Cardiology | Issue 4/2023

Login to get access

Abstract

Background

Cardiovascular risk models are based on traditional risk factors and investigations such as imaging tests. External validation is important to determine reproducibility and generalizability of a prediction model. We performed an external validation of t the Japanese Assessment of Cardiac Events and Survival Study by Quantitative Gated SPECT (J-ACCESS) model, developed from a cohort of patients undergoing stress myocardial perfusion imaging.

Methods

We included 3623 patients with suspected or known coronary artery disease undergoing stress single-photon emission computer tomography (SPECT) myocardial perfusion imaging at our academic center between January 2001 and December 2019.

Results

In our study population, the J-ACCESS model underestimated the risk of major adverse cardiac events (cardiac death, nonfatal myocardial infarction, and severe heart failure requiring hospitalization) within three-year follow-up. The recalibrations and updated of the model slightly improved the initial performance: C-statistics increased from 0.664 to 0.666 and Brier score decreased from 0.075 to 0.073. Hosmer–Lemeshow test indicated a logistic regression fit only for the calibration slope (P = .45) and updated model (P = .22). In the update model, the intercept, diabetes, and severity of myocardial perfusion defects categorized coefficients were comparable with J-ACCESS.

Conclusion

The external validation of the J-ACCESS model as well as recalibration models have a limited value for predicting of three-year major adverse cardiac events in our patients. The performance in predicting risk of the updated model resulted superimposable to the calibration slope model.
Appendix
Available only for authorised users
Literature
1.
go back to reference Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary-artery disease. N Engl J Med 1979;300:1350‐8.CrossRefPubMed Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary-artery disease. N Engl J Med 1979;300:1350‐8.CrossRefPubMed
2.
go back to reference Genders TS, Steyerberg EW, Alkadhi H, Leschka S, Desbiolles L, Nieman K, et al. CAD Consortium. A clinical prediction rule for the diagnosis of coronary artery disease: Validation, updating, and extension. Eur Heart J 2011;32:1316-30. Genders TS, Steyerberg EW, Alkadhi H, Leschka S, Desbiolles L, Nieman K, et al. CAD Consortium. A clinical prediction rule for the diagnosis of coronary artery disease: Validation, updating, and extension. Eur Heart J 2011;32:1316-30.
3.
go back to reference Genders TS, Steyerberg EW, Hunink MG, Nieman K, Galema TW, Mollet NR. Prediction model to estimate presence of coronary artery disease: Retrospective pooled analysis of existing cohorts. BMJ 2012;344:e3485.CrossRefPubMedPubMedCentral Genders TS, Steyerberg EW, Hunink MG, Nieman K, Galema TW, Mollet NR. Prediction model to estimate presence of coronary artery disease: Retrospective pooled analysis of existing cohorts. BMJ 2012;344:e3485.CrossRefPubMedPubMedCentral
4.
go back to reference Reeh J, Therming CB, Heitmann M, Højberg S, Sørum C, Bech J, et al. Prediction of obstructive coronary artery disease and prognosis in patients with suspected stable angina. Eur Heart J 2019;40:1426‐35.CrossRefPubMed Reeh J, Therming CB, Heitmann M, Højberg S, Sørum C, Bech J, et al. Prediction of obstructive coronary artery disease and prognosis in patients with suspected stable angina. Eur Heart J 2019;40:1426‐35.CrossRefPubMed
5.
go back to reference Megna R, Assante R, Zampella E, Gaudieri V, Nappi C, Cuocolo R, et al. Pretest models for predicting abnormal stress single-photon emission computed tomography myocardial perfusion imaging. J Nucl Cardiol 2021;28:1891‐902.CrossRefPubMed Megna R, Assante R, Zampella E, Gaudieri V, Nappi C, Cuocolo R, et al. Pretest models for predicting abnormal stress single-photon emission computed tomography myocardial perfusion imaging. J Nucl Cardiol 2021;28:1891‐902.CrossRefPubMed
7.
go back to reference Nishimura T, Nakajima K, Kusuoka H, Yamashina A, Nishimura S. Prognostic study of risk stratification among Japanese patients with ischemic heart disease using gated myocardial perfusion SPECT: J-ACCESS study. Eur J Nucl Med Mol Imaging 2008;35:319‐28.CrossRefPubMed Nishimura T, Nakajima K, Kusuoka H, Yamashina A, Nishimura S. Prognostic study of risk stratification among Japanese patients with ischemic heart disease using gated myocardial perfusion SPECT: J-ACCESS study. Eur J Nucl Med Mol Imaging 2008;35:319‐28.CrossRefPubMed
8.
go back to reference Nakajima K, Nishimura T. Prognostic table for predicting major cardiac events based on J-ACCESS investigation. Ann Nucl Med 2008;22:891‐7.CrossRefPubMed Nakajima K, Nishimura T. Prognostic table for predicting major cardiac events based on J-ACCESS investigation. Ann Nucl Med 2008;22:891‐7.CrossRefPubMed
9.
go back to reference Sakatani T, Nakajima K, Fujita H, Nishimura T. Cardiovascular event risk estimated after coronary revascularization and optimal medical therapy: J-ACCESS4 prognostic study. Ann Nucl Med 2021;35:241‐52.CrossRefPubMedPubMedCentral Sakatani T, Nakajima K, Fujita H, Nishimura T. Cardiovascular event risk estimated after coronary revascularization and optimal medical therapy: J-ACCESS4 prognostic study. Ann Nucl Med 2021;35:241‐52.CrossRefPubMedPubMedCentral
10.
go back to reference Gibbons RJ, Miller TD. Declining accuracy of the traditional Diamond-Forrester estimates of pretest probability of coronary artery disease: Time for new methods. JAMA Intern Med 2021;181:579‐80.CrossRefPubMed Gibbons RJ, Miller TD. Declining accuracy of the traditional Diamond-Forrester estimates of pretest probability of coronary artery disease: Time for new methods. JAMA Intern Med 2021;181:579‐80.CrossRefPubMed
12.
go back to reference Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons C, Catapano AL, et al. 2016 European Guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts) Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Eur Heart J 2016;37:2315‐81.CrossRefPubMedPubMedCentral Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons C, Catapano AL, et al. 2016 European Guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts) Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Eur Heart J 2016;37:2315‐81.CrossRefPubMedPubMedCentral
13.
go back to reference Mensah GA, Wei GS, Sorlie PD, Fine LJ, Rosenberg Y, Kaufmann PG, et al. Decline in cardiovascular mortality: Possible causes and implications. Circ Res 2017;120:366‐80.CrossRefPubMedPubMedCentral Mensah GA, Wei GS, Sorlie PD, Fine LJ, Rosenberg Y, Kaufmann PG, et al. Decline in cardiovascular mortality: Possible causes and implications. Circ Res 2017;120:366‐80.CrossRefPubMedPubMedCentral
14.
go back to reference Rozanski A, Gransar H, Hayes SW, Min J, Friedman JD, Thomson LE, et al. Temporal trends in the frequency of inducible myocardial ischemia during cardiac stress testing: 1991 to 2009. J Am Coll Cardiol 2013;61:1054‐65.CrossRefPubMed Rozanski A, Gransar H, Hayes SW, Min J, Friedman JD, Thomson LE, et al. Temporal trends in the frequency of inducible myocardial ischemia during cardiac stress testing: 1991 to 2009. J Am Coll Cardiol 2013;61:1054‐65.CrossRefPubMed
15.
go back to reference Duvall WL, Rai M, Ahlberg AW, O’Sullivan DM, Henzlova MJ. A multi-center assessment of the temporal trends in myocardial perfusion imaging. J Nucl Cardiol 2015;22:539‐51.CrossRefPubMed Duvall WL, Rai M, Ahlberg AW, O’Sullivan DM, Henzlova MJ. A multi-center assessment of the temporal trends in myocardial perfusion imaging. J Nucl Cardiol 2015;22:539‐51.CrossRefPubMed
16.
go back to reference Thompson RC, Allam AH. More risk factors, less ischemia, and the relevance of MPI testing. J Nucl Cardiol 2015;22:552‐4.CrossRefPubMed Thompson RC, Allam AH. More risk factors, less ischemia, and the relevance of MPI testing. J Nucl Cardiol 2015;22:552‐4.CrossRefPubMed
17.
go back to reference Jouni H, Askew JW, Crusan DJ, Miller TD, Gibbons RJ. Temporal trends of single-photon emission computed tomography myocardial perfusion imaging in patients without prior coronary artery disease: A 22-year experience at a tertiary academic medical center. Am Heart J 2016;176:127‐33.CrossRefPubMed Jouni H, Askew JW, Crusan DJ, Miller TD, Gibbons RJ. Temporal trends of single-photon emission computed tomography myocardial perfusion imaging in patients without prior coronary artery disease: A 22-year experience at a tertiary academic medical center. Am Heart J 2016;176:127‐33.CrossRefPubMed
18.
go back to reference Jouni H, Askew JW, Crusan DJ, Miller TD, Gibbons RJ. Temporal trends of single-photon emission computed tomography myocardial perfusion imaging in patients with coronary artery disease: A 22-year experience from a tertiary academic medical center. Circ Cardiovasc Imaging 2017;10:e005628.CrossRefPubMed Jouni H, Askew JW, Crusan DJ, Miller TD, Gibbons RJ. Temporal trends of single-photon emission computed tomography myocardial perfusion imaging in patients with coronary artery disease: A 22-year experience from a tertiary academic medical center. Circ Cardiovasc Imaging 2017;10:e005628.CrossRefPubMed
19.
go back to reference Megna R, Zampella E, Assante R, Nappi C, Gaudieri V, Mannarino T. Temporal trends of abnormal myocardial perfusion imaging in a cohort of Italian subjects: Relation with cardiovascular risk factors. J Nucl Cardiol 2020;27:2167‐77.CrossRefPubMed Megna R, Zampella E, Assante R, Nappi C, Gaudieri V, Mannarino T. Temporal trends of abnormal myocardial perfusion imaging in a cohort of Italian subjects: Relation with cardiovascular risk factors. J Nucl Cardiol 2020;27:2167‐77.CrossRefPubMed
20.
go back to reference Huang Y, Li W, Macheret F, Gabriel RA, Ohno-Machado L. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc 2020;27:621‐33.CrossRefPubMedPubMedCentral Huang Y, Li W, Macheret F, Gabriel RA, Ohno-Machado L. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc 2020;27:621‐33.CrossRefPubMedPubMedCentral
21.
go back to reference Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M. External validation of prognostic models: What, why, how, when and where? Clin Kidney J 2020;14:49‐58.CrossRefPubMedPubMedCentral Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M. External validation of prognostic models: What, why, how, when and where? Clin Kidney J 2020;14:49‐58.CrossRefPubMedPubMedCentral
22.
go back to reference Megna R, Petretta M, Alfano B, Cantoni V, Green R, Daniele S, et al. A New relational database including clinical data and myocardial perfusion imaging findings in coronary artery disease. Curr Med Imaging Rev 2019;15:661‐71.CrossRefPubMed Megna R, Petretta M, Alfano B, Cantoni V, Green R, Daniele S, et al. A New relational database including clinical data and myocardial perfusion imaging findings in coronary artery disease. Curr Med Imaging Rev 2019;15:661‐71.CrossRefPubMed
23.
go back to reference Verberne HJ, Acampa W, Anagnostopoulos C, Ballinger J, Bengel F, De Bondt P, et al. European Association of Nuclear Medicine (EANM). EANM procedural guidelines for radionuclide myocardial perfusion imaging with SPECT and SPECT/CT: 2015 revision. Eur J Nucl Med Mol Imaging 2015;42:1929-40. Verberne HJ, Acampa W, Anagnostopoulos C, Ballinger J, Bengel F, De Bondt P, et al. European Association of Nuclear Medicine (EANM). EANM procedural guidelines for radionuclide myocardial perfusion imaging with SPECT and SPECT/CT: 2015 revision. Eur J Nucl Med Mol Imaging 2015;42:1929-40.
24.
go back to reference Berman DS, Abidov A, Kang X, Hayes SW, Friedman JD, Sciammarella MG, et al. Prognostic validation of a 17-segment score derived from a 20-segment score for myocardial perfusion SPECT interpretation. J Nucl Cardiol 2004;11:414‐23.CrossRefPubMed Berman DS, Abidov A, Kang X, Hayes SW, Friedman JD, Sciammarella MG, et al. Prognostic validation of a 17-segment score derived from a 20-segment score for myocardial perfusion SPECT interpretation. J Nucl Cardiol 2004;11:414‐23.CrossRefPubMed
25.
go back to reference Hosmer DW, Lemesbow S. Goodness of fit tests for the multiple logistic regression model. Commun Stat Theory Methods 1980;9:1043‐69.CrossRef Hosmer DW, Lemesbow S. Goodness of fit tests for the multiple logistic regression model. Commun Stat Theory Methods 1980;9:1043‐69.CrossRef
26.
go back to reference Roth GA, Mensah GA, Johnson CO, Addolorato G, Ammirati E, Baddour LM, et al. GBD-NHLBI-JACC Global Burden of Cardiovascular Diseases Writing Group. Global Burden of Cardiovascular Diseases and Risk Factors, 1990–2019: Update From the GBD 2019 Study. J Am Coll Cardiol 2020;76:2982-3021. Roth GA, Mensah GA, Johnson CO, Addolorato G, Ammirati E, Baddour LM, et al. GBD-NHLBI-JACC Global Burden of Cardiovascular Diseases Writing Group. Global Burden of Cardiovascular Diseases and Risk Factors, 1990–2019: Update From the GBD 2019 Study. J Am Coll Cardiol 2020;76:2982-3021.
28.
29.
go back to reference Megna R, Cuocolo A, Petretta M. Applications of machine learning in medicine. Biomed J Sci & Tech Res 2019;20:15350‐2. Megna R, Cuocolo A, Petretta M. Applications of machine learning in medicine. Biomed J Sci & Tech Res 2019;20:15350‐2.
30.
go back to reference Stevens LM, Mortazavi BJ, Deo RC, Curtis L, Kao DP. Recommendations for reporting machine learning analyses in clinical research. Circ Cardiovasc Qual Outcomes 2020;13:e006556.CrossRefPubMedPubMedCentral Stevens LM, Mortazavi BJ, Deo RC, Curtis L, Kao DP. Recommendations for reporting machine learning analyses in clinical research. Circ Cardiovasc Qual Outcomes 2020;13:e006556.CrossRefPubMedPubMedCentral
31.
go back to reference Ricciardi C, Cuocolo R, Megna R, Cesarelli M, Petretta M. Machine learning analysis: General features, requirements and cardiovascular applications. Minerva Cardiol Angiol 2022;70:67‐74.CrossRefPubMed Ricciardi C, Cuocolo R, Megna R, Cesarelli M, Petretta M. Machine learning analysis: General features, requirements and cardiovascular applications. Minerva Cardiol Angiol 2022;70:67‐74.CrossRefPubMed
32.
go back to reference SCORE2 working group and ESC Cardiovascular risk collaboration. SCORE2 risk prediction algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe. Eur Heart J 2021;42:2439‐54.CrossRef SCORE2 working group and ESC Cardiovascular risk collaboration. SCORE2 risk prediction algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe. Eur Heart J 2021;42:2439‐54.CrossRef
33.
go back to reference Tillmann T, Läll K, Dukes O, Veronesi G, Pikhart H, Peasey A, et al. Development and validation of two SCORE-based cardiovascular risk prediction models for Eastern Europe: a multicohort study. Eur Heart J 2020;41:3325‐33.CrossRefPubMedPubMedCentral Tillmann T, Läll K, Dukes O, Veronesi G, Pikhart H, Peasey A, et al. Development and validation of two SCORE-based cardiovascular risk prediction models for Eastern Europe: a multicohort study. Eur Heart J 2020;41:3325‐33.CrossRefPubMedPubMedCentral
34.
go back to reference Megna R, Nappi C, Gaudieri V, Mannarino T, Assante R, Zampella E, et al. Diagnostic value of clinical risk scores for predicting normal stress myocardial perfusion imaging in subjects without coronary artery calcium. J Nucl Cardiol 2022;29:323‐33.CrossRefPubMed Megna R, Nappi C, Gaudieri V, Mannarino T, Assante R, Zampella E, et al. Diagnostic value of clinical risk scores for predicting normal stress myocardial perfusion imaging in subjects without coronary artery calcium. J Nucl Cardiol 2022;29:323‐33.CrossRefPubMed
35.
go back to reference Nakajima K, Matsuo S, Okuyama C, Hatta T, Tsukamoto K, Nishimura S, et al. Cardiac event risk in Japanese subjects estimated using gated myocardial perfusion imaging, in conjunction with diabetes mellitus and chronic kidney disease. Circ J 2012;76:168‐75.CrossRefPubMed Nakajima K, Matsuo S, Okuyama C, Hatta T, Tsukamoto K, Nishimura S, et al. Cardiac event risk in Japanese subjects estimated using gated myocardial perfusion imaging, in conjunction with diabetes mellitus and chronic kidney disease. Circ J 2012;76:168‐75.CrossRefPubMed
36.
go back to reference Aburadani I, Usuda K, Sumiya H, Sakagami S, Kiyokawa H, Matsuo S, et al. Ability of the prognostic model of J-ACCESS study to predict cardiac events in a clinical setting: The APPROACH study. J Cardiol 2018;72:81‐6.CrossRefPubMed Aburadani I, Usuda K, Sumiya H, Sakagami S, Kiyokawa H, Matsuo S, et al. Ability of the prognostic model of J-ACCESS study to predict cardiac events in a clinical setting: The APPROACH study. J Cardiol 2018;72:81‐6.CrossRefPubMed
37.
go back to reference Nakajima K, Nakamura S, Hase H, Takeishi Y, Nishimura S, Kawano Y, et al. Risk stratification based on J-ACCESS risk models with myocardial perfusion imaging: Risk versus outcomes of patients with chronic kidney disease. J Nucl Cardiol 2020;27:41‐50.CrossRefPubMed Nakajima K, Nakamura S, Hase H, Takeishi Y, Nishimura S, Kawano Y, et al. Risk stratification based on J-ACCESS risk models with myocardial perfusion imaging: Risk versus outcomes of patients with chronic kidney disease. J Nucl Cardiol 2020;27:41‐50.CrossRefPubMed
Metadata
Title
External validation and update of the J-ACCESS model in an Italian cohort of patients undergoing stress myocardial perfusion imaging
Authors
Mario Petretta, MD
Rosario Megna, PhD
Roberta Assante, MD, PhD
Emilia Zampella, MD, PhD
Carmela Nappi, MD, PhD
Valeria Gaudieri, MD, PhD
Teresa Mannarino, MD
Roberta Green, PhD
Valeria Cantoni, PhD
Adriana D’Antonio, MD
Mariarosaria Panico, PhD
Wanda Acampa, MD, PhD
Alberto Cuocolo, MD
Publication date
04-01-2023
Publisher
Springer International Publishing
Published in
Journal of Nuclear Cardiology / Issue 4/2023
Print ISSN: 1071-3581
Electronic ISSN: 1532-6551
DOI
https://doi.org/10.1007/s12350-022-03173-4

Other articles of this Issue 4/2023

Journal of Nuclear Cardiology 4/2023 Go to the issue