Skip to main content
Top
Published in: The International Journal of Cardiovascular Imaging 5/2008

01-06-2008 | Original Paper

Angiographic core laboratory reproducibility analyses: implications for planning clinical trials using coronary angiography and left ventriculography end-points

Authors: Terje K. Steigen, Cheryl Claudio, David Abbott, Michael Schulzer, Jeff Burton, Wayne Tymchak, Christopher E. Buller, G. B. John Mancini

Published in: The International Journal of Cardiovascular Imaging | Issue 5/2008

Login to get access

Abstract

Objectives To assess reproducibility of core laboratory performance and impact on sample size calculations. Background Little information exists about overall reproducibility of core laboratories in contradistinction to performance of individual technicians. Also, qualitative parameters are being adjudicated increasingly as either primary or secondary end-points. The comparative impact of using diverse indexes on sample sizes has not been previously reported. Methods We compared initial and repeat assessments of five quantitative parameters [e.g., minimum lumen diameter (MLD), ejection fraction (EF), etc.] and six qualitative parameters [e.g., TIMI myocardial perfusion grade (TMPG) or thrombus grade (TTG), etc.], as performed by differing technicians and separated by a year or more. Sample sizes were calculated from these results. TMPG and TTG were also adjudicated by a second core laboratory. Results MLD and EF were the most reproducible, yielding the smallest sample size calculations, whereas percent diameter stenosis and centerline wall motion require substantially larger trials. Of the qualitative parameters, all except TIMI flow grade gave reproducibility characteristics yielding sample sizes of many 100’s of patients. Reproducibility of TMPG and TTG was only moderately good both within and between core laboratories, underscoring an intrinsic difficulty in assessing these. Conclusions Core laboratories can be shown to provide reproducibility performance that is comparable to performance commonly ascribed to individual technicians. The differences in reproducibility yield huge differences in sample size when comparing quantitative and qualitative parameters. TMPG and TTG are intrinsically difficult to assess and conclusions based on these parameters should arise only from very large trials.
Literature
1.
go back to reference Bolson EL, Kilman S, Sheehan FH, Dodge HT (1980) Left ventricular segmental wall motion: a new method using local direction information. Proceedings of computers in cardiology, IEEE Press, pp 245–248 Bolson EL, Kilman S, Sheehan FH, Dodge HT (1980) Left ventricular segmental wall motion: a new method using local direction information. Proceedings of computers in cardiology, IEEE Press, pp 245–248
2.
go back to reference Mancini GBJ, Simon SB, McGillem MJ, Lefree MT, Friedman HZ, Vogel RA (1987) Automated quantitative coronary arteriography: morphologic and physiologic validation in vivo of a rapid digital angiographic method. Circulation 75:452–460PubMed Mancini GBJ, Simon SB, McGillem MJ, Lefree MT, Friedman HZ, Vogel RA (1987) Automated quantitative coronary arteriography: morphologic and physiologic validation in vivo of a rapid digital angiographic method. Circulation 75:452–460PubMed
3.
go back to reference Mancini GB, Yeoh E, Kamimura C, Abbott D (2001) A comparison of quantitative coronary angiography systems using a unique set of in vivo coronary stenosis images. Can J Cardiol 17:785–791PubMed Mancini GB, Yeoh E, Kamimura C, Abbott D (2001) A comparison of quantitative coronary angiography systems using a unique set of in vivo coronary stenosis images. Can J Cardiol 17:785–791PubMed
4.
go back to reference Gibson CM, Cannon CP, Daley WL, Dodge JT Jr, Alexander B Jr, Marble SJ, McCabe CH, Raymond L, Fortin T, Poole WK, Braunwald E (1996) TIMI frame count: a quantitative method of assessing coronary artery flow. Circulation 93:879–888PubMed Gibson CM, Cannon CP, Daley WL, Dodge JT Jr, Alexander B Jr, Marble SJ, McCabe CH, Raymond L, Fortin T, Poole WK, Braunwald E (1996) TIMI frame count: a quantitative method of assessing coronary artery flow. Circulation 93:879–888PubMed
5.
go back to reference Kalbfleisch SJ, McGillem MJ, Pinto IM, Kavanaugh KM, DeBoe SF, Mancini GB (1990) Comparison of automated quantitative coronary angiography with caliper measurements of percent diameter stenosis. Am J Cardiol 65:1181–1184PubMedCrossRef Kalbfleisch SJ, McGillem MJ, Pinto IM, Kavanaugh KM, DeBoe SF, Mancini GB (1990) Comparison of automated quantitative coronary angiography with caliper measurements of percent diameter stenosis. Am J Cardiol 65:1181–1184PubMedCrossRef
6.
go back to reference Baim DS, Grossman W (2000) Cardiac catheterization, angiography, and intervention, 6th edn. Lippincott Williams & Wilkins, Philadelphia, pp 766–767 Baim DS, Grossman W (2000) Cardiac catheterization, angiography, and intervention, 6th edn. Lippincott Williams & Wilkins, Philadelphia, pp 766–767
7.
go back to reference Bashore TM (ed) (1990) Invasive cardiology, principles and techniques. BC Decker Inc, Toronto Bashore TM (ed) (1990) Invasive cardiology, principles and techniques. BC Decker Inc, Toronto
8.
go back to reference The Thrombolysis in Myocardial Infarction (TIMI) trial (1985) Phase I findings. TIMI Study Group. N Engl J Med 312:932–936 The Thrombolysis in Myocardial Infarction (TIMI) trial (1985) Phase I findings. TIMI Study Group. N Engl J Med 312:932–936
9.
go back to reference Gibson CM, Ryan K, Sparano A, Moynihan JL, Rizzo M, Kelley M, Marble SJ, Laham R, Simons M, McClusky TR, Dodge JT Jr (1999) Angiographic methods to assess human coronary angiogenesis. Am Heart J 137:169–179PubMedCrossRef Gibson CM, Ryan K, Sparano A, Moynihan JL, Rizzo M, Kelley M, Marble SJ, Laham R, Simons M, McClusky TR, Dodge JT Jr (1999) Angiographic methods to assess human coronary angiogenesis. Am Heart J 137:169–179PubMedCrossRef
10.
go back to reference Gibson CM, Cannon CP, Murphy SA, Ryan KA, Mesley R, Marble SJ, McCabe CH, Van De Werf F, Braunwald E (2000) Relationship of TIMI myocardial perfusion grade to mortality after administration of thrombolytic drugs. Circulation 101:125–130PubMed Gibson CM, Cannon CP, Murphy SA, Ryan KA, Mesley R, Marble SJ, McCabe CH, Van De Werf F, Braunwald E (2000) Relationship of TIMI myocardial perfusion grade to mortality after administration of thrombolytic drugs. Circulation 101:125–130PubMed
11.
go back to reference Gibson CM, de Lemos JA, Murphy SA, Marble SJ, McCabe CH, Cannon CP, Antman EM, Braunwald E (2001) Combination therapy with Abciximab reduces angiographically evident thrombus in acute myocardial infarction: A TIMI 14 substudy. Circulation 103:2550–2554PubMed Gibson CM, de Lemos JA, Murphy SA, Marble SJ, McCabe CH, Cannon CP, Antman EM, Braunwald E (2001) Combination therapy with Abciximab reduces angiographically evident thrombus in acute myocardial infarction: A TIMI 14 substudy. Circulation 103:2550–2554PubMed
12.
go back to reference Bland JM, Altman DG (1986) Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet 346:1085–1087CrossRef Bland JM, Altman DG (1986) Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet 346:1085–1087CrossRef
13.
go back to reference Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46CrossRef Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46CrossRef
14.
go back to reference Sim J, Wright CC (2005) The kappa statistic in reliability studies. Use, interpretation, and sample size requirements. Phys Ther 85:257–268PubMed Sim J, Wright CC (2005) The kappa statistic in reliability studies. Use, interpretation, and sample size requirements. Phys Ther 85:257–268PubMed
15.
go back to reference Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174. (in Sim J, Wright CC. The Kappa statistic in reliability studies: use, interpretation, and sample size requirements)PubMedCrossRef Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174. (in Sim J, Wright CC. The Kappa statistic in reliability studies: use, interpretation, and sample size requirements)PubMedCrossRef
16.
go back to reference Winer BJ, Brown DR, Michels KM (1991) Statistical principles in experimental design. 3rd edn. McGraw Hill, Boston, pp 92–100 Winer BJ, Brown DR, Michels KM (1991) Statistical principles in experimental design. 3rd edn. McGraw Hill, Boston, pp 92–100
17.
go back to reference Fleiss JL (1999) The design and analysis of clinical experiments. Wiley, NewYork Fleiss JL (1999) The design and analysis of clinical experiments. Wiley, NewYork
18.
go back to reference Machin D, Campbell M, Fayers P, Pinol A (1997) Sample size tables for clinical studies, 2nd edn. Blackwell Science, Oxford Machin D, Campbell M, Fayers P, Pinol A (1997) Sample size tables for clinical studies, 2nd edn. Blackwell Science, Oxford
19.
go back to reference International Organization for Standardization (1993) Guide to the expression of uncertainty in measurement, 1st edn. ISO/TAG/WG3; ISO: Geneva, Switzerland International Organization for Standardization (1993) Guide to the expression of uncertainty in measurement, 1st edn. ISO/TAG/WG3; ISO: Geneva, Switzerland
20.
go back to reference Moer R, Van Weert AWM, Myreng Y, Mølstad P (2003) Variability of quantitative coronary angiography: an evaluation of on-site versus core laboratory analysis. Int J Cardiovasc Imaging 19(6):457–464PubMedCrossRef Moer R, Van Weert AWM, Myreng Y, Mølstad P (2003) Variability of quantitative coronary angiography: an evaluation of on-site versus core laboratory analysis. Int J Cardiovasc Imaging 19(6):457–464PubMedCrossRef
21.
go back to reference Popma JJ, Lansky AJ, Yeh W, Kennard ED, Keller MB, Merritt AJ, DeFalco RA, Desai A, Pacera JH, Schnabel JF, Niedermeyer V, Baim DS, Detre KM (1997) Reliability of the quantitative angiographic measurements in the new approaches to coronary intervention (NACI) registry: a comparison of clinical site and repeated angiographic core laboratory readings. Am J Cardiol 80(10A):19K–25KPubMedCrossRef Popma JJ, Lansky AJ, Yeh W, Kennard ED, Keller MB, Merritt AJ, DeFalco RA, Desai A, Pacera JH, Schnabel JF, Niedermeyer V, Baim DS, Detre KM (1997) Reliability of the quantitative angiographic measurements in the new approaches to coronary intervention (NACI) registry: a comparison of clinical site and repeated angiographic core laboratory readings. Am J Cardiol 80(10A):19K–25KPubMedCrossRef
22.
go back to reference Brown BG, Hillger LA, Lewis C, Zhao X-Q, Sacco D, Bisson B, Fisher L (1993) Quantitative imaging, risk factors, prevalence, and change: chairman’s discussion of session 2: a maximum confidence approach for measuring progression and regression of coronary artery disease in clinical trials. Circulation 87(3S):II66–II73PubMed Brown BG, Hillger LA, Lewis C, Zhao X-Q, Sacco D, Bisson B, Fisher L (1993) Quantitative imaging, risk factors, prevalence, and change: chairman’s discussion of session 2: a maximum confidence approach for measuring progression and regression of coronary artery disease in clinical trials. Circulation 87(3S):II66–II73PubMed
23.
go back to reference Reiber JHC, Serruys PW, Kooijman CJ, Wijns W, Slager CJ, Gerbrands JJ, Schuurbiers JCH, den Boer A, Hugenholtz PG (1985) Assessment of short-, medium-, and long-term variations in arterial dimensions from computer-assisted quantitation of coronary cineangiograms. Circulation 71:280–288PubMed Reiber JHC, Serruys PW, Kooijman CJ, Wijns W, Slager CJ, Gerbrands JJ, Schuurbiers JCH, den Boer A, Hugenholtz PG (1985) Assessment of short-, medium-, and long-term variations in arterial dimensions from computer-assisted quantitation of coronary cineangiograms. Circulation 71:280–288PubMed
24.
go back to reference Herrman JP, Azar A, Umans VA, Boersma E, von Es GA, Serruys PW (1996) Inter- and intra-observer variability in the qualitative categorization of coronary angiograms. Int J Cardiovasc Imaging 12(1):21–30CrossRef Herrman JP, Azar A, Umans VA, Boersma E, von Es GA, Serruys PW (1996) Inter- and intra-observer variability in the qualitative categorization of coronary angiograms. Int J Cardiovasc Imaging 12(1):21–30CrossRef
25.
go back to reference Beauman GJ, Vogel RA (1990) Accuracy of individual and panel visual interpretations of coronary arteriograms: implications for clinical decision making. J Am Coll Cardiol 16(1):108–113PubMedCrossRef Beauman GJ, Vogel RA (1990) Accuracy of individual and panel visual interpretations of coronary arteriograms: implications for clinical decision making. J Am Coll Cardiol 16(1):108–113PubMedCrossRef
26.
go back to reference van ‘t Hof AW, Liem A, Suryapranata H, Hoorntje JC, de Boer MJ, Zijlstra F (1998) Angiographic assessment of myocardial reperfusion in patients treated with primary angioplasty for acute myocardial infarction: myocardial blush grade. Zwolle Myocardial Infarction Study Group. Circulation 97(23):2302–2306PubMed van ‘t Hof AW, Liem A, Suryapranata H, Hoorntje JC, de Boer MJ, Zijlstra F (1998) Angiographic assessment of myocardial reperfusion in patients treated with primary angioplasty for acute myocardial infarction: myocardial blush grade. Zwolle Myocardial Infarction Study Group. Circulation 97(23):2302–2306PubMed
27.
go back to reference Stone GW, Peterson MA, Lansky AJ, Dangas G, Mehran R, Leon MB (2002) Impact of normalized myocardial perfusion after successful angioplasty in acute myocardial infarction. J Am Coll Cardiol 39(4):591–597PubMedCrossRef Stone GW, Peterson MA, Lansky AJ, Dangas G, Mehran R, Leon MB (2002) Impact of normalized myocardial perfusion after successful angioplasty in acute myocardial infarction. J Am Coll Cardiol 39(4):591–597PubMedCrossRef
28.
go back to reference Costantini CO, Stone GW, Mehran R, Aymong E, Grines CL, Cox DA, Stuckey T, Turco M, Gersh BJ, Tcheng JE, Garcia E, Griffin JJ, Guagliumi G, Leon MB, Lansky AJ (2004) Frequency, correlates, and clinical implications of myocardial perfusion after primary angioplasty and stenting, with and without glycoprotein IIb/IIIa inhibition, in acute myocardial infarction. J Am Coll Cardiol 44(2):305–312PubMedCrossRef Costantini CO, Stone GW, Mehran R, Aymong E, Grines CL, Cox DA, Stuckey T, Turco M, Gersh BJ, Tcheng JE, Garcia E, Griffin JJ, Guagliumi G, Leon MB, Lansky AJ (2004) Frequency, correlates, and clinical implications of myocardial perfusion after primary angioplasty and stenting, with and without glycoprotein IIb/IIIa inhibition, in acute myocardial infarction. J Am Coll Cardiol 44(2):305–312PubMedCrossRef
29.
go back to reference Gibson CM, Cannon CP, Murphy SA, Ryan KA, Mesley R, Marble SJ, McCabe CH, Van De Werf F, Braunwald E (2000) Relationship of TIMI myocardial perfusion grade to mortality after administration of thrombolytic drugs. Circulation 101(2):125–130PubMed Gibson CM, Cannon CP, Murphy SA, Ryan KA, Mesley R, Marble SJ, McCabe CH, Van De Werf F, Braunwald E (2000) Relationship of TIMI myocardial perfusion grade to mortality after administration of thrombolytic drugs. Circulation 101(2):125–130PubMed
30.
go back to reference Kirtane AJ, Bui A, Murphy SA, Karmpaliotis D, Kosmidou I, Boundy K, Rahman A, Pinto DS, Aroesty JM, Giugliano RP, Cannon CP, Antman EM, Gibson CM (2004) TIMI Study Group. Association of epicardial and tissue-level reperfusion with left ventricular end-diastolic pressures in ST-elevation myocardial infarction. J Thromb Thrombolysis 17(3):177–184PubMedCrossRef Kirtane AJ, Bui A, Murphy SA, Karmpaliotis D, Kosmidou I, Boundy K, Rahman A, Pinto DS, Aroesty JM, Giugliano RP, Cannon CP, Antman EM, Gibson CM (2004) TIMI Study Group. Association of epicardial and tissue-level reperfusion with left ventricular end-diastolic pressures in ST-elevation myocardial infarction. J Thromb Thrombolysis 17(3):177–184PubMedCrossRef
31.
go back to reference Gibson CM, Cannon CP, Murphy SA, Marble SJ, Barron HV, Braunwald E (2002) TIMI Study Group. Relationship of the TIMI myocardial perfusion grades, flow grades, frame count, and percutaneous coronary intervention to long-term outcomes after thrombolytic administration in acute myocardial infarction. Circulation 105(16):1909–1913PubMedCrossRef Gibson CM, Cannon CP, Murphy SA, Marble SJ, Barron HV, Braunwald E (2002) TIMI Study Group. Relationship of the TIMI myocardial perfusion grades, flow grades, frame count, and percutaneous coronary intervention to long-term outcomes after thrombolytic administration in acute myocardial infarction. Circulation 105(16):1909–1913PubMedCrossRef
32.
go back to reference Gibson CM, Schomig A (2004) Coronary and myocardial angiography: angiographic assessment of both epicardial and myocardial perfusion. Circulation 109(25):3096–3105PubMedCrossRef Gibson CM, Schomig A (2004) Coronary and myocardial angiography: angiographic assessment of both epicardial and myocardial perfusion. Circulation 109(25):3096–3105PubMedCrossRef
33.
go back to reference Gibson CM, Murphy SA, Morrow DA, Aroesty JM, Gibbons RJ, Gourlay SG, Barron HV, Giugliano RP, Antman EM, Braunwald E (2004) Angiographic perfusion score: an angiographic parameter that integrates both epicardial and tissue level perfusion before and after facilitated percutaneous coronary intervention in acute myocardial infarction. Am Heart J 148(2):336–340PubMedCrossRef Gibson CM, Murphy SA, Morrow DA, Aroesty JM, Gibbons RJ, Gourlay SG, Barron HV, Giugliano RP, Antman EM, Braunwald E (2004) Angiographic perfusion score: an angiographic parameter that integrates both epicardial and tissue level perfusion before and after facilitated percutaneous coronary intervention in acute myocardial infarction. Am Heart J 148(2):336–340PubMedCrossRef
34.
go back to reference Gibson CM, Kirtane AJ, Boundy K, Ly H, Karmpaliotis D, Murphy SA, Giugliano RP, Cannon CP, Antman EM, Braunwald E (2005) TIMI Study Group. Association of a negative residual stenosis following rescue/adjunctive percutaneous coronary intervention with impaired myocardial perfusion and adverse outcomes among ST-segment elevation myocardial infarction patients. J Am Coll Cardiol 45(3):357–362PubMedCrossRef Gibson CM, Kirtane AJ, Boundy K, Ly H, Karmpaliotis D, Murphy SA, Giugliano RP, Cannon CP, Antman EM, Braunwald E (2005) TIMI Study Group. Association of a negative residual stenosis following rescue/adjunctive percutaneous coronary intervention with impaired myocardial perfusion and adverse outcomes among ST-segment elevation myocardial infarction patients. J Am Coll Cardiol 45(3):357–362PubMedCrossRef
35.
go back to reference Gibson CM, de Lemos JA, Murphy SA, Marble SJ, Dauterman KW, Michaels A, Barron HV, Antman EM (2002) TIMI Study Group. Methodologic and clinical validation of the TIMI myocardial perfusion grade in acute myocardial infarction. J Thromb Thrombolysis 14(3):233–237PubMedCrossRef Gibson CM, de Lemos JA, Murphy SA, Marble SJ, Dauterman KW, Michaels A, Barron HV, Antman EM (2002) TIMI Study Group. Methodologic and clinical validation of the TIMI myocardial perfusion grade in acute myocardial infarction. J Thromb Thrombolysis 14(3):233–237PubMedCrossRef
36.
go back to reference Tuinenburg JC, Koning G, Hekking E, Desjardins C, Harel F, Bilodeau L, van Weert AW, Lesperance J, Reiber JH (2002) One core laboratory at two international sites, is that feasible? An inter-core laboratory and intra-observer variability study. Catheter Cardiovasc Interv 56(3):333–340PubMedCrossRef Tuinenburg JC, Koning G, Hekking E, Desjardins C, Harel F, Bilodeau L, van Weert AW, Lesperance J, Reiber JH (2002) One core laboratory at two international sites, is that feasible? An inter-core laboratory and intra-observer variability study. Catheter Cardiovasc Interv 56(3):333–340PubMedCrossRef
37.
go back to reference Beauman GJ, Reiber JH, Koning G, Vogel RA (1996) Comparisons of angiographic core laboratory analyses of phantom and clinical images: interlaboratory variability. Cathet Cardiovasc Diagn 37(1):24–31PubMedCrossRef Beauman GJ, Reiber JH, Koning G, Vogel RA (1996) Comparisons of angiographic core laboratory analyses of phantom and clinical images: interlaboratory variability. Cathet Cardiovasc Diagn 37(1):24–31PubMedCrossRef
Metadata
Title
Angiographic core laboratory reproducibility analyses: implications for planning clinical trials using coronary angiography and left ventriculography end-points
Authors
Terje K. Steigen
Cheryl Claudio
David Abbott
Michael Schulzer
Jeff Burton
Wayne Tymchak
Christopher E. Buller
G. B. John Mancini
Publication date
01-06-2008
Publisher
Springer Netherlands
Published in
The International Journal of Cardiovascular Imaging / Issue 5/2008
Print ISSN: 1569-5794
Electronic ISSN: 1875-8312
DOI
https://doi.org/10.1007/s10554-007-9285-x

Other articles of this Issue 5/2008

The International Journal of Cardiovascular Imaging 5/2008 Go to the issue