Top

Published in:

Open Access 01-12-2019 | Stroke | Research

Stroke aetiological classification reliability and effect on trial sample size: systematic review, meta-analysis and statistical modelling

Authors: Azmil H. Abdul-Rahim, David Alexander Dickie, Johann R. Selvarajah, Kennedy R. Lees, Terence J. Quinn, On behalf of VISTA Collaborators

Published in: Trials | Issue 1/2019

Abstract

Background

Inter-observer variability in stroke aetiological classification may have an effect on trial power and estimation of treatment effect. We modelled the effect of misclassification on required sample size in a hypothetical cardioembolic (CE) stroke trial.

Methods

We performed a systematic review to quantify the reliability (inter-observer variability) of various stroke aetiological classification systems. We then modelled the effect of this misclassification in a hypothetical trial of anticoagulant in CE stroke contaminated by patients with non-cardioembolic (non-CE) stroke aetiology. Rates of misclassification were based on the summary reliability estimates from our systematic review. We randomly sampled data from previous acute trials in CE and non-CE participants, using the Virtual International Stroke Trials Archive. We used bootstrapping to model the effect of varying misclassification rates on sample size required to detect a between-group treatment effect across 5000 permutations. We described outcomes in terms of survival and stroke recurrence censored at 90 days.

Results

From 4655 titles, we found 14 articles describing three stroke classification systems. The inter-observer reliability of the classification systems varied from ‘fair’ to ‘very good’ and suggested misclassification rates of 5% and 20% for our modelling. The hypothetical trial, with 80% power and alpha 0.05, was able to show a difference in survival between anticoagulant and antiplatelet in CE with a sample size of 198 in both trial arms. Contamination of both arms with 5% misclassified participants inflated the required sample size to 237 and with 20% misclassification inflated the required sample size to 352, for equivalent trial power. For an outcome of stroke recurrence using the same data, base-case estimated sample size for 80% power and alpha 0.05 was n = 502 in each arm, increasing to 605 at 5% contamination and 973 at 20% contamination.

Conclusions

Stroke aetiological classification systems suffer from inter-observer variability, and the resulting misclassification may limit trial power.

Trial registration

Protocol available at reviewregistry540.

Available only for authorised users

McArthur KS, Johnson PC, Quinn TJ, Higgins P, Langhorne P, Walters MR, Weir CJ, Dawson J, Lees KR. Improving the efficiency of stroke trials: feasibility and efficacy of group adjudication of functional end points. Stroke. 2013;44:3422–8.CrossRef

Muller MJ, Szegedi A. Effects of interrater reliability of psychopathologic assessment on power and sample size calculations in clinical trials. J Clin Psychopharmacol. 2002;22:318–25.CrossRef

Diener HC, Easton JD, Granger CB, Cronin L, Duffy C, Cotton D, Brueckmann M, Sacco RL. Design of Randomized, double-blind, Evaluation in secondary Stroke Prevention comparing the EfficaCy and safety of the oral Thrombin inhibitor dabigatran etexilate vs. acetylsalicylic acid in patients with Embolic Stroke of Undetermined Source (RE-SPECT ESUS). Int J Stroke. 2015;10:1309–12.CrossRef

Hart RG, Sharma M, Mundl H, Kasner SE, Bangdiwala SI, Berkowitz SD, Swaminathan B, Lavados P, Wang Y, Wang Y, Davalos A, Shamalov N, Mikulik R, Cunha L, Lindgren A, Arauz A, Lang W, Czlonkowska A, Eckstein J, Gagliardi RJ, Amarenco P, Ameriso SF, Tatlisumak T, Veltkamp R, Hankey GJ, Toni D, Bereczki D, Uchiyama S, Ntaios G, Yoon B-W, Brouns R, Endres M, Muir KW, Bornstein N, Ozturk S, O’Donnell MJ, De Vries Basson MM, Pare G, Pater C, Kirsch B, Sheridan P, Peters G, Weitz JI, Peacock WF, Shoamanesh A, Benavente OR, Joyner C, Themeles E, Connolly SJ. Rivaroxaban for stroke prevention after embolic stroke of undetermined source. N Engl J Med. 2018;378:2191–201.CrossRef

Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, Hrobjartsson A, Roberts C, Shoukri M, Streiner DL. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64:96–106.CrossRef

McGinn T, Wyer PC, Newman TB, Keitz S, Leipzig R, Guyat G, et al. Tips for learners of evidence-based medicine: 3. Measures of observer variability (kappa statistic). CMAJ. 2004;171:1369–73.CrossRef

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRef

McHugh ML. Interrater reliability: the kappa statistic. Biochemia medica. 2012;22:276–82.CrossRef

Hylek EM, Go AS, Chang Y, Jensvold NG, Henault LE, Selby JV, Singer DE. Effect of intensity of oral anticoagulation on stroke severity and mortality in atrial fibrillation. N Engl J Med. 2003;349:1019–26.CrossRef

10.

Rothwell PM, Algra A, Chen Z, Diener HC, Norrving B, Mehta Z. Effects of aspirin on risk and severity of early recurrent stroke after transient ischaemic attack and ischaemic stroke: time-course analysis of randomised trials. Lancet. 2016;388:365–75.CrossRef

11.

Ali M, Bath P, Brady M, Davis S, Diener HC, Donnan G, Fisher M, Hacke W, Hanley DF, Luby M, Tsivgoulis G, Wahlgren N, Warach S, Lees KR, Comm VS. Development, expansion, and use of a stroke clinical trials resource for novel exploratory analyses. Int J Stroke. 2012;7:133–8.CrossRef

12.

Ali M, Bath PM, Curram J, Davis SM, Diener HC, Donnan GA, Fisher M, Gregson BA, Grotta J, Hacke W, Hennerici MG, Hommel M, Kaste M, Marler JR, Sacco RL, Teal P, Wahlgren NG, Warach S, Weir CJ, Lees KR. The Virtual International Stroke Trials Archive. Stroke. 2007;38:1905–10.CrossRef

13.

Ali M, Fulton R, Quinn T, Brady M. How well do standard stroke outcome measures reflect quality of life? A retrospective analysis of clinical trial data. Stroke. 2013;44:3161–5.CrossRef

14.

MacIsaac RL, Ali M, Taylor-Rowan M, Rodgers H, Lees KR, Quinn TJ. Use of a 3-item short-form version of the Barthel Index for use in stroke. Stroke. 2017;48:618.CrossRef

15.

Han SW, Kim SH, Lee JY, Chu CK, Yang JH, Shin HY, Nam HS, Lee BI, Heo JH. A new subtype classification of ischemic stroke based on treatment and etiologic mechanism. Eur Neurol. 2007;57:96–102.CrossRef

16.

Fure B, Wyller TB, Thommessen B. TOAST criteria applied in acute ischemic stroke. Acta Neurol Scand. 2005;112:254–8.CrossRef

17.

Ay H, Furie KL, Singhal A, Smith WS, Sorensen AG, Koroshetz WJ. An evidence-based causative classification system for acute ischemic stroke. Ann Neurol. 2005;58:688–97.CrossRef

18.

Selvarajah JR, Glaves M, Wainwright J, Jha A, Vail A, Tyrrell PJ. Classification of minor stroke: intra- and inter-observer reliability. Cerebrovasc Dis. 2009;27:209–14.CrossRef

19.

Goldstein LB, Jones MR, Matchar DB, Edwards LJ, Hoff J, Chilukuri V, Armstrong SB, Horner RD. Improving the reliability of stroke subgroup classification using the Trial of ORG 10172 in Acute Stroke Treatment (TOAST) criteria. Stroke. 2001;32:1091–8.CrossRef

20.

Meschia JF, Barrett KM, Chukwudelunzu F, Brown WM, Case LD, Kissela BM, Brown RD Jr, Brott TG, Olson TS, Rich SS, Silliman S, Worrall BB. Interobserver agreement in the Trial of Org 10172 in Acute Stroke Treatment classification of stroke based on retrospective medical record review. J Stroke Cerebrovasc Dis. 2006;15:266–72.CrossRef

21.

Nam HS, Cha MJ, Kim YD, Kim EH, Park E, Lee HS, Nam CM, Heo JH. Use of a handheld, computerized device as a decision support tool for stroke classification. Eur J Neurol. 2012;19:426–30.CrossRef

22.

Gordon DL, Bendixen BH, Adams HP Jr, Clarke W, Kappelle LJ, Woolson RF. Interphysician agreement in the diagnosis of subtypes of acute ischemic stroke: implications for clinical trials. The TOAST Investigators. Neurology. 1993;43:1021–7.CrossRef

23.

Wolf ME, Sauer T, Alonso A, Hennerici MG. Comparison of the new ASCO classification with the TOAST classification in a population with acute ischemic stroke. J Neurol. 2012;259:1284–9.CrossRef

24.

Cotter PE, Belham M, Martin PJ. Towards understanding the cause of stroke in young adults utilising a new stroke classification system (A-S-C-O). Cerebrovasc Dis. 2012;33:123–7.CrossRef

25.

Chen N, Zhou M, Wang Y, Wang H, Yang M, Guo J, Yang X, Zheng H, Zhou D, He L. Inter-rater reliability of the A-S-C-O classification system for ischemic stroke. J Clin Neurosci. 2013;20:410–2.CrossRef

26.

Marnane M, Duggan CA, Sheehan OC, Merwick A, Hannon N, Curtin D, Harris D, Williams EB, Horgan G, Kyne L, McCormack PM, Duggan J, Moore A, Crispino-O'Connell G, Kelly PJ. Stroke subtype classification to mechanism-specific and undetermined categories by TOAST, A-S-C-O, and causative classification system: direct comparison in the North Dublin population stroke study. Stroke. 2010;41:1579–86.CrossRef

27.

Arsava EM, Ballabio E, Benner T, Cole JW, Delgado-Martinez MP, Dichgans M, Fazekas F, Furie KL, Illoh K, Jood K, Kittner S, Lindgren AG, Majersik JJ, Macleod MJ, Meurer WJ, Montaner J, Olugbodi AA, Pasdar A, Redfors P, Schmidt R, Sharma P, Singhal AB, Sorensen AG, Sudlow C, Thijs V, Worrall BB, Rosand J, Ay H. The Causative Classification of Stroke system: an international reliability and optimization study. Neurology. 2010;75:1277–84.CrossRef

28.

Ay H, Benner T, Arsava EM, Furie KL, Singhal AB, Jensen MB, Ayata C, Towfighi A, Smith EE, Chong JY, Koroshetz WJ, Sorensen AG. A computerized algorithm for etiologic classification of ischemic stroke: the Causative Classification of Stroke System. Stroke. 2007;38:2979–84.CrossRef

29.

Quinn TJ, Dawson J, Walters MR, Lees KR. Functional outcome measures in contemporary stroke trials. Int J Stroke. 2009;4:200–5.CrossRef

30.

Quinn TJ, Dawson J, Walters MR, Lees KR. Reliability of the modified Rankin Scale: a systematic review. Stroke. 2009;40:3393–5.CrossRef

31.

Wilson JTL, Hareendran A, Hendry A, Potter J, Bone I, Muir KW. Reliability of the modified Rankin Scale across multiple raters: benefits of a structured interview. Stroke. 2005;36:777–81.CrossRef

32.

Hanley DF, Lane K, McBee N, Ziai W, Tuhrim S, Lees KR, Dawson J, Gandhi D, Ullman N, Mould WA, Mayo SW, Mendelow AD, Gregson B, Butcher K, Vespa P, Wright DW, Kase CS, Carhuapoma JR, Keyl PM, Diener-West M, Muschelli J, Betz JF, Thompson CB, Sugar EA, Yenokyan G, Janis S, John S, Harnof S, Lopez GA, Aldrich EF, Harrigan MR, Ansari S, Jallo J, Caron J-L, LeDoux D, Adeoye O, Zuccarello M, Adams HP Jr, Rosenblum M, Thompson RE, Awad IA. Thrombolytic removal of intraventricular haemorrhage in treatment of severe stroke: results of the randomised, multicentre, multiregion, placebo-controlled CLEAR III trial. Lancet. 2017;389:603–11.CrossRef

33.

van der Worp HB, Macleod MR, Bath PM, Demotes J, Durand-Zaleski I, Gebhardt B, Gluud C, Kollmar R, Krieger DW, Lees KR, Molina C, Montaner J, Roine RO, Petersson J, Staykov D, Szabo I, Wardlaw JM, Schwab S. EuroHYP-1: European multicenter, randomized, phase III clinical trial of therapeutic hypothermia plus best medical treatment vs. best medical treatment alone for acute ischemic stroke. Int J Stroke. 2014;9:642–5.CrossRef

34.

Fam MD, Hanley D, Stadnik A, Zeineddine HA, Girard R, Jesselson M, Cao Y, Money L, McBee N, Bistran-Hall AJ, Mould WA, Lane K, Camarata PJ, Zuccarello M, Awad IA. Surgical performance in Minimally Invasive Surgery Plus Recombinant Tissue Plasminogen Activator for Intracerebral Hemorrhage Evacuation phase III clinical trial. Neurosurgery. 2017;81:860–6.PubMedPubMedCentral

35.

Reinink H, de Jonge JC, Bath PM, van de Beek D, Berge E, Borregaard S, Ciccone A, Csiba L, Demotes J, Dippel DW, Kõrv J, Kurkowska-Jastrzebska I, Lees KR, Macleod MR, Ntaios G, Randall G, Thomalla G, van der Worp HB. PRECIOUS: PREvention of Complications to Improve OUtcome in elderly patients with acute Stroke. Rationale and design of a randomised, open, phase III, clinical trial with blinded outcome assessment. Eur Stroke J. 2018;3:291–8.CrossRef

36.

Duffy L, Gajree S, Langhorne P, Stott DJ, Quinn TJ. Reliability (inter-rater agreement) of the Barthel Index for assessment of stroke survivors: systematic review and meta-analysis. Stroke. 2013;44:462–8.CrossRef

37.

Connolly SJ, Ezekowitz MD, Yusuf S, Eikelboom J, Oldgren J, Parekh A, Pogue J, Reilly PA, Themeles E, Varrone J, Wang S, Alings M, Xavier D, Zhu J, Diaz R, Lewis BS, Darius H, Diener HC, Joyner CD, Wallentin L. Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med. 2009;361:1139–51.CrossRef

38.

Patel MR, Mahaffey KW, Garg J, Pan G, Singer DE, Hacke W, Breithardt G, Halperin JL, Hankey GJ, Piccini JP, Becker RC, Nessel CC, Paolini JF, Berkowitz SD, Fox KA, Califf RM. Rivaroxaban versus warfarin in nonvalvular atrial fibrillation. N Engl J Med. 2011;365:883–91.CrossRef

39.

Granger CB, Alexander JH, McMurray JJ, Lopes RD, Hylek EM, Hanna M, Al-Khalidi HR, Ansell J, Atar D, Avezum A, Bahit MC, Diaz R, Easton JD, Ezekowitz JA, Flaker G, Garcia D, Geraldes M, Gersh BJ, Golitsyn S, Goto S, Hermosillo AG, Hohnloser SH, Horowitz J, Mohan P, Jansky P, Lewis BS, Lopez-Sendon JL, Pais P, Parkhomenko A, Verheugt FW, Zhu J, Wallentin L. Apixaban versus warfarin in patients with atrial fibrillation. N Engl J Med. 2011;365:981–92.CrossRef

40.

Sanna T, Diener H-C, Passman RS, Di Lazzaro V, Bernstein RA, Morillo CA, Rymer MM, Thijs V, Rogers T, Beckers F, Lindborg K, Brachmann J. Cryptogenic stroke and underlying atrial fibrillation. N Engl J Med. 2014;370:2478–86.CrossRef

41.

Gladstone DJ, Spring M, Dorian P, Panzov V, Thorpe KE, Hall J, Vaid H, O'Donnell M, Laupacis A, Côté R, Sharma M, Blakely JA, Shuaib A, Hachinski V, Coutts SB, Sahlas DJ, Teal P, Yip S, Spence JD, Buck B, Verreault S, Casaubon LK, Penn A, Selchen D, Jin A, Howse D, Mehdiratta M, Boyle K, Aviv R, Kapral MK, Mamdani M. Atrial fibrillation in patients with cryptogenic stroke. N Engl J Med. 2014;370:2467–77.CrossRef

Title: Stroke aetiological classification reliability and effect on trial sample size: systematic review, meta-analysis and statistical modelling
Authors: Azmil H. Abdul-Rahim
David Alexander Dickie
Johann R. Selvarajah
Kennedy R. Lees
Terence J. Quinn
On behalf of VISTA Collaborators
Publication date: 01-12-2019
Publisher: BioMed Central
Keywords: Stroke
Anticoagulant
Published in: Trials / Issue 1/2019
Electronic ISSN: 1745-6215
DOI: https://doi.org/10.1186/s13063-019-3222-x

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Stroke aetiological classification reliability and effect on trial sample size: systematic review, meta-analysis and statistical modelling

Abstract

Background

Methods

Results

Conclusions

Trial registration

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Trial registration

Please log in to get access to this content

Other articles of this Issue 1/2019

Sensory integration therapy versus usual care for sensory processing difficulties in autism spectrum disorder in children: study protocol for a pragmatic randomised controlled trial

Managing uncertain recovery for patients nearing the end of life in hospital: a mixed-methods feasibility cluster randomised controlled trial of the AMBER care bundle

This is a platform alteration: a trial management perspective on the operational aspects of adaptive and platform and umbrella protocols

The effect of repeated remote ischemic postconditioning on infarct size in patients with an ischemic stroke (REPOST): study protocol for a randomized clinical trial

Tranexamic acid for acute gastrointestinal bleeding (the HALT-IT trial): statistical analysis plan for an international, randomised, double-blind, placebo-controlled trial

Correction to: Dexamethasone for adult patients with a symptomatic chronic subdural haematoma (Dex-CSDH) trial: study protocol for a randomised controlled trial