Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2016

Open Access 01-12-2016 | Research article

Cohort Multiple Randomised Controlled Trials (cmRCT) design: efficient but biased? A simulation study to evaluate the feasibility of the Cluster cmRCT design

Authors: Alexander Pate, Jane Candlish, Matthew Sperrin, Tjeerd Pieter Van Staa, on behalf of GetReal Work Package 2

Published in: BMC Medical Research Methodology | Issue 1/2016

Login to get access

Abstract

Background

The Cohort Multiple Randomised Controlled Trial (cmRCT) is a newly proposed pragmatic trial design; recently several cmRCT have been initiated. This study tests the unresolved question of whether differential refusal in the intervention arm leads to bias or loss of statistical power and how to deal with this.

Methods

We conduct simulations evaluating a hypothetical cluster cmRCT in patients at risk of cardiovascular disease (CVD). To deal with refusal, we compare the analysis methods intention to treat (ITT), per protocol (PP) and two instrumental variable (IV) methods: two stage predictor substitution (2SPS) and two stage residual inclusion (2SRI) with respect to their bias and power. We vary the correlation between treatment refusal probability and the probability of experiencing the outcome to create different scenarios.

Results

We found ITT to be biased in all scenarios, PP the most biased when correlation is strong and 2SRI the least biased on average. Trials suffer a drop in power unless the refusal rate is factored into the power calculation.

Conclusions

The ITT effect in routine practice is likely to lie somewhere between the ITT and IV estimates from the trial which differ significantly depending on refusal rates. More research is needed on how refusal rates of experimental interventions correlate with refusal rates in routine practice to help answer the question of which analysis more relevant. We also recommend updating the required sample size during the trial as more information about the refusal rate is gained.
Appendix
Available only for authorised users
Literature
1.
2.
Zwarenstein M, Oxman A. Why are so few randomized trials useful, and what can we do about it? J Clin Epidemiol. 2006;59(11):1125–6.CrossRefPubMed
3.
Zwarenstein M, Treweek S. What kind of randomised trials do patients and clinicians need? Evid Based Med. 2009;14(4):101–3.CrossRefPubMed
4.
Luce BR, Kramer JM, Goodman SN, Connor JT, Tunis S, Whicher D, Schwartz JS. Medicine and Public Issues Annals of Internal Medicine Rethinking Randomized Clinical Trials for Comparative Effectiveness Research: The Need for Transformational Change. Ann Intern Med. 2009;151:206–209.CrossRefPubMed
5.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62(5):464–75.CrossRefPubMed
6.
van Staa T-P, Goldacre B, Gulliford M, Cassell J, Pirmohamed M, Taweel A, Dalaney B, Smeeth L. Pragmatic randomised trials using routine electronic health records: putting them to the test. BMJ. 2012;344(1):e55.CrossRefPubMedPubMedCentral
7.
Purgato M, Barbui C, Stroup S, Adams C. Pragmatic design in randomized controlled trials. Psychol Med. 2014;45(02):225–30.CrossRefPubMed
8.
Relton C, Torgerson D, O’Cathain A, Nicholl J. Rethinking pragmatic randomised controlled trials: introducing the ‘cohort multiple randomised controlled trial’ design. BMJ. 2010;340:c1066.CrossRefPubMed
9.
Burbach JM, Verkooijen HM, Intven M, Kleijnen J-PJ, Bosman ME, Raaymakers BW, van Grevenstein WM, Koopman M, Seravalli E, van Asselen B, Reerink O. RandomizEd controlled trial for pre-operAtive dose-escaLation BOOST in locally advanced rectal cancer (RECTAL BOOST study): study protocol for a randomized controlled trial. Trials. 2015;16(1):1–8.CrossRef
10.
Mitchell N, Hewitt C, Adamson J, Parrott S, Torgerson D, Ekers D, Holmes J, Lester H, McMillan D, Richards D, Spilsbury K, Godfrey C, Gilbody S. A randomised evaluation of CollAborative care and active surveillance for Screen-Positive EldeRs with sub-threshold depression (CASPER): study protocol for a randomized controlled trial. Trials. 2011;12(1):225.CrossRefPubMedPubMedCentral
11.
Kwakkenbos L, Jewett LR, Baron M, Bartlett SJ, Furst D, Gottesman K, Khanna D, Malcarne VL, Mayes MD, Mouthon L, Poiraudeau S, Sauve M, Nielson WR, Poole JL, Assassi S, Boutron I, Ells C, van den Ende CH, Hudson M, Impens A, Körner A, Leite C, Costa Maia A, Mendelson C, Pope J, Steele RJ, Suarez-Almazor ME, Ahmed S, Coronado-Montoya S, Delisle VC, Gholizadeh S, Jang Y, Levis B, Milette K, Mills SD, Razykov I, Fox RS, Thombs BD. The Scleroderma Patient-centered Intervention Network (SPIN) Cohort: protocol for a cohort multiple randomised controlled trial (cmRCT) design to support trials of psychosocial and rehabilitation interventions in a rare disease context. BMJ Open. 2013;3(8):1–11.CrossRef
12.
Uher R, Cumby J, Mackenzie LE, Morash-conway J, Glover JM, Aylott A, Propper L, Abidi S, Bagnell A, Pavlova B, Hajek T, Lovas D, Pajer K, Gardner W, Levy A, Alda M. A familial risk enriched cohort as a platform for testing early interventions to prevent severe mental illness. BMC Psychiatry. 2014;14(1):344.CrossRefPubMedPubMedCentral
13.
Young-Afat HMVDA, van Gils CH, van den Bongard HJ, van Vulpen M. Introducing the new ‘cohort multiple Randomised Controlled Trial’ design for evaluation of interventions for breast cancer: The UMBRELLA study. Eur J Cancer. 2014;50:S210–9.
14.
15.
Bower P. ISRCTN - ISRCTN12286422: CLASSIC Proactive Telephone Coaching and Tailored Support (PROTECTS). 2014.
16.
17.
van Staa TP, Dyson L, McCann G, Padmanabhan S, Belatri R, Goldacre B, Cassell J, Pirmohamed M, Torgerson D, Ronaldson S, Adamson J, Taweel A, Delaney B, Mahmood S, Baracaia S, Round T, Fox R, Hunter T, Gulliford M, Smeeth L. The opportunities and challenges of pragmatic point-of-care randomised trials using routinely collected electronic records: Evaluations of two exemplar trials. Health Technol Assess (Rockv). 2014;18(43):1–146.
18.
Angrist JD, Imbens GW, Rubin DB, Association S, Jun N. Identification of Causal Effects Using Instrumental Variables. J Am Stat Assoc. 1996;91(434):444–55.CrossRef
19.
Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol. 2000;29(4):722–9.CrossRefPubMed
20.
Frangakis BCE, Rubin DB. Addressing complications of intention-to-treat analysis in the combined presence of all-or-none treatment-noncompliance and subsequent missing outcomes. Biometrika. 1999;86(2):365–379.CrossRef
21.
Dunn G, Maracy M, Tomenson B. Estimating treatment effects from randomized clinical trials with noncompliance and loss to follow-up: the role of instrumental variable methods. Stat Methods Med Res. 2005;14(4):369–95.CrossRefPubMed
22.
Wu J, Zhu S, Yao GL, Mohammed MA, Marshall T. Patient Factors Influencing the Prescribing of Lipid Lowering Drugs for Primary Prevention of Cardiovascular Disease in UK General Practice: A National Retrospective Cohort Study. PLoS One. 2013;8(7):1–10.
23.
24.
Armijo-Olivo S, Warren S, Magee D. Intention to treat analysis, compliance, drop-outs and how to deal with missing data in clinical research: a review. Phys Ther Rev. 2009;14(1):36–49.CrossRef
25.
Terza JV, Basu A, Rathouz PJ. Two-stage residual inclusion estimation: Addressing endogeneity in health econometric modeling. J Health Econ. 2008;27(3):531–43.CrossRefPubMed
26.
Efron BYB. Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods. Biometrika. 1981;68(3):589–599.CrossRef
27.
Groenwold RHH, Uddin MJ, Roes KCB, De Boer A, Martin E, Gatto NM, Klungel OH. Instrumental variable analysis in randomized trials with non- compliance and observational pharmacoepidemiologic studies. OA Epidemiology. 2014;2(1):9.
28.
Hernán MA, Robins JM. Instruments for causal inference: an epidemiologist’s dream? Epidemiology. 2006;17(4):360–72.CrossRefPubMed
29.
30.
Townsend N, Williams J, Bhatnagar P, Wickramasinghe K, Rayner M (2014). Cardiovascular disease statistics, 2014. British Heart Foundation: London.
31.
Eldridge SM, Ashby D, Kerry S. Sample size for cluster randomized trials: Effect of coefficient of variation of cluster size and analysis method. Int J Epidemiol. 2006;35:1292–300.CrossRefPubMed
32.
Kerry SM, Martin Bland J. Unequal cluster sizes for trials in English and Welsh general practice: Implications for sample size calculations. Stat Med. 2001;20:377–90.CrossRefPubMed
33.
Arnold BF, Hogan DR, Colford JMJ, Hubbard AE. Simulation methods to estimate design power: an overview for applied research. BMC Med Res Methodol. 2011;11:94.CrossRefPubMedPubMedCentral
34.
O’Quigley J, Stare J. Proportional hazards models with frailties and random effects. Stat Med. 2002;21(21):3219–33.CrossRefPubMed
35.
Glidden DV, Vittinghoff E. Modelling clustered survival data from multicentre clinical trials. Stat Med. 2004;23(3):369–88.CrossRefPubMed
36.
Greenland S, Robins J, Pearl J. Confounding and collapsibility in causal inference. Stat Sci. 1999;14:29–46.CrossRef
37.
Adamson J, Cockayne S, Puffer S, Torgerson DJ. Review of randomised trials using the post-randomised consent (Zelen’s) design. Contemp Clin Trials. 2006;27:305–19.CrossRefPubMed
38.
39.
40.
Ye C, Beyene J, Browne G, Thabane L. Estimating treatment effects in randomised controlled trials with non-compliance: a simulation study. BMJ Open. 2014;4:e005362. CrossRefPubMedPubMedCentral
41.
Goldsmith LP, Lewis SW, Dunn G, Bentall RP. Psychological treatments for early psychosis can be beneficial or harmful, depending on the therapeutic alliance: an instrumental variable analysis. Psychol Med. 2015;45(11):2365–73.CrossRefPubMedPubMedCentral
42.
Jago R, Edwards MJ, Sebire SJ, Tomkinson K, Bird EL, Banfield K, May T, Kesten JM, Cooper AR, Powell JE, Blair PS. Effect and cost of an after-school dance programme on the physical activity of 11–12 year old girls: The Bristol Girls Dance Project, a school-based cluster randomised controlled trial. Int J Behav Nutr Phys Act. 2015;12(1):128.CrossRefPubMedPubMedCentral
43.
Halpern SD, French B, Small DS, Saulsgiver K, Harhay MO, Audrain-McGovern J, Loewenstein G, Brennan TA, Asch DA, Volpp KG. Randomized Trial of Four Financial-Incentive Programs for Smoking Cessation. N Engl J Med. 2015;372(22):2108–17.CrossRefPubMedPubMedCentral
44.
van der Velden JM, Verkooijen HM, Young-Afat DA, Burbach JPM, van Vulpen M, Relton C, van Gils CH, May AM, Groenwold RHH. The cohort multiple randomized controlled trial design: a valid and efficient alternative to pragmatic trials?. Int J Epidemiol. 2016. doi:10.​1093/​ije/​dyw050.
45.
Lakatos E. Sample size determination in clinical trials with time-dependent rates of losses and noncompliance. Control Clin Trials. 1986;7(3):189–99.CrossRefPubMed
46.
47.
Jiang Q, Snapinn S, Iglewicz B. Calculation of sample size in survival trials: the impact of informative noncompliance. Biometrics. 2004;60(3):800–6.CrossRefPubMed
48.
Wan F, Small D, Bekelman JE, Mitra N. Bias in estimating the causal hazard ratio when using two-stage instrumental variable methods. Stat Med. 2015;34(14):2235–2265.CrossRefPubMedPubMedCentral
49.
Metadata
Title
Cohort Multiple Randomised Controlled Trials (cmRCT) design: efficient but biased? A simulation study to evaluate the feasibility of the Cluster cmRCT design
Authors
Alexander Pate
Jane Candlish
Matthew Sperrin
Tjeerd Pieter Van Staa
on behalf of GetReal Work Package 2
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0208-1

Other articles of this Issue 1/2016

BMC Medical Research Methodology 1/2016 Go to the issue

Research article

Does size really matter? A sensitivity analysis of number of seeds in a respondent-driven sampling study of gay, bisexual and other men who have sex with men in Vancouver, Canada

Research article

Predictor characteristics necessary for building a clinically useful risk prediction model: a simulation study

Publisher's Erratum

Erratum to: Statistical methodology for age-adjustment of the GH-2000 score detecting growth hormone misuse

Technical advance

Comparing survival curves based on medians

Research article

A method for sensitivity analysis to assess the effects of measurement error in multiple exposure variables using external validation data

Research article

A simulation study on matched case-control designs in the perspective of causal diagrams