Skip to main content
Top
Published in: Trials 1/2017

Open Access 01-12-2017 | Methodology

Analysis of cluster randomised stepped wedge trials with repeated cross-sectional samples

Authors: Karla Hemming, Monica Taljaard, Andrew Forbes

Published in: Trials | Issue 1/2017

Login to get access

Abstract

Background

The stepped wedge cluster randomised trial (SW-CRT) is increasingly being used to evaluate policy or service delivery interventions. However, there is a dearth of trials literature addressing analytical approaches to the SW-CRT. Perhaps as a result, a significant number of published trials have major methodological shortcomings, including failure to adjust for secular trends at the analysis stage. Furthermore, the commonly used analytical framework proposed by Hussey and Hughes makes several assumptions.

Methods

We highlight the assumptions implicit in the basic SW-CRT analytical model proposed by Hussey and Hughes. We consider how simple modifications of the basic model, using both random and fixed effects, can be used to accommodate deviations from the underlying assumptions. We consider the implications of these modifications for the intracluster correlation coefficients. In a case study, the importance of adjusting for the secular trend is illustrated.

Results

The basic SW-CRT model includes a fixed effect for time, implying a common underlying secular trend across steps and clusters. It also includes a single term for treatment, implying a constant shift in this trend under the treatment. When these assumptions are not realistic, simple modifications can be implemented to allow the secular trend to vary across clusters and the treatment effect to vary across clusters or time. In our case study, the naïve treatment effect estimate (adjusted for clustering but unadjusted for time) suggests a beneficial effect. However, after adjusting for the underlying secular trend, we demonstrate a reversal of the treatment effect.

Conclusion

Due to the inherent confounding of the treatment effect with time, analysis of a SW-CRT should always account for secular trends or risk-biased estimates of the treatment effect. Furthermore, the basic model proposed by Hussey and Hughes makes a number of important assumptions. Consideration needs to be given to the appropriate model choice at the analysis stage. We provide a Stata code to implement the proposed analyses in the illustrative case study.
Appendix
Available only for authorised users
Literature
2.
go back to reference Mdege ND, Man MS, Taylor (nee Brown) CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64(9):936–48.CrossRefPubMed Mdege ND, Man MS, Taylor (nee Brown) CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64(9):936–48.CrossRefPubMed
4.
go back to reference The Gambia Hepatitis Study Group. The Gambia Hepatitis Intervention Study. Cancer Res. 1987;47(21):5782–7. The Gambia Hepatitis Study Group. The Gambia Hepatitis Intervention Study. Cancer Res. 1987;47(21):5782–7.
5.
go back to reference King G, Gakidou E, Ravishankar N, Moore RT, Lakin J, Vargas M, et al. A ‘politically robust’ experimental design for public policy evaluation, with application to the Mexican universal health insurance program. J Policy Anal Manage. 2007;26:479–506.CrossRefPubMed King G, Gakidou E, Ravishankar N, Moore RT, Lakin J, Vargas M, et al. A ‘politically robust’ experimental design for public policy evaluation, with application to the Mexican universal health insurance program. J Policy Anal Manage. 2007;26:479–506.CrossRefPubMed
6.
go back to reference Leontjevas R, Gerritsen DL, Smalbrugge M, Teerenstra S, Vernooij-Dassen MJ, Koopmans RT. A structural multidisciplinary approach to depression management in nursing-home residents: a multicentre, stepped-wedge cluster-randomised trial. Lancet. 2013;381:2255–64.CrossRefPubMed Leontjevas R, Gerritsen DL, Smalbrugge M, Teerenstra S, Vernooij-Dassen MJ, Koopmans RT. A structural multidisciplinary approach to depression management in nursing-home residents: a multicentre, stepped-wedge cluster-randomised trial. Lancet. 2013;381:2255–64.CrossRefPubMed
7.
go back to reference Pearse R, Pedan C, Bion J, Faiz O, Holt P, Girling A, et al. HS&DR—12/5005/10. Enhanced Peri-Operative Care for High-risk patients (EPOCH) trial: a stepped wedge cluster randomised trial of a quality improvement intervention for patients undergoing emergency laparotomy [protocol]. 2013. www.nets.nihr.ac.uk/projects/hsdr/12500510. Pearse R, Pedan C, Bion J, Faiz O, Holt P, Girling A, et al. HS&DR—12/5005/10. Enhanced Peri-Operative Care for High-risk patients (EPOCH) trial: a stepped wedge cluster randomised trial of a quality improvement intervention for patients undergoing emergency laparotomy [protocol]. 2013. www.​nets.​nihr.​ac.​uk/​projects/​hsdr/​12500510.
8.
go back to reference Davey C, Hargreaves J, Thompson JA, Copas AJ, Beard E, Lewis JJ, Fielding KL. Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials. 2015;16:358. doi:10.1186/s13063-015-0838-3. PubMed PMID: 26278667, PubMed Central PMCID: PMC4538923.CrossRefPubMedPubMedCentral Davey C, Hargreaves J, Thompson JA, Copas AJ, Beard E, Lewis JJ, Fielding KL. Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials. 2015;16:358. doi:10.​1186/​s13063-015-0838-3. PubMed PMID: 26278667, PubMed Central PMCID: PMC4538923.CrossRefPubMedPubMedCentral
10.
go back to reference Barker D, et al. Stepped wedge cluster randomised trials: a review of the statistical methodology used and available. BMC Med Res Methodol. 2016;16:69.CrossRefPubMedPubMedCentral Barker D, et al. Stepped wedge cluster randomised trials: a review of the statistical methodology used and available. BMC Med Res Methodol. 2016;16:69.CrossRefPubMedPubMedCentral
12.
go back to reference Ivers NM, Taljaard M, Dixon S, Bennett C, McRae A, Taleban J, Skea Z, Brehaut JC, Boruch RF, Eccles MP, Grimshaw JM, Weijer C, Zwarenstein M, Donner A. Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000-8. BMJ. 2011;343:d5886. doi:10.1136/bmj.d5886. PubMed PMID: 21948873, PubMed Central PMCID: PMC3180203, Review.CrossRefPubMedPubMedCentral Ivers NM, Taljaard M, Dixon S, Bennett C, McRae A, Taleban J, Skea Z, Brehaut JC, Boruch RF, Eccles MP, Grimshaw JM, Weijer C, Zwarenstein M, Donner A. Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000-8. BMJ. 2011;343:d5886. doi:10.​1136/​bmj.​d5886. PubMed PMID: 21948873, PubMed Central PMCID: PMC3180203, Review.CrossRefPubMedPubMedCentral
13.
go back to reference Rutterford C, Taljaard M, Dixon S, Copas A, Eldridge S. Reporting and methodological quality of sample size calculations in cluster randomised trials could be improved: a review. J Clin Epidemiol. In Press. Rutterford C, Taljaard M, Dixon S, Copas A, Eldridge S. Reporting and methodological quality of sample size calculations in cluster randomised trials could be improved: a review. J Clin Epidemiol. In Press.
14.
15.
go back to reference Heo M, Kim N, Rinke ML, Wylie-Rosett J. Sample size determinations for stepped-wedge clinical trials from a three-level data hierarchy perspective. Stat Methods Med Res. 2016. [Epub ahead of print]. Heo M, Kim N, Rinke ML, Wylie-Rosett J. Sample size determinations for stepped-wedge clinical trials from a three-level data hierarchy perspective. Stat Methods Med Res. 2016. [Epub ahead of print].
16.
go back to reference Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–191. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–191.
17.
go back to reference Parienti JJ, Kuss O. Cluster-crossover design: a method for limiting clusters level effect in community-intervention studies. Contemp Clin Trials. 2007;28(3):316–23. Epub 2006 Oct 6.CrossRefPubMed Parienti JJ, Kuss O. Cluster-crossover design: a method for limiting clusters level effect in community-intervention studies. Contemp Clin Trials. 2007;28(3):316–23. Epub 2006 Oct 6.CrossRefPubMed
19.
go back to reference S Kenyon, S Dann, L Ingram, P Clarke, A Hogan, D Jenkinson, K Hemming. Evaluation of bespoke training to midwifery teams to increase membrane sweeping to reduce induction of labour: a stepped wedge cluster randomised trial. Submitted for publication. S Kenyon, S Dann, L Ingram, P Clarke, A Hogan, D Jenkinson, K Hemming. Evaluation of bespoke training to midwifery teams to increase membrane sweeping to reduce induction of labour: a stepped wedge cluster randomised trial. Submitted for publication.
20.
go back to reference Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeated cross-sectional binary measurements. Stat Med. 2001;20(3):417–33.7. 28(2):182-91.CrossRefPubMed Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeated cross-sectional binary measurements. Stat Med. 2001;20(3):417–33.7. 28(2):182-91.CrossRefPubMed
21.
go back to reference Turner RM, White IR, Croudace T, PIP Study Group. Analysis of cluster randomized cross-over trial data: a comparison of methods. Stat Med. 2007;26(2):274–89.CrossRefPubMed Turner RM, White IR, Croudace T, PIP Study Group. Analysis of cluster randomized cross-over trial data: a comparison of methods. Stat Med. 2007;26(2):274–89.CrossRefPubMed
23.
go back to reference Girling AJ, Hemming K. Statistical efficiency and optimal design for stepped cluster studies under linear mixed effects models. Stat Med. 2016. doi:10.1002/sim.6850 [Epub ahead of print]. Girling AJ, Hemming K. Statistical efficiency and optimal design for stepped cluster studies under linear mixed effects models. Stat Med. 2016. doi:10.1002/sim.6850 [Epub ahead of print].
24.
25.
go back to reference Taljaard M, Teerenstra S, Ivers NM, Fergusson DA. Substantial risks associated with few clusters in cluster randomized and stepped wedge designs. Clin Trials. 2016. [Epub ahead of print]. Taljaard M, Teerenstra S, Ivers NM, Fergusson DA. Substantial risks associated with few clusters in cluster randomized and stepped wedge designs. Clin Trials. 2016. [Epub ahead of print].
26.
go back to reference Brown H, Prescott R. Generalised linear mixed models, Applied mixed models. In: Medicine. 3rd ed. Chichester: Wiley; 2014. doi:10.1002/9781118778210.ch3. Brown H, Prescott R. Generalised linear mixed models, Applied mixed models. In: Medicine. 3rd ed. Chichester: Wiley; 2014. doi:10.1002/9781118778210.ch3.
28.
go back to reference Eldridge SM, Ukoumunne OC, Carlin JB. The intra-cluster correlation coefficient in cluster randomized trials: a review of definitions. Int Stat Rev. 2009;77(3):378–94.CrossRef Eldridge SM, Ukoumunne OC, Carlin JB. The intra-cluster correlation coefficient in cluster randomized trials: a review of definitions. Int Stat Rev. 2009;77(3):378–94.CrossRef
29.
go back to reference van den Broek IV, van Bergen JE, Brouwers EE, Fennema JS, Götz HM, Hoebe CJ, Koekenbier RH, Kretzschmar M, Over EA, Schmid BV, Pars LL, van Ravesteijn SM, van der Sande MA, de Wit GA, Low N, Op de Coul EL. Effectiveness of yearly, register based screening for chlamydia in the Netherlands: controlled trial with randomised stepped wedge implementation. BMJ. 2012;345:e4316. doi:10.1136/bmj.e4316. PubMed PMID: 22767614; PubMed Central PMCID: PMC3390168.CrossRefPubMedPubMedCentral van den Broek IV, van Bergen JE, Brouwers EE, Fennema JS, Götz HM, Hoebe CJ, Koekenbier RH, Kretzschmar M, Over EA, Schmid BV, Pars LL, van Ravesteijn SM, van der Sande MA, de Wit GA, Low N, Op de Coul EL. Effectiveness of yearly, register based screening for chlamydia in the Netherlands: controlled trial with randomised stepped wedge implementation. BMJ. 2012;345:e4316. doi:10.​1136/​bmj.​e4316. PubMed PMID: 22767614; PubMed Central PMCID: PMC3390168.CrossRefPubMedPubMedCentral
31.
go back to reference Rothman KJ, Greenland S, Lash TL. Modern epidemiology. 3rd ed. Philadelphia: Lippincott, Williams & Wilkins; 2008. Rothman KJ, Greenland S, Lash TL. Modern epidemiology. 3rd ed. Philadelphia: Lippincott, Williams & Wilkins; 2008.
33.
go back to reference Fitzmaurice GM, Laird N, Ware JH. Applied longitudinal analysis. London: Wiley; 2004. Fitzmaurice GM, Laird N, Ware JH. Applied longitudinal analysis. London: Wiley; 2004.
34.
go back to reference Twisk JWR, Hoogendijk EO, Zwijsen SA, de Boer MR. Different methods to analyze stepped wedge trial designs revealed different aspects of intervention effects. J Clin Epidemiol. 2016;72:75–83.CrossRefPubMed Twisk JWR, Hoogendijk EO, Zwijsen SA, de Boer MR. Different methods to analyze stepped wedge trial designs revealed different aspects of intervention effects. J Clin Epidemiol. 2016;72:75–83.CrossRefPubMed
36.
go back to reference Morgan KE, Forbes AB, Keogh RH, Jairath V, Kahan BC. Choosing appropriate analysis methods for cluster randomised cross-over trials with a binary outcome. Stat Med. 2017;36(2):318–33. doi:10.1002/sim.7137.CrossRefPubMed Morgan KE, Forbes AB, Keogh RH, Jairath V, Kahan BC. Choosing appropriate analysis methods for cluster randomised cross-over trials with a binary outcome. Stat Med. 2017;36(2):318–33. doi:10.​1002/​sim.​7137.CrossRefPubMed
37.
go back to reference Hubbard AE, Ahern J, Fleischer NL, Van der Laan M, Lippman SA, Jewell N, Bruckner T, Satariano WA. To GEE or not to GEE: comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology. 2010;21(4):467–74. doi:10.1097/EDE.0b013e3181caeb90.CrossRefPubMed Hubbard AE, Ahern J, Fleischer NL, Van der Laan M, Lippman SA, Jewell N, Bruckner T, Satariano WA. To GEE or not to GEE: comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology. 2010;21(4):467–74. doi:10.​1097/​EDE.​0b013e3181caeb90​.CrossRefPubMed
38.
go back to reference Scott JM, deCamp A, Juraska M, Fay MP, Gilbert PB. Finite-sample corrected generalized estimating equation of population average treatment effects in stepped wedge cluster randomized trials. Stat Methods Med Res. 2014. [Epub ahead of print] PubMed PMID: 25267551; PubMed Central PMCID: PMC4411204. Scott JM, deCamp A, Juraska M, Fay MP, Gilbert PB. Finite-sample corrected generalized estimating equation of population average treatment effects in stepped wedge cluster randomized trials. Stat Methods Med Res. 2014. [Epub ahead of print] PubMed PMID: 25267551; PubMed Central PMCID: PMC4411204.
39.
go back to reference Donner A, Klar N. Design and analysis of cluster randomization trials in health research. London: Arnold; 2000. 0-340-69153-0. Donner A, Klar N. Design and analysis of cluster randomization trials in health research. London: Arnold; 2000. 0-340-69153-0.
40.
go back to reference Hayes RJ, Moulton LH. Cluster randomised trials. London: Chapman & Hall/CRC Press. 2009. Hayes RJ, Moulton LH. Cluster randomised trials. London: Chapman & Hall/CRC Press. 2009.
Metadata
Title
Analysis of cluster randomised stepped wedge trials with repeated cross-sectional samples
Authors
Karla Hemming
Monica Taljaard
Andrew Forbes
Publication date
01-12-2017
Publisher
BioMed Central
Published in
Trials / Issue 1/2017
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/s13063-017-1833-7

Other articles of this Issue 1/2017

Trials 1/2017 Go to the issue