Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Technical advance

Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR)

Authors: Simon Lewin, Maggie Hendry, Jackie Chandler, Andrew D. Oxman, Susan Michie, Sasha Shepperd, Barnaby C. Reeves, Peter Tugwell, Karin Hannes, Eva A. Rehfuess, Vivien Welch, Joanne E. Mckenzie, Belinda Burford, Jennifer Petkovic, Laurie M. Anderson, Janet Harris, Jane Noyes

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing ‘complex interventions’, but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews.

Methods

We developed the iCAT_SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool.

Results

The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1–6 are considered ‘core’ dimensions. Dimensions 7–10 are optional and may not be useful for all interventions.

Conclusions

The iCAT_SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.
Appendix
Available only for authorised users
Literature
1.
Clark AM, Briffa TG, Thirsk L, Neubeck L, Redfern J. What football teaches us about researching complex health interventions. BMJ. 2012;345:e8316.CrossRefPubMed
2.
Petticrew M. When are complex interventions ‘complex’? When are simple interventions ‘simple’? Eur J Public Health. 2011;21(4):397–8.CrossRefPubMed
3.
Shepperd S, Lewin S, Straus S, Clarke M, Eccles MP, Fitzpatrick R, Wong G, Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Med. 2009;6(8):e1000086.CrossRefPubMedPubMedCentral
4.
Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, et al. Process evaluation of complex interventions: medical research council guidance. London: MRC Population Health Science Research Network; 2014.
5.
Richards D, Hallberg I, editors. Complex interventions in health: an overview of research methods. Oxford and New York: Routledge; 2015.
6.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council G. Developing and evaluating complex interventions: the new medical research council guidance. BMJ. 2008;337:a1655.CrossRefPubMedPubMedCentral
7.
Lewin S, Oxman A, Glenton C. Assessing healthcare interventions along the complex-simple continuum: a proposal. Abstract P100. Dublin: 14th Cochrane Colloquium; 2006.
8.
Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13(1):1.CrossRef
9.
Clark AM. What are the components of complex interventions in healthcare? Theorizing approaches to parts, powers and the whole intervention. Soc Sci Med. 2013;93:185–93.CrossRefPubMed
10.
Rogers P. Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation. 2008;14(1):29–48.CrossRef
11.
12.
Hooper R, Froud RJ, Bremner SA, Perera R, Eldridge S. Cascade diagrams for depicting complex interventions in randomised trials. BMJ. 2013;347:f6681.CrossRefPubMed
13.
14.
Noyes J, Gough D, Lewin S, Mayhew A, Michie S, Pantoja T, Petticrew M, Pottie K, Rehfuessk E, Shemilt I, et al. A research and development agenda for systematic reviews that ask complex questions about complex interventions. J Clin Epidemiol. 2013;66(11):1262–70.CrossRefPubMed
15.
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Altman DG, Barbour V, Macdonald H, Johnston M, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.CrossRefPubMed
16.
Higgins JPT, Deeks JJ. Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration; 2011. Available from www.​handbook.​cochrane.​org. Accessed 22 Apr 2017.
17.
18.
Anderson LM, Oliver SR, Michie S, Rehfuess E, Noyes J, Shemilt I. Investigating complexity in systematic reviews of interventions by using a spectrum of methods. J Clin Epidemiol. 2013;66(11):1223–9.CrossRefPubMed
19.
Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, Ogedegbe G, Orwig D, Ernst D, Czajkowski S, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23(5):443–51.CrossRefPubMed
20.
Borrelli B, Sepinwall D, Ernst D, Bellg AJ, Czajkowski S, Breger R, DeFrancesco C, Levesque C, Sharp DL, Ogedegbe G, et al. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73(5):852–60.CrossRefPubMed
21.
Pfadenhauer LM, Mozygemba K, Gerhardus A, Hofmann B, Booth A, Lysdahl KB, Tummers M, Burns J, Rehfuess EA. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes. 2015;109(2):103–14.CrossRefPubMed
22.
Sutcliffe K, Thomas J, Stokes G, Hinds K, Bangpan M. Intervention component analysis (ICA): a pragmatic approach for identifying the critical features of complex interventions. Syst Rev. 2015;4:140.CrossRefPubMedPubMedCentral
23.
Anderson LM, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, Francis D, Tugwell P. Using logic models to capture complexity in systematic reviews. Res Synth Methods. 2011;2(1):33–42.CrossRefPubMed
24.
De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C, Patel V. Theory of change: a theory-driven approach to enhance the medical research Council’s framework for complex interventions. Trials. 2014;15:267.CrossRefPubMedPubMedCentral
25.
Glenton C, Colvin CJ, Carlsen B, Swartz A, Lewin S, Noyes J, Rashidian A. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database Syst Rev. 2013;10:CD010414.
26.
Lewin S, Munabi-Babigumira S, Glenton C, Daniels K, Bosch-Capblanch X, van Wyk BE, Odgaard-Jensen J, Johansen M, Aja GN, Zwarenstein M, et al. Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev. 2010;3:CD004015.
27.
Ames HMR, Glenton C, Lewin S. Parents’ and informal caregivers’ views and experiences of communication about routine childhood vaccination: a synthesis of qualitative evidence. Cochrane Database Syst Rev. 2017;2:CD011787.PubMed
28.
Saeterdal I, Lewin S, Austvoll-Dahlgren A, Glenton C, Munabi-Babigumira S. Interventions aimed at communities to inform and/or educate about early childhood vaccination. Cochrane Database Syst Rev. 2014;11:CD010232.
29.
Petticrew M, Anderson L, Elder R, Grimshaw J, Hopkins D, Hahn R, Krause L, Kristjansson E, Mercer S, Sipe T, et al. Complex interventions and their implications for systematic reviews: a pragmatic approach. J Clin Epidemiol. 2013;66(11):1209–14.CrossRefPubMed
30.
Burford B, Lewin S, Welch V, Rehfuess E, Waters E. Assessing the applicability of findings in systematic reviews of complex interventions can enhance the utility of reviews for decision making. J Clin Epidemiol. 2013;66(11):1251–61.CrossRefPubMed
31.
Petticrew M, Rehfuess E, Noyes J, Higgins JPT, Mayhew A, Pantoja T, Shemilt I, Sowden A. Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute. J Clin Epidemiol. 2013;66(11):1230–43.CrossRefPubMed
32.
Lamb SE, Becker C, Gillespie LD, Smith JL, Finnegan S, Potter R, Pfeiffer K, Taxonomy I. Reporting of complex interventions in clinical trials: development of a taxonomy to classify and describe fall-prevention interventions. Trials. 2011;12:125.CrossRefPubMedPubMedCentral
33.
Schulz R, Czaja SJ, McKay JR, Ory MG, Belle SH. Intervention taxonomy (ITAX): describing essential features of interventions. Am J Health Behav. 2010;34(6):811–21.CrossRefPubMedPubMedCentral
34.
Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42.CrossRefPubMedPubMedCentral
35.
Michie S, Hyder N, Walia A, West R. Development of a taxonomy of behaviour change techniques used in individual behavioural support for smoking cessation. Addict Behav. 2011;36(4):315–9.CrossRefPubMed
36.
Michie S, Whittington C, Hamoudi Z, Zarnani F, Tober G, West R. Identification of behaviour change techniques to reduce excessive alcohol consumption. Addiction. 2012;107(8):1431–40.CrossRefPubMed
37.
Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, Eccles MP, Cane J, Wood CE. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1):81–95.CrossRefPubMed
38.
Hildon Z, Allwood D, Black N. Impact of format and content of visual display of data on comprehension, choice and preference: a systematic review. Int J Qual Health Care. 2012;24(1):55–64.CrossRefPubMed
39.
Glasziou P, Chalmers I, Altman DG, Bastian H, Boutron I, Brice A, Jamtvedt G, Farmer A, Ghersi D, Groves T, et al. Taking healthcare interventions from trial to practice. BMJ. 2010;341:c3852.CrossRefPubMed
40.
Squires JE, Valentine JC, Grimshaw JM. Systematic reviews of complex interventions: framing the review question. J Clin Epidemiol. 2013;66(11):1215–22.CrossRefPubMed
41.
Grimshaw JM, Freemantle N, Langhorne P, Song F. Complexity and systematic reviews: report to US congress office of technology assessment. Washington DC: Office of Technology Assessment; 1995.
42.
Bullock LF, Wells JE, Duff GB, Hornblow AR. Telephone support for pregnant women: outcome in late pregnancy. N Z Med J. 1995;108(1012):476–8.PubMed
43.
Hardy JB, Streett R. Family support and parenting education in the home: an effective extension of clinic-based preventive health care services for poor children. J Pediatr. 1989;115(6):927–31.CrossRefPubMed
44.
Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.CrossRefPubMed
Metadata
Title
Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR)
Authors
Simon Lewin
Maggie Hendry
Jackie Chandler
Andrew D. Oxman
Susan Michie
Sasha Shepperd
Barnaby C. Reeves
Peter Tugwell
Karin Hannes
Eva A. Rehfuess
Vivien Welch
Joanne E. Mckenzie
Belinda Burford
Jennifer Petkovic
Laurie M. Anderson
Janet Harris
Jane Noyes
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-017-0349-x

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue

Research Article

On comparison of net survival curves

Research article

Estimating parameters for probabilistic linkage of privacy-preserved datasets

Research article

An experimental comparison of web-push vs. paper-only survey procedures for conducting an in-depth health survey of military spouses

Research article

Impact of correlation of predictors on discrimination of risk models in development and external populations

Research article

Participant retention practices in longitudinal clinical research studies with high retention rates

Research article

The development of CHAMP: a checklist for the appraisal of moderators and predictors