Top

BMC Public Health

Published in:

Open Access 01-12-2018 | Research article

Pathways leading to success and non-success: a process evaluation of a cluster randomized physical activity health promotion program applying fuzzy-set qualitative comparative analysis

Authors: Christina Kien, Ludwig Grillich, Barbara Nussbaumer-Streit, Rudolf Schoberberger

Published in: BMC Public Health | Issue 1/2018

Abstract

Background

Health promotion programs can only lead to improvements in health outcomes if they are effectively implemented. However, most studies assessing implementation success focus on only one condition, although more conditions influence this process. Therefore, evidence is scarce on what conditions play a role in successful implementation and how they interact. Hence, we aimed to identify which combinations of teacher and implementation process characteristics affected the emotional and social school experience (SCE) of pupils participating in a school-based health promotion program.

Methods

This study was part of an effectiveness and process evaluation including 24 intervention and 27 control classes. We used fuzzy-set qualitative comparative analysis (fsQCA) to identify combinations of conditions that were associated with either an increase or no increase in the outcome SCE in comparison to the control group at 20 months post intervention. We deductively selected five conditions based on the Consolidated Framework for Implementation Research: teachers’ perceived self-efficacy, teachers’ expectations of the benefits of the intervention, teachers’ previous knowledge about the intervention, dosage of physical activity breaks, and quality of the implementation.

Results

We identified five different pathways that led to no increase in the pupils’ outcome (parameters of fit: consistency 94%, coverage 66%). The combination of an unsatisfying quality of implementing the intervention and a low previous knowledge about the intervention showed the highest empirical relevance. Similarly, fewer physical activity breaks in combination with other conditions impeded the program’s success. Furthermore, we identified two different pathways characterizing ways to success (consistency: 81%, coverage: 52%). The most relevant combination was good quality implementation of physical activity breaks, implemented by teachers with a high self-efficacy, and a good previous knowledge about the intervention.

Conclusions

QCA has potential for an in-depth analysis of complex interventions as it can rely on small to medium sample sizes and analyze pathways to success and non-success separately. The investigated program can be improved by considering the following suggestions: The quality of the implementation process should be monitored during the implementation phase, and regular feedback loops and learning opportunities for teachers should accompany a program. Clear recommendations regarding the dosage should be established.

Trial registration

German register of clinical studies: DRKS00000622. Retrospectively registered: December 3, 2010, (http://www.drks.de/drks_web/setLocale_EN.do). Approved by the Ethics Committee of Lower Austria (GS4-EK-4/107–2010).

Available only for authorised users

Janssen I, Leblanc AG. Systematic review of the health benefits of physical activity and fitness in school-aged children and youth. Int J Behav Nutr Phys Act. 2010. https://doi.org/10.1186/1479-5868-7-40.

Lubans D, Richards J, Hillman C, Faulkner G, Beauchamp M, Nilsson M, et al. Physical activity for cognitive and mental health in youth: a systematic review of mechanisms. Pediatrics. 2016. https://doi.org/10.1542/peds.2016-1642.

Ahn JV, Sera F, Cummins S, Flouri E. Associations between objectively measured physical activity and later mental health outcomes in children: findings from the UK millennium cohort study. J Epidemiol Community Health. 2018. https://doi.org/10.1136/jech-2017-209455.

Smedegaard S, Brondeel R, Christiansen LB, Skovgaard T. What happened in the 'Move for well-being in School': a process evaluation of a cluster randomized physical activity intervention using the RE-AIM framework. Int J Behav Nutr Phys Act. 2017. https://doi.org/10.1186/s12966-017-0614-8.

Liu M, Wu L, Ming Q. How does physical activity intervention improve self-esteem and self-concept in children and adolescents? Evidence from a meta-analysis. PLoS One. 2015. https://doi.org/10.1371/journal.pone.0134804.

Martin R, Murtagh EM. Effect of active lessons on physical activity, academic, and health outcomes: a systematic review. Res Q Exerc Sports. 2017. https://doi.org/10.1080/02701367.2017.1294244.

Williams HG, Pfeiffer KA, O'Neill JR, Dowda M, McIver KL, Brown WH, et al. Motor skill performance and physical activity in preschool children. Obesity (Silver Spring). 2008. https://doi.org/10.1038/oby.2008.214.

Gray CE, Larouche R, Barnes JD, Colley RC, Bonne JC, Arthur M, et al. Are we driving our kids to unhealthy habits? Results of the active healthy kids Canada 2013 report card on physical activity for children and youth. Int J Environ Res Public Health. 2014. https://doi.org/10.3390/ijerph110606009.

Griffiths LJ, Cortina-Borja M, Sera F, Pouliou T, Geraci M, Rich C, et al. How active are our children? Findings from the millennium cohort study. BMJ Open. 2013. https://doi.org/10.1136/bmjopen-2013-002893.

10.

Hoffmann B, Kettner S, Wirt T, Wartha O, Hermeling L, Steinacker JM, et al. Sedentary time among primary school children in south-West Germany: amounts and correlates. Arch Public Health. 2017. https://doi.org/10.1186/s13690-017-0230-8.

11.

Lloyd J, Creanor S, Price L, Abraham C, Dean S, Green C, et al. Trial baseline characteristics of a cluster randomised controlled trial of a school-located obesity prevention programme; the healthy lifestyles Programme (HeLP) trial. BMC Public Health. 2017. https://doi.org/10.1186/s12889-017-4196-9.

12.

Titze S, Ring-Dimitriou S, Schober PH, Halbwachs C, Samitz G, Miko HC, et al. Österreichische Empfehlungen für gesundheitswirksame Bewegung. Vienna: Bundesministerium für Gesundheit, Gesundheit Österreich GmbH, Geschäftsbereich Fonds Gesundes Österreich; 2010. Available from: https://www.bmgf.gv.at/cms/home/attachments/1/6/5/CH1357/CMS1405438552027/oe_empfehlung_gesundheitswirksamebewegung.pdf. Accessed: 1 June 2018.

13.

World Health Organization (WHO). Global Recommendations on physical activity for health: World Health Organization (WHO); 2010. Available from: http://www.who.int/dietphysicalactivity/factsheet_young_people/en/. Accessed: 1 June 2018.

14.

van Sluijs EM, Kriemler S. Reflections on physical activity intervention research in young people - dos, don'ts, and critical thoughts. Int J Behav Nutr Phys Act. 2016. https://doi.org/10.1186/s12966-016-0348-z.

15.

Beets MW, Okely A, Weaver RG, Webster C, Lubans D, Brusseau T, et al. The theory of expanded, extended, and enhanced opportunities for youth physical activity promotion. Int J Behav Nutr Phys Act. 2016. https://doi.org/10.1186/s12966-016-0442-2.

16.

Naylor PJ, McKay HA. Prevention in the first place: schools a setting for action on physical inactivity. Br J Sports Med. 2009. https://doi.org/10.1136/bjsm.2008.053447.

17.

Watson A, Timperio A, Brown H, Best K, Hesketh KD. Effect of classroom-based physical activity interventions on academic and physical activity outcomes: a systematic review and meta-analysis. Int J Behav Nutr Phys Act. 2017. https://doi.org/10.1186/s12966-017-0569-9.

18.

Daly-Smith AJ, Zwolinsky S, McKenna J, Tomporowski PD, Defeyter MA, Manley A. Systematic review of acute physically active learning and classroom movement breaks on children's physical activity, cognition, academic performance and classroom behaviour: understanding critical design features. BMJ Open Sport Exerc Med. 2018. https://doi.org/10.1136/bmjsem-2018-000341.

19.

Norris E, Shelton N, Dunsmuir S, Duke-Williams O, Stamatakis E. Physically active lessons as physical activity and educational interventions: a systematic review of methods and results. Prev Med. 2015. https://doi.org/10.1016/j.ypmed.2014.12.027.

20.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015. https://doi.org/10.1136/bmj.h1258.

21.

Fixsen D, Naoom S, Blase K, Friedman RM, Wallace F. Implementation Research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005.

22.

Keshavarz N, Nutbeam D, Rowling L, Khavarpour F. Schools as social complex adaptive systems: a new way to understand the challenges of introducing the health promoting schools concept. Soc Sci Med. 2010. https://doi.org/10.1016/j.socscimed.2010.01.034.

23.

Turunen H, Sormunen M, Jourdan D, von Seelen J, Buijs G. Health promoting schools-a complex approach and a major means to health improvement. Health Promot Int. 2017. https://doi.org/10.1093/heapro/dax001.

24.

Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008. https://doi.org/10.1007/s10464-008-9165-0.

25.

Greenhalgh T, Kristjansson E, Robinson V. Realist review to understand the efficacy of school feeding programmes. BMJ. 2007. https://doi.org/10.1136/bmj.39359.525174.AD.

26.

Carlson JA, Engelberg JK, Cain KL, Conway TL, Geremia C, Bonilla E, et al. Contextual factors related to implementation of classroom physical activity breaks. Transl Behav Med. 2017. https://doi.org/10.1007/s13142-017-0509-x.

27.

Langford R, Bonell C, Jones H, Campbell R. Obesity prevention and the health promoting schools framework: essential components and barriers to success. Int J Behav Nutr Phys Act. 2015. https://doi.org/10.1186/s12966-015-0167-7.

28.

Naylor PJ, Nettlefold L, Race D, Hoy C, Ashe MC, Wharf Higgins J, et al. Implementation of school based physical activity interventions: a systematic review. Prev Med. 2015. https://doi.org/10.1016/j.ypmed.2014.12.034.

29.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009. https://doi.org/10.1186/1748-5908-4-50.

30.

Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014. https://doi.org/10.1186/s13012-014-0118-8.

31.

Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A glossary for dissemination and implementation research in health. J Public Health Manag Pract. 2008. https://doi.org/10.1097/01.PHH.0000311888.06252.bb.

32.

Grillich L, Kien C, Takuya Y, Weber M, Gartlehner G. Effectiveness evaluation of a health promotion programme in primary schools: a cluster randomised controlled trial. BMC Public Health. 2016. https://doi.org/10.1186/s12889-016-3330-4.

33.

Spencer B, Broesskamp-Stone U, Ruckstuhl B, Ackermann G, Spoerri A, Cloetta B. Modelling the results of health promotion activities in Switzerland: development of the Swiss model for outcome classification in health promotion and prevention. Health Promot Int. 2008. https://doi.org/10.1093/heapro/dam038.

34.

Ragin CC. Redesigning social inquiry: fuzzy sets and beyond. Chicago: The University of Chicago Press; 2008.CrossRef

35.

Kane H, Lewis MA, Williams PA, Kahwati LC. Using qualitative comparative analysis to understand and quantify translation and implementation. Transl Behav Med. 2014. https://doi.org/10.1007/s13142-014-0251-6.

36.

Kahwati LC, Lewis MA, Kane H, Williams PA, Nerz P, Jones KR, et al. Best practices in the veterans health Administration's MOVE! Weight management program. Am J Prev Med. 2011. https://doi.org/10.1016/j.amepre.2011.06.047.

37.

Weiner BJ, Jacobs SR, Minasian LM, Good MJ. Organizational designs for achieving high treatment trial enrollment: a fuzzy-set analysis of the community clinical oncology program. Journal of Oncology Practice. 2012. https://doi.org/10.1200/JOP.2011.000507.

38.

Kane H, Hinnant L, Day K, Council M, Tzeng J, Soler R, et al. Pathways to program success: a qualitative comparative analysis (QCA) of communities putting prevention to work case study programs. J Public Health Manag Pract. 2017. https://doi.org/10.1097/phh.0000000000000449.

39.

Rauer W, Schuck KD. FEESS 3–4. Fragebogen zur Erfassung emotionaler und sozialer Schulerfahrungen von Grundschulkindern dritter und vierter Klassen. Göttingen: Hogrefe; 2003.

40.

Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013. https://doi.org/10.1186/1748-5908-8-51.

41.

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015. https://doi.org/10.1186/s13012-015-0242-0.

42.

Schmitz GS, Schwarzer R. Perceived self-efficacy of teachers: longitudinal findings with a new instrument. Z Padagog Psychol. 2000. https://doi.org/10.1024//1010-0652.14.1.12.

43.

Tomporowski PD, Lambourne K, Okumura MS. Physical activity interventions and children's mental function: an introduction and overview. Prev Med. 2011. https://doi.org/10.1016/j.ypmed.2011.01.028.

44.

Rollett B. Lernen und Lehren. Eine Einführung in die Pädagogische Psychologie und ihre entwicklungspsychologischen Grundlagen. Wien: Facultas Universitätsverlag; 1997.

45.

Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977. https://doi.org/10.1037/0033-295x.84.2.191.

46.

Ragin CC. Fuzzy-set qualitative comparative analysis software 2008. Available from:http://www.u.arizona.edu/~cragin/fsQCA/software.shtml. Accessed: 6 June 2018.

47.

Kahwati L, Jacobs S, Kane H, Lewis M, Viswanathan M, Golin CE. Using qualitative comparative analysis in a systematic review of a complex intervention. Syst Rev. 2016. https://doi.org/10.1186/s13643-016-0256-y.

48.

Kahwati L, Viswanathan M, Golin CE, Kane H, Lewis M, Jacobs S. Identifying configurations of behavior change techniques in effective medication adherence interventions: a qualitative comparative analysis. Syst Rev. 2016. https://doi.org/10.1186/s13643-016-0255-z.

49.

Ragin CC. Using qualitative comparative analysis to study causal complexity. Health Serv Res. 1999;34(5 Pt 2):1225–39.

50.

Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011. https://doi.org/10.1186/1748-5908-6-42.

51.

Prestwich A, Kellar I, Parker R, MacRae S, Learmonth M, Sykes B, et al. How can self-efficacy be increased? Meta-analysis of dietary interventions. Health Psychol Rev. 2014. https://doi.org/10.1080/17437199.2013.813729.

52.

Latouche AP, Gascoigne M. In-service training for increasing teachers' ADHD knowledge and self-efficacy. J Atten Disord. 2017. https://doi.org/10.1177/1087054717707045.

53.

Donnelly JE, Greene JL, Gibson CA, Smith BK, Washburn RA, Sullivan DK, et al. Physical activity across the curriculum (PAAC): a randomized controlled trial to promote physical activity and diminish overweight and obesity in elementary school children. Prev Med. 2009. https://doi.org/10.1016/j.ypmed.2009.07.022.

54.

Haerens L, Deforche B, Maes L, Stevens V, Cardon G, De Bourdeaudhuij I. Body mass effects of a physical activity and healthy food intervention in middle schools. Obesity (Silver Spring). 2006. https://doi.org/10.1038/oby.2006.98.

55.

Sutherland RL, Nathan NK, Lubans DR, Cohen K, Davies LJ, Desmet C, et al. An RCT to facilitate implementation of school practices known to increase physical activity. Am J Prev Med. 2017. https://doi.org/10.1016/j.amepre.2017.08.009.

56.

Klug J, Schultes MT, Spiel C. Assessment at school - teachers' diary-supported implementation of a training program. Teach Teach Educ. 2017. https://doi.org/10.1016/j.tate.2017.10.014.

57.

Resnicow K, Davis M, Smith M, Lazarus-Yaroch A, Baranowski T, Baranowski J, et al. How best to measure implementation of school health curricula: a comparison of three measures. Health Educ Res. 1998. https://doi.org/10.1093/her/13.2.239.

58.

Dyrstad SM, Kvalo SE, Alstveit M, Skage I. Physically active academic lessons: acceptance, barriers and facilitators for implementation. BMC Public Health. 2018. https://doi.org/10.1186/s12889-018-5205-3.

59.

Adamowitsch M, Gugglberger L, Dur W. Implementation practices in school health promotion: findings from an Austrian multiple-case study. Health Promot Int. 2014. https://doi.org/10.1093/heapro/dau018.

60.

Schneider CQ, Wagemann C. Standards of good practice in qualitative comparative analysis (QCA) and fuzzy-sets. Comp Sociol. 2010. https://doi.org/10.1163/156913210X12493538729793.

Title: Pathways leading to success and non-success: a process evaluation of a cluster randomized physical activity health promotion program applying fuzzy-set qualitative comparative analysis
Authors: Christina Kien
Ludwig Grillich
Barbara Nussbaumer-Streit
Rudolf Schoberberger
Publication date: 01-12-2018
Publisher: BioMed Central
Published in: BMC Public Health / Issue 1/2018
Electronic ISSN: 1471-2458
DOI: https://doi.org/10.1186/s12889-018-6284-x

At a glance: The STEP trials

Springer Medicine

Pathways leading to success and non-success: a process evaluation of a cluster randomized physical activity health promotion program applying fuzzy-set qualitative comparative analysis

Abstract

Background

Methods

Results

Conclusions

Trial registration

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Trial registration

Please log in to get access to this content

Other articles of this Issue 1/2018

Direct healthcare costs associated with device assessed and self-reported physical activity: results from a cross-sectional population-based study

A qualitative study of the drivers of socioeconomic inequalities in men’s eating behaviours

Undocumented adult migrants in Sweden: mental health and associated factors

Changing the housing environment to reduce obesity in public housing residents: a cluster randomized trial

Exploring the care provided to mothers and children by community health workers in South Africa: missed opportunities to provide comprehensive care

The TALENT II study: a randomized controlled trial assessing the impact of an individual health management (IHM) on stress reduction