Skip to main content
Top
Published in: BMC Public Health 1/2014

Open Access 01-12-2014 | Debate

Supporting health promotion practitioners to undertake evaluation for program development

Authors: Roanna Lobo, Mark Petrich, Sharyn K Burns

Published in: BMC Public Health | Issue 1/2014

Login to get access

Abstract

Background

The vital role of evaluation as integral to program planning and program development is well supported in the literature, yet we find little evidence of this in health promotion practice. Evaluation is often a requirement for organisations supported by public funds, and is duly undertaken, however the quality, comprehensiveness and use of evaluation findings are lacking. Practitioner peer-reviewed publications presenting evaluation work are also limited. There are few published examples where evaluation is conducted as part of a comprehensive program planning process or where evaluation findings are used for program development in order to improve health promotion practice.

Discussion

For even the smallest of programs, there is a diverse array of evaluation that is possible before, during and after program implementation. Some types of evaluation are less prevalent than others. Data that are easy to collect or that are required for compliance purposes are common. Data related to how and why programs work which could be used to refine and improve programs are less commonly collected. This finding is evident despite numerous resources and frameworks for practitioners on how to conduct effective evaluation and increasing pressure from funders to provide evidence of program effectiveness. We identify several organisational, evaluation capacity and knowledge translation factors which contribute to the limited collection of some types of data. In addition, we offer strategies for improving health promotion program evaluation and we identify collaboration of a range of stakeholders as a critical enabler for improved program evaluation.

Summary

Evaluation of health promotion programs does occur and resources for how to conduct evaluation are readily available to practitioners. For the purposes of program development, multi-level strategies involving multiple stakeholders are required to address the organisational, capacity and translational factors that affect practitioners’ ability to undertake adequate evaluation.
Appendix
Available only for authorised users
Literature
1.
go back to reference Stufflebeam D: The CIPP Model for Evaluation. The International Handbook of Educational Evaluation. Edited by: Kellaghan T, Stufflebeam DL. 2003, Dordrech: Kluwer Stufflebeam D: The CIPP Model for Evaluation. The International Handbook of Educational Evaluation. Edited by: Kellaghan T, Stufflebeam DL. 2003, Dordrech: Kluwer
2.
go back to reference Dunt D: Levels of project evaluation and evaluation study designs. Population Health, Communities and Health Promotion. Edited by: Jirowong S, Liamputtong P. 2009, Sydney: Oxford University Press, 1 Dunt D: Levels of project evaluation and evaluation study designs. Population Health, Communities and Health Promotion. Edited by: Jirowong S, Liamputtong P. 2009, Sydney: Oxford University Press, 1
3.
go back to reference Green L, Kreuter M: Health Promotion Planning: An Educational and Ecological Approach. 2005, New York: McGraw Hill, 4 Green L, Kreuter M: Health Promotion Planning: An Educational and Ecological Approach. 2005, New York: McGraw Hill, 4
4.
go back to reference Howat P, Brown G, Burns S, McManus A: Project planning using the PRECEDE-PROCEED model. Edited by: Jirowong S, Liamputtong P. 2009, Sydney: Oxford University Press, 1 Howat P, Brown G, Burns S, McManus A: Project planning using the PRECEDE-PROCEED model. Edited by: Jirowong S, Liamputtong P. 2009, Sydney: Oxford University Press, 1
5.
go back to reference O'Connor-Fleming M-L, Parker E, Higgins H, Gould T: A framework for evaluating health promotion programs. Health Promot J Aust. 2006, 17 (1): 61-66. O'Connor-Fleming M-L, Parker E, Higgins H, Gould T: A framework for evaluating health promotion programs. Health Promot J Aust. 2006, 17 (1): 61-66.
6.
go back to reference Smith B: Evaluation of health promotion programs: are we making progress?. Health Promot J Aust. 2011, 22 (3): 165-CrossRef Smith B: Evaluation of health promotion programs: are we making progress?. Health Promot J Aust. 2011, 22 (3): 165-CrossRef
7.
go back to reference Hawe P, Degeling D, Hall J: Evaluating Health Promotion: A Health Workers Guide. 1990, MacLennan and Petty: Sydney Hawe P, Degeling D, Hall J: Evaluating Health Promotion: A Health Workers Guide. 1990, MacLennan and Petty: Sydney
8.
go back to reference Project Management Institute (PMI) Inc: A Guide to the Project Management Body of Knowledge (PMBOK Guide). 2013, Newtown Square, PA: Project Management Institute Inc., 5 Project Management Institute (PMI) Inc: A Guide to the Project Management Body of Knowledge (PMBOK Guide). 2013, Newtown Square, PA: Project Management Institute Inc., 5
9.
go back to reference Bauman A, Nutbeam D: Evaluation in a nutshell: a practical guide to the evaluation of health promotion programs. 2013, N.S.W.: McGraw-Hill Bauman A, Nutbeam D: Evaluation in a nutshell: a practical guide to the evaluation of health promotion programs. 2013, N.S.W.: McGraw-Hill
10.
go back to reference Eggar G, Spark R, Donovan R: Health promotion strategies and methods. 2005, Sydney: McGraw-Hill, 2 Eggar G, Spark R, Donovan R: Health promotion strategies and methods. 2005, Sydney: McGraw-Hill, 2
11.
go back to reference Glasgow R, Vogt T, Boles S: Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999, 89 (9): 1322-1327. 10.2105/AJPH.89.9.1322.CrossRefPubMedPubMedCentral Glasgow R, Vogt T, Boles S: Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999, 89 (9): 1322-1327. 10.2105/AJPH.89.9.1322.CrossRefPubMedPubMedCentral
12.
go back to reference Nutbeam D: The challenge to provide 'evidence' in health promotion. Health Promot Int. 1999, 14 (2): 99-101. 10.1093/heapro/14.2.99.CrossRef Nutbeam D: The challenge to provide 'evidence' in health promotion. Health Promot Int. 1999, 14 (2): 99-101. 10.1093/heapro/14.2.99.CrossRef
13.
go back to reference Valente T: Evaluating health promotion programs. 2002, USA: Oxford University Press Valente T: Evaluating health promotion programs. 2002, USA: Oxford University Press
14.
go back to reference Fagen M, Redman S, Stacks J, Barrett V, Thullen B, Altenor S, Neiger B: Developmental evaluation: building innovations in complex environments. Health Promot Practice. 2011, 12 (5): 645-650. 10.1177/1524839911412596.CrossRef Fagen M, Redman S, Stacks J, Barrett V, Thullen B, Altenor S, Neiger B: Developmental evaluation: building innovations in complex environments. Health Promot Practice. 2011, 12 (5): 645-650. 10.1177/1524839911412596.CrossRef
15.
go back to reference Harris A, Mortimer D: Funding illness prevention and health promotion in Australia: a way forward. Austr NZ Health Policy. 2009, 6 (1): 25-10.1186/1743-8462-6-25.CrossRef Harris A, Mortimer D: Funding illness prevention and health promotion in Australia: a way forward. Austr NZ Health Policy. 2009, 6 (1): 25-10.1186/1743-8462-6-25.CrossRef
16.
go back to reference Hildebrand J, Lobo R, Hallett J, Brown G, Maycock B: My-Peer Toolkit [1.0]: developing an online resource for planning and evaluating peer-based youth programs. Youth Stud Aus. 2012, 31 (2): 53-61. Hildebrand J, Lobo R, Hallett J, Brown G, Maycock B: My-Peer Toolkit [1.0]: developing an online resource for planning and evaluating peer-based youth programs. Youth Stud Aus. 2012, 31 (2): 53-61.
17.
go back to reference Jolley G, Lawless A, Hurley C: Framework and tools for planning and evaluating community participation, collaborative partnerships and equity in health promotion. Health Promot J Aust. 2008, 19 (2): 152-157. Jolley G, Lawless A, Hurley C: Framework and tools for planning and evaluating community participation, collaborative partnerships and equity in health promotion. Health Promot J Aust. 2008, 19 (2): 152-157.
18.
go back to reference Klinner C, Carter S, Rychetnik L, Li V, Daley M, Zask A, Lloyd B: Integrating relationship- and research-based approaches in Australian health promotion practice. Health Promot Int. 2014, doi:10.1093/heapro/dau026 Klinner C, Carter S, Rychetnik L, Li V, Daley M, Zask A, Lloyd B: Integrating relationship- and research-based approaches in Australian health promotion practice. Health Promot Int. 2014, doi:10.1093/heapro/dau026
19.
go back to reference Pettman T, Armstrong R, Doyle J, Burford B, Anderson L, Hillgrove T: Honey N. Waters E: Strengthening evaluation to capture the breadth of public health practice: ideal vs. real. J Public Health. 2012, 34 (1): 151-155. Pettman T, Armstrong R, Doyle J, Burford B, Anderson L, Hillgrove T: Honey N. Waters E: Strengthening evaluation to capture the breadth of public health practice: ideal vs. real. J Public Health. 2012, 34 (1): 151-155.
20.
go back to reference Barry M, Allegrante J, Lamarre M, Auld M, Taub A: The Galway Consensus Conference: international collaboration on the development of core competencies for health promotion and health education. Global Health Promot. 2009, 16 (2): 5-11. 10.1177/1757975909104097.CrossRef Barry M, Allegrante J, Lamarre M, Auld M, Taub A: The Galway Consensus Conference: international collaboration on the development of core competencies for health promotion and health education. Global Health Promot. 2009, 16 (2): 5-11. 10.1177/1757975909104097.CrossRef
21.
go back to reference Demspey C, Battel-Kirk B, Barry M: The CompHP Core Competencies Framework for Health Promotion handbook. 2011, Galway: National University of Ireland Demspey C, Battel-Kirk B, Barry M: The CompHP Core Competencies Framework for Health Promotion handbook. 2011, Galway: National University of Ireland
22.
go back to reference Thurston W, Potvin L: Evaluability assessment: a tool for incorporating evaluation in social change programmes. Evaluation. 2003, 9 (4): 453-469. 10.1177/1356389003094006.CrossRef Thurston W, Potvin L: Evaluability assessment: a tool for incorporating evaluation in social change programmes. Evaluation. 2003, 9 (4): 453-469. 10.1177/1356389003094006.CrossRef
23.
go back to reference Wise M, Signal L: Health promotion development in Australia and New Zealand. Health Promot Int. 2000, 15 (3): 237-248. 10.1093/heapro/15.3.237.CrossRef Wise M, Signal L: Health promotion development in Australia and New Zealand. Health Promot Int. 2000, 15 (3): 237-248. 10.1093/heapro/15.3.237.CrossRef
24.
go back to reference Maycock B, Jackson R, Howat P, Burns S, Collins J: Orienting health promotion course structure to maximise competency development. 13th Annual Teaching and Learning Forum. 2004, Perth: Murdoch University Maycock B, Jackson R, Howat P, Burns S, Collins J: Orienting health promotion course structure to maximise competency development. 13th Annual Teaching and Learning Forum. 2004, Perth: Murdoch University
25.
go back to reference Zechmeister I, Kilian R, McDaid D: Is it worth investing in mental health promotion and prevention of mental illness? A systematic review of the evidence from economic evaluations. BMC Public Health. 2008, 8 (1): 20-10.1186/1471-2458-8-20.CrossRefPubMedPubMedCentral Zechmeister I, Kilian R, McDaid D: Is it worth investing in mental health promotion and prevention of mental illness? A systematic review of the evidence from economic evaluations. BMC Public Health. 2008, 8 (1): 20-10.1186/1471-2458-8-20.CrossRefPubMedPubMedCentral
26.
go back to reference W.K. Kellogg Foundation: The W. K. Kellogg Foundation Evaluation Handbook. 2010, Michigan: W.K. Kellogg Foundation W.K. Kellogg Foundation: The W. K. Kellogg Foundation Evaluation Handbook. 2010, Michigan: W.K. Kellogg Foundation
27.
go back to reference Hanusaik N, O’Loughlin J, Kishchuk N, Paradis G, Cameron R: Organizational capacity for chronic disease prevention: a survey of Canadian public health organizations. Eur J Public Health. 2010, 20 (2): 195-201. 10.1093/eurpub/ckp140.CrossRefPubMed Hanusaik N, O’Loughlin J, Kishchuk N, Paradis G, Cameron R: Organizational capacity for chronic disease prevention: a survey of Canadian public health organizations. Eur J Public Health. 2010, 20 (2): 195-201. 10.1093/eurpub/ckp140.CrossRefPubMed
28.
go back to reference Jolley G: Evaluating complex community-based health promotion: addressing the challenges. Eval Program Plann. 2014, 45: 71-81.CrossRefPubMed Jolley G: Evaluating complex community-based health promotion: addressing the challenges. Eval Program Plann. 2014, 45: 71-81.CrossRefPubMed
29.
go back to reference Harris N, Sandor M: Defining sustainable practice in community-based health promotion: A Delphi study of practitioner perspectives. Health Promot J Aust. 2013, 24: 53-60. Harris N, Sandor M: Defining sustainable practice in community-based health promotion: A Delphi study of practitioner perspectives. Health Promot J Aust. 2013, 24: 53-60.
31.
go back to reference Cross D, Waters S, Pearce N, Shaw T, Hall M, Erceg E, Burns S, Roberts C, Hamilton G: The Friendly Schools Friendly Families programme: three-year bullying behaviour outcomes in primary school children. Int J Educ Res. 2012, 53: 394-406.CrossRef Cross D, Waters S, Pearce N, Shaw T, Hall M, Erceg E, Burns S, Roberts C, Hamilton G: The Friendly Schools Friendly Families programme: three-year bullying behaviour outcomes in primary school children. Int J Educ Res. 2012, 53: 394-406.CrossRef
32.
go back to reference Maycock B, Binns C, Dhaliwal S, Tohotoa J, Hauck Y, Burns S, Howat P: Education and support for fathers improves breastfeeding rates: a randomized controlled trial. J Hum Lact. 2013, 29 (4): 484-490. 10.1177/0890334413484387.CrossRefPubMed Maycock B, Binns C, Dhaliwal S, Tohotoa J, Hauck Y, Burns S, Howat P: Education and support for fathers improves breastfeeding rates: a randomized controlled trial. J Hum Lact. 2013, 29 (4): 484-490. 10.1177/0890334413484387.CrossRefPubMed
33.
go back to reference Kemm J: The limitations of 'evidence-based' public health. J Eval Clin Pract. 2006, 12 (3): 319-324. 10.1111/j.1365-2753.2006.00600.x.CrossRefPubMed Kemm J: The limitations of 'evidence-based' public health. J Eval Clin Pract. 2006, 12 (3): 319-324. 10.1111/j.1365-2753.2006.00600.x.CrossRefPubMed
34.
go back to reference Armstrong R, Waters E, Crockett B, Keleher H: The nature of evidence resources and knowledge translation for health promotion practitioners. Health Promot Int. 2007, 22 (3): 254-260. 10.1093/heapro/dam017.CrossRefPubMed Armstrong R, Waters E, Crockett B, Keleher H: The nature of evidence resources and knowledge translation for health promotion practitioners. Health Promot Int. 2007, 22 (3): 254-260. 10.1093/heapro/dam017.CrossRefPubMed
35.
go back to reference Green J, South J: Evaluation with hard-to-reach groups. Evaluation: Key concepts for public health practice. 2006, Berkshire, England: Open University Press, 113-128. 1 Green J, South J: Evaluation with hard-to-reach groups. Evaluation: Key concepts for public health practice. 2006, Berkshire, England: Open University Press, 113-128. 1
36.
go back to reference Preskill H, Boyle S: A multidisciplinary model of evaluation capacity building. Am J Public Health. 2008, 29 (4): 443-459. Preskill H, Boyle S: A multidisciplinary model of evaluation capacity building. Am J Public Health. 2008, 29 (4): 443-459.
37.
go back to reference Lobo R, McManus A, Brown G, Hildebrand J, Maycock B: Evaluating peer-based youth programs: barriers and enablers. Eval J Australas. 2010, 10 (1): 36-43. Lobo R, McManus A, Brown G, Hildebrand J, Maycock B: Evaluating peer-based youth programs: barriers and enablers. Eval J Australas. 2010, 10 (1): 36-43.
38.
go back to reference Preskill H, Catsambas T: Reframing evaluation through appreciative inquiry. 2006, Thousand Oaks, CA: Sage Publications Preskill H, Catsambas T: Reframing evaluation through appreciative inquiry. 2006, Thousand Oaks, CA: Sage Publications
39.
go back to reference Dart J, Davies R: A dialogical, story-based evaluation tool: the most significant change technique. Am J Eval. 2003, 24 (2): 137-155. 10.1177/109821400302400202.CrossRef Dart J, Davies R: A dialogical, story-based evaluation tool: the most significant change technique. Am J Eval. 2003, 24 (2): 137-155. 10.1177/109821400302400202.CrossRef
40.
go back to reference Brinkerhoff R: The success case method: a strategic evaluation approach to increasing the value and effect of training. Adv Dev Hum Resour. 2005, 7: 86-101. 10.1177/1523422304272172.CrossRef Brinkerhoff R: The success case method: a strategic evaluation approach to increasing the value and effect of training. Adv Dev Hum Resour. 2005, 7: 86-101. 10.1177/1523422304272172.CrossRef
41.
go back to reference Davies C, Knuiman M, Wright P, Rosenberg M: The art of being healthy: a qualitative study to develop a thematic framework for understanding the relationship between health and the arts. BMJ Open. 2014, 4: e004790-10.1136/bmjopen-2014-004790. doi:10.1136/bmjopen-2014-004790CrossRefPubMedPubMedCentral Davies C, Knuiman M, Wright P, Rosenberg M: The art of being healthy: a qualitative study to develop a thematic framework for understanding the relationship between health and the arts. BMJ Open. 2014, 4: e004790-10.1136/bmjopen-2014-004790. doi:10.1136/bmjopen-2014-004790CrossRefPubMedPubMedCentral
43.
go back to reference Lobo R, Brown G, Maycock B, Burns S: Peer-based Outreach for Same Sex Attracted Youth (POSSAY) project final report. 2006, Perth: Western Australian Centre for Health Promotion Research, Curtin University Lobo R, Brown G, Maycock B, Burns S: Peer-based Outreach for Same Sex Attracted Youth (POSSAY) project final report. 2006, Perth: Western Australian Centre for Health Promotion Research, Curtin University
44.
go back to reference Roberts M, Lobo R, Sorenson A: Evaluating the Sharing Stories youth drama program. In Australasian Sexual Health Conference, 23-25 October 2013. 2013, Darwin Australia: Australasian Sexual Health Alliance (ASHA) Roberts M, Lobo R, Sorenson A: Evaluating the Sharing Stories youth drama program. In Australasian Sexual Health Conference, 23-25 October 2013. 2013, Darwin Australia: Australasian Sexual Health Alliance (ASHA)
45.
go back to reference Liket K, Rey-Garcia M, Maas K: Why aren't evaluations working and what to do about it: a framework for negotiating meaningful evaluation in nonprofits. Am J Eval. 2014, 35: 171-188. 10.1177/1098214013517736.CrossRef Liket K, Rey-Garcia M, Maas K: Why aren't evaluations working and what to do about it: a framework for negotiating meaningful evaluation in nonprofits. Am J Eval. 2014, 35: 171-188. 10.1177/1098214013517736.CrossRef
46.
go back to reference Lobo R, Doherty M, Crawford G, Hallett J, Comfort J, Tilley P: SiREN – building health promotion capacity in Western Australian sexual health services. International Union for Health Promotion and Education, 21st World Conference on Health Promotion, 25–29 August 2013. 2013, Pattaya, Thailand: International Union for Health Promotion and Education (IUHPE) Lobo R, Doherty M, Crawford G, Hallett J, Comfort J, Tilley P: SiREN – building health promotion capacity in Western Australian sexual health services. International Union for Health Promotion and Education, 21st World Conference on Health Promotion, 25–29 August 2013. 2013, Pattaya, Thailand: International Union for Health Promotion and Education (IUHPE)
48.
go back to reference Walker R, Lobo R: Asking, listening and changing direction: guiding youth sector capacity building for youth sexual health promotion. Australasian Sexual Health Conference, 15–17 October 2012. 2012, Melbourne: International Union against Sexually Transmitted Infections (IUSTI) Walker R, Lobo R: Asking, listening and changing direction: guiding youth sector capacity building for youth sexual health promotion. Australasian Sexual Health Conference, 15–17 October 2012. 2012, Melbourne: International Union against Sexually Transmitted Infections (IUSTI)
49.
go back to reference Brown G, Johnston K: REACH partnership - final report. 2013, Melbourne: Australian Research Centre in Sex, Health and Society, La Trobe University Brown G, Johnston K: REACH partnership - final report. 2013, Melbourne: Australian Research Centre in Sex, Health and Society, La Trobe University
50.
go back to reference Lobo R: An evaluation framework for peer-based youth programs. PhD thesis. 2012, Perth: Curtin University Lobo R: An evaluation framework for peer-based youth programs. PhD thesis. 2012, Perth: Curtin University
51.
go back to reference Kemmis S, McTaggart R: Participatory action research. Handbook of Qualitative Research. Edited by: Denzin N, Lincoln YS. 2000, Thousand Oaks, CA: Sage, 567-606. 2 Kemmis S, McTaggart R: Participatory action research. Handbook of Qualitative Research. Edited by: Denzin N, Lincoln YS. 2000, Thousand Oaks, CA: Sage, 567-606. 2
Metadata
Title
Supporting health promotion practitioners to undertake evaluation for program development
Authors
Roanna Lobo
Mark Petrich
Sharyn K Burns
Publication date
01-12-2014
Publisher
BioMed Central
Published in
BMC Public Health / Issue 1/2014
Electronic ISSN: 1471-2458
DOI
https://doi.org/10.1186/1471-2458-14-1315

Other articles of this Issue 1/2014

BMC Public Health 1/2014 Go to the issue