Top

BMC Health Services Research

Published in:

Open Access 01-12-2018 | Research article

Developing a framework to evaluate knowledge into action interventions

Authors: Sarah Morton, Suzanne Wilson, Sheila Inglis, Karen Ritchie, Ann Wales

Published in: BMC Health Services Research | Issue 1/2018

Abstract

Background

There are many challenges in delivering and evaluating knowledge for healthcare, but the lack of clear routes from knowledge to practice is a root cause of failures in safety within healthcare. Various types and sources of knowledge are relevant at different levels within the healthcare system. These need to be delivered in a timely way that is useful and actionable for those providing services or developing policies. How knowledge is taken up and used through networks and relationships, and the difficulties in attributing change to knowledge-based interventions, present challenges to understanding how knowledge into action (K2A) work influences healthcare outcomes. This makes it difficult to demonstrate the importance of K2A work, and harness support for its development and resourcing. This paper presents the results from a project commissioned by NHS Education for Scotland (NES) and Healthcare Improvement Scotland (HIS) to create an evaluation framework to help understand the NHS Scotland Knowledge into Action model.

Methods

The team took a developmental approach to creating an evaluation framework that would be useful and practical. This included a literature review to ensure the evaluation was evidence-based; adaptation of contribution analysis for K2A project; action research with K2A project leads to refine the work and develop suitable measures.

Results

Principles for evaluation and an evaluation framework based on contribution analysis were developed and implemented on a trial project. An outcomes chain was developed for the K2A programme and specific projects. This was used to design, collect and collate evidence of the K2A intervention. Data collected routinely by the intervention was supplemented with specific feedback measures from K2A project users.

Conclusions

The evaluation approach allowed for scrutiny of both processes and outcomes and was adaptable to projects on different scales. This framework has proved useful as a planning, reflecting and evaluation tool for K2A, and could be more widely used to evidence the ways in which knowledge to action work helps improve healthcare outcomes.

For example NHS England Knowledge for Healthcare; A Development Framework https://hee.nhs.uk/sites/default/files/documents/Knowledge%20for%20healthcare%20-%20a%20development%20framework.pdf. Accessed 15 Mar 2017 or Karamitri, I., Talias, M. A., and Bellali, T. Knowledge management practices in healthcare settings: a systematic review. Int J Health Plann Mgmt, 2017;32:

Morton S. Progressing research impact assessment: a ‘contributions’ approach. Research Evaluation. 2015;24(4):405–19.CrossRef

Daley BJ. Learning and Professional Practice: A Study of Four Professions. Adult Education Quarterly. 2001;52:1.CrossRef

Williams I. Receptive contexts and the role of knowledge management in evidence-based practice. In: Glasby J, editor. Evidence, policy and practice. Bristol: Policy Press; 2011.

Donovan C. State of the art in assessing research impact: introduction to a special issue. Research Evaluation. 2011;20:3.CrossRef

Penfield T, Baker MJ, Scoble R, Wykes MC. Assessment, evaluations, and definitions of research impact: a review. Research Evaluation. 2014;23:1.CrossRef

Donovan C. The Australian research quality framework: a live experiment in capturing the social, economic, environmental, and cultural returns of publicly funded research. N Dir Eval. 2008;118:47–60.

Bell S, et al. Real-world approaches to assessing the impact of environmental research on policy. Research Evaluation. 2011;20:3.CrossRef

NHS Education for Scotland and Healthcare Improvement Scotland. Getting Knowledge into Action to Improve Healthcare Quality: Report of Strategic Review and Recommendations. 2012. http://www.knowledge.scot.nhs.uk/media/CLT/ResourceUploads/4031966/K2AReview2012.pdf. Accessed 1 Dec 2015.

10.

Scottish Government. The Healthcare Quality Strategy for NHSScotland; 2010. http://www.gov.scot/resource/doc/311667/0098354.pdf. Accessed 1 Dec 2015.

11.

Institute of Medicine Crossing the quality chasm: a new health system for the 21st century. Washington DC: institute of Medicine; 2001.

12.

Francis R. Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. London: 2013. https://www.gov.uk/government/publications/report-of-the-mid-staffordshire-nhs-foundation-trust-public-inquiry. Accessed 1 Dec 2015.

13.

Patton MQ. Developmental evaluation: applying complexity concepts to enhance innovation and use. New York: Guilford Press; 2011.

14.

Montague S. Results Logic for Complex Systems. Presentation May 13 2008, Performance management network. https://www.pmn.net/wp-content/uploads/w45-Results-Logic-for-Complex-Systems-May-13-08-Read-Only.pdf. Accessed 1 Dec 2015.

15.

Mayne J. Contribution analysis: an approach to exploring cause and effect: ILAC institutional learning and change institute; 2008.

16.

Van Eerd D, Cole D, Keown K, Irvin E, Kramer D, Gibson J, Kohn M, Mahood Q, Slack T, Amick B, Phipps D, Garcia J, Morassaei S. Report on knowledge transfer and exchange practices: A systematic review of the quality and types of instruments used to assess KTE implementation and impact. Institute for Work and Health; 2011. http://www.iwh.on.ca/sys-reviews/kte-evaluation-tools. Accessed 1 Dec 2015.

17.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:1.CrossRef

18.

Given L. The SAGE Encyclopedia of Qualitative Research Methods. Los Angeles: SAGE; 2008.CrossRef

19.

Weiss CH. Evaluation : methods for studying programs and policies. Upper Saddle River: Prentice Hall; 1998.

20.

Molas-Gallart J, Tang P. Tracing 'productive interactions' to identify social impacts: an example from the social sciences. Research Evaluation. 2011;20(3):219–26.CrossRef

21.

Ammendolia C, Hogg-Johnson S, Bombardier C, Pennick V, Glazier R. Implementing evidence-based guidelines for radiography in acute low back pain: a pilot study in a chiropractic community. J Manip Physiol Ther. 2004;27:3.

22.

Okon TR, Evans JM, Gomez CF, Blackhall LJ. Palliative educational outcome with implementation of PEACE tool integrated clinical pathway. J Palliat Med. 2004;7(2).

23.

Bonetti D, Eccles M, Johnston M, Steen N, Grimshaw J, Baker R, Walker A, Pitts N. Guiding the design and selection of interventions to influence the implementation of evidence-based practice: an experimental simulation of a complex intervention trial. Soc Sci Med. 2005;60(9).

24.

Mikhael J, Baker L, et al. Using a pocket card to improve end-of-life care on internal medicine clinical teaching units: a cluster-randomized controlled trial. J Gen Intern Med. 2008;23(8).

25.

Price SK, Gardner P, et al. Changing hospital newborn nursery practice: results from a statewide “back to sleep” nurses training program. Matern Child Health J. 2008;12:3.CrossRef

26.

Neitzel JJ, Miller EH, et al. Improving pain management after total joint replacement surgery Orthopaedic nursing / National Association of Orthopaedic Nurses. 1999;18:4.

27.

Lumley J, Watson L, et al. PRISM (program of resources, information and support for mothers): a community-randomised trial to reduce depression and improve women's physical health six months after birth [ISRCTN03464021]. BMC Public Health. 2006;6(1):37.CrossRefPubMedPubMedCentral

28.

Titler MG, Herr K, et al. Translating research into practice intervention improves Management of Acute Pain in older hip fracture patients. Health Serv Res. 2009;44(1).

29.

Légaré F, Labrecque M, et al. Training family physicians in shared decision making for the use of antibiotics for acute respiratory infections: a pilot clustered randomized controlled trial. Health Expect. 2011;14.

30.

Ethical Procedure for the University of Edinburgh http://www.sps.ed.ac.uk/research/ethics.

31.

Developed by Morton S. Progressing research impact assessment: A ‘contributions’ approach. Research Evaluation. 2015;24:4.CrossRef

32.

Rogers PJ. Using Programme theory to evaluate complicated and complex aspects of interventions. Evaluation. 2008;14(1):29–48.CrossRef

33.

Montague S. Results logic for complex systems. Presentation May 13, Performance Management network. 2008. http://wwwpmnnet/library/. Accessed 1 Dec 2015.

34.

Oliver K, Innvar S, Lorenc T, Woodman J, Thomas JA. Systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14(1):1–12.CrossRef

35.

CLEAR: clinical enquiry and response service. http://www.knowledge.scot.nhs.uk/clear.aspx. Accessed 1 Dec 2015.

36.

Mansfield W, Grunewald P. The use of Indicators for the Monitoring and Evaluation of Knowledge Management and Knowledge Brokering in International Development, Institute of Development Studies... Report of a workshop held at Institute Development Studies 8 March 2013. 2013. http://www.ids.ac.uk/files/dmfile/Knowledgeindicatorsworkshopreport8thMarch2013.pdf. Accessed 1 Dec 2015.

Title: Developing a framework to evaluate knowledge into action interventions
Authors: Sarah Morton
Suzanne Wilson
Sheila Inglis
Karen Ritchie
Ann Wales
Publication date: 01-12-2018
Publisher: BioMed Central
Published in: BMC Health Services Research / Issue 1/2018
Electronic ISSN: 1472-6963
DOI: https://doi.org/10.1186/s12913-018-2930-3

At a glance: The STEP trials

Springer Medicine

Developing a framework to evaluate knowledge into action interventions

Abstract

Background

Methods

Results

Conclusions

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2018

Assessment of the development of assisted reproductive technology in Liaoning province of China, from 2012 to 2016

Curriculum and training needs of mid-level health workers in Africa: a situational review from Kenya, Nigeria, South Africa and Uganda

Value co-creation in healthcare: evidence from innovative therapeutic alternatives for hereditary angioedema

Independent medical evaluation for sick-listed patients: a focus group study of GPs´ expectations and experiences

Impact of a genetic counseling requirement prior to genetic testing

Disabled people’s experiences of accessing reasonable adjustments in hospitals: a qualitative study