Skip to main content
Top
Published in: Implementation Science 1/2017

Open Access 01-12-2017 | Research

Five years’ experience of an annual course on implementation science: an evaluation among course participants

Authors: Siw Carlfjord, Kerstin Roback, Per Nilsen

Published in: Implementation Science | Issue 1/2017

Login to get access

Abstract

Background

Increasing interest in implementation science has generated a demand for education and training opportunities for researchers and practitioners in the field. However, few implementation science courses have been described or evaluated in the scientific literature. The aim of the present study was to provide a short- and long-term evaluation of the implementation training at Linköping University, Sweden.

Methods

Two data collections were carried out. In connection with the final seminar, a course evaluation form, including six items on satisfaction and suggestions for improvement, was distributed to the course participants, a total of 101 students from 2011 to 2015 (data collection 1), response rate 72%. A questionnaire including six items was distributed by e-mail to the same students in autumn 2016 (data collection 2), response rate 63%. Data from the two data collections were presented descriptively and analysed using the Kirkpatrick model consisting of four levels: reaction, learning, behaviour and results.

Results

The students were very positive immediately after course participation, rating high on overall perception of the course and the contents (reaction). The students also rated high on achievement of the course objectives and considered their knowledge in implementation science to be very good and to a high degree due to course participation (learning). Knowledge gained from the course was viewed to be useful (behaviour) and was applied to a considerable extent in research projects and work apart from research activities (results).

Conclusions

The evaluation of the doctoral-level implementation science course provided by Linköping University showed favourable results, both in the short and long term. The adapted version of the Kirkpatrick model was useful because it provided a structure for evaluation of the short- and long-term learning outcomes.
Appendix
Available only for authorised users
Literature
1.
go back to reference Brownson RC, Colditz GA, Proctor EK. Dissemination and implementation research in health: translating science to practice. New York: Oxford University Press; 2012.CrossRef Brownson RC, Colditz GA, Proctor EK. Dissemination and implementation research in health: translating science to practice. New York: Oxford University Press; 2012.CrossRef
2.
go back to reference Grol R, Wensing M, Eccles M, Davis D. Improving patient care—the implementation of change in health care. 2nd ed. Oxford: Wiley Blackwell; 2013.CrossRef Grol R, Wensing M, Eccles M, Davis D. Improving patient care—the implementation of change in health care. 2nd ed. Oxford: Wiley Blackwell; 2013.CrossRef
4.
go back to reference Chambers DA, Proctor EK, Brownson RC, Straus SE. Mapping training needs for dissemination and implementation research: lessons from a synthesis of existing D&I research training programs. Transl Behav Med 2016 [Epub ahead of print]. DOI:10.1007/s13142-016-0399-3. Chambers DA, Proctor EK, Brownson RC, Straus SE. Mapping training needs for dissemination and implementation research: lessons from a synthesis of existing D&I research training programs. Transl Behav Med 2016 [Epub ahead of print]. DOI:10.1007/s13142-016-0399-3.
5.
go back to reference Ullrich C, Mahler C, Forstner J, Szecsenyi J, Wensing M. Teaching implementation science in a new Master of Science Program in Germany: a survey of stakeholders expectations. Implement Sci. 2017;12:55.CrossRefPubMedPubMedCentral Ullrich C, Mahler C, Forstner J, Szecsenyi J, Wensing M. Teaching implementation science in a new Master of Science Program in Germany: a survey of stakeholders expectations. Implement Sci. 2017;12:55.CrossRefPubMedPubMedCentral
7.
go back to reference Proctor EK, Landsverk J, Baumann AA, Mittman BS, Aarons GA, Brownson RC, et al. The implementation research institute: training mental health implementation researchers in the United States. Implement Sci. 2013;8:105.CrossRefPubMedPubMedCentral Proctor EK, Landsverk J, Baumann AA, Mittman BS, Aarons GA, Brownson RC, et al. The implementation research institute: training mental health implementation researchers in the United States. Implement Sci. 2013;8:105.CrossRefPubMedPubMedCentral
8.
go back to reference Meissner HI, Glasgow RE, Vinson CA, Chambers D, Brownson RC, Green LW, et al. The U.S. training institute for dissemination and implementation research in health. Implement Sci. 2013;8:12.CrossRefPubMedPubMedCentral Meissner HI, Glasgow RE, Vinson CA, Chambers D, Brownson RC, Green LW, et al. The U.S. training institute for dissemination and implementation research in health. Implement Sci. 2013;8:12.CrossRefPubMedPubMedCentral
9.
go back to reference Norton WE. Advancing the science and practice of dissemination and implementation in health: a novel course for public health students and academic researchers. Public Health Rep. 2014;129:536–42.CrossRefPubMedPubMedCentral Norton WE. Advancing the science and practice of dissemination and implementation in health: a novel course for public health students and academic researchers. Public Health Rep. 2014;129:536–42.CrossRefPubMedPubMedCentral
10.
go back to reference Straus SE, Sales A, Wensing M, Michie S, Kent B, Foy R. Education and training for implementation science: our interest in manuscripts describing education and training materials. Implement Sci. 2015;10:136.CrossRefPubMedPubMedCentral Straus SE, Sales A, Wensing M, Michie S, Kent B, Foy R. Education and training for implementation science: our interest in manuscripts describing education and training materials. Implement Sci. 2015;10:136.CrossRefPubMedPubMedCentral
11.
go back to reference Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs. The four levels. 3rd ed. (1st ed. 1994). San Francisco: Berrett-Koehler Publishers; 2006. Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs. The four levels. 3rd ed. (1st ed. 1994). San Francisco: Berrett-Koehler Publishers; 2006.
12.
go back to reference Lindhe Söderlund L, Madson M, Rubak S, Nilsen P. A systematic review of motivational interviewing training for general health care practitioners. Patient Educ Couns. 2010;89:16–26. Lindhe Söderlund L, Madson M, Rubak S, Nilsen P. A systematic review of motivational interviewing training for general health care practitioners. Patient Educ Couns. 2010;89:16–26.
13.
go back to reference Boud D, Feletti G. The challenge of problem based-learning. London: Kogan Page; 1991. Boud D, Feletti G. The challenge of problem based-learning. London: Kogan Page; 1991.
16.
go back to reference Praslova L. Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. Educ Assess Eval Account. 2010;22:215–25.CrossRef Praslova L. Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. Educ Assess Eval Account. 2010;22:215–25.CrossRef
17.
go back to reference Yardley S, Dornan T. Kirkpatrick’s levels and education ‘evidence’. Med Educ. 2012;46:97–106.CrossRefPubMed Yardley S, Dornan T. Kirkpatrick’s levels and education ‘evidence’. Med Educ. 2012;46:97–106.CrossRefPubMed
18.
go back to reference Dorri S, Akbari M, Dorri SM. Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iran J Nurs Midwifery Res. 2016;21:493–7.CrossRefPubMedPubMedCentral Dorri S, Akbari M, Dorri SM. Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iran J Nurs Midwifery Res. 2016;21:493–7.CrossRefPubMedPubMedCentral
19.
go back to reference Throgmorton C, Mitchell T, Morley T, Snyder M. Evaluating a physician leadership development program—a mixed methods approach. J Health Organ Manag. 2016;30:390–407.CrossRefPubMed Throgmorton C, Mitchell T, Morley T, Snyder M. Evaluating a physician leadership development program—a mixed methods approach. J Health Organ Manag. 2016;30:390–407.CrossRefPubMed
20.
go back to reference Paull M, Whitsed C, Girardi A. Applying the Kirkpatrick model: evaluating an Interaction for Learning Framework curriculum intervention. Issues Educ Res. 2016;26:490–507. Paull M, Whitsed C, Girardi A. Applying the Kirkpatrick model: evaluating an Interaction for Learning Framework curriculum intervention. Issues Educ Res. 2016;26:490–507.
21.
go back to reference Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24:105–12.CrossRefPubMed Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24:105–12.CrossRefPubMed
22.
go back to reference Deci EL, Ryan RM. Intrinsic motivation and self-determination in human behavior. New York: Plenum; 1985.CrossRef Deci EL, Ryan RM. Intrinsic motivation and self-determination in human behavior. New York: Plenum; 1985.CrossRef
23.
go back to reference Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55:68–78.CrossRefPubMed Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55:68–78.CrossRefPubMed
24.
go back to reference Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;2:CD003030. Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;2:CD003030.
25.
go back to reference Hmelo-Silver CE. Problem-based learning: what and how do students learn? Educ Psychol Rev. 2004;3:235–66.CrossRef Hmelo-Silver CE. Problem-based learning: what and how do students learn? Educ Psychol Rev. 2004;3:235–66.CrossRef
26.
go back to reference Gould BH, Brodie L, Carver F, Logan P. Not just ticking all the boxes. Problem based learning and mental health nursing. A review. Nurse Educ Today. 2015;35:e1–5.CrossRefPubMed Gould BH, Brodie L, Carver F, Logan P. Not just ticking all the boxes. Problem based learning and mental health nursing. A review. Nurse Educ Today. 2015;35:e1–5.CrossRefPubMed
27.
go back to reference Vernon DTA, Blake RL. Does problem-based learning work. A meta analysis of evaluative research. Acad Med. 1993;68:550–63.CrossRefPubMed Vernon DTA, Blake RL. Does problem-based learning work. A meta analysis of evaluative research. Acad Med. 1993;68:550–63.CrossRefPubMed
28.
go back to reference Dewey J. How we think. A restatement of the relation of reflective thinking to the educative process (revised ed. 1982). Boston: D.C. Heath; 1933. Dewey J. How we think. A restatement of the relation of reflective thinking to the educative process (revised ed. 1982). Boston: D.C. Heath; 1933.
29.
go back to reference Schön DA. The reflective practitioner—how professionals think in action. New York: Basic Books; 1983. Schön DA. The reflective practitioner—how professionals think in action. New York: Basic Books; 1983.
30.
go back to reference Peytchev A, Baxter DK, Carley-Baxter LR. Not all survey effort is equal: reduction of nonresponse bias and nonresponse error. Public Opin Q. 2009;73:785–806.CrossRef Peytchev A, Baxter DK, Carley-Baxter LR. Not all survey effort is equal: reduction of nonresponse bias and nonresponse error. Public Opin Q. 2009;73:785–806.CrossRef
31.
go back to reference Paulhus DL. Measurement and control of response bias. In: Robinson JP, Shaver PR, Wrightsman LS, editors. Measures of personality and social psychological attitudes. San Diego: Academic Press; 1991. p. 17–59.CrossRef Paulhus DL. Measurement and control of response bias. In: Robinson JP, Shaver PR, Wrightsman LS, editors. Measures of personality and social psychological attitudes. San Diego: Academic Press; 1991. p. 17–59.CrossRef
32.
go back to reference Padek M, Colditz G, Dobbins M, Koscielniak N, Proctor EK, Sales AE, et al. Developing educational competencies and implementation research training programs: an exploratory analysis using card sorts. Implement Sci. 2015;10:114.CrossRefPubMedPubMedCentral Padek M, Colditz G, Dobbins M, Koscielniak N, Proctor EK, Sales AE, et al. Developing educational competencies and implementation research training programs: an exploratory analysis using card sorts. Implement Sci. 2015;10:114.CrossRefPubMedPubMedCentral
Metadata
Title
Five years’ experience of an annual course on implementation science: an evaluation among course participants
Authors
Siw Carlfjord
Kerstin Roback
Per Nilsen
Publication date
01-12-2017
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2017
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-017-0618-4

Other articles of this Issue 1/2017

Implementation Science 1/2017 Go to the issue