Skip to main content
Top
Published in: Perspectives on Medical Education 5/2016

Open Access 01-10-2016 | Eye-opener

On the issue of costs in programmatic assessment

Authors: Cees P. M. van der Vleuten, Sylvia Heeneman

Published in: Perspectives on Medical Education | Issue 5/2016

Login to get access

Abstract

Programmatic assessment requires labour and cost intensive activities such as feedback in a quantitative and qualitative form, a system of learner support in guiding feedback uptake and self-directed learning, and a decision-making arrangement that includes committees of experts making a holistic professional judgment while using due process measures to achieve trustworthy decisions. This can only be afforded if we redistribute the resources of assessment in a curriculum. Several strategies are suggested. One is to introduce progress testing as a replacement for costly cognitive assessment formats in modules. In addition, all assessments should be replaced by assessment formats that are maximally aligned with the learning tasks. For performance-based assessment, OSCEs should be sparsely used, while education and work-embedded assessment should be maximized as part of the routine of ongoing instruction and assessment. Information technology may support affordable feedback strategies, as well as the creation of a paper trail on performance. By making more dramatic choices in the way we allocate resources to assessment, the cost-intensive activities of programmatic assessment may be realized.
Literature
4.
go back to reference Kluger AN, DeNisi A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254–84.CrossRef Kluger AN, DeNisi A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254–84.CrossRef
5.
6.
go back to reference Van der Vleuten CP, Schuwirth LW, Scheele F, Driessen EW, Hodges B. The assessment of professional competence: building blocks for theory development. Best Pract Res Clin Obstet Gynaecol. 2010;24:703–19.CrossRef Van der Vleuten CP, Schuwirth LW, Scheele F, Driessen EW, Hodges B. The assessment of professional competence: building blocks for theory development. Best Pract Res Clin Obstet Gynaecol. 2010;24:703–19.CrossRef
7.
go back to reference Muijtjens AM, Timmermans I, Donkers J, et al. Flexible electronic feedback using the virtues of progress testing. Med Teach. 2010;32:491–5.CrossRef Muijtjens AM, Timmermans I, Donkers J, et al. Flexible electronic feedback using the virtues of progress testing. Med Teach. 2010;32:491–5.CrossRef
8.
go back to reference Govaerts MJB, van der Vleuten CPM. Validity in work-based assessment: expanding our horizons. Med Educ. 2013;47:1164–74.CrossRef Govaerts MJB, van der Vleuten CPM. Validity in work-based assessment: expanding our horizons. Med Educ. 2013;47:1164–74.CrossRef
9.
go back to reference Ginsburg S, Eva K, Regehr G. Do in-training evaluation reports deserve their bad reputations? A study of the reliability and predictive ability of ITER scores and narrative comments. Acad Med. 2013;88:1539–44.CrossRef Ginsburg S, Eva K, Regehr G. Do in-training evaluation reports deserve their bad reputations? A study of the reliability and predictive ability of ITER scores and narrative comments. Acad Med. 2013;88:1539–44.CrossRef
10.
go back to reference Watling C, Driessen E, van der Vleuten CP, Lingard L. Learning from clinical work: the roles of learning cues and credibility judgements. Med Educ. 2012;46:192–200.CrossRef Watling C, Driessen E, van der Vleuten CP, Lingard L. Learning from clinical work: the roles of learning cues and credibility judgements. Med Educ. 2012;46:192–200.CrossRef
11.
go back to reference Driessen EW, Overeem K. Mentoring. In: Walsh K, editor. Oxford textbook of medical education. Oxford: Oxford University Press; 2013. pp. 265–84. Driessen EW, Overeem K. Mentoring. In: Walsh K, editor. Oxford textbook of medical education. Oxford: Oxford University Press; 2013. pp. 265–84.
12.
go back to reference Harrison CJ, Könings KD, Schuwirth L, Wass V, Vleuten C van der. Barriers to the uptake and use of feedback in the context of summative assessment. Adv Health Sci Educ. 2015;20:229–45.CrossRef Harrison CJ, Könings KD, Schuwirth L, Wass V, Vleuten C van der. Barriers to the uptake and use of feedback in the context of summative assessment. Adv Health Sci Educ. 2015;20:229–45.CrossRef
13.
go back to reference Sargeant J, Eva KW, Armson H, et al. Features of assessment learners use to make informed self-assessments of clinical performance. Med Teach. 2011;45:636–47. Sargeant J, Eva KW, Armson H, et al. Features of assessment learners use to make informed self-assessments of clinical performance. Med Teach. 2011;45:636–47.
14.
go back to reference Carpenter JL. Cost analysis of objective structured clinical examinations. Acad Med. 1995;70:828–33. Carpenter JL. Cost analysis of objective structured clinical examinations. Acad Med. 1995;70:828–33.
15.
go back to reference Schuwirth LWT, Vleuten CPM van der. Cost-effective assessment. In: Walsh K, editor. Cost effectiveness in medical education. Oxford: Radcliffe; 2010. pp. 94–100. Schuwirth LWT, Vleuten CPM van der. Cost-effective assessment. In: Walsh K, editor. Cost effectiveness in medical education. Oxford: Radcliffe; 2010. pp. 94–100.
16.
go back to reference Wrigley W, Vleuten CP van der, Freeman A, Muijtjens A. A systemic framework for the progress test: strengths, constraints and issues: AMEE Guide. Med Teach. 2012;34(71):683–97.CrossRef Wrigley W, Vleuten CP van der, Freeman A, Muijtjens A. A systemic framework for the progress test: strengths, constraints and issues: AMEE Guide. Med Teach. 2012;34(71):683–97.CrossRef
17.
go back to reference Boardman AE, Greenberg DH, Vining AR, Weimer DL. Cost-benefit analysis: concepts and practice. New York: Pearson; 2011. Boardman AE, Greenberg DH, Vining AR, Weimer DL. Cost-benefit analysis: concepts and practice. New York: Pearson; 2011.
18.
go back to reference Siegel JE, Weinstein MC, Russell LB, Gold MR. Recommendations for reporting cost-effectiveness analyses. JAMA. 1996;276:1339–41.CrossRef Siegel JE, Weinstein MC, Russell LB, Gold MR. Recommendations for reporting cost-effectiveness analyses. JAMA. 1996;276:1339–41.CrossRef
19.
go back to reference Ricketts C, Bligh J. Developing a “frequent look and rapid remediation” assessment system for a new medical school. Acad Med. 2011;86:67–71.CrossRef Ricketts C, Bligh J. Developing a “frequent look and rapid remediation” assessment system for a new medical school. Acad Med. 2011;86:67–71.CrossRef
20.
go back to reference Harrison CJ, Könings KD, Dannefer EF, Schuwirth LWT, Wass V, Vleuten CPM van der. Factors influencing students’ receptivity to formative feedback emerging from different assessment cultures. Perspect Med Educ. 2016. doi:10.1007/s40037-016-0297-x Harrison CJ, Könings KD, Dannefer EF, Schuwirth LWT, Wass V, Vleuten CPM van der. Factors influencing students’ receptivity to formative feedback emerging from different assessment cultures. Perspect Med Educ. 2016. doi:10.​1007/​s40037-016-0297-x
21.
go back to reference Karpicke JD, Roediger HL 3rd. The critical importance of retrieval for learning. Science. 2008;319:966–8.CrossRef Karpicke JD, Roediger HL 3rd. The critical importance of retrieval for learning. Science. 2008;319:966–8.CrossRef
22.
go back to reference Dannefer EF, Henson LC. The portfolio approach to competency-based assessment at the Cleveland Clinic Lerner College of Medicine. Acad Med. 2007;82:493–502.CrossRef Dannefer EF, Henson LC. The portfolio approach to competency-based assessment at the Cleveland Clinic Lerner College of Medicine. Acad Med. 2007;82:493–502.CrossRef
23.
go back to reference Cilliers FJ, Schuwirth LW, Herman N, Adendorff HJ, van der Vleuten CP. A model of the pre-assessment learning effects of summative assessment in medical education. Adv Health Sci Educ Theory Pract. 2012;17:39–53.CrossRef Cilliers FJ, Schuwirth LW, Herman N, Adendorff HJ, van der Vleuten CP. A model of the pre-assessment learning effects of summative assessment in medical education. Adv Health Sci Educ Theory Pract. 2012;17:39–53.CrossRef
24.
go back to reference Cohen-Schotanus J, van der Vleuten CP. A standard setting method with the best performing students as point of reference: practical and affordable. Med Teach. 2010;32:154–60.CrossRef Cohen-Schotanus J, van der Vleuten CP. A standard setting method with the best performing students as point of reference: practical and affordable. Med Teach. 2010;32:154–60.CrossRef
25.
go back to reference Cusimano MD, Cohen R, Tucker W, Murnaghan J, Kodama R, Reznick R. A comparative analysis of the costs of administration of an OSCE (objective structured clinical examination). Acad Med. 1994;69:571–6.CrossRef Cusimano MD, Cohen R, Tucker W, Murnaghan J, Kodama R, Reznick R. A comparative analysis of the costs of administration of an OSCE (objective structured clinical examination). Acad Med. 1994;69:571–6.CrossRef
26.
go back to reference Daelmans HE, Overmeer RM, van der Hem-Stokroos HH, Scherpbier AJ, Stehouwer CD, van der Vleuten CP. In-training assessment: qualitative study of effects on supervision and feedback in an undergraduate clinical rotation. Med Educ. 2006;40:51–8.CrossRef Daelmans HE, Overmeer RM, van der Hem-Stokroos HH, Scherpbier AJ, Stehouwer CD, van der Vleuten CP. In-training assessment: qualitative study of effects on supervision and feedback in an undergraduate clinical rotation. Med Educ. 2006;40:51–8.CrossRef
27.
go back to reference Watling C, Driessen E, van der Vleuten CP, Vanstone M, Lingard L. Beyond individualism: professional culture and its influence on feedback. Med Educ. 2013;47:585–94.CrossRef Watling C, Driessen E, van der Vleuten CP, Vanstone M, Lingard L. Beyond individualism: professional culture and its influence on feedback. Med Educ. 2013;47:585–94.CrossRef
28.
go back to reference Heeneman S, Oudkerk Pool A, Schuwirth LW, van der Vleuten CP, Driessen EW. The impact of programmatic assessment on student learning: theory versus practice. Med Educ. 2015;49:487–98.CrossRef Heeneman S, Oudkerk Pool A, Schuwirth LW, van der Vleuten CP, Driessen EW. The impact of programmatic assessment on student learning: theory versus practice. Med Educ. 2015;49:487–98.CrossRef
29.
go back to reference Könings KD, van Berlo J, Koopmans R, et al. Using a smartphone app and coaching group sessions to promote residents’ reflection in the workplace. Acad Med. 2016;91:365–70.CrossRef Könings KD, van Berlo J, Koopmans R, et al. Using a smartphone app and coaching group sessions to promote residents’ reflection in the workplace. Acad Med. 2016;91:365–70.CrossRef
30.
go back to reference Van der Vleuten CPM. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ. 1996;1:41–67.CrossRef Van der Vleuten CPM. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ. 1996;1:41–67.CrossRef
Metadata
Title
On the issue of costs in programmatic assessment
Authors
Cees P. M. van der Vleuten
Sylvia Heeneman
Publication date
01-10-2016
Publisher
Bohn Stafleu van Loghum
Published in
Perspectives on Medical Education / Issue 5/2016
Print ISSN: 2212-2761
Electronic ISSN: 2212-277X
DOI
https://doi.org/10.1007/s40037-016-0295-z

Other articles of this Issue 5/2016

Perspectives on Medical Education 5/2016 Go to the issue