Skip to main content
Top
Published in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie 12/2016

01-12-2016 | Reports of Original Investigations

Perceptions of purpose, value, and process of the mini-Clinical Evaluation Exercise in anesthesia training

Authors: Damian J. Castanelli, MBBS, Tanisha Jowsey, PhD, Yan Chen, PhD, Jennifer M. Weller, MD

Published in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie | Issue 12/2016

Login to get access

Abstract

Introduction

Workplace-based assessment is integral to programmatic assessment in a competency-based curriculum. In 2013, one such assessment, a mini-Clinical Evaluation Exercise (mini-CEX) with a novel “entrustability scale”, became compulsory for over 1,200 Australia and New Zealand College of Anaesthetists (ANZCA) trainees. We explored trainees’ and supervisors’ understanding of the mini-CEX, their experience with the assessment, and their perceptions of its influence on learning and supervision.

Methods

We conducted semi-structured telephone interviews with anesthesia supervisors and trainees and performed an inductive thematic analysis of the verbatim transcripts.

Results

Eighteen supervisors and 17 trainees participated (n = 35). Interrelated themes concerned the perceived purpose of the mini-CEX, its value in trainee learning and supervision, and the process of performing the assessment. While few participants saw the mini-CEX primarily as an administrative burden, most focused on its potential for facilitating trainee improvement and reported positive impacts on the quantity and quality of feedback, trainee learning, and supervision. Finding time to schedule assessments and deliver timely feedback proved to be difficult in busy clinical workplaces. Views on case selection were divided and driven by contrasting goals – i.e., receiving useful feedback on challenging cases or receiving a high score by choosing lenient assessors or easy cases. Whether individual mini-CEXs were summative or formative was subject to intense debate, while the intended summative use of multiple mini-CEXs in programmatic assessment was poorly understood.

Conclusion

Greater clarity of purpose and consistency of time commitment are necessary to embed the mini-CEX in the culture of the workplace, to realize the full potential for trainee learning, and to reach decisions on trainee progression.
Appendix
Available only for authorised users
Literature
1.
go back to reference Iobst WF, Sherbino J, Cate OT, et al. Competency-based medical education in postgraduate medical education. Med Teach 2010; 32: 651-6.CrossRefPubMed Iobst WF, Sherbino J, Cate OT, et al. Competency-based medical education in postgraduate medical education. Med Teach 2010; 32: 651-6.CrossRefPubMed
2.
go back to reference Miller GE. The assessment of clinical skills/ competence/ performance. Acad Med 1990; 65(9 Suppl): S63-7.CrossRefPubMed Miller GE. The assessment of clinical skills/ competence/ performance. Acad Med 1990; 65(9 Suppl): S63-7.CrossRefPubMed
3.
go back to reference Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 2007; 29: 855-71.CrossRefPubMed Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 2007; 29: 855-71.CrossRefPubMed
4.
go back to reference Scheele F, Teunissen P, Van Luijk S, et al. Introducing competency-based postgraduate medical education in the Netherlands. Med Teach 2008; 30: 248-53.CrossRefPubMed Scheele F, Teunissen P, Van Luijk S, et al. Introducing competency-based postgraduate medical education in the Netherlands. Med Teach 2008; 30: 248-53.CrossRefPubMed
5.
go back to reference Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach 2010; 32: 676-82.CrossRefPubMed Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach 2010; 32: 676-82.CrossRefPubMed
6.
go back to reference Schuwirth LW, Van der Vleuten CP. Programmatic assessment: From assessment of learning to assessment for learning. Med Teach 2011; 33: 478-85.CrossRefPubMed Schuwirth LW, Van der Vleuten CP. Programmatic assessment: From assessment of learning to assessment for learning. Med Teach 2011; 33: 478-85.CrossRefPubMed
7.
go back to reference Kogan JR, Holmboe E. Realizing the promise and importance of performance-based assessment. Teach Learn Med 2013; 25(Suppl 1): S68-74.CrossRefPubMed Kogan JR, Holmboe E. Realizing the promise and importance of performance-based assessment. Teach Learn Med 2013; 25(Suppl 1): S68-74.CrossRefPubMed
8.
go back to reference Schuwirth LW, van der Vleuten CP. General overview of the theories used in assessment: AMEE Guide No. 57. Med Teach 2011; 33: 783-97.CrossRefPubMed Schuwirth LW, van der Vleuten CP. General overview of the theories used in assessment: AMEE Guide No. 57. Med Teach 2011; 33: 783-97.CrossRefPubMed
9.
go back to reference van der Vleuten CP, Schuwirth LW, Driessen EW, et al. A model for programmatic assessment fit for purpose. Med Teach 2012; 34: 205-14.CrossRefPubMed van der Vleuten CP, Schuwirth LW, Driessen EW, et al. A model for programmatic assessment fit for purpose. Med Teach 2012; 34: 205-14.CrossRefPubMed
10.
go back to reference Bindal N, Goodyear H, Bindal T, Wall D. DOPS assessment: a study to evaluate the experience and opinions of trainees and assessors. Med Teach 2013; 35: e1230-4.CrossRefPubMed Bindal N, Goodyear H, Bindal T, Wall D. DOPS assessment: a study to evaluate the experience and opinions of trainees and assessors. Med Teach 2013; 35: e1230-4.CrossRefPubMed
11.
go back to reference Bindal T, Wall D, Goodyear HM. Trainee doctors’ views on workplace-based assessments: are they just a tick box exercise? Med Teach 2011; 33: 919-27.CrossRefPubMed Bindal T, Wall D, Goodyear HM. Trainee doctors’ views on workplace-based assessments: are they just a tick box exercise? Med Teach 2011; 33: 919-27.CrossRefPubMed
12.
go back to reference Bok HG, Teunissen PW, Favier RP, et al. Programmatic assessment of competency-based workplace learning: when theory meets practice. BMC Med Educ 2013; 13: 123.CrossRefPubMedPubMedCentral Bok HG, Teunissen PW, Favier RP, et al. Programmatic assessment of competency-based workplace learning: when theory meets practice. BMC Med Educ 2013; 13: 123.CrossRefPubMedPubMedCentral
14.
go back to reference Cho SP, Parry D, Wade W. Lessons learnt from a pilot of assessment for learning. Clin Med (Lond) 2014; 14: 577-84.CrossRef Cho SP, Parry D, Wade W. Lessons learnt from a pilot of assessment for learning. Clin Med (Lond) 2014; 14: 577-84.CrossRef
15.
go back to reference Rees CE, Cleland JA, Dennis A, Kelly N, Mattick K, Monrouxe LV. Supervised learning events in the foundation programme: a UK-wide narrative interview study. BMJ Open 2014; 4: e005980.CrossRefPubMedPubMedCentral Rees CE, Cleland JA, Dennis A, Kelly N, Mattick K, Monrouxe LV. Supervised learning events in the foundation programme: a UK-wide narrative interview study. BMJ Open 2014; 4: e005980.CrossRefPubMedPubMedCentral
16.
go back to reference Ringsted C, Ostergaard D, Scherpbier A. Embracing the new paradigm of assessment in residency training: an assessment programme for first-year residency training in anaesthesiology. Med Teach 2003; 25: 54-62.CrossRefPubMed Ringsted C, Ostergaard D, Scherpbier A. Embracing the new paradigm of assessment in residency training: an assessment programme for first-year residency training in anaesthesiology. Med Teach 2003; 25: 54-62.CrossRefPubMed
20.
go back to reference Weller JM, Jones A, Merry AF, Jolly B, Saunders D. Investigation of trainee and specialist reactions to the mini-clinical evaluation exercise in anaesthesia: implications for implementation. Br J Anaesth 2009; 103: 524-30.CrossRefPubMed Weller JM, Jones A, Merry AF, Jolly B, Saunders D. Investigation of trainee and specialist reactions to the mini-clinical evaluation exercise in anaesthesia: implications for implementation. Br J Anaesth 2009; 103: 524-30.CrossRefPubMed
21.
go back to reference Weller JM, Jolly B, Misur MP, et al. Mini-clinical evaluation exercise in anaesthesia training. Br J Anaesth 2009; 102: 633-41.CrossRefPubMed Weller JM, Jolly B, Misur MP, et al. Mini-clinical evaluation exercise in anaesthesia training. Br J Anaesth 2009; 102: 633-41.CrossRefPubMed
22.
go back to reference Weller JM, Misur M, Nicolson S, et al. Can I leave the theatre? A key to more reliable workplace-based assessment. Br J Anaesth 2014; 112: 1083-91.CrossRefPubMed Weller JM, Misur M, Nicolson S, et al. Can I leave the theatre? A key to more reliable workplace-based assessment. Br J Anaesth 2014; 112: 1083-91.CrossRefPubMed
23.
go back to reference Rekman J, Gofton W, Dudek N, Gofton T, Hamstra SJ. Entrustability scales: outlining their usefulness for competency-based clinical assessment. Acad Med 2016; 91: 186-90.CrossRefPubMed Rekman J, Gofton W, Dudek N, Gofton T, Hamstra SJ. Entrustability scales: outlining their usefulness for competency-based clinical assessment. Acad Med 2016; 91: 186-90.CrossRefPubMed
24.
go back to reference Schwandt TA. Three epistemological stances for qualitative inquiry: interpretivism, hermeneutics and social constructionism. In: Denzin NK, Lincoln YS, editors. Handbook of Qualitative Research. 2nd ed. Newbury Park, CA: Sage; 2000 . Schwandt TA. Three epistemological stances for qualitative inquiry: interpretivism, hermeneutics and social constructionism. In: Denzin NK, Lincoln YS, editors. Handbook of Qualitative Research. 2nd ed. Newbury Park, CA: Sage; 2000 .
26.
go back to reference Liamputtong P, Ezzy D. Qualitative Research Methods. 2nd ed. South Melbourne: Oxford University Press; 2005 . Liamputtong P, Ezzy D. Qualitative Research Methods. 2nd ed. South Melbourne: Oxford University Press; 2005 .
27.
go back to reference Kitto SC, Chesters J, Grbich C. Quality in qualitative research. Med J Aust 2008; 188: 243-6.PubMed Kitto SC, Chesters J, Grbich C. Quality in qualitative research. Med J Aust 2008; 188: 243-6.PubMed
28.
go back to reference Taylor SJ, Bogdan R. Introduction to Qualitative Research Methods: The Search for Meaning. NY: John Wiley & Sons; 1984 . Taylor SJ, Bogdan R. Introduction to Qualitative Research Methods: The Search for Meaning. NY: John Wiley & Sons; 1984 .
29.
go back to reference Savin-Baden M, Major CH. Qualitative Research: the Essential Guide to Theory and Practice: Routledge; 2013. Savin-Baden M, Major CH. Qualitative Research: the Essential Guide to Theory and Practice: Routledge; 2013.
30.
go back to reference Morse JM, Field PA. Qualitative Research Methods for Health Professionals, 2nd ed. California, USA. Sage Publications; 1995: 254. Morse JM, Field PA. Qualitative Research Methods for Health Professionals, 2nd ed. California, USA. Sage Publications; 1995: 254.
31.
go back to reference Saldana J. The Coding Manual for Qualitative Researchers. London: Sage Publications; 2009 . Saldana J. The Coding Manual for Qualitative Researchers. London: Sage Publications; 2009 .
33.
go back to reference Heeneman S, Oudkerk Pool A, Schuwirth LW, van der Vleuten CP, Driessen EW. The impact of programmatic assessment on student learning: theory versus practice. Med Educ 2015; 49: 487-98.CrossRefPubMed Heeneman S, Oudkerk Pool A, Schuwirth LW, van der Vleuten CP, Driessen EW. The impact of programmatic assessment on student learning: theory versus practice. Med Educ 2015; 49: 487-98.CrossRefPubMed
34.
go back to reference Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Adv Health Sci Educ Pract 2016; 21: 455-73.CrossRef Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Adv Health Sci Educ Pract 2016; 21: 455-73.CrossRef
35.
go back to reference van der Vleuten CP, Schuwirth LW, Scheele F, Driessen EW, Hodges B. The assessment of professional competence: building blocks for theory development. Best Pract Res Clin Obstet Gynaecol 2010; 24: 703-19.CrossRefPubMed van der Vleuten CP, Schuwirth LW, Scheele F, Driessen EW, Hodges B. The assessment of professional competence: building blocks for theory development. Best Pract Res Clin Obstet Gynaecol 2010; 24: 703-19.CrossRefPubMed
36.
go back to reference Taras M. Assessment for learning: assessing the theory and evidence. Procedia - Social and Behavioral Sciences 2010; 2: 3015-22.CrossRef Taras M. Assessment for learning: assessing the theory and evidence. Procedia - Social and Behavioral Sciences 2010; 2: 3015-22.CrossRef
37.
go back to reference Downing SM, Yudkowsky R. Assessment in Health Professions Education. Routledge; 2009. Downing SM, Yudkowsky R. Assessment in Health Professions Education. Routledge; 2009.
39.
go back to reference Ramsden P. Learning to Teach in Higher Education. 2nd ed. London: Routledge; 2003 . Ramsden P. Learning to Teach in Higher Education. 2nd ed. London: Routledge; 2003 .
40.
go back to reference Boud D. Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education 2000; 22: 151-67.CrossRef Boud D. Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education 2000; 22: 151-67.CrossRef
41.
go back to reference Olupeliyawa A, Balasooriya C. The impact of programmatic assessment on student learning: what can the students tell us? Med Educ 2015; 49: 453-6.CrossRefPubMed Olupeliyawa A, Balasooriya C. The impact of programmatic assessment on student learning: what can the students tell us? Med Educ 2015; 49: 453-6.CrossRefPubMed
43.
go back to reference Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004; 79: S70-81.CrossRefPubMed Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004; 79: S70-81.CrossRefPubMed
44.
go back to reference Verenikina I. Scaffolding and learning: its role in nurturing new learners. In: Kell P, Vialle W, Konza D, Vogl G, editors. Learning and the Learner: Exploring Learning for New Times. Wollongong, NSW: University of Wollongong; 2008 . Verenikina I. Scaffolding and learning: its role in nurturing new learners. In: Kell P, Vialle W, Konza D, Vogl G, editors. Learning and the Learner: Exploring Learning for New Times. Wollongong, NSW: University of Wollongong; 2008 .
45.
go back to reference Slater RJ, Castanelli DJ, Barrington MJ. Learning and teaching motor skills in regional anesthesia: a different perspective. Reg Anesth Pain Med 2014; 39: 230-9.CrossRefPubMed Slater RJ, Castanelli DJ, Barrington MJ. Learning and teaching motor skills in regional anesthesia: a different perspective. Reg Anesth Pain Med 2014; 39: 230-9.CrossRefPubMed
46.
go back to reference Driessen E, Scheele F. What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research. Med Teach 2013; 35: 569-74.CrossRefPubMed Driessen E, Scheele F. What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research. Med Teach 2013; 35: 569-74.CrossRefPubMed
47.
go back to reference Gingerich A, Kogan J, Yeates P, Govaerts M, Holmboe E. Seeing the ‘black box’ differently: assessor cognition from three research perspectives. Med Educ 2014; 48: 1055-68.CrossRefPubMed Gingerich A, Kogan J, Yeates P, Govaerts M, Holmboe E. Seeing the ‘black box’ differently: assessor cognition from three research perspectives. Med Educ 2014; 48: 1055-68.CrossRefPubMed
48.
go back to reference Govaerts M, van der Vleuten CP. Validity in work-based assessment: expanding our horizons. Med Educ 2013; 47: 1164-74.CrossRefPubMed Govaerts M, van der Vleuten CP. Validity in work-based assessment: expanding our horizons. Med Educ 2013; 47: 1164-74.CrossRefPubMed
50.
go back to reference Tavares W, Eva KW. Exploring the impact of mental workload on rater-based assessments. Adv Health Sci Educ Theory Pract 2013; 18: 291-303.CrossRefPubMed Tavares W, Eva KW. Exploring the impact of mental workload on rater-based assessments. Adv Health Sci Educ Theory Pract 2013; 18: 291-303.CrossRefPubMed
51.
go back to reference Driessen EW, van Tartwijk J, Govaerts M, Teunissen P, van der Vleuten CP. The use of programmatic assessment in the clinical workplace: a Maastricht case report. Med Teach 2012; 34: 226-31.CrossRefPubMed Driessen EW, van Tartwijk J, Govaerts M, Teunissen P, van der Vleuten CP. The use of programmatic assessment in the clinical workplace: a Maastricht case report. Med Teach 2012; 34: 226-31.CrossRefPubMed
52.
go back to reference Fokkema JP, Teunissen PW, Westerman M, et al. Exploration of perceived effects of innovations in postgraduate medical education. Med Educ 2013; 47: 271-81.CrossRefPubMed Fokkema JP, Teunissen PW, Westerman M, et al. Exploration of perceived effects of innovations in postgraduate medical education. Med Educ 2013; 47: 271-81.CrossRefPubMed
53.
go back to reference Steinert Y, Macdonald ME, Boillat M, et al. Faculty development: if you build it, they will come. Med Educ 2010; 44: 900-7.CrossRefPubMed Steinert Y, Macdonald ME, Boillat M, et al. Faculty development: if you build it, they will come. Med Educ 2010; 44: 900-7.CrossRefPubMed
54.
go back to reference Castanelli DJ, Smith NA, Noonan CL. Do anaesthetists believe their teaching is evidence-based? Med Teach 2015; 37: 1098-105.CrossRefPubMed Castanelli DJ, Smith NA, Noonan CL. Do anaesthetists believe their teaching is evidence-based? Med Teach 2015; 37: 1098-105.CrossRefPubMed
55.
go back to reference Dijksterhuis MG, Schuwirth LW, Braat DD, Teunissen PW, Scheele F. A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education. Med Teach 2013; 35: e1396-402.CrossRefPubMed Dijksterhuis MG, Schuwirth LW, Braat DD, Teunissen PW, Scheele F. A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education. Med Teach 2013; 35: e1396-402.CrossRefPubMed
56.
go back to reference Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA 2009; 302: 1277-83.CrossRefPubMed Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA 2009; 302: 1277-83.CrossRefPubMed
57.
go back to reference Johnston MJ, Singh P, Pucher PH, et al. Systematic review with meta-analysis of the impact of surgical fellowship training on patient outcomes. Br J Surg 2015; 102: 1156-66.CrossRefPubMed Johnston MJ, Singh P, Pucher PH, et al. Systematic review with meta-analysis of the impact of surgical fellowship training on patient outcomes. Br J Surg 2015; 102: 1156-66.CrossRefPubMed
Metadata
Title
Perceptions of purpose, value, and process of the mini-Clinical Evaluation Exercise in anesthesia training
Authors
Damian J. Castanelli, MBBS
Tanisha Jowsey, PhD
Yan Chen, PhD
Jennifer M. Weller, MD
Publication date
01-12-2016
Publisher
Springer US
Published in
Canadian Journal of Anesthesia/Journal canadien d'anesthésie / Issue 12/2016
Print ISSN: 0832-610X
Electronic ISSN: 1496-8975
DOI
https://doi.org/10.1007/s12630-016-0740-9

Other articles of this Issue 12/2016

Canadian Journal of Anesthesia/Journal canadien d'anesthésie 12/2016 Go to the issue

Guest Reviewers

Guest Reviewers