Top

World Journal of Surgery

Published in:

01-03-2015 | Original Scientific Report

The Modern Surgeon and Competency Assessment: Are the Workplace-Based Assessments Evidence-Based?

Authors: K. M. Torsney, D. M. Cocker, A. A. P. Slesser

Published in: World Journal of Surgery | Issue 3/2015

Abstract

Introduction

The assessment of higher surgical training has changed in the last decade or two, with a greater emphasis on work-based assessments (WBAs) to prove competency. The aim of this study was to determine the evidence underpinning the use and number of WBAs in surgical training.

Methods

In July 2013, a systematic electronic literature review was undertaken using PubMed (Medline), Embase, Google Scholar and the Cochrane library.

Results

A total of 27 studies met the inclusion criteria of which 25 were observational studies and only five assessed WBAs in a surgical setting. Validity and feasibility in surgical training were assessed in two studies, respectively, with the results suggesting that WBAs maybe neither feasible nor valid in surgical training. The number required to achieve reliability in surgical training was demonstrated to be three in two separate studies. The evidence for the reliability, feasibility and validity of WBAs in other non-surgical fields was conflicting.

Conclusion

There is a paucity of evidence supporting the use of WBAs as a tool to determine competency in surgical training, and as such, they should only have a limited role in training until more evidence is available. There appears to be no justification or evidence underpinning the use of a specific number of WBAs to determine surgical competency.

Fitzgerald JEF, Giddings CEB, Khera G et al (2012) Improving the future of surgical training and education: consensus recommendations from the Association of Surgeons in Training. Int J Surg 10(8):389–392CrossRefPubMed

Shalhoub J, Vesey AT, Fitzgerald JE (2014) What evidence is there for the use of Workplace-based assessment in surgical training? J Surg Educ. doi:10.1016/j.jsurg.2014.03.013

General Medical Council (2011) Learning and assessment in the clinical environment: the way forward. General Medical Council, United Kingdom

Munsch C (2009) British surgeons’ experiences of mandatory online workplace-based assessment—reply to authors. J R Soc Med 102(9):357–358CrossRefPubMedCentralPubMed

Pentlow A (2013) Workplace-based assessments in surgery: are we heading in the wrong direction? Med Teach 35(1):80CrossRefPubMed

Bindal T, Wall D, Goodyear HM (2011) Trainee doctors’ views on workplace-based assessments: Are they just a tick box exercise? Med Teach 33(11):919–927CrossRefPubMed

Moher D, Liberati A, Tetzlaff J et al (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339:b2535CrossRefPubMedCentralPubMed

Eraut M (2009) The ISCP evaluation report. Joint Committee on Surgical Training

ISCP Management Group (2012) ISCP evaluation report. http://www.jcst.org/ISCP%20Evaluation%20/full_evaluation_report

10.

Buckley S, Coleman J, Davison I et al (2009) The educational effects of portfolios on undergraduate student learning: a best evidence medical education (BEME) systematic review. BEME Guide No 11. Med Teach 31:282–298CrossRefPubMed

11.

Barton JR, Corbett S, van der Vleuten CP (2012) The validity and reliability of a direct observation of procedural skills assessment tool: assessing colonoscopic skills of senior endoscopists. Gastrointest Endosc 75(3):591–597CrossRefPubMed

12.

Brazil V, Ratcliffe L, Zhang J et al (2012) Mini-CEX as a workplace-based assessment tool for interns in an emergency department—does cost outweight value? Med Teach 34:1017–1023CrossRefPubMed

13.

Beard JD (2008) Assessment of surgical skills of trainees in the UK. Ann R Coll Surg Engl 90:282–285CrossRefPubMedCentralPubMed

14.

Crossley J, Johnson G, Booth J et al (2011) Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ 45:560–569CrossRefPubMed

15.

De Lima AA, Barrero C, Bararra S et al (2007) Validity, reliability, feasibility and satisfaction of the mini-clinical evaluation exercise (Mini-CEX) for cardiology residency training. Med Teach 29:785–790CrossRef

16.

Durning SJ, Cation LJ, Markert RJ (2002) Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med 77(9):900–904CrossRefPubMed

17.

Hassan S (2011) Faculty development: mini-CEX as workplace-based assessment. Educ Med J 3(1):12–21CrossRef

18.

Hill F, Kendall K, Galbraith K et al (2009) Implementing the undergraduate mini-CEX: a tailored approach at Southampton University. Med Educ 43:326–334CrossRefPubMed

19.

Holmboe ES, Huot S, Chung J et al (2003) Construct validity of the mini-clinical evaluation exercise (minCEX). Acad Med 78(8):826–830CrossRefPubMed

20.

James K, Cross K, Lucarotti ME et al (2009) Undertaking procedure-based assessment is feasible in clinical practice. Ann R Coll Surg Engl 91:110–112CrossRefPubMedCentralPubMed

21.

Johnson G, Booth J, Crossley J et al (2011) Assessing trainees in the workplace: results of a pilot study. Clin Med 11(1):48–53CrossRefPubMed

22.

Kogan JR, Bellini LM, Shea JA (2003) Feasibility, reliability and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med 78(10):S33–S35CrossRefPubMed

23.

Kogan JR, Bellini LM, Shea JA (2002) Implementation of the mini-CEX to evaluate medical students’ clinical skills. Acad Med 77(11):1156–1157CrossRefPubMed

24.

Malhotra S, Hatalal R, Courneya CA (2008) Internal medicine residents’ perceptions of the mini-clinical evaluation exercise. Med Teach 30:414–419CrossRefPubMed

25.

Marriott J, Purdie H, Crossley J et al (2011) Evaluation of procedure-based assessment for assessing trainees’ skills in the operating theatre. Br J Surg 98:450–457CrossRefPubMed

26.

Mitchell C, Bhat S, Herbert A et al (2011) Workplace-based assessments of junior doctors: do scores predict training difficulties? Med Educ 45:1190–1198CrossRefPubMed

27.

Moonen-van Loon JMW, Overeem K, Donkers HHLM et al (2013) Composite reliability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ 18(5):1087–1102CrossRef

28.

Morris A, Hewitt J, Roberts CM (2006) Practical experience of using directly observed procedures, mini clinical evaluation examinations, and peer observation in pre-registration house officer (FY1) trainees. Postgrad Med J 82:285–288CrossRefPubMedCentralPubMed

29.

Nair B, Hensley MJ, Parvathy MS et al (2012) A systematic approach to workplace-based assessment for international medical graduates. Med J Aust 196:399–402CrossRefPubMed

30.

Nair BR, Alexander HG, McGrath BP et al (2008) The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates. Med Educ 189(3):159–161

31.

Norcini JJ, Blank LL, Arnold GK et al (1995) The mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med 123:795–799CrossRefPubMed

32.

Norcini JJ, Blank LL, Arnold GK et al (1997) Examiner differences in the mini-Cex. Adv Health Sci Educ 2:27–33CrossRef

33.

Norcini JJ, Blank LL, Duffy DF et al (2003) The mini-CEX: a method for assessing clinical skills. Ann Intern Med 138:476–481CrossRefPubMed

34.

Playford D, Kirke A, Maley M et al (2013) Longitudinal assessment in an undergraduate longitudinal integrated clerkship: the mini clinical evaluation exercise (mCEX) profile. Med Teach 35(8):1412–1421CrossRef

35.

Satpathy A, Kneebone R (2011) Workplace-based assessment in surgical training: a qualitative study. Ann R Coll Surg Engl 93(5):1–5CrossRef

36.

Wilkinson JR, Crossley JGM, Wragg A et al (2008) Implementing workplace-based assessment across the medical specialties in the United Kingdom. Med Educ 42:364–373CrossRefPubMed

37.

Weller JM, Jolly B, Misur MP et al (2009) Mini-clinical evaluation exercise in anaesthesia training. Br J Anaesth 102:633–641CrossRefPubMed

38.

Beard J, Rowley D, Bussey M et al (2009) Workplace-based assessment: assessing technical skill throughout the continuum of surgical training. ANZ J Surg 79(3):148–153CrossRefPubMed

Title: The Modern Surgeon and Competency Assessment: Are the Workplace-Based Assessments Evidence-Based?
Authors: K. M. Torsney
D. M. Cocker
A. A. P. Slesser
Publication date: 01-03-2015
Publisher: Springer US
Published in: World Journal of Surgery / Issue 3/2015
Print ISSN: 0364-2313
Electronic ISSN: 1432-2323
DOI: https://doi.org/10.1007/s00268-014-2875-6

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

The Modern Surgeon and Competency Assessment: Are the Workplace-Based Assessments Evidence-Based?

Abstract

Introduction

Methods

Results

Conclusion

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Introduction

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 3/2015

Inventing the Future of Surgery

A Comparison of Video Laryngoscopy to Direct Laryngoscopy for the Emergency Intubation of Trauma Patients

Pedestrian Injuries-Related Deaths: A Global Evaluation

Improving Outcomes in Breast Cancer for Low and Middle Income Countries

Rate of Pulmonary Metastasis Varies with Location of Rectal Cancer in the Patients Undergoing Curative Resection

Big Data as the Universal Language for Barrett’s Esophagus