Skip to main content
Top
Published in: BMC Psychiatry 1/2011

Open Access 01-12-2011 | Debate

Objective Structured Clinical Examinations (OSCEs), psychiatry and the Clinical assessment of Skills and Competencies (CASC)Same Evidence, Different Judgement

Author: Steven Marwaha

Published in: BMC Psychiatry | Issue 1/2011

Login to get access

Abstract

Background

The Objective Structured Clinical Examination (OSCE), originally developed in the 1970's, has been hailed as the "gold standard" of clinical assessments for medical students and is used within medical schools throughout the world. The Clinical assessment of Skills and Competencies (CASC) is an OSCE used as a clinical examination gateway, granting access to becoming a senior Psychiatrist in the UK.

Discussion

Van der Vleuten's utility model is used to examine the CASC from the viewpoint of a senior psychiatrist. Reliability may be equivalent to more traditional examinations. Whilst the CASC is likely to have content validity, other forms of validity are untested and authenticity is poor. Educational impact has the potential to change facets of psychiatric professionalism and influence future patient care. There are doubts about acceptability from candidates and more senior psychiatrists.

Summary

Whilst OSCEs may be the best choice for medical student examinations, their use in post graduate psychiatric examination in the UK is subject to challenge on the grounds of validity, authenticity and educational impact.
Literature
1.
2.
go back to reference Rothman AI, Cohen R: Proceedings of the 6th Ottawa Conference on medical Education. Toronto: University of Toronto Bookstore Publishing, 292-321. Rothman AI, Cohen R: Proceedings of the 6th Ottawa Conference on medical Education. Toronto: University of Toronto Bookstore Publishing, 292-321.
3.
go back to reference Miller GE: The assessment of clinical skills/competence/performance. Academic Medicine. 1990, 65: 63-67. 10.1097/00001888-199002000-00001.CrossRef Miller GE: The assessment of clinical skills/competence/performance. Academic Medicine. 1990, 65: 63-67. 10.1097/00001888-199002000-00001.CrossRef
4.
go back to reference Wilson GM, Lever R, Harden RMcG, Roberston JIL, MacRitchie J: Examination of Clinical Examiners. The Lancet. 1969, 1: 37-40.CrossRef Wilson GM, Lever R, Harden RMcG, Roberston JIL, MacRitchie J: Examination of Clinical Examiners. The Lancet. 1969, 1: 37-40.CrossRef
5.
go back to reference Maxim BR, Dielman TE: Dimensionality, Internal consistency and inter rater reliability of clinical performance ratings. Medical Education. 1987, 21: 130-137. 10.1111/j.1365-2923.1987.tb00679.x.CrossRefPubMed Maxim BR, Dielman TE: Dimensionality, Internal consistency and inter rater reliability of clinical performance ratings. Medical Education. 1987, 21: 130-137. 10.1111/j.1365-2923.1987.tb00679.x.CrossRefPubMed
6.
go back to reference Harden RMcG, Stevenson M, Downie WW, Wilson GM: Assessment of Clinical Competence using Objective Structured Examination. British Medical Journal. 1975, 1: 447-451. 10.1136/bmj.1.5955.447.CrossRefPubMedPubMedCentral Harden RMcG, Stevenson M, Downie WW, Wilson GM: Assessment of Clinical Competence using Objective Structured Examination. British Medical Journal. 1975, 1: 447-451. 10.1136/bmj.1.5955.447.CrossRefPubMedPubMedCentral
7.
go back to reference Van der Vleuten CPM: The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education. 1996, 1: 41-67. 10.1007/BF00596229.CrossRefPubMed Van der Vleuten CPM: The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education. 1996, 1: 41-67. 10.1007/BF00596229.CrossRefPubMed
8.
go back to reference Van der Vleuten CPM, Schuwirth LWT: Assessing professional competence: from methods to programmes. Medical Education. 2005, 39: 309-317. 10.1111/j.1365-2929.2005.02094.x.CrossRefPubMed Van der Vleuten CPM, Schuwirth LWT: Assessing professional competence: from methods to programmes. Medical Education. 2005, 39: 309-317. 10.1111/j.1365-2929.2005.02094.x.CrossRefPubMed
9.
go back to reference Vu NV, Barrows HS: Use of standardized Patients in Clinical Assessments: Recent Developments and Measurements Findings. Educational Researcher. 1994, 23: 23-30.CrossRef Vu NV, Barrows HS: Use of standardized Patients in Clinical Assessments: Recent Developments and Measurements Findings. Educational Researcher. 1994, 23: 23-30.CrossRef
10.
go back to reference Turner JL, Dankoski ME: Objective Structured Clinical exam: A Critical Review. Family Medicine. 2008, 40: 574-8.PubMed Turner JL, Dankoski ME: Objective Structured Clinical exam: A Critical Review. Family Medicine. 2008, 40: 574-8.PubMed
11.
go back to reference Boulet JR, McKinley DW, Whelan GP, Hambleton RK: Quality assurance methods for performance-based assessments. Advances Health Science Education Theory Practice. 2003, 8: 27-47. 10.1023/A:1022639521218.CrossRef Boulet JR, McKinley DW, Whelan GP, Hambleton RK: Quality assurance methods for performance-based assessments. Advances Health Science Education Theory Practice. 2003, 8: 27-47. 10.1023/A:1022639521218.CrossRef
12.
go back to reference Swanson DB, Norman Linn RL: Performance-based assessment: lessons from health professions. Educational research. 1995, 24: 5-11.CrossRef Swanson DB, Norman Linn RL: Performance-based assessment: lessons from health professions. Educational research. 1995, 24: 5-11.CrossRef
13.
go back to reference Sloan DA, Donnelly MB, Schwartz Strodel WE: The Objective Structured Clinical Examination. Annals of Surgery. 1995, 222: 735-742. 10.1097/00000658-199512000-00007.CrossRefPubMedPubMedCentral Sloan DA, Donnelly MB, Schwartz Strodel WE: The Objective Structured Clinical Examination. Annals of Surgery. 1995, 222: 735-742. 10.1097/00000658-199512000-00007.CrossRefPubMedPubMedCentral
14.
go back to reference Swanson DB: A measurement framework for performance based tests. Further developments in assessing clinical competence. Edited by: Hart IR, Harden RM. 1987, Montreal: Can-Heal, 13-45. Swanson DB: A measurement framework for performance based tests. Further developments in assessing clinical competence. Edited by: Hart IR, Harden RM. 1987, Montreal: Can-Heal, 13-45.
15.
go back to reference Van der Vleuten CPM, Swanson DB: Assessment of clinical skills with standardized patients: state of the art. Teaching and Learning in Medicine. 1990, 2: 58-76. 10.1080/10401339009539432.CrossRef Van der Vleuten CPM, Swanson DB: Assessment of clinical skills with standardized patients: state of the art. Teaching and Learning in Medicine. 1990, 2: 58-76. 10.1080/10401339009539432.CrossRef
16.
go back to reference Swanson DB, Norcini JJ: Factors influencing reliability of tests using standardized patients. Teach Learn Med. 1989, 1: 158-66. 10.1080/10401338909539401.CrossRef Swanson DB, Norcini JJ: Factors influencing reliability of tests using standardized patients. Teach Learn Med. 1989, 1: 158-66. 10.1080/10401338909539401.CrossRef
17.
go back to reference Wass V, Jones R, van der Vleuten : Standardised or real patients to test clinical competence? The long case revisited. Medical Education. 2001, 35: 321-5. 10.1046/j.1365-2923.2001.00928.x.CrossRefPubMed Wass V, Jones R, van der Vleuten : Standardised or real patients to test clinical competence? The long case revisited. Medical Education. 2001, 35: 321-5. 10.1046/j.1365-2923.2001.00928.x.CrossRefPubMed
18.
go back to reference Van der Vleuten CPM, Normal GR, DeGraff E: Pitfalls in the persuit of objectivity: issues of reliability. Medical Education. 1991, 25: 110-118. 10.1111/j.1365-2923.1991.tb00036.x.CrossRefPubMed Van der Vleuten CPM, Normal GR, DeGraff E: Pitfalls in the persuit of objectivity: issues of reliability. Medical Education. 1991, 25: 110-118. 10.1111/j.1365-2923.1991.tb00036.x.CrossRefPubMed
19.
go back to reference Regehr G, MacRae H, Reznick R, Szalay D: Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Academic Medicine. 1998, 73: 993-7. 10.1097/00001888-199809000-00020.CrossRefPubMed Regehr G, MacRae H, Reznick R, Szalay D: Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Academic Medicine. 1998, 73: 993-7. 10.1097/00001888-199809000-00020.CrossRefPubMed
20.
go back to reference Carraccio C, Englander R: The objective structured clinical examination: a step in the direction of competency-based evaluation. Arch Pediatr Adolesc Med. 2000, 154: 736-41.CrossRefPubMed Carraccio C, Englander R: The objective structured clinical examination: a step in the direction of competency-based evaluation. Arch Pediatr Adolesc Med. 2000, 154: 736-41.CrossRefPubMed
21.
go back to reference Pierre R, Wierenga A, Barton M, Branday JM, Christie CDC: Student evaluation of an OSCE in Paediatric at the Univsertisty of the West Indies, Jamaica. BMC Medical Education. 2004, 4: 22-10.1186/1472-6920-4-22.CrossRefPubMedPubMedCentral Pierre R, Wierenga A, Barton M, Branday JM, Christie CDC: Student evaluation of an OSCE in Paediatric at the Univsertisty of the West Indies, Jamaica. BMC Medical Education. 2004, 4: 22-10.1186/1472-6920-4-22.CrossRefPubMedPubMedCentral
22.
go back to reference Auewarakul C, Downing SM, Jaturatamrong U, Praditsuwan R: Sources of validity evidence for an internal medicine student evaluation system: an evaluative study of assessment methods. Medical Education. 2005, 39: 276-83. 10.1111/j.1365-2929.2005.02090.x.CrossRefPubMed Auewarakul C, Downing SM, Jaturatamrong U, Praditsuwan R: Sources of validity evidence for an internal medicine student evaluation system: an evaluative study of assessment methods. Medical Education. 2005, 39: 276-83. 10.1111/j.1365-2929.2005.02090.x.CrossRefPubMed
23.
go back to reference Barman A: Critiques on the Objective Structured Clinical examination. 2005, Annals of the Academy of Medicine of Singapore, 34: 478-82. Barman A: Critiques on the Objective Structured Clinical examination. 2005, Annals of the Academy of Medicine of Singapore, 34: 478-82.
24.
go back to reference Hodges B, Regehr G, Hanson M, McNaughton N: Validation of an objective structured clinical examination in psychiatry. Academic medicine. 1998, 73: 910-2. 10.1097/00001888-199808000-00019.CrossRefPubMed Hodges B, Regehr G, Hanson M, McNaughton N: Validation of an objective structured clinical examination in psychiatry. Academic medicine. 1998, 73: 910-2. 10.1097/00001888-199808000-00019.CrossRefPubMed
25.
go back to reference Park RS, Chibnall JT, Blaskiewicz , Furman G, Powell J, Mohr CJ: Construct Validity of an Objective Structured Clinical examination (OSCE) in Psychiatry: Associations with the Clinical Skillls examination and other indicators. Academic Psychiatry. 2004, 28: 122-128. 10.1176/appi.ap.28.2.122.CrossRefPubMed Park RS, Chibnall JT, Blaskiewicz , Furman G, Powell J, Mohr CJ: Construct Validity of an Objective Structured Clinical examination (OSCE) in Psychiatry: Associations with the Clinical Skillls examination and other indicators. Academic Psychiatry. 2004, 28: 122-128. 10.1176/appi.ap.28.2.122.CrossRefPubMed
26.
go back to reference Hodges B, Lofchy J: Evaluating psychiatry clinical clerks with a mini-objective structured clinical examination. Academic psychiatry. 1997, 21: 219-225.CrossRefPubMed Hodges B, Lofchy J: Evaluating psychiatry clinical clerks with a mini-objective structured clinical examination. Academic psychiatry. 1997, 21: 219-225.CrossRefPubMed
27.
go back to reference Hodges B, Hanson M, McNaughton N, Regehr : What do Psychiatry residents think of an Objective Structured Clinical Examination. Academic Psychiatry. 1999, 23: 198-204.CrossRef Hodges B, Hanson M, McNaughton N, Regehr : What do Psychiatry residents think of an Objective Structured Clinical Examination. Academic Psychiatry. 1999, 23: 198-204.CrossRef
28.
go back to reference Norman GR, Tugwell P, Feighter JW: A comparison of residents performance on real and simulated patients. Journal of Medical Education. 1982, 57: 708-715.PubMed Norman GR, Tugwell P, Feighter JW: A comparison of residents performance on real and simulated patients. Journal of Medical Education. 1982, 57: 708-715.PubMed
29.
go back to reference Rethans JJ, Drop R, Sturmans F, van der Vleuten C: A method for introducing standardized patients into general practice consultations. British Journal of General Practice. 1991, 41: 94-96.PubMedPubMedCentral Rethans JJ, Drop R, Sturmans F, van der Vleuten C: A method for introducing standardized patients into general practice consultations. British Journal of General Practice. 1991, 41: 94-96.PubMedPubMedCentral
30.
go back to reference Badger LW, DeGruy F, Hartman J, et al: Stability of standardised patients performance in a study of clinical decision making. Family Medicine. 1995, 27: 126-133.PubMed Badger LW, DeGruy F, Hartman J, et al: Stability of standardised patients performance in a study of clinical decision making. Family Medicine. 1995, 27: 126-133.PubMed
31.
go back to reference Benning T, Broadhurt M: The long case is dead. Long live the long case. Loss of the MRCPsych long case and holism in Psychiatry. Psychiatric Bulletin. 2007, 31: 441-42. 10.1192/pb.bp.107.014951.CrossRef Benning T, Broadhurt M: The long case is dead. Long live the long case. Loss of the MRCPsych long case and holism in Psychiatry. Psychiatric Bulletin. 2007, 31: 441-42. 10.1192/pb.bp.107.014951.CrossRef
32.
go back to reference Hodges B: Variations on a theme by Harden. Medical Education. 2003, 37: 1134-1140. 10.1111/j.1365-2923.2003.01717.x.CrossRefPubMed Hodges B: Variations on a theme by Harden. Medical Education. 2003, 37: 1134-1140. 10.1111/j.1365-2923.2003.01717.x.CrossRefPubMed
33.
go back to reference Van Thiel J, van der Vleuten CPM, Kraan H: Assessment of medical interviewing skills: generalisibility of scores using successive MAAS-versions. Approaches to assessment of Clinical Competence-Part II. Edited by: Harden RM, Hart IR, Mulholland H. 1992, Norwich: Page Brothers, 536-540. Van Thiel J, van der Vleuten CPM, Kraan H: Assessment of medical interviewing skills: generalisibility of scores using successive MAAS-versions. Approaches to assessment of Clinical Competence-Part II. Edited by: Harden RM, Hart IR, Mulholland H. 1992, Norwich: Page Brothers, 536-540.
34.
go back to reference Norman GR, Tugwell P, Feightner W, Muzzin LJ, Jacoby LL: Knowledge and clinical problem solving. Medical Education. 1985, 19: 344-56. 10.1111/j.1365-2923.1985.tb01336.x.CrossRefPubMed Norman GR, Tugwell P, Feightner W, Muzzin LJ, Jacoby LL: Knowledge and clinical problem solving. Medical Education. 1985, 19: 344-56. 10.1111/j.1365-2923.1985.tb01336.x.CrossRefPubMed
35.
go back to reference Blaskiewicz RJ, Park RS, Chibnall JT, Powell JK: The influence of testing context and clinical rotation order on student OSCE performance. Academic Medicine. 2004, 79: 597-601. 10.1097/00001888-200406000-00020.CrossRefPubMed Blaskiewicz RJ, Park RS, Chibnall JT, Powell JK: The influence of testing context and clinical rotation order on student OSCE performance. Academic Medicine. 2004, 79: 597-601. 10.1097/00001888-200406000-00020.CrossRefPubMed
36.
go back to reference Cangelosi JS: Designing tests for evaluating student achievement. 1990, White plains New York, Longman Cangelosi JS: Designing tests for evaluating student achievement. 1990, White plains New York, Longman
38.
go back to reference Cohen R, Rothman AI, Poldre P, Ross J: Validity and generalisibility of global ratings in an objective structured clinical examination. Academic Medicine. 1991, 66: 545-8. 10.1097/00001888-199109000-00012.CrossRefPubMed Cohen R, Rothman AI, Poldre P, Ross J: Validity and generalisibility of global ratings in an objective structured clinical examination. Academic Medicine. 1991, 66: 545-8. 10.1097/00001888-199109000-00012.CrossRefPubMed
39.
go back to reference Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M: OSCE Checklists do not capture increasing levels of expertise. Academic Medicine. 1999, 74: 1129-1134. 10.1097/00001888-199910000-00017.CrossRefPubMed Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M: OSCE Checklists do not capture increasing levels of expertise. Academic Medicine. 1999, 74: 1129-1134. 10.1097/00001888-199910000-00017.CrossRefPubMed
40.
go back to reference Gleeson F: Assessment of Clinical Competence using the Objective Structured Long Examination Record (OSLER). Medical Teacher. 1997, 19: 7-14. 10.3109/01421599709019339.CrossRef Gleeson F: Assessment of Clinical Competence using the Objective Structured Long Examination Record (OSLER). Medical Teacher. 1997, 19: 7-14. 10.3109/01421599709019339.CrossRef
41.
go back to reference Duerson MC, Romrell LJ, Steven CB: Impacting faculty teaching and student performance: nine years experience with the objective structured clinical examination. Teaching and Learning in Medicine. 2000, 12: 176-82. 10.1207/S15328015TLM1204_3.CrossRefPubMed Duerson MC, Romrell LJ, Steven CB: Impacting faculty teaching and student performance: nine years experience with the objective structured clinical examination. Teaching and Learning in Medicine. 2000, 12: 176-82. 10.1207/S15328015TLM1204_3.CrossRefPubMed
42.
go back to reference Newble DI, Jaeger K: The effect of assessments and examinations on the learning of medical students. Medical education. 1983, 17: 165-171. 10.1111/j.1365-2923.1983.tb00657.x.CrossRefPubMed Newble DI, Jaeger K: The effect of assessments and examinations on the learning of medical students. Medical education. 1983, 17: 165-171. 10.1111/j.1365-2923.1983.tb00657.x.CrossRefPubMed
43.
go back to reference Van Luijk SJ, Van der Vleuten CPM, Schelven RM: The relation between content and psychometric characteristics in performance-based testing. Teaching and assessing clinical competence. Edited by: bender W, Hiemstra RJ, Scherpbier AIJA, Zwierstra RP. 1990, Groningen: Boekwerk Publications, 202-7. Van Luijk SJ, Van der Vleuten CPM, Schelven RM: The relation between content and psychometric characteristics in performance-based testing. Teaching and assessing clinical competence. Edited by: bender W, Hiemstra RJ, Scherpbier AIJA, Zwierstra RP. 1990, Groningen: Boekwerk Publications, 202-7.
45.
go back to reference Epstein RM: Assessment in Medical Education. 2007, MThe New England Journal of Medicine, 356: 387-96. Epstein RM: Assessment in Medical Education. 2007, MThe New England Journal of Medicine, 356: 387-96.
46.
go back to reference Hodges B, Regehr G, Hanson M, McNaughton N: An objective Structured Clinical examination for Evaluating Psychiatric Clinical Clerks. Academic Medicine. 1997, 72: 715-721. 10.1097/00001888-199708000-00019.CrossRefPubMed Hodges B, Regehr G, Hanson M, McNaughton N: An objective Structured Clinical examination for Evaluating Psychiatric Clinical Clerks. Academic Medicine. 1997, 72: 715-721. 10.1097/00001888-199708000-00019.CrossRefPubMed
47.
go back to reference Cusimano MD, Cohen R, Tucker W, Murnaghan J, Kodama R, Reznick R: A comparative analysis of the costs of administration of an OSCE (objective structured clinical examination). Academic Medicine. 1994, 69: 571-6. 10.1097/00001888-199407000-00014.CrossRefPubMed Cusimano MD, Cohen R, Tucker W, Murnaghan J, Kodama R, Reznick R: A comparative analysis of the costs of administration of an OSCE (objective structured clinical examination). Academic Medicine. 1994, 69: 571-6. 10.1097/00001888-199407000-00014.CrossRefPubMed
48.
go back to reference Varkey P, natt N, Lesnick T, Downing S, Yudkowsky : Validity evidence for an OSCE to assess competency in systems-based practice and practice-based learning and improvement: a preliminary investigation. Academic Medicine. 2008, 83: 775-80. 10.1097/ACM.0b013e31817ec873.CrossRefPubMed Varkey P, natt N, Lesnick T, Downing S, Yudkowsky : Validity evidence for an OSCE to assess competency in systems-based practice and practice-based learning and improvement: a preliminary investigation. Academic Medicine. 2008, 83: 775-80. 10.1097/ACM.0b013e31817ec873.CrossRefPubMed
49.
go back to reference Whelan P, Larence-Smith G, Church L, Woolcock C, Meerten M, Rao R: Good bye OSCE, Hello CASC: a mock CASC course and examination. Psychiatric Bulletin. 2009, 33: 149-153. 10.1192/pb.bp.108.021527.CrossRef Whelan P, Larence-Smith G, Church L, Woolcock C, Meerten M, Rao R: Good bye OSCE, Hello CASC: a mock CASC course and examination. Psychiatric Bulletin. 2009, 33: 149-153. 10.1192/pb.bp.108.021527.CrossRef
Metadata
Title
Objective Structured Clinical Examinations (OSCEs), psychiatry and the Clinical assessment of Skills and Competencies (CASC)Same Evidence, Different Judgement
Author
Steven Marwaha
Publication date
01-12-2011
Publisher
BioMed Central
Published in
BMC Psychiatry / Issue 1/2011
Electronic ISSN: 1471-244X
DOI
https://doi.org/10.1186/1471-244X-11-85

Other articles of this Issue 1/2011

BMC Psychiatry 1/2011 Go to the issue