Skip to main content
Top
Published in: Perspectives on Medical Education 1/2019

Open Access 01-02-2019 | Original Article

Design and evaluation of a clinical competency committee

Authors: Marrigje E. Duitsman, Cornelia R. M. G. Fluit, Janiëlle A. E. M. van Alfen-van der Velden, Marieke de Visser, Marianne ten Kate-Booij, Diana H. J. M. Dolmans, Debbie A. D. C. Jaarsma, Jacqueline de Graaf

Published in: Perspectives on Medical Education | Issue 1/2019

Login to get access

Abstract

Introduction

In postgraduate medical education, group decision-making has emerged as an essential tool to evaluate the clinical progress of residents. Clinical competency committees (CCCs) have been set up to ensure informed decision-making and provide feedback regarding performance of residents. Despite this important task, it remains unclear how CCCs actually function in practice and how their performance should be evaluated.

Methods

In the prototyping phase of a design-based approach, a CCC meeting was developed, using three theoretical design principles: (1) data from multiple assessment tools and multiple perspectives, (2) a shared mental model and (3) structured discussions. The meetings were held in a university children’s hospital and evaluated using observations, interviews with CCC members and an open-ended questionnaire among residents.

Results

The structured discussions during the meetings provided a broad outline of resident performance, including identification of problematic and excellent residents. A shared mental model about the assessment criteria had developed over time. Residents were not always satisfied with the feedback they received after the meeting. Feedback that had been provided to a resident after the first CCC meeting was not addressed in the second meeting.

Discussion

The principles that were used to design the CCC meeting were feasible in practice. Structured discussions, based on data from multiple assessment tools and multiple perspectives, provided a broad outline of resident performance. Residency programs that wish to implement CCCs can build on our design principles and adjust the prototype to their particular context. When running a CCC, it is important to consider feedback that has been provided to a resident after the previous meeting and to evaluate whether it has improved the resident’s performance.
Appendix
Available only for authorised users
Literature
2.
go back to reference Andolsek K, Padmore J, Hauer KE, Holmboe E. Clinical competency committees. A guidebook for programs. Chicago: The Accreditation Council for Graduate Medical Education; 2015. Andolsek K, Padmore J, Hauer KE, Holmboe E. Clinical competency committees. A guidebook for programs. Chicago: The Accreditation Council for Graduate Medical Education; 2015.
3.
go back to reference Michaelsen LK, Watson WE, Black RH. A realistic test of individual versus group consensus decision making. J Appl Psychol. 1989;74:834.CrossRef Michaelsen LK, Watson WE, Black RH. A realistic test of individual versus group consensus decision making. J Appl Psychol. 1989;74:834.CrossRef
4.
go back to reference Hauer KE, ten Cate O, Boscardin CK, et al. Ensuring resident competence: a narrative review of the literature on group decision making to inform the work of clinical competency committees. J Grad Med Educ. 2016;8:156–64.CrossRef Hauer KE, ten Cate O, Boscardin CK, et al. Ensuring resident competence: a narrative review of the literature on group decision making to inform the work of clinical competency committees. J Grad Med Educ. 2016;8:156–64.CrossRef
5.
go back to reference Hemmer PA, Hawkins R, Jackson JL, Pangaro LN. Assessing how well three evaluation methods detect deficiencies in medical students’ professionalism in two settings of an internal medicine clerkship. Acad Med. 2000;75:167–73.CrossRef Hemmer PA, Hawkins R, Jackson JL, Pangaro LN. Assessing how well three evaluation methods detect deficiencies in medical students’ professionalism in two settings of an internal medicine clerkship. Acad Med. 2000;75:167–73.CrossRef
6.
go back to reference Thomas MR, Beckman TJ, Mauck KF, Cha SS, Thomas KG. Group assessments of resident physicians improve reliability and decrease halo error. J Gen Intern Med. 2011;26:759–64.CrossRef Thomas MR, Beckman TJ, Mauck KF, Cha SS, Thomas KG. Group assessments of resident physicians improve reliability and decrease halo error. J Gen Intern Med. 2011;26:759–64.CrossRef
7.
go back to reference Schwind CJ, Williams RG, Boehler ML, Dunnington GL. Do individual attendings’ post-rotation performance ratings detect residents’ clinical performance deficiencies? Acad Med. 2004;79:453–7.CrossRef Schwind CJ, Williams RG, Boehler ML, Dunnington GL. Do individual attendings’ post-rotation performance ratings detect residents’ clinical performance deficiencies? Acad Med. 2004;79:453–7.CrossRef
8.
go back to reference Hill GW. Group versus individual performance: are N+ 1 heads better than one? Psychol Bull. 1982;91:517.CrossRef Hill GW. Group versus individual performance: are N+ 1 heads better than one? Psychol Bull. 1982;91:517.CrossRef
9.
go back to reference Hauer KE, Chesluk B, Iobst W, et al. Reviewing residents’ competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. 2015;90:1084–92.CrossRef Hauer KE, Chesluk B, Iobst W, et al. Reviewing residents’ competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. 2015;90:1084–92.CrossRef
10.
go back to reference Promes SB, Wagner MJ. Starting a clinical competency committee. J Grad Med Educ. 2014;6:163–4.CrossRef Promes SB, Wagner MJ. Starting a clinical competency committee. J Grad Med Educ. 2014;6:163–4.CrossRef
11.
go back to reference Doty CI, Roppolo LP, Asher S, et al. How do emergency medicine residency programs structure their clinical competency committees? A survey. Acad Emerg Med. 2015;22:1351–4.CrossRef Doty CI, Roppolo LP, Asher S, et al. How do emergency medicine residency programs structure their clinical competency committees? A survey. Acad Emerg Med. 2015;22:1351–4.CrossRef
12.
go back to reference Ketteler ER, Auyang ED, Beard KE, et al. Competency champions in the clinical competency committee: a successful strategy to implement milestone evaluations and competency coaching. J Surg Educ. 2014;71:36–8.CrossRef Ketteler ER, Auyang ED, Beard KE, et al. Competency champions in the clinical competency committee: a successful strategy to implement milestone evaluations and competency coaching. J Surg Educ. 2014;71:36–8.CrossRef
13.
go back to reference French JC, Dannefer EF, Colbert CY. A systematic approach toward building a fully operational clinical competency committee. J Surg Educ. 2014;71:e22–e7.CrossRef French JC, Dannefer EF, Colbert CY. A systematic approach toward building a fully operational clinical competency committee. J Surg Educ. 2014;71:e22–e7.CrossRef
14.
go back to reference Donato AA, Alweis R, Wenderoth S. Design of a clinical competency committee to maximize formative feedback. J Community Hosp Intern Med Perspect. 2016;6:33533.CrossRef Donato AA, Alweis R, Wenderoth S. Design of a clinical competency committee to maximize formative feedback. J Community Hosp Intern Med Perspect. 2016;6:33533.CrossRef
15.
go back to reference Chahine S, Cristancho S, Padgett J, Lingard L. How do small groups make decisions? Perspect Med Educ. 2017;6:192–8.CrossRef Chahine S, Cristancho S, Padgett J, Lingard L. How do small groups make decisions? Perspect Med Educ. 2017;6:192–8.CrossRef
16.
go back to reference Dickey CC, Thomas C, Feroze U, Nakshabandi F, Cannon B. Cognitive demands and bias: challenges facing clinical competency committees. J Grad Med Educ. 2017;9:162–4.CrossRef Dickey CC, Thomas C, Feroze U, Nakshabandi F, Cannon B. Cognitive demands and bias: challenges facing clinical competency committees. J Grad Med Educ. 2017;9:162–4.CrossRef
17.
go back to reference Badley G. The crisis in educational research: a pragmatic approach. Eur Edu Res J. 2003;2:296–308.CrossRef Badley G. The crisis in educational research: a pragmatic approach. Eur Edu Res J. 2003;2:296–308.CrossRef
18.
go back to reference Dolmans DH, Tigelaar D. Building bridges between theory and practice in medical education using a design-based research approach: AMEE Guide No. 60. Med Teach. 2012;34:1–10.CrossRef Dolmans DH, Tigelaar D. Building bridges between theory and practice in medical education using a design-based research approach: AMEE Guide No. 60. Med Teach. 2012;34:1–10.CrossRef
19.
go back to reference Hodges B. Assessment in the post-psychometric era: learning to love the subjective and collective. Med Teach. 2013;35:564–8.CrossRef Hodges B. Assessment in the post-psychometric era: learning to love the subjective and collective. Med Teach. 2013;35:564–8.CrossRef
20.
go back to reference Laughlin PR, Adamopoulos J. Social combination processes and individual learning for six-person cooperative groups on an intellective task. J Pers Soc Psychol. 1980;38:941.CrossRef Laughlin PR, Adamopoulos J. Social combination processes and individual learning for six-person cooperative groups on an intellective task. J Pers Soc Psychol. 1980;38:941.CrossRef
21.
go back to reference Surowiecki J, Silverman MP. The wisdom of crowds. Am J Phys. 2007;75:190–2.CrossRef Surowiecki J, Silverman MP. The wisdom of crowds. Am J Phys. 2007;75:190–2.CrossRef
22.
go back to reference Eva KW, Hodges BD. Scylla or Charybdis? Can we navigate between objectification and judgement in assessment? Med Educ. 2012;46:914–9.CrossRef Eva KW, Hodges BD. Scylla or Charybdis? Can we navigate between objectification and judgement in assessment? Med Educ. 2012;46:914–9.CrossRef
23.
go back to reference Lu L, Yuan YC, McLeod PL. Twenty-five years of hidden profiles in group decision making: a meta-analysis. Pers Soc Psychol Rev. 2012;16:54–75.CrossRef Lu L, Yuan YC, McLeod PL. Twenty-five years of hidden profiles in group decision making: a meta-analysis. Pers Soc Psychol Rev. 2012;16:54–75.CrossRef
24.
go back to reference Janis IL. Groupthink. Psychol Today. 1971;5:43–6. Janis IL. Groupthink. Psychol Today. 1971;5:43–6.
25.
go back to reference Jonker CM, Van Riemsdijk MB, Vermeulen B. Shared mental models. In: De Vos M, Fornara N, Pit JV, Vouros G, editors. Coordination, organizations, institutions, and norms in agent systems VI. Coin 2010. Lecture Notes in Computer Science, Vol. 6541. Berlin, Heidelberg: Springer; 2011. pp. 132–51.CrossRef Jonker CM, Van Riemsdijk MB, Vermeulen B. Shared mental models. In: De Vos M, Fornara N, Pit JV, Vouros G, editors. Coordination, organizations, institutions, and norms in agent systems VI. Coin 2010. Lecture Notes in Computer Science, Vol. 6541. Berlin, Heidelberg: Springer; 2011. pp. 132–51.CrossRef
26.
go back to reference Schultze T, Mojzisch A, Schulz-Hardt S. Why groups perform better than individuals at quantitative judgment tasks: group-to-individual transfer as an alternative to differential weighting. Organ Behav Hum Decis Process. 2012;118:24–36.CrossRef Schultze T, Mojzisch A, Schulz-Hardt S. Why groups perform better than individuals at quantitative judgment tasks: group-to-individual transfer as an alternative to differential weighting. Organ Behav Hum Decis Process. 2012;118:24–36.CrossRef
27.
go back to reference Chernyshenko OS, Miner AG, Baumann MR, Sniezek JA. The impact of information distribution, ownership, and discussion on group member judgment: the differential cue weighting model. Organ Behav Hum Decis Process. 2003;91:12–25.CrossRef Chernyshenko OS, Miner AG, Baumann MR, Sniezek JA. The impact of information distribution, ownership, and discussion on group member judgment: the differential cue weighting model. Organ Behav Hum Decis Process. 2003;91:12–25.CrossRef
28.
go back to reference Stasser G, Titus W. Pooling of unshared information in group decision making: biased information sampling during discussion. J Pers Soc Psychol. 1985;48:1467.CrossRef Stasser G, Titus W. Pooling of unshared information in group decision making: biased information sampling during discussion. J Pers Soc Psychol. 1985;48:1467.CrossRef
29.
go back to reference Greenhalgh L, Chapman DI. Negotiator relationships: construct measurement, and demonstration of their impact on the process and outcomes of negotiation. Group Decis Negot. 1998;7:465–89.CrossRef Greenhalgh L, Chapman DI. Negotiator relationships: construct measurement, and demonstration of their impact on the process and outcomes of negotiation. Group Decis Negot. 1998;7:465–89.CrossRef
30.
go back to reference Schittekatte M, Van Hiel A. Effects of partially shared information and awareness of unshared information on information sampling. Small Group Res. 1996;27:431–49.CrossRef Schittekatte M, Van Hiel A. Effects of partially shared information and awareness of unshared information on information sampling. Small Group Res. 1996;27:431–49.CrossRef
31.
go back to reference Mesmer-Magnus JR, DeChurch LA. Information sharing and team performance: a meta-analysis. J Appl Psychol. 2009;94:534–46.CrossRef Mesmer-Magnus JR, DeChurch LA. Information sharing and team performance: a meta-analysis. J Appl Psychol. 2009;94:534–46.CrossRef
32.
go back to reference Webster M Jr, Rashotte LS. Behavior, expectations and status. Soc Forces. 2010;88:1021–49.CrossRef Webster M Jr, Rashotte LS. Behavior, expectations and status. Soc Forces. 2010;88:1021–49.CrossRef
33.
go back to reference Kameda T, Sugimori S. Psychological entrapment in group decision making: an assigned decision rule and a groupthink phenomenon. J Pers Soc Psychol. 1993;65:282.CrossRef Kameda T, Sugimori S. Psychological entrapment in group decision making: an assigned decision rule and a groupthink phenomenon. J Pers Soc Psychol. 1993;65:282.CrossRef
34.
go back to reference Stasson MF, Kameda T, Davis JH. A model of agenda influences on group decisions. Group Dyn Theory Res Pract. 1997;1:316.CrossRef Stasson MF, Kameda T, Davis JH. A model of agenda influences on group decisions. Group Dyn Theory Res Pract. 1997;1:316.CrossRef
35.
go back to reference Waller BM, Hope L, Burrowes N, Morrison ER. Twelve (not so) angry men: managing conversational group size increases perceived contribution by decision makers. Group Process Intergr Relat. 2011;14:835–43.CrossRef Waller BM, Hope L, Burrowes N, Morrison ER. Twelve (not so) angry men: managing conversational group size increases perceived contribution by decision makers. Group Process Intergr Relat. 2011;14:835–43.CrossRef
36.
go back to reference Laughlin PR, Hatch EC, Silver JS, Boh L. Groups perform better than the best individuals on letters-to-numbers problems: effects of group size. J Pers Soc Psychol. 2006;90:644.CrossRef Laughlin PR, Hatch EC, Silver JS, Boh L. Groups perform better than the best individuals on letters-to-numbers problems: effects of group size. J Pers Soc Psychol. 2006;90:644.CrossRef
37.
go back to reference Karotkin D, Paroush J. Optimum committee size: quality-versus-quantity dilemma. Soc Choice Welfare. 2003;20:429–41.CrossRef Karotkin D, Paroush J. Optimum committee size: quality-versus-quantity dilemma. Soc Choice Welfare. 2003;20:429–41.CrossRef
38.
go back to reference Kerr NL, Tindale RS. Group performance and decision making. Annu Rev Psychol. 2004;55:623–55.CrossRef Kerr NL, Tindale RS. Group performance and decision making. Annu Rev Psychol. 2004;55:623–55.CrossRef
39.
go back to reference Hsieh H‑F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.CrossRef Hsieh H‑F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.CrossRef
40.
go back to reference Bengtsson M. How to plan and perform a qualitative study using content analysis. Nurs Plus Open. 2016;2:8–14.CrossRef Bengtsson M. How to plan and perform a qualitative study using content analysis. Nurs Plus Open. 2016;2:8–14.CrossRef
41.
go back to reference Stasser G, Titus W. Effects of information load and percentage of shared information on the dissemination of unshared information during group discussion. J Pers Soc Psychol. 1987;53:81.CrossRef Stasser G, Titus W. Effects of information load and percentage of shared information on the dissemination of unshared information during group discussion. J Pers Soc Psychol. 1987;53:81.CrossRef
42.
go back to reference Boud D, Molloy E. Rethinking models of feedback for learning: the challenge of design. Assess Eval High Educ. 2013;38:698–712.CrossRef Boud D, Molloy E. Rethinking models of feedback for learning: the challenge of design. Assess Eval High Educ. 2013;38:698–712.CrossRef
43.
go back to reference Boud D. Feedback: ensuring that it leads to enhanced learning. Clin Teach. 2015;12:3–7.CrossRef Boud D. Feedback: ensuring that it leads to enhanced learning. Clin Teach. 2015;12:3–7.CrossRef
44.
go back to reference Cantillon P, Sargeant J. Giving feedback in clinical settings. BMJ. 2008;337:a1961.CrossRef Cantillon P, Sargeant J. Giving feedback in clinical settings. BMJ. 2008;337:a1961.CrossRef
45.
go back to reference Sargeant JM, Mann KV, Van der Vleuten CP, Metsemakers JF. Reflection: a link between receiving and using assessment feedback. Adv Health Sci Educ Theory Pract. 2009;14:399–410.CrossRef Sargeant JM, Mann KV, Van der Vleuten CP, Metsemakers JF. Reflection: a link between receiving and using assessment feedback. Adv Health Sci Educ Theory Pract. 2009;14:399–410.CrossRef
46.
go back to reference Bonnefon J‑F, Feeney A, De Neys W. The risk of polite misunderstandings. Curr Dir Psychol Sci. 2011;20:321–4.CrossRef Bonnefon J‑F, Feeney A, De Neys W. The risk of polite misunderstandings. Curr Dir Psychol Sci. 2011;20:321–4.CrossRef
47.
48.
go back to reference Marks MA, Zaccaro SJ, Mathieu JE. Performance implications of leader briefings and team-interaction training for team adaptation to novel environments. J Appl Psychol. 2000;85:971.CrossRef Marks MA, Zaccaro SJ, Mathieu JE. Performance implications of leader briefings and team-interaction training for team adaptation to novel environments. J Appl Psychol. 2000;85:971.CrossRef
Metadata
Title
Design and evaluation of a clinical competency committee
Authors
Marrigje E. Duitsman
Cornelia R. M. G. Fluit
Janiëlle A. E. M. van Alfen-van der Velden
Marieke de Visser
Marianne ten Kate-Booij
Diana H. J. M. Dolmans
Debbie A. D. C. Jaarsma
Jacqueline de Graaf
Publication date
01-02-2019
Publisher
Bohn Stafleu van Loghum
Published in
Perspectives on Medical Education / Issue 1/2019
Print ISSN: 2212-2761
Electronic ISSN: 2212-277X
DOI
https://doi.org/10.1007/s40037-018-0490-1

Other articles of this Issue 1/2019

Perspectives on Medical Education 1/2019 Go to the issue

Thanks to Reviewers

Thanks to reviewers