Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medicine
Published in: BMC Medicine 1/2018

Open Access 01-12-2018 | Research article

Fitness to practise sanctions in UK doctors are predicted by poor performance at MRCGP and MRCP(UK) assessments: data linkage study

Authors: Richard Wakeford, Kasia Ludka, Katherine Woolf, I. C. McManus

Published in: BMC Medicine | Issue 1/2018

Login to get access

Abstract

Background

The predictive validity of postgraduate examinations, such as MRCGP and MRCP(UK) in the UK, is hard to assess, particularly for clinically relevant outcomes. The sanctions imposed on doctors by the UK’s General Medical Council (GMC), including erasure from the Medical Register, are indicators of serious problems with fitness to practise (FtP) that threaten patient safety or wellbeing. This data linkage study combined data on GMC sanctions with data on postgraduate examination performance.

Methods

Examination results were obtained for UK registered doctors taking the MRCGP Applied Knowledge Test (AKT; n = 27,561) or Clinical Skills Assessment (CSA; n = 17,365) at first attempt between 2010 and 2016 or taking MRCP(UK) Part 1 (MCQ; n = 37,358), Part 2 (MCQ; n = 28,285) or Practical Assessment of Clinical Examination Skills (PACES; n = 27,040) at first attempt between 2001 and 2016. Exam data were linked with GMC actions on a doctor’s registration from September 2008 to January 2017, sanctions including Erasure, Suspension, Conditions on Practice, Undertakings or Warnings (ESCUW). Examination results were only considered at first attempts. Multiple logistic regression assessed the odds ratio for ESCUW in relation to examination results. Multiple imputation was used for structurally missing values.

Results

Doctors sanctioned by the GMC performed substantially less well on MRCGP and MRCP(UK), with a mean Cohen’s d across the five exams of − 0.68. Doctors on the 2.5th percentile of exam performance were about 12 times more likely to have FtP problems than those on the 97.5th percentile. Knowledge assessments and clinical assessments were independent predictors of future sanctions, with clinical assessments predicting ESCUW significantly better. The log odds of an FtP sanction were linearly related to examination marks over the entire range of performance, additional performance increments lowering the risk of FtP sanctions at all performance levels.

Conclusions

MRCGP and MRCP(UK) performance are valid predictors of professionally important outcomes that transcend simple knowledge or skills and the GMC puts under the headings of conduct and trust. Postgraduate examinations may predict FtP sanctions because the psychological processes involved in successfully studying, understanding and practising medicine at a high level share similar mechanisms to those underlying conduct and trust.
Literature
1.
2.
Unwin E, Woolf K, Wadlow C, Dacre J. Disciplined doctors: does the sex of a doctor matter? A cross-sectional study examining the association between a doctor's sex and receiving sanctions against their medical registration. BMJ Open. 2014;4(8):e005405.CrossRefPubMedPubMedCentral
3.
4.
Archer J, Lynn N, Coombes L, Gale T, de Bere SR. The medical licensing examination debate. Regul Gov. 2016;11:315–22.CrossRef
5.
Flier LA, Henderson CR, Treasure CL. Time to eliminate the Step 2 Clinical Skills examination for US medical graduates. JAMA Intern Med. 2016;176:1245–6.CrossRefPubMed
6.
7.
8.
Ludka-Stempien K, Woolf K, McManus IC. Poor performance on the MRCP(UK) examination predicts license limitations in subsequent medical practice. Prague: Proceedings of AMEE (Association for Medical Education in Europe), Prague August 2013; 2013.
9.
Ludka-Stempien K. The predictive validity of the MRCP(UK) examination (PhD thesis). London: University college London; 2015.
10.
11.
Messick S. Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. AP. 1995;50:749.
12.
Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37:830–7.CrossRefPubMed
13.
Lissitz RW. The concept of validity: revisions, new directions, and applications. Charlotte: Information Age Publishing; 2009.
14.
Kane MT. Validating the interpretations and uses of test scores. JEM. 2014;50:1–73.
15.
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.
16.
Norcini JJ, Boulet JR, Opalek A, Dauphinee WD. The relationship between licensing examination performance and the outcomes of care by international medical school graduates. Acad Med. 2014;89:1–6.CrossRef
17.
Papadakis MA, Arnold J, Blanton H, Holmboe ES, Lipner RS. Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Ann Intern Med. 2008;148:869–76.CrossRefPubMed
18.
Cuddy MM, Young A, Gelman A, Swanson DB, Johnson DA, Dillon GF, et al. Exploring the relationships between USMLE performance and disciplinary action in practice: a validity study of score inferences from a licensure examination. Acad Med. 2017;92:1780–5.CrossRefPubMed
19.
Cuddy MM, Winward ML, Johnston MM, Lipner RS, Clauser BE. Evaluating validity evidence for USMLE step 2 clinical skills data gathering and data interpretation scores: does performance predict history-taking and physical examination ratings for first-year internal medicine residents? Acad Med. 2016;91:133–9.CrossRefPubMed
20.
21.
Wakeford R, Denney ML, Ludka-Stempien K, Dacre J, McManus IC. Cross-comparison of MRCGP & MRCP(UK) in a database linkage study of 2,284 candidates taking both examinations: assessment of validity and differential performance by ethnicity. BMC Med Educ. 2015;15:1.CrossRefPubMedPubMedCentral
22.
McManus IC, Ludka K. Resitting a high-stakes postgraduate medical examination on multiple occasions: nonlinear multilevel modelling of performance in the MRCP(UK) examinations. BMC Med. 2012;10:60.CrossRefPubMedPubMedCentral
23.
24.
25.
Elder A, McAlpine L, Bateman N, Dacre J, Kopelman P, McManus IC. Changing PACES: developments to the examination in 2009. Clin Med. 2011;11:231–4.CrossRef
26.
27.
Hope D, Adamson K, McManus IC, Chis L, Elder A. Using differential item functioning to evaluate potential bias in a high stakes postgraduate knowledge based assessment. BMC Med Educ. 2018;18:1–7.CrossRef
28.
29.
30.
Arneson JJ, Sackett PR, Beatty AS. Ability-performance relationships in education and employment settings: critical tests of the more-is-better and good-enough hypotheses. Psychol Sci. 2011;22:1336–42.CrossRefPubMed
31.
Outliers: the story of success. New York: Little, Brown; 2008.
32.
Schwartz B. Top colleges should select randomly from a pool of ‘good enough’. Chron High Educ. 2005;2005:B20–5.
33.
Major LE, Machin S. Social mobility and its enemies. London: Penguin Random House; 2018.
34.
McManus IC, Vincent CA. Can future poor performance be identified during selection? In: Lens P, van der Waal G, editors. Problem doctors: a conspiracy of silence. Amsterdam: IOS Press; 1997. p. 213–36.
35.
Handfield-Jones RS, Mann KV, Challis ME, Hobma SO, Klass DJ, McManus IC, et al. Linking assessment to learning: a new route to quality assurance in medical practice. Med Educ. 2002;36:949–58.CrossRefPubMed
36.
37.
Cresswell M. Heaps, prototypes and ethics: the consequence of using judgments of student performance to set examination standards in a time of change. London: Institute of Education; 2003.
38.
Sainsbury RM. Departing from Frege. Essays in the philosophy of language. London: Routledge; 2018.CrossRef
39.
40.
41.
42.
Bassett K. Operating theatre. CAM. 1995:(14)7–11.
43.
44.
Elder A, Verghese A. Bedside matters – putting the patient at the centre of teaching and learning. J R Coll Physicians Edinb. 2015;45:186–7.CrossRefPubMed
45.
Elder A, McManus IC, Patrick A, Nair K, Vaughan L, Dacre J. The value of the physical examination in clinical practice: an international survey. Clin Med. 2017;17:490–8.CrossRef
46.
Reilly BM. Physical examination in the care of medical inpatients: an observational study. Lancet. 2003;362:1100–5.CrossRefPubMed
47.
Verghese A, Charlton B, Kassirer JP, Ramsey M, Ioannidis JPA. Inadequacies of physical examination as a cause of medical errors and adverse events: a collection of vignettes. AJM. 2015;128:1322–4.CrossRef
48.
Verghese A, Horwitz RI: In praise of the physical examination [Editorial]. Brit Med J 2009, 339b5448.
49.
50.
Hampton JR, Harrison MJG, Mitchell JRA, Prichard JS, Seymour C. Relative contributions of history-taking, physical examination, and laboratory investigation to diagnosis and management of medical outpatients. Brit Med J. 1975;2:486–9.CrossRefPubMed
Metadata
Title
Fitness to practise sanctions in UK doctors are predicted by poor performance at MRCGP and MRCP(UK) assessments: data linkage study
Authors
Richard Wakeford
Kasia Ludka
Katherine Woolf
I. C. McManus
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medicine / Issue 1/2018
Electronic ISSN: 1741-7015
DOI
https://doi.org/10.1186/s12916-018-1214-4

Other articles of this Issue 1/2018

BMC Medicine 1/2018 Go to the issue

Opinion

Mapping malaria by combining parasite genomic and epidemiologic data

Commentary

Communicating health risks in science publications: time for everyone to take responsibility

Research article

Four-year effects of exercise on fatigue and physical activity in patients with cancer

Research article

An exploratory study examining how nano-liquid chromatography–mass spectrometry and phosphoproteomics can differentiate patients with advanced fibrosis and higher percentage collagen in non-alcoholic fatty liver disease

Research article

Human antibodies activate complement against Plasmodium falciparum sporozoites, and are associated with protection against malaria in children

Research article

Differences in the health transition patterns of migrants and non-migrants aged 50 and older in southern and western Europe (2004–2015)