Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Journal of General Internal Medicine
Published in: Journal of General Internal Medicine 5/2016

01-05-2016 | Original Research

Rating the Quality of Entrustable Professional Activities: Content Validation and Associations with the Clinical Context

Authors: Jason A. Post, M.D., Christopher M. Wittich, M.D., Pharm.D., Kris G. Thomas, M.D., Denise M. Dupras, M.D., Ph.D., Andrew J. Halvorsen, M.S., Jay N. Mandrekar, Ph.D., Amy S. Oxentenko, M.D., Thomas J. Beckman, M.D.

Published in: Journal of General Internal Medicine | Issue 5/2016

Login to get access

ABSTRACT

BACKGROUND

Entrustable professional activities (EPAs) have been developed to assess resident physicians with respect to Accreditation Council for Graduate Medical Education (ACGME) competencies and milestones. Although the feasibility of using EPAs has been reported, we are unaware of previous validation studies on EPAs and potential associations between EPA quality scores and characteristics of educational programs.

OBJECTIVES

Our aim was to validate an instrument for assessing the quality of EPAs for assessment of internal medicine residents, and to examine associations between EPA quality scores and features of rotations.

DESIGN

This was a prospective content validation study to design an instrument to measure the quality of EPAs that were written for assessing internal medicine residents.

PARTICIPANTS

Residency leadership at Mayo Clinic, Rochester participated in this study. This included the Program Director, Associate program directors and individual rotation directors.

INTERVENTIONS

The authors reviewed salient literature. Items were developed to reflect domains of EPAs useful for assessment. The instrument underwent further testing and refinement. Each participating rotation director created EPAs that they felt would be meaningful to assess learner performance in their area. These 229 EPAs were then assessed with the QUEPA instrument to rate the quality of each EPA.

MAIN MEASURES

Performance characteristics of the QUEPA are reported. Quality ratings of EPAs were compared to the primary ACGME competency, inpatient versus outpatient setting and specialty type.

KEY RESULTS

QUEPA tool scores demonstrated excellent reliability (ICC range 0.72 to 0.94). Higher ratings were given to inpatient versus outpatient (3.88, 3.66; p = 0.03) focused EPAs. Medical knowledge EPAs scored significantly lower than EPAs assessing other competencies (3.34, 4.00; p < 0.0001).

CONCLUSIONS

The QUEPA tool is supported by good validity evidence and may help in rating the quality of EPAs developed by individual programs. Programs should take care when writing EPAs for the outpatient setting or to assess medical knowledge, as these tended to be rated lower.
Appendix
Available only for authorised users
Literature
1.
2.
Jones MD Jr, Rosenberg AA, Gilhooly JT, Carraccio CL. Perspective: Competencies, outcomes, and controversy—linking professional activities to competencies to improve resident education and practice. Acad Med. 2011;86:161–5.CrossRefPubMed
3.
Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366:1051–6.CrossRefPubMed
4.
(ACGME) ACfGME. Frequently asked questions: Milestones. 2015.
5.
ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82:542–7.CrossRefPubMed
6.
7.
8.
ten Cate O, Snell L, Carraccio C. Medical competence: the interplay between individual ability and the health care environment. Med Teach. 2010;32:669–75.CrossRefPubMed
9.
ten Cate O, Young JQ. The patient handover as an entrustable professional activity: adding meaning in teaching and practice. BMJ Qual Saf. 2012;21(Suppl 1):i9–12.CrossRefPubMed
10.
Warm EJ, Mathis BR, Held JD, et al. Entrustment and mapping of observable practice activities for resident assessment. J Gen Intern Med. 2014;29:1177–82.CrossRefPubMedPubMedCentral
11.
Jonker G, Hoff RG, Ten Cate OT et al. A case for competency-based anaesthesiology training with entrustable professional activities: An agenda for development and research. Eur J Anaesthesiol 2014.
12.
Aylward M, Nixon J, Gladding S. et al. An Entrustable Professional Activity (EPA) for handoffs as a model for EPA Assessment Development. Acad Med. 2014.
13.
14.
Hauer KE, Soni K, Cornett P, et al. Developing entrustable professional activities as the basis for assessment of competence in an internal medicine residency: a feasibility study. J Gen Intern Med. 2013;28:1110–4.CrossRefPubMedPubMedCentral
15.
Hauer KE, Ten Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ : Theory Pract. 2013.
16.
Colleges AoAM. Core entrustable professional activities for entering residency. Association of American Medical Colleges. 2014.
17.
Caverzagie KJ, Cooney TG, Hemmer PA, Berkowitz L. The development of entrustable professional activities for internal medicine residency training: a report from the Education Redesign Committee of the Alliance for Academic Internal Medicine. Acad Med. 2014.
18.
Chang A, Bowen JL, Buranosky RA, et al. Transforming primary care training—patient-centered medical home entrustable professional activities for internal medicine residents. J Gen Intern Med. 2013;28:801–9.CrossRefPubMedPubMedCentral
19.
Mulder H, Ten Cate O, Daalder R, Berkvens J. Building a competency-based workplace curriculum around entrustable professional activities: the case of physician assistant training. Med Teach. 2010;32:e453–9.CrossRefPubMed
20.
Ten Cate O. AM last page: what entrustable professional activities add to a competency-based curriculum. Acad Med. 2014;89:691.CrossRefPubMed
21.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRefPubMed
22.
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–81.CrossRefPubMedPubMedCentral
23.
24.
Hawkins RE, Sumption KF, Gaglione MM, Holmboe ES. The in-training examination in internal medicine: resident perceptions and lack of correlation between resident scores and faculty predictions of resident performance. Am J Med. 1999;106:206–10.CrossRefPubMed
25.
Kolars JC, McDonald FS, Subhiyah RG, Edson RS. Knowledge base evaluation of medicine residents on the gastroenterology service: implications for competency assessments by faculty. Clin Gastroenterol Hepatol. 2003;1:64–8.CrossRefPubMed
26.
Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119:166 e7–16.CrossRefPubMed
27.
APA, AERA, and National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Education Research Association.; 2014.
28.
Beckman TJ, Cook DA. Developing scholarly projects in education: a primer for medical teachers. Med Teach. 2007;29:210–8.CrossRefPubMed
29.
Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. Jama. 2007;298:1002–9.CrossRefPubMed
30.
Shea JA. Mind the gap: some reasons why medical education research is different from health services research. Med Educ. 2001;35:319–20.CrossRefPubMed
31.
Beckman TJ, Cook DA, Mandrekar JN. Factor instability of clinical teaching assessment scores among general internists and cardiologists. Med Educ. 2006;40:1209–16.CrossRefPubMed
Metadata
Title
Rating the Quality of Entrustable Professional Activities: Content Validation and Associations with the Clinical Context
Authors
Jason A. Post, M.D.
Christopher M. Wittich, M.D., Pharm.D.
Kris G. Thomas, M.D.
Denise M. Dupras, M.D., Ph.D.
Andrew J. Halvorsen, M.S.
Jay N. Mandrekar, Ph.D.
Amy S. Oxentenko, M.D.
Thomas J. Beckman, M.D.
Publication date
01-05-2016
Publisher
Springer US
Published in
Journal of General Internal Medicine / Issue 5/2016
Print ISSN: 0884-8734
Electronic ISSN: 1525-1497
DOI
https://doi.org/10.1007/s11606-016-3611-8

Other articles of this Issue 5/2016

Journal of General Internal Medicine 5/2016 Go to the issue

Erratum

Erratum to: Provider patient-sharing networks and multiple-provider prescribing of benzodiazepines

Original Research

VA and Medicare Utilization Among Dually Enrolled Veterans with Type 2 Diabetes: A Latent Class Analysis

Editorial

Chronic Pain and Prescription Drug Use and Abuse: Emerging Research in General Internal Medicine

Capsule Commentary

Capsule Commentary on Gaither et al., the Association Between Receipt of Guideline-Concordant Long-Term Opioid Therapy and All-Cause Mortality

Review Paper

Benefits and Harms of Screening Mammography by Comorbidity and Age: A Qualitative Synthesis of Observational Studies and Decision Analyses

Capsule Commentary

Capsule Commentary on Meredith et al., Impact of Collaborative Care for Underserved Patients with PTSD in Primary Care: A Randomized Controlled Trial
Live Webinar | 27-06-2024 | 18:00 (CEST)

Keynote webinar | Spotlight on medication adherence

Reserve your place

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

WHO estimates that half of all patients worldwide are non-adherent to their prescribed medication. The consequences of poor adherence can be catastrophic, on both the individual and population level.

Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.

Prof. Kevin Dolgin
Prof. Florian Limbourg
Prof. Anoop Chauhan
Developed by: Springer Medicine
Obesity Clinical Trial Summary

At a glance: The STEP trials

Read more

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine
Image Credits