Skip to main content
Top
Published in: Perspectives on Medical Education 6/2015

Open Access 01-12-2015 | Original Article

Student-directed retrieval practice is a predictor of medical licensing examination performance

Authors: Francis Deng, Jeffrey A. Gluckstein, Douglas P. Larsen

Published in: Perspectives on Medical Education | Issue 6/2015

Login to get access

Abstract

Introduction

A large body of evidence indicates that retrieval practice (test-enhanced learning) and spaced repetition increase long-term information retention. Implementation of these strategies in medical curricula is unfortunately limited. However, students may choose to apply them autonomously when preparing for high-stakes, cumulative assessments, such as the United States Medical Licensing Examination Step 1. We examined the prevalence of specific self-directed methods of testing, with or without spaced repetition, among preclinical students and assessed the relationship between these methods and licensing examination performance.

Methods

Seventy-two medical students at one institution completed a survey concerning their use of user-generated (Anki) or commercially-available (Firecracker) flashcards intended for spaced repetition and of boards-style multiple-choice questions (MCQs). Other information collected included Step 1 score, past academic performance (Medical College Admission Test [MCAT] score, preclinical grades), and psychological factors that may have affected exam preparation or performance (feelings of depression, burnout, and test anxiety).

Results

All students reported using practice MCQs (mean 3870, SD 1472). Anki and Firecracker users comprised 31 and 49 % of respondents, respectively. In a multivariate regression model, significant independent predictors of Step 1 score included MCQs completed (unstandardized beta coefficient [B] = 2.2 × 10− 3, p < 0.001), unique Anki flashcards seen (B = 5.9 × 10− 4, p = 0.024), second-year honours (B = 1.198, p = 0.002), and MCAT score (B = 1.078, p = 0.003). Test anxiety was a significant negative predictor (B= − 1.986, p < 0.001). Unique Firecracker flashcards seen did not predict Step 1 score. Each additional 445 boards-style practice questions or 1700 unique Anki flashcards was associated with an additional point on Step 1 when controlling for other academic and psychological factors.

Conclusions

Medical students engage extensively in self-initiated retrieval practice, often with spaced repetition. These practices are associated with superior performance on a medical licensing examination and should be considered for formal support by educators.
Literature
1.
go back to reference Roediger HL, Karpicke JD. The power of testing memory: basic research and implications for educational practice. Perspect Psychol Sci. 2006;1(3):181–210.CrossRef Roediger HL, Karpicke JD. The power of testing memory: basic research and implications for educational practice. Perspect Psychol Sci. 2006;1(3):181–210.CrossRef
2.
go back to reference Karpicke JD, Roediger HL. The critical importance of retrieval for learning. Science. 2008;319(5865):966–8.CrossRef Karpicke JD, Roediger HL. The critical importance of retrieval for learning. Science. 2008;319(5865):966–8.CrossRef
3.
go back to reference Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychol Sci. 2006;17(3):249–55.CrossRef Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychol Sci. 2006;17(3):249–55.CrossRef
4.
go back to reference Larsen DP, Butler AC, Roediger III HL. Test-enhanced learning in medical education. Med Educ. 2008;42(10):959–66.CrossRef Larsen DP, Butler AC, Roediger III HL. Test-enhanced learning in medical education. Med Educ. 2008;42(10):959–66.CrossRef
5.
go back to reference Larsen DP, Butler AC, Roediger HL III. Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Med Educ. 2009;43(12):1174–81.CrossRef Larsen DP, Butler AC, Roediger HL III. Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Med Educ. 2009;43(12):1174–81.CrossRef
6.
go back to reference Larsen DP, Butler AC, Roediger HL. Comparative effects of test-enhanced learning and self-explanation on long-term retention. Med Educ. 2013;47(7):674–82.CrossRef Larsen DP, Butler AC, Roediger HL. Comparative effects of test-enhanced learning and self-explanation on long-term retention. Med Educ. 2013;47(7):674–82.CrossRef
7.
go back to reference Kerfoot BP, DeWolf WC, Masser BA, Church PA, Federman DD. Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial. Med Educ. 2007;41(1):23–31.CrossRef Kerfoot BP, DeWolf WC, Masser BA, Church PA, Federman DD. Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial. Med Educ. 2007;41(1):23–31.CrossRef
8.
go back to reference Schmidmaier R, Ebersbach R, Schiller M, Hege I, Holzer M, Fischer MR. Using electronic flashcards to promote learning in medical students: retesting versus restudying. Med Educ. 2011;45(11):1101–10.CrossRef Schmidmaier R, Ebersbach R, Schiller M, Hege I, Holzer M, Fischer MR. Using electronic flashcards to promote learning in medical students: retesting versus restudying. Med Educ. 2011;45(11):1101–10.CrossRef
9.
go back to reference Dobson JL. Retrieval practice is an efficient method of enhancing the retention of anatomy and physiology information. Adv Physiol Educ. 2013;37(2):184–91.CrossRef Dobson JL. Retrieval practice is an efficient method of enhancing the retention of anatomy and physiology information. Adv Physiol Educ. 2013;37(2):184–91.CrossRef
10.
go back to reference Sleight DA, Mavis BE. Study skills and academic performance among second-year medical students in problem-based learning. Med Educ Online. 2009;11. Sleight DA, Mavis BE. Study skills and academic performance among second-year medical students in problem-based learning. Med Educ Online. 2009;11.
12.
go back to reference West C, Kurz T, Smith S, Graham L. Are study strategies related to medical licensing exam performance? Int J Med Educ. 2014;5:199–204.CrossRef West C, Kurz T, Smith S, Graham L. Are study strategies related to medical licensing exam performance? Int J Med Educ. 2014;5:199–204.CrossRef
14.
go back to reference Morrison CA, Ross LP, Fogle T, Butler A, Miller J, Dillon GF. Relationship between performance on the NBME comprehensive basic sciences self-assessment and USMLE step 1 for U.S. and Canadian medical school students. Acad Med. 2010;85(10 Suppl):S98–101.CrossRef Morrison CA, Ross LP, Fogle T, Butler A, Miller J, Dillon GF. Relationship between performance on the NBME comprehensive basic sciences self-assessment and USMLE step 1 for U.S. and Canadian medical school students. Acad Med. 2010;85(10 Suppl):S98–101.CrossRef
15.
go back to reference Elmes D. Anki—powerful, intelligent flashcards. 2015. http://ankisrs.net/. Elmes D. Anki—powerful, intelligent flashcards. 2015. http://​ankisrs.​net/​.​
17.
go back to reference West CP, Dyrbye LN, Sloan JA, Shanafelt TD. Single item measures of emotional exhaustion and depersonalization are useful for assessing burnout in medical professionals. J Gen Intern Med. 2009;24(12):1318–21.CrossRef West CP, Dyrbye LN, Sloan JA, Shanafelt TD. Single item measures of emotional exhaustion and depersonalization are useful for assessing burnout in medical professionals. J Gen Intern Med. 2009;24(12):1318–21.CrossRef
18.
go back to reference Taylor J, Deane FP. Development of a short form of the Test Anxiety Inventory (TAI). J Gen Psychol. 2002;129(2):127–36.CrossRef Taylor J, Deane FP. Development of a short form of the Test Anxiety Inventory (TAI). J Gen Psychol. 2002;129(2):127–36.CrossRef
19.
go back to reference Frierson HT, Hoban JD. The effects of acute test anxiety on NBME Part I performance. J Natl Med Assoc. 1992;84(8):686–9. Frierson HT, Hoban JD. The effects of acute test anxiety on NBME Part I performance. J Natl Med Assoc. 1992;84(8):686–9.
20.
go back to reference Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007;82(1):100–6.CrossRef Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007;82(1):100–6.CrossRef
21.
go back to reference Kleshinski J, Khuder SA, Shapiro JI, Gold JP. Impact of preadmission variables on USMLE step 1 and step 2 performance. Adv Health Sci Educ Theory Pract. 2009;14(1):69–78.CrossRef Kleshinski J, Khuder SA, Shapiro JI, Gold JP. Impact of preadmission variables on USMLE step 1 and step 2 performance. Adv Health Sci Educ Theory Pract. 2009;14(1):69–78.CrossRef
22.
go back to reference Zhang C, Rauchwarger A, Toth C, O’Connell M. Student USMLE step 1 preparation and performance. Adv Health Sci Educ Theory Pract. 2004;9(4):291–7.CrossRef Zhang C, Rauchwarger A, Toth C, O’Connell M. Student USMLE step 1 preparation and performance. Adv Health Sci Educ Theory Pract. 2004;9(4):291–7.CrossRef
23.
go back to reference Green M, Angoff N, Encandela J. Test anxiety and United States medical licensing examination scores. Clin Teach. 2015. doi:10.1111/tct.12386. Green M, Angoff N, Encandela J. Test anxiety and United States medical licensing examination scores. Clin Teach. 2015. doi:10.1111/tct.12386.
24.
go back to reference Thadani RA, Swanson DB, Galbraith RM. A preliminary analysis of different approaches to preparing for the USMLE step 1. Acad Med. 2000;75(10 Suppl):S40–2.CrossRef Thadani RA, Swanson DB, Galbraith RM. A preliminary analysis of different approaches to preparing for the USMLE step 1. Acad Med. 2000;75(10 Suppl):S40–2.CrossRef
25.
go back to reference Werner LS, Bull BS. The effect of three commercial coaching courses on step one USMLE performance. Med Educ. 2003;37(6):527–31.CrossRef Werner LS, Bull BS. The effect of three commercial coaching courses on step one USMLE performance. Med Educ. 2003;37(6):527–31.CrossRef
26.
go back to reference Kang SHK, McDermott KB, Roediger HL. Test format and corrective feedback modify the effect of testing on long-term retention. Eur J Cogn Psychol. 2007;19(4–5):528–58.CrossRef Kang SHK, McDermott KB, Roediger HL. Test format and corrective feedback modify the effect of testing on long-term retention. Eur J Cogn Psychol. 2007;19(4–5):528–58.CrossRef
27.
go back to reference Nguyen K, McDaniel MA. Using quizzing to assist student learning in the classroom the good, the bad, and the ugly. Teach Psychol. 2015;42(1):87–92.CrossRef Nguyen K, McDaniel MA. Using quizzing to assist student learning in the classroom the good, the bad, and the ugly. Teach Psychol. 2015;42(1):87–92.CrossRef
28.
go back to reference McConnell MM, St-Onge C, Young ME. The benefits of testing for learning on later performance. Adv Health Sci Educ Theory Pract. 2015;20(2):305–20.CrossRef McConnell MM, St-Onge C, Young ME. The benefits of testing for learning on later performance. Adv Health Sci Educ Theory Pract. 2015;20(2):305–20.CrossRef
29.
go back to reference Tucker P, Jeon-Slaughter H, Sener U, Arvidson M, Khalafian A. Do medical student stress, health, or quality of life foretell step 1 scores? A comparison of students in traditional and revised preclinical curricula. Teach Learn Med. 2015;27(1):63–70.CrossRef Tucker P, Jeon-Slaughter H, Sener U, Arvidson M, Khalafian A. Do medical student stress, health, or quality of life foretell step 1 scores? A comparison of students in traditional and revised preclinical curricula. Teach Learn Med. 2015;27(1):63–70.CrossRef
30.
go back to reference Strowd RE, Lambros A. Impacting student anxiety for the USMLE Step 1 through process-oriented preparation. Med Educ Online. 2010;15. Strowd RE, Lambros A. Impacting student anxiety for the USMLE Step 1 through process-oriented preparation. Med Educ Online. 2010;15.
31.
go back to reference Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002–9. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002–9.
Metadata
Title
Student-directed retrieval practice is a predictor of medical licensing examination performance
Authors
Francis Deng
Jeffrey A. Gluckstein
Douglas P. Larsen
Publication date
01-12-2015
Publisher
Bohn Stafleu van Loghum
Published in
Perspectives on Medical Education / Issue 6/2015
Print ISSN: 2212-2761
Electronic ISSN: 2212-277X
DOI
https://doi.org/10.1007/s40037-015-0220-x

Other articles of this Issue 6/2015

Perspectives on Medical Education 6/2015 Go to the issue