Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
Perspectives on Medical Education
Published in: Perspectives on Medical Education 1/2019

Open Access 01-02-2019 | Original Article

Putting post-decision wagering to the test: a measure of self-perceived knowledge in basic sciences?

Authors: Marjolein Versteeg, Paul Steendijk

Published in: Perspectives on Medical Education | Issue 1/2019

Login to get access

Abstract

Introduction

Students learn more effectively when they know what they do not know. Gaining insight into students’ metacognitive awareness is needed as misalignment between actual and self-perceived knowledge impedes their learning process. The optimal method of measuring self-perceived knowledge is still under debate. In this study, we evaluate the use of psychology-derived post-decision wagering for mapping students self-perceived knowledge.

Methods

Students (n = 71) performed a pre-test on medical physiology, followed by a teacher-moderated discussion and a post-test with isomorph questions. Half of the students rated their self-perceived knowledge on each question using post-decision wagering, i. e. betting 1–5 points on the correctness of their answer, whereas the other half used a 5-point Likert scale to rate their confidence.

Results

Self-perceived knowledge scores were higher for post-decision wagering (pre: 3.75 ± 0.14, post: 4.60 ± 0.07) compared with Likert scales (pre: 3.13 ± 0.08, post: 3.92 ± 0.08) despite similar actual knowledge scores. Furthermore, Likert ratings showed a near-normal distribution, whereas wagers were placed preferentially using the outer ends of the scale. Correlations between mean actual and self-perceived knowledge scores were low in both groups. On average, 8.5% of responses were classified as misconceptions, defined as highly confident incorrect answers.

Discussion

Despite the presumed reliability of post-decision wagering, our findings suggest that we should adhere to the use of Likert scales as a balanced measure for self-perceived knowledge in medical education. Moreover, the prevalence of misconceptions did not alter after instruction, indicating a need for instructional designs that enhance students’ conceptual understanding in basic sciences.
Literature
1.
Ziegler B, Montplaisir L. Student Perceived and Determined Knowledge of Biology Concepts in an Upper-Level Biology Course. CBE Life Sci Educ. IEEE Trans Med Imaging. 2014;13:322:30.
2.
Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77:1121–34.CrossRef
3.
Flavell JH. Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. Am Psychol. 1979;34:906.CrossRef
4.
Zohar A, Barzilai S. A review of research on metacognition in science education: Current and future directions. Stud Sci Educ. IEEE Trans Med Imaging. 2013;49:121:69.
5.
Pintrich PR. The role of metacognitive knowledge in learning, teaching, and assessing. Theory Pract. 2002;41:219–25.CrossRef
6.
Cavers M, Ling J. Confidence Weighting Procedures for Multiple-Choice Tests. In: Chen DG, Chen J, Lu X, Yi G, Yu H, editors. Advanced Statistical Methods in Data Science. ICSA Book Series in Statistics. Singapore: Springer; 2016.
7.
Sparck EM, Bjork EL, Bjork RA. On the learning benefits of confidence-weighted testing. Cogn Res Princ Implic. IEEE Trans Med Imaging. 2016;1:3.
8.
Swartz SM. Acceptance and Accuracy of Multiple Choice, Confidence-Level, and Essay Question Formats for Graduate Students. J Educ Bus. 2006;81:215–20.CrossRef
9.
Hopkins KD, Hakstian AR, Hopkins BR. Validity and reliability consequences of confidence weighting. Educ Psychol Meas. 1973;33:135–41.CrossRef
10.
Bokhorst FD. Confidence-weighting and the validity of achievement tests. Psych Rep. IEEE Trans Med Imaging. 1986;59:383:6.
11.
Brinkman DJ, Tichelaar J, van Agtmael MA, de Vries TP, Richir MC. Self-reported confidence in prescribing skills correlates poorly with assessed competence in fourth-year medical students. J Clin Pharmacol. 2015;55:825–30.CrossRef
12.
Kampmeyer D, Matthes J, Herzig S. Lucky guess or knowledge: A cross-sectional study using the Bland and Altman analysis to compare confidence-based testing of pharmacological knowledge in 3rd and 5th year medical students. Adv Health Sci Educ. IEEE Trans Med Imaging. 2015;20:431:40.
13.
Rangel RH, Möller L, Sitter H, Stibane T, Strzelczyk A. Sure, or unsure? Measuring students’ confidence and the potential impact on patient safety in multiple-choice questions. Med Teach. 2017;39:1189–94.CrossRef
14.
Curtis DA, Lind SL, Boscardin CK, Dellinges M. Does student confidence on multiple-choice question assessments provide useful information? Med Educ. 2013;47:578–84.CrossRef
15.
Lai NM, Teng CL. Self-perceived competence correlates poorly with objectively measured competence in evidence based medicine among medical students. Bmc Med Educ. 2011;11:25.CrossRef
16.
Brydges R, Butler D. A reflective analysis of medical education research on self-regulation in learning and practice. Med Educ. 2012;46:71–9.CrossRef
17.
Sandars J. The use of reflection in medical education: AMEE Guide. Med Teach. 2009;31(44):685–95.CrossRef
18.
Leggett H, Sandars J, Burns P. Helping students to improve their academic performance: a pilot study of a workbook with self-monitoring exercises. Med Teach. 2012;34:751–3.CrossRef
19.
Tanner K, Allen D. Approaches to biology teaching and learning: understanding the wrong answers—teaching toward conceptual change. CBE Life Sci Educ. IEEE Trans Med Imaging. 2005;4:112:7.
20.
Woods NN. Science is fundamental: the role of biomedical knowledge in clinical reasoning. Med Educ. 2007;41:1173–7.CrossRef
21.
Dawson-Saunders B, Feltovich PJ, Coulson RL, Steward DE. A survey of medical school teachers to identify basic biomedical concepts medical students should understand. Acad Med. 1990;65:448–54.CrossRef
22.
Michael JA, Wenderoth MP, Modell HI, et al. Undergraduates’ understanding of cardiovascular phenomena. Adv Physiol Educ. 2002;26:72–84.CrossRef
23.
Michael J. What makes physiology hard for students to learn? Results of a faculty survey. Adv Physiol Educ. 2007;31:34–40.CrossRef
24.
Versteeg M, Wijnen-Meijer M, Steendijk P. Informing the uninformed: a multitier approach to uncover students’ misconceptions on cardiovascular physiology. Adv Physiol Educ. 2018;43:7–14.CrossRef
25.
Badenhorst E, Hartman N, Mamede S. How Biomedical Misconceptions May Arise and Affect Medical Students’ Learning: A Review of Theoretical Perspectives and Empirical Evidence. Health Prof Educ. IEEE Trans Med Imaging. 2016;2:10:7.
26.
Posner GJ, Strike KA, Hewson PW, Gertzog WA. Accommodation of a scientific conception: Toward a theory of conceptual change. Sci Educ. IEEE Trans Med Imaging. 1982;66:211:27.
27.
Hasan S, Bagayoko D, Kelley EL. Misconceptions and the certainty of response index (CRI). Phys Educ. IEEE Trans Med Imaging. 1999;34:294:9.
28.
Gunstone RF, Mitchell IJ. Metacognition and conceptual change. In: Mintzes JJ, Wandersee JH, Novak JD, editors. Teaching science for understanding. A human constructivist view. San Diego, CA: Academic Press; 2005. pp. 133–63.CrossRef
29.
Schraw G, Dennison RS. Assessing metacognitive awareness. Contemp Educ Psychol. 1994;19:460–75.CrossRef
30.
Tippett CD. Refutation text in science education: A review of two decades of research. Int J Sci Math Educ. 2010;8:951–70.CrossRef
31.
Lin LM, Zabrucky KM. Calibration of comprehension: Research and implications for education and instruction. Contemp Educ Psychol. 1998;23:345–91.CrossRef
32.
Caleon IS, Subramaniam R. Do students know what they know and what they don’t know? Using a four-tier diagnostic test to assess the nature of students’ alternative conceptions. Res Sci Educ. IEEE Trans Med Imaging. 2010;40:313:37.
33.
Sreenivasulu B, Subramaniam R. Exploring undergraduates’ understanding of transition metals chemistry with the use of cognitive and confidence measures. Res Sci Educ. IEEE Trans Med Imaging. 2014;44:801:28.
34.
Yan YK, Subramaniam R. Using a multi-tier diagnostic test to explore the nature of students’ alternative conceptions on reaction kinetics. Chem Educ Res Pract. IEEE Trans Med Imaging. 2018;19:213:26.
35.
Brandriet AR, Bretz SL. The development of the redox concept inventory as a measure of students’ symbolic and particulate redox understandings and confidence. J Chem Educ. 2014;91:1132–44.CrossRef
36.
Koch C, Preuschoff K. Betting the house on consciousness. Nat Neurosci. 2007;10:140–1.CrossRef
37.
Persaud N, McLeod P, Cowey A. Post-decision wagering objectively measures awareness. Nat Neurosci. 2007;10:257–61.CrossRef
38.
Dienes Z, Scott R. Measuring unconscious knowledge: Distinguishing structural knowledge and judgment knowledge. Psychol Res. 2005;69:338–51.CrossRef
39.
Wierzchoń M, Wronka E, Paulewicz B, Szczepanowski R. Post-decision wagering affects metacognitive awareness of emotional stimuli: an event related potential study. PLoS ONE. 2016;11:e159516.CrossRef
40.
Sandberg K, Timmermans B, Overgaard M, Cleeremans A. Measuring consciousness: is one measure better than the other? Conscious Cogn. 2010;19:1069–78.CrossRef
41.
Dienes Z, Seth A. Gambling on the unconscious: A comparison of wagering and confidence ratings as measures of awareness in an artificial grammar task. Conscious Cogn. 2010;19:674–81.CrossRef
42.
Fleming SM, Dolan RJ. Effects of loss aversion on post-decision wagering: implications for measures of awareness. Conscious Cogn. 2010;19:352–63.CrossRef
43.
Palizvan MR, Nejad MR, Jand A, Rafeie M. Cardiovascular physiology misconceptions and the potential of cardiovascular physiology teaching to alleviate these. Med Teach. 2013;35:454–8.CrossRef
44.
Lisk K, Agur AM, Woods NN. Exploring cognitive integration of basic science and its effect on diagnostic reasoning in novices. Perspect Med Educ. 2016;5:147–53.CrossRef
45.
Goldszmidt M, Minda JP, Devantier SL, Skye AL, Woods NN. Expanding the basic science debate: the role of physics knowledge in interpreting clinical findings. Adv Health Sci Educ. IEEE Trans Med Imaging. 2012;17:547:55.
46.
Bell P, Volckmann D. Knowledge surveys in general chemistry: confidence, overconfidence, and performance. J Chem Educ. 2011;88:1469–76.CrossRef
47.
Favazzo L, Willford JD, Watson RM. Correlating student knowledge and confidence using a graded knowledge survey to assess student learning in a general microbiology classroom. J Microbiol Biol Educ. 2014;15:251.CrossRef
48.
Lindsey BA, Nagel ML. Do students know what they know? Exploring the accuracy of students’ self-assessments. Phys Rev Spec Top-Phys Educ R. IEEE Trans Med Imaging. 2015;11:20103.
49.
Potgieter M, Malatje E, Gaigher E, Venter E. Confidence versus performance as an indicator of the presence of alternative conceptions and inadequate problem-solving skills in mechanics. Int J Sci Educ. 2010;32:1407–29.CrossRef
50.
Ravesloot CJ, Van der Schaaf MF, Muijtjens AM, et al. The don’t know option in progress testing. Adv Health Sci Educ. IEEE Trans Med Imaging. 2015;20:1325:38.
51.
Anderson RC, Pearson PD. A schema-theoretic view of basic processes in reading comprehension. In: Pearson PD, Barr R, Kamil ML, Mosenthal P, editors. Handbook of reading research. New York: Longman; 1984. pp. 255–91.
52.
Bransford JD, Johnson MK. Contextual prerequisites for understanding: Some investigations of comprehension and recall. J Verbal Learn Verbal Behav. 1972;11:717–26.CrossRef
53.
54.
Tse D, Langston RF, Kakeyama M, et al. Schemas and memory consolidation. Science. 2007;316:76–82.CrossRef
55.
van Kesteren MT, Ruiter DJ, Fernández G, Henson RN. How schema and novelty augment memory formation. Trends Neurosci. 2012;35:211–9.CrossRef
56.
Hewson MG, Hewson PW. Effect of instruction using students’ prior knowledge and conceptual change strategies on science learning. J Res Sci Teach. 1983;20:731–43.CrossRef
57.
Hailikari T, Katajavuori N, Lindblom-Ylanne S. The relevance of prior knowledge in learning and instructional design. Am J Pharm Educ. 2008;72:113.CrossRef
Metadata
Title
Putting post-decision wagering to the test: a measure of self-perceived knowledge in basic sciences?
Authors
Marjolein Versteeg
Paul Steendijk
Publication date
01-02-2019
Publisher
Bohn Stafleu van Loghum
Published in
Perspectives on Medical Education / Issue 1/2019
Print ISSN: 2212-2761
Electronic ISSN: 2212-277X
DOI
https://doi.org/10.1007/s40037-019-0495-4

Other articles of this Issue 1/2019

Perspectives on Medical Education 1/2019 Go to the issue

Eye-Opener

Problems of problem-based learning: Towards transformative critical pedagogy in medical education

Thanks to Reviewers

Thanks to reviewers

A Qualitative Space

Why institutional ethnography? Why now? Institutional ethnography in health professions education

Show and Tell

The Clinical Reasoning Mapping Exercise (CResME): a new tool for exploring clinical reasoning

Eye-Opener

A critical perspective on the modified personal interview

Show and Tell

Implementing an interprofessional narrative medicine program in academic clinics: Feasibility and program evaluation