A CURE Lab in Introductory Biology at a Regional Comprehensive University Negatively Impacts Student Success in the Associated Lecture Course Among Students from Groups Underrepresented in Science
Abstract
Course-based undergraduate research experiences (CUREs) have been proposed as a mechanism to democratize access to the benefits of apprentice-style scientific research to a broader diversity of students, promoting inclusivity and increasing student success and retention. As we evaluate CUREs, it is essential to explore their effectiveness within the environments of regional comprehensive universities and community colleges, because they are important access points for a wide variety of students. It is also important to address the potential influence of volunteer bias, where students can opt to enroll in either the CURE or a traditional lab, on the outcomes of CUREs. We evaluated a CURE at a regional comprehensive university under conditions both with and without volunteer bias. We find that nonvolunteer students report a lower sense of discovery and relevance of the CURE compared with students who volunteered for the course. Importantly, we also find that our replacement of the traditional lab class with a CURE resulted in lower scores on exams in the associated lecture course among students who are both BIPOC and Pell eligible. We call for additional research on the effects of CUREs at nonresearch-intensive institutions and without volunteer bias, to better understand the impact of these classes.
When a traditional lab was replaced with a CURE a Regional Comprehensive Institution, students who are both BIPOC and Pell eligible have lower scores on the exams in the associated lecture course. Also, students who did not volunteer for the CURE report a lower sense of discovery and relevance in the course.
INTRODUCTION
Course-Based Undergraduate Research Experiences (CUREs) are a powerful pedagogical approach in undergraduate science education. The recent shift from traditional laboratory instruction to authentic in-class research experiences has been met with optimism for its potential to enhance student engagement, success, and persistence in the sciences. CUREs were first designed and implemented at research-intensive institutions as an outgrowth of faculty research programs (Buchanan and Fisher, 2022). The reported positive outcomes of CUREs at these institutions include increased student motivation, learning gains, improved retention in STEM disciplines, and the development of essential scientific competencies (Drew and Triplett, 2008; Harrison et al., 2011; Bascom-Slack et al., 2012; Kloser et al., 2013; Auchincloss et al., 2014; Bangera and Brownell, 2014; Jordan et al., 2014; Shaffer et al., 2014; Rodenbusch et al., 2016; Hanauer et al., 2017; Indorf et al., 2019; Lopatto et al., 2022; Newell and Ulrich, 2022).
One of the most compelling promises of CUREs lies in their potential to democratize access to authentic research experiences for all students at the college level (Smith et al., 2023). Unlike apprentice-style undergraduate research experiences in which students are mentored one-on-one by a senior-level researcher on a project outside of class and are thus limited to a select few individuals, CUREs integrate authentic research into the curriculum, making it accessible to a broader student population. By embedding research within coursework, particularly at the first-year level, CUREs can dismantle barriers related to prior research experience, socioeconomic status, or institutional resources and thus could enhance diversity and equity in STEM fields (Olson and Riordan, 2012; Integrating Discovery-Based Research into the Undergraduate Curriculum, 2015; Barabino et al., 2023).
As we evaluate CUREs for democratizing access to research opportunities, promoting inclusivity, and increasing student success and retention in the sciences, it is essential to explore their effectiveness within the environments of regional comprehensive universities and community colleges because these institutions are important access points for a diversity of students, including students with low socioeconomic status and students who identify as Black, Indigenous, or Persons of Color (BIPOC; García et al., 2022). Thus, regional comprehensive universities and community colleges provide fertile ground for assessing the broad applicability, effectiveness, and inclusivity of CUREs (Bangera and Brownell, 2014; Estrada et al., 2016; Ing et al., 2021; Martin et al., 2021; Killpack and Popolizio, 2023).
There are a small number of reports that evaluate CUREs at regional comprehensive universities and community colleges. In three studies at community colleges of volunteer CUREs (in which students self-select the option of enrolling in a CURE rather than a traditional lab class), students were observed to demonstrate learning gains, to report increased positive attitudes toward research, and to report greater intention to seek a higher academic degree after the CURE, though none of these studies included traditional lab data for comparison (Beagley, 2013; Ashcroft et al., 2021; Meyer et al., 2023). At a community college with a nonvolunteer CURE (in which no traditional lab class was offered during the same semester), students reported greater enjoyment of the CURE compared with traditional lab students (Wolkow et al., 2014). A study from a Hispanic-serving institution found that CURE students had higher grades in the affiliated lecture course; placement in the CURE was random in this study, but the students were selected from among those who had already volunteered to be part of a highly structured first-year success in science program (Ing et al., 2021). One study presents conflicting data; a non-volunteer CURE at a comprehensive, minority-serving institution indicates no change in students’ career goals following two semesters of the CURE (Martin et al., 2021).
CUREs, unlike traditional labs, are not often designed to promote student understanding of lecture content, so it is also essential to evaluate the effect of replacing a traditional lab with a CURE on student performance in the lecture course to ensure that we are not detracting from students’ ability to succeed in associated lectures. A small number of CURE studies have explicitly evaluated lecture course outcomes, but none of these reports evaluate the effect on student subgroups. Students in an Introductory Biology CURE at the University of Washington have no difference in score on three out of four lecture exams (and 2.6% lower on one lecture exam) compared with traditional lab students, but the data set was not large enough to detect differences by subgroup (Freeman et al., 2023). A study at the University of Alaska Anchorage reported that lab participation does not affect course outcomes in the associated nonmajors Introductory Biology lecture; individual student subgroups are included as controls in their regression analysis, but the effects of belonging to multiple subgroups were not evaluated (DeFeo et al., 2020). At the University of California-Riverside, a Hispanic-serving institution, students randomly assigned to a biology CURE had significantly higher overall grades in a lecture course compared with students in the traditional lab; however all of the students included in this study had volunteered to be part of a highly structured mentoring program (Ing et al., 2021). At the University of Texas at El Paso, there were no negative effects on biological content knowledge after a CURE was implemented; this CURE was unique in that it addressed the potential disconnect from the lecture class by intentionally incorporating supplemental instruction active-learning exercises focused on topics discussed in lecture that week during the first 30–45 min of each laboratory session (Olimpo et al., 2016; Fisher et al., 2018).
It is also imperative that we address the challenges posed by enrollment conditions, particularly in the context of volunteer versus nonvolunteer participation. Volunteer bias in research studies, such as where students can opt to enroll in either the CURE or a traditional version of the lab, introduces a potential source of distortion in the outcomes (Ball et al., 1993; Holden et al., 1993; Rosnow et al., 1969). Students who choose to participate in CUREs may possess distinct characteristics, such as higher self-confidence, preexisting interest in research, or stronger academic backgrounds, compared with those who opt for the traditional lab setting (Rosenthal and Rosnow, 1975). This self-selection can influence the observed outcomes, potentially leading to an overestimation of the benefits associated with CUREs. One study of a nonvolunteer CURE at a research institution indicates an increase in self-reported enjoyment and intrinsic scientific motivation compared with the traditional lab (Olimpo et al., 2016). In contrast, two reports from nonvolunteer CUREs indicate no change in student-reported interest in scientific research after the CURE; however, sample sizes in these studies were small (Brownell et al., 2013; Kloser et al., 2013). More studies that compare CUREs with traditional labs under conditions without volunteer bias using large sample sizes are needed to provide a more accurate understanding of the true impact of these classes.
Here, we aim to explore the context-dependent effects of CURE implementation in a regional comprehensive university, with emphasis on the consequences of volunteer bias in enrollment. We have conducted a large-scale, multisemester study of a CURE in Introductory Biology at Eastern Michigan University (EMU), which is a nonselective institution with high student diversity. To our knowledge, this is the largest study to date of a CURE at a regional comprehensive university. We addressed the following research questions, under conditions both with and without volunteer bias: (1) Do students perceive experiencing the components of a high-quality CURE? (2) Does the CURE at EMU have the same impact on student perception of research project ownership as has been published for CUREs at research-intensive institutions? (3) Does the lack of association between the CURE lab and the lecture course at EMU negatively affect student success in the lecture course? (4) Does the CURE at EMU have differential positive impacts on students who identify as BIPOC, students with low socioeconomic status, or students who are both BIPOC and have low socioeconomic status?
This research is guided self-determination theory (SDT). This theory proposes a self-determination continuum for humans that ranges from amotivation (action without intent or “going through the motions”) through extrinsic motivation (action in response to external demand or personal understanding of importance), finally reaching intrinsic motivation (action due to personal interest/enjoyment; Ryan and Deci, 2000). Motivation that is intrinsic, rather than extrinsic, produces enhanced performance, persistence, and self-esteem (Deci and Ryan, 1991, 1995). This SDT framework influences our research questions, methodology, and analyses (Luft et al., 2022). Drawing from SDT, we anticipate that volunteer bias will affect student perceptions of the research project and overall performance, because students who volunteer are more likely to have intrinsic motivation for the CURE, rather than extrinsic motivation. Through this investigation, we seek to contribute valuable insights into the efficacy of CUREs in enhancing student learning and contributing to the democratization of research experiences within nonresearch-intensive institutions.
MATERIALS AND METHODS
Characteristics of EMU
EMU is a regional comprehensive institution with an enrollment of approximately 14,500 undergraduates and 2,400 graduate students, primarily in Masters programs. EMU is an Institution of Opportunity: Enrollment is open to any student who received a 2.0 or better GPA in high school or community college. Approximately 34% of EMU students are Pell grant recipients, and approximately 28% are the first generation in their family to attend college. EMU has a highly diverse student enrollment; 17% of students are Black/African-American, 9% are Hispanic/Latino/a, and 59% are White. A large proportion of students at EMU in the “White” category are of Middle Eastern or North African descent; EMU does not collect information to identify students in this demographic group thus we do not have a specific percentage to report. EMU is primarily a commuter institution; only 20% of students live on campus. Approximately 64% of students are enrolled full-time. EMU is not a research-intensive institution and faculty at EMU teach 12 contact hours per semester.
Format of Introductory Biology Lecture, Traditional Lab, and Tiny Earth CURE at EMU
Students taking Introductory Biology: Cells & Molecules at EMU are required to enroll in both the lecture and lab courses simultaneously. The Introductory Biology: Cells & Molecules lecture course meets for two 75-min class sessions each week. The lecture course content includes biological molecules, cell structure and function, energetics, cell division, Mendelian and population genetics, evolution, and ecology. Students must earn a C or better in both the lecture and the laboratory courses in order to move to courses beyond the introductory level.
The “traditional” version of the Introductory Biology: Cells & Molecules lab at EMU was fully designed by the faculty who teach the associated lecture course, and was taught with very little change from 2010–2020. The enrollment cap on each lab section is 24 students, with multiple sections offered every semester. The traditional lab is taught by Masters students, who are trained by faculty. The traditional version of the Introductory Biology: Cells & Molecules lab was explicitly designed to do three things: teach technical skills “at the bench,” teach skills in scientific communication, and reinforce lecture class content. The traditional lab meets for one, 4-h class per week. The first hour of the lab is a recitation reviewing that week’s lecture class material, and the remaining time is a hands-on experiment directly related to that week’s lecture class material. A representative schedule from the traditional lab is provided in the Supplemental Materials; its design was stable during the semesters studied for this research.
The CURE version of the Introductory Biology: Cells & Molecules lab at EMU is associated with the Tiny Earth network, https://tinyearth.wisc.edu (Hurley et al., 2021). The implementation of this CURE at EMU was designed jointly by two faculty, one who teaches the associated Introductory Biology: Cells & Molecules lecture course, and one who teaches EMU courses in microbiology. Like the traditional lab, the CURE at EMU was designed to teach technical skills and scientific communication. Unlike the traditional lab, the CURE does not reinforce lecture content. The CURE was designed to increase students’ sense of research project ownership by having each student bring in their own soil sample of interest and work with bacteria from their soil sample throughout the semester, with the hope of increasing student retention and persistence in biology (Hanauer and Dolan, 2014; Corwin et al., 2018). Like the traditional lab, the CURE enrollment cap on each section is 24 students, with multiple sections offered every semester, and it is taught by Masters students, who are trained by faculty. The Tiny Earth CURE lab meets twice/week for 2 h each time. None of the CURE class time is spent on that week’s lecture class material; all of the time is spent working on the semester-long research project and understanding the concepts necessary to understand the context of the project, many of which are not covered in the associated lecture. In the Tiny Earth CURE students isolate and characterize antibiotic-producing microbes from local soils. The details of these microbes are added to a national Tiny Earth database and frozen stocks are preserved locally. This research project is completed entirely within the scheduled class time; students do not come to the laboratory outside of class time to carry out research. Further investigation of these frozen stocks by EMU faculty together with undergraduate researchers (in the apprentice-style research model) has resulted in the publication of one newly identified antibiotic (Mohamed et al., 2021).
The operational definition of a CURE and the essential features of these courses were initially shaped by a meeting report of the CUREnet (Auchincloss et al., 2014). This report identified five dimensions that together make a learning experience a CURE: (1) use of scientific practices, (2) discovery, in that the outcome of the research is unknown to both the students and the instructor, (3) relevance, in that the outcome of the research is useful and valued by individuals outside of the classroom (other scientists and/or the local community), (4) collaboration among students in the class, and (5) iteration both for revising work to address problems and for gathering additional data to support claims (Auchincloss et al., 2014). Brownell and Kloser (2015) recommended grouping together discovery and relevance, and Corwin et al. (2015) developed the Laboratory Course Assessment Survey (LCAS), a tool that has been widely used to evaluate the latent constructs of Collaboration, Iteration, and Discovery/Relevance in CUREs. When we designed the Tiny Earth course at EMU, we followed these published best practices in CUREs to implement the three key features of Collaboration, Iteration, and Discovery/Relevance. In our course, student collaborated in stable groups of two to four throughout the semester, the schedule was created with time explicitly built in for Iteration (both for repeating failed experiments and expanding on earlier results), and the new antibiotic-producing microbes discovered by students in the course are shared with scientists throughout the Tiny Earth network. A representative schedule from the CURE lab is provided in the Supplemental Materials; its design was stable during the semesters studied for this research.
Sample Population for Experiment #1 (Volunteer CURE)
Our first experiment evaluated the effects of the Tiny Earth CURE at EMU under conditions of volunteer bias. We collected data for this experiment during four semesters: Fall 2018, Winter 2019, Fall 2019, and Winter 2020. In all four of these semesters, multiple sections of both the traditional lab and the CURE were offered for Introductory Biology: Cells & Molecules, and students could choose in which version of the lab course they wished to enroll. Students were informed about the options they had for their introductory biology lab (i.e., CURE or traditional) through explanatory posters in the science buildings on campus and through a 3-min video that was given to academic advisors to share with students when they met with them for registration. This video was made by one of the EMU faculty teaching the CURE and it described the differences between the CURE and the traditional lab in such a way that neither the CURE nor the traditional lab was intentionally favored. Over the Experiment #1 semesters, a total of 239 students chose to enroll in the CURE and a total of 684 students chose to enroll in the traditional lab.
| Experiment #1 | Experiment #2 | |||
|---|---|---|---|---|
| Traditional lab | Volunteer CURE | Traditional lab | Nonvolunteer CURE | |
| Semesters included in survey analysis | Fall 2018Winter 2019Fall 2019Winter 2020 | Fall 2018Winter 2019Fall 2019Winter 2020 | Fall 2016Winter 2017Fall 2017Winter 2018 | Fall 2021 |
| Total number of survey responsesa(overall response rate) | 113 (16%) | 61 (26%) | 82 (8%) | 59 (26%) |
| Responses from students who are neither Pell eligible nor BIPOC | 51 | 29 | 44 | 34 |
| Responses from students who are Pell eligible only | 23 | 16 | 19 | 12 |
| Responses from students who are BIPOC only | 13 | 5 | 7 | 6 |
| Responses from students who are both Pell eligible and BIPOC | 18 | 6 | 8 | 6 |
Sample Population for Experiment #2 (Nonvolunteer CURE)
Our second experiment evaluated the effects of the Tiny Earth CURE at EMU when volunteer bias was not present. We collected data from four semesters in which only the traditional lab was offered for Introductory Biology: Cells & Molecules (Fall 2016, Winter 2017, Fall 2017, Winter 2018), and from three semesters in which only the Tiny Earth CURE was offered (Fall 2021, Winter 2022, Fall 2022). Over the Experiment #2 semesters, a total of 619 students were enrolled in the CURE and a total of 1091 students were enrolled in the traditional lab.
Demographic Data and Concurrent GPA
Data on student sex, race/ethnicity, Pell grant eligibility, declared major at the time of taking the Introductory Biology lab class, and grades in other courses taken concurrently with Introductory Biology were obtained from the EMU Office of Institutional Research and Information Management (IRIM). One limitation of this dataset is that EMU data on student sex is only female or male; there was not an option for students to indicate their gender identity. We used Pell grant eligibility as a proxy for low socioeconomic status. We excluded first generation status from our analysis because approximately 20% of EMU students do not respond to this question when asked on their financial aid forms. Additionally, students are not given the opportunity to update this information if they initially reported it incorrectly. Using the race/ethnicity data obtained from IRIM, students were coded as “Yes” for BIPOC (Black, Indigenous, People of Color) if they selected Black/African American, Hispanic/Latino, Native American, Native Hawaiian, or Two or More Races. We created a “subgroup” category in which students were coded Pell eligible only, BIPOC only, both Pell eligible and BIPOC, or neither Pell eligible nor BIPOC. Concurrent GPA was calculated for each student by determining the GPA from all courses taken the same semester as the Introductory Biology, without including the Introductory Biology lecture or lab. We used concurrent GPA as a predictor in our regression models to control for variation in the academic preparation of EMU students (Theobald and Freeman, 2014). Summarized student demographics from both experiments are reported in Supplemental Data.
Survey Data Collection
A postcourse survey was used to address the first two research questions, (1) Do students perceive experiencing the components of a high-quality CURE? and (2) Does the CURE at EMU have the same impact on student perception of research project ownership as has been published for CUREs at research-intensive institutions? This survey consisted of all questions from the Laboratory Course Assessment Survey (LCAS; Corwin et al., 2015) and all questions from the Project Ownership Survey (POS; Hanauer and Dolan, 2014; Supplemental Data). The LCAS has 17 Likert-scale items and three sub-scales that measure students’ perceptions of the key design features of CURE labs, which are Collaboration, Iteration, and Discovery/Relevance. The POS has 16 Likert-scale items that can significantly differentiate students’ sense of ownership of a research project. It has two subscales: 1) cognitive ownership, which evaluates students’ commitment to a research project, and 2) emotional ownership, which evaluates students’ personal identification with a research project.
For Experiment #1 (volunteer CURE), all students in all semesters of this experiment in both the traditional and CURE versions of the lab were asked to complete this survey online, postcourse, through REDCap software. We received a total of 174 Experiment #1 responses (113 traditional lab; 61 CURE). For Experiment #2 (nonvolunteer CURE), the Fall 2021 CURE semester students, and all traditional semester students for this experiment, were asked to complete the survey online, postcourse, through REDCap software. We received a total of 141 Experiment #2 responses (82 traditional lab; 59 CURE). Because the surveys were distributed after the laboratory class ended and there was no incentive offered for taking the survey, response rates were low. The response rates of student subgroups, however, were consistent with the overall response rate for each experiment and lab type (Table 1).
Confirmatory Factory Analysis (CFA) of Survey Data
We determined the average response per student on each survey subscale (students’ perception of Collaboration, Iteration, Discovery/Relevance, cognitive ownership, emotional ownership). We used confirmatory factor analysis (CFA; Knekta et al., 2019) to determine whether our survey data fit the expected three-factor solution for the LCAS (Corwin et al., 2015) and the expected two-factor solution for the POS (Hanauer and Dolan, 2014). CFA was run using the R package lavaan (Rosseel, 2012). We evaluated the fit of the model using the cutoffs recommended by Hu and Bentler (1999).
CFA confirms that our LCAS data fit within the published three-factor model (Knekta et al., 2019). The LCAS data we obtained has a comparative fit index (CFI) = 0.932, root-mean-square error of approximation (RMSEA) = 0.076 with 90% confidence interval of 0.063–0.090, and standardized root-mean-square residual (SRMR) = 0.046. These values are within or very close to the cutoff recommendations of CFI ≥ 0.95, RMSEA < 0.06, SRMR < 0.08 (Hu and Bentler, 1999).
CFA of our POS data indicates a CFI = 0.832, RMSEA = 0.160 with 90% confidence interval of 0.149–0.172, and SRMR = 0.060. Thus, we find that only the SRMR is within the cutoff recommendations of CFI ≥ 0.95, RMSEA < 0.06, SRMR < 0.08 (Hu and Bentler, 1999) indicating that our POS survey data do not fit within the published two-factor model of cognitive ownership and emotional ownership (Hanauer and Dolan, 2014). Other publications have also reported a poor fit for the expected two-factor solution for the POS (Corwin et al., 2018; Hester et al., 2018). The POS questions for the latent factor of emotional ownership may be a blend of enjoyment and surprise (Corwin et al., 2018). The six survey questions that address this factor are formatted as follows, “To what extent does the word _____ describe your experience of the laboratory course?” There are six words, three of which (delighted, happy, joyful) fit the latent factor of enjoyment, and three of which (astonished, surprised, and amazed) fit the latent factor of surprise (Corwin et al., 2018). We evaluated the three-factor model for the POS of cognitive ownership, enjoyment, and surprise and found that the CFA data are better but not within traditional cutoff values; CFI = 0.893, RMSEA = 0.130 with 90% confidence interval of 0.118 – 0.142, SRMR = 0.050. We next tested the cognitive project ownership items on their own as a single factor; the CFA data indicate that these items do not load onto a single factor, CFI = 0.816, RMSEA = 0.207 with 90% confidence interval of 0.190 – 0.225. SRMR = 0.065. Finally, we tested the emotional ownership items on their own split into the two sub-factors of enjoyment and surprise; CFI and SRMR are well within cutoff values but RMSEA remains high, CFI = 0.973, RMSEA = 0.145 with 90% confidence interval of 0.101–0.192, SRMR = 0.033. Because the cognitive ownership items do not function as expected for our sample, only the emotional ownership subfactors of enjoyment and surprise on the POS were further evaluated. Student responses to each item in the cognitive ownership subscale are presented in the Supplemental Methods.
Power Analysis of Survey Data
To determine whether our survey data was sufficient to detect meaningful differences in how lab type impacted the perceptions of BIPOC or Pell-eligible students, we did a power analysis of each experiment. We measured effect sizes for each subgroup (Pell eligible only, BIPOC only, or both Pell eligible, and BIPOC) compared with students who are neither Pell eligible nor BIPOC in each experiment using Cohen’s d and interpreted them using educational interventions guidelines recommended by Kraft: effect size less than 0.05 is small, 0.05 to less than 0.20 is medium, and 0.20 and above is large (Kraft, 2020; Supplemental Materials). Our power analysis for Experiment #1 (volunteer CURE) indicated that our LCAS and POS data is sufficient to detect medium effect sizes for latent constructs in most of the student subgroups; however, for Experiment #2 (nonvolunteer CURE), we can detect medium effect sizes only in the subgroup of students who are both BIPOC and Pell eligible while the effect sizes for other groups are small (Supplemental Materials).
Regression Analysis of Survey Data
The average response per student on each survey subscale was used as the outcome variable in linear regression. Model selection was carried out as recommended for discipline-based education research (Theobald, 2018). Because students are nested in lab sections, and lab sections are nested in semesters, we evaluated whether these would be appropriate to include as random effects in a multilevel regression model. We calculated the intraclass correlation (ICC) for lab section and semester in both Experiment #1 and Experiment #2 for each survey subscale. All ICC values were < 0.001 (Supplemental Data). The fixed effects that we tested included Lab Type (Traditional or CURE), student subgroup (Pell only, BIPOC only, Both, or Neither), and sex (male or female). After determining the AIC values for all possible models, the best fitting model (with the lowest AIC value) was selected (Supplemental Data). The best-fitted models for both Experiments #1 and #2 are as follows:
LCAS:
Collaboration ∼ Lab Type * Subgroup
Iteration ∼ Lab Type * Subgroup
Discovery Relevance ∼ Lab Type * Subgroup
POS:
Enjoyment ∼ Lab Type * Subgroup
Surprise ∼ Lab Type * Subgroup
Lecture Course Exam Score Data Collection
Exam scores in the associated lecture course were used to address the third research question, does the lack of association between the CURE lab and the lecture course at EMU negatively affect student success in the lecture course? For all students in Experiment #1 (volunteer CURE), raw exam scores were requested from instructors of the Introductory Biology: Cells & Molecules lecture course. Within each lecture class there was a mixture of students enrolled in each lab type. Four of the six instructors who taught the lecture course during the semesters of Experiment #1 provided raw exam scores. All four of these instructors have taught the Introductory Biology: Cells & Molecules lecture course for multiple years, including many semesters before the data collection semesters. Of the students in Experiment #1, lecture exam scores were provided by instructors for 59% of all students who took the traditional lab (n = 407) and 46% of all students who took the Tiny Earth CURE (n = 110; Table 2).
| Experiment #1(volunteer CURE) | Experiment #2(nonvolunteer CURE) | |
|---|---|---|
| Semesters included in lecture exam analysis | Fall 2018Winter 2019Fall 2019Winter 2020 | Fall 2017 (Trad)Winter 2018 (Trad)Fall 2021 (CURE)Winter 2022 (CURE)Fall 2022 (CURE) |
| Number of lecture instructors who provided exam data | 4 (of 6 total possible) | 2a (of 4 total possible) |
| Number of students with lecture exam score data (proportion)b | Trad n = 407 (60%)CURE n = 110 (46%) | Trad n = 219 (39%)CURE n = 219 (35%) |
| Number of students in exam dataset who are neither Pell eligible nor BIPOC | Trad n = 171CURE n = 40 | Trad n = 95CURE n = 96 |
| Number of students in exam dataset who are Pell eligible only | Trad n = 85CURE n = 22 | Trad n = 45CURE n = 38 |
| Number of students in exam dataset who are BIPOC only | Trad n = 38CURE n = 11 | Trad n = 32CURE n = 32 |
| Number of students in exam dataset who are both Pell eligible and BIPOC | Trad n = 67CURE n = 25 | Trad n = 32CURE n = 40 |
There were two instructors who taught the Introductory Biology: Cells & Molecules lecture course during both traditional lab semesters and CURE semesters for Experiment #2 (nonvolunteer CURE). These instructors are two of the four that were included in lecture exam analysis for Experiment #1. From these two instructors, we were able to obtain lecture exam score data from the 2017–2018 academic year and from the Fall 2021, Winter 2022, and Fall 2022 semesters (Table 2). Beginning in Fall 2021, the EMU Biology department made broad changes to the topics presented in our two semesters of Introductory Biology. Due to this change, approximately 25% of the material that had been included in the Introductory Biology: Cells & Molecules lecture course (particularly the ecology topics) was removed. Thus, starting in Fall 2021 this lecture course overall covered fewer content topics, and more class time was available to spend on each topic. The ecology content that was removed from the Introductory Biology: Cells & Molecules lecture class had been included on Lecture Exam #4 by both of the instructors included in this analysis. Thus, for the Experiment #2 lecture exams analysis, we excluded Exam #4 from the pre-Fall 2021 data, so that our comparison is only between lecture topics that were common to all students in this data set. The sample size for Experiment #2 lecture exam score analysis is 219 students who took the traditional lab and 219 students who took the Tiny Earth CURE.
Regression Analysis of Lecture Course Exam Score Data
The proportion of exam points earned from the total possible was calculated for each student. Model selection was carried out as recommended for discipline-based education research (Theobald, 2018). We evaluated whether lab section, semester, or lecture instructor by semester (because lecture instructors taught in multiple semesters) would be appropriate to include as random effects in a multilevel regression model. We calculated the ICC for each random effect in both Experiment #1 and Experiment #2 (Supplemental Data). The fixed effects that we tested included Lab Type (Traditional or CURE), student subgroup (Pell only, BIPOC only, Both, or Neither), sex (male or female), and concurrent GPA. To reduce negative skew in the concurrent GPA, we reflected each value by subtracting from five, and then took the natural log of the reflected value. After determining the AIC values for all possible models, the best fitting model (with the lowest AIC value) was selected (Supplemental Data). The best-fitted model for both Experiments #1 and #2 is as follows:
Percent of Lecture Exam Points ∼ Lab Type * Subgroup + Sex + ln(5-Concurrent GPA)
We also ran this model using a different outcome variable, the natural log of the odds of exam score, to avoid having the exam score dependent variable trapped between 0 and 1 (Supplemental Data).
RESULTS
Volunteer Bias Impacts EMU Students’ Perception of Discovery/Relevance in the Tiny Earth Lab
Our Tiny Earth lab class at EMU was intentionally designed to meet the three key features that define a high-quality CURE: Collaboration, Iteration, and Discovery/Relevance, and we used the LCAS to evaluate whether students perceive these three key aspects (Corwin et al., 2015). We observe from our linear regression analyses of LCAS data that students who volunteered to enroll in the Tiny Earth CURE at EMU perceive all three key aspects of the CURE (Collaboration, Iteration, and Discovery/Relevance) significantly more strongly than students in the traditional lab (p < 0.01 for all comparisons; Figure 1; Table 3). However, in Experiment #2 (nonvolunteer CURE), students perceive only two aspects – Collaboration and Iteration – significantly more strongly than students in the traditional lab (p < 0.01 for all comparisons) (Figure 1; Table 3). Students in the CURE who did not volunteer to enroll (Experiment #2) did not report a significantly stronger sense of Discovery/Relevance compared with students in the traditional lab. There are multiple individual items on the Discovery/Relevance subscale of the LCAS in which students in the non-volunteer CURE responded with lower scores (Supplemental Materials). Yet, the course structure and content of both the Tiny Earth CURE and the traditional lab were unchanged between Experiment #1 and Experiment #2. Our regression models showed no statistically significant subgroup effects or interactions in either experiment.
Figure 1. All students in the Tiny Earth CURE perceive greater Collaboration and Iteration, but perception of Discovery/Relevance is impacted by volunteer bias. For each LCAS survey subscale (Collaboration, Iteration, Discovery/Relevance) we determined the average response per student. Items on the Collaboration subscale can be rated from 1 = Never to 4 = Weekly. Items on the Iteration sub-scale and Discovery/Relevance sub-scale can be rated from 1 = Strongly Disagree to 6 = Strongly Agree. There is a significant difference (p < 0.01 by linear regression analysis) for comparisons of both Collaboration and Iteration between students in the traditional lab and in the CURE in both Experiment #1 (Volunteer CURE) and Experiment #2 (Nonvolunteer CURE). Only students in Experiment #1 (Volunteer CURE) perceive greater Discovery/Relevance in the CURE compared with the traditional lab (p < 0.01 by linear regression analysis). Significant differences between the CURE and the traditional lab (p < 0.01) are indicated by an asterisk.
| Experiment #1 (Volunteer) | Experiment #2 (Non–volunteer) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Predictors | Collaboration Estimates | p | Iteration Estimates | p | Discovery Relevance Estimates | p | Collaboration Estimates | p | Iteration Estimates | p | Discovery Relevance Estimates | p |
| (Intercept) | 3.09 | <0.001 | 3.62 | <0.001 | 3.70 | <0.001 | 3.26 | <0.001 | 4.26 | <0.001 | 4.19 | <0.001 |
| CURE (ref = Trad) | 0.57 | 0.003 | 1.41 | <0.001 | 1.23 | <0.001 | 0.45 | 0.005 | 0.86 | 0.001 | 0.37 | 0.177 |
| Pell only (ref = neither Pell elig nor BIPOC) | –0.48 | 0.034 | –0.50 | 0.127 | –0.21 | 0.541 | –0.05 | 0.787 | –0.30 | 0.286 | –0.13 | 0.676 |
| BIPOC only | 0.08 | 0.785 | 0.57 | 0.175 | 0.73 | 0.078 | 0.09 | 0.718 | 0.34 | 0.407 | 0.12 | 0.788 |
| Both Pell elig and BIPOC | 0.13 | 0.575 | 0.04 | 0.908 | 0.51 | 0.152 | –0.38 | 0.107 | –0.21 | 0.580 | 0.10 | 0.814 |
| CURE * Pell only | 0.34 | 0.309 | 0.21 | 0.667 | 0.33 | 0.503 | –0.09 | 0.758 | –0.13 | 0.775 | 0.32 | 0.540 |
| CURE * BIPOC only | 0.16 | 0.733 | –0.87 | 0.208 | –0.59 | 0.402 | –0.48 | 0.230 | –0.42 | 0.511 | –0.65 | 0.360 |
| CURE * Both Pell elig and BIPOC | –0.03 | 0.952 | 0.20 | 0.753 | –0.66 | 0.329 | 0.20 | 0.613 | 0.48 | 0.439 | 0.26 | 0.706 |
| Observations | 142 | 140 | 140 | 152 | 151 | 152 | ||||||
| R2/R2 adjusted | 0.174/0.131 | 0.288/0.251 | 0.221/0.179 | 0.104/0.060 | 0.150/0.108 | 0.041/–0.006 | ||||||
EMU Students in the CURE have Higher Emotional Ownership of the Research Project Than Those in the Traditional Lab Regardless of Whether They Volunteered for the CURE
Our regression analysis of POS data indicates that student enjoyment and surprise (aspects of emotional project ownership) are significantly greater in the CURE than the traditional lab (p < 0.01 for all comparisons), irrespective of whether students volunteered for the CURE (Figure 2; Table 4). We observed that Pell-eligible students report significantly greater enjoyment and surprise in the volunteer CURE version of the lab compared with non-Pell-eligible students (Experiment #1; Table 4). No differential effect for Pell-eligible students was observed in Experiment #2 (nonvolunteer CURE) and no other significant interactions between subgroups and the CURE lab were observed in either experiment.
Figure 2. Enjoyment and Surprise are greater in the CURE than in the traditional lab, regardless of whether students volunteered for the CURE. For the POS survey subscales of enjoyment and surprise (which are aspects of emotional ownership of the research project), we determined the average response per student. Items on these POS subscales can be rated from 1 = Very slightly to 5 = Very strongly. Average survey responses indicate that students in both (A) Experiment #1 (volunteer CURE) and (B) Experiment #2 (nonvolunteer CURE) report a greater sense of enjoyment and surprise of the research project. Our linear regression analyses indicate there is a significant difference (p < 0.01) for all comparisons between students in the traditional lab and in the CURE. Significant differences between the CURE and the traditional lab (p < 0.01) are indicated by an asterisk.
| Experiment #1 (Volunteer) | Experiment #2 (Non-volunteer) | |||||||
|---|---|---|---|---|---|---|---|---|
| Predictors | Enjoyment estimates | p | Surprise estimates | p | Enjoyment estimates | p | Surprise estimates | p |
| Intercept | 2.31 | <0.001 | 2.37 | <0.001 | 2.75 | <0.001 | 2.55 | <0.001 |
| CURE (ref = Trad) | 1.15 | <0.001 | 1.08 | <0.001 | 1.26 | <0.001 | 1.27 | <0.001 |
| Pell only (ref = neither Pell elig nor BIPOC) | –0.62 | 0.032 | –0.76 | 0.009 | –0.11 | 0.692 | –0.08 | 0.778 |
| BIPOC only | 0.62 | 0.081 | 0.63 | 0.078 | 0.14 | 0.719 | 0.26 | 0.495 |
| Both Pell elig and BIPOC | 0.22 | 0.463 | 0.07 | 0.809 | –0.35 | 0.333 | –0.43 | 0.221 |
| CURE * Pell only | 0.85 | 0.048 | 0.96 | 0.026 | 0.10 | 0.829 | 0.03 | 0.947 |
| CURE * BIPOC only | 0.25 | 0.679 | 0.32 | 0.593 | –0.37 | 0.550 | –0.70 | 0.251 |
| CURE * Both Pell elig and BIPOC | –0.75 | 0.195 | –0.59 | 0.308 | 0.34 | 0.572 | 0.61 | 0.300 |
| Observations | 141 | 141 | 151 | 151 | ||||
| R2/R2 adjusted | 0.340/0.305 | 0.344/0.310 | 0.265/0.229 | 0.281/0.246 | ||||
Replacing the Traditional Lab with a CURE at EMU Negatively Impacts Lecture Course Performance of Some Student Subgroups
Lecture exam scores for students in the traditional and CURE versions of the lab in each experiment are summarized in Figure 3. We observed in Experiment #1 (volunteer CURE) that individuals in the CURE who are both Pell eligible and BIPOC earn significantly lower scores on lecture exams compared with students who are neither Pell eligible nor BIPOC (p < 0.05), while controlling for concurrent GPA and student sex (Table 5). This negative interaction is even greater in magnitude and more significant in Experiment #2 (nonvolunteer CURE); while controlling for concurrent GPA and student sex, students who are both Pell eligible and BIPOC earn 12 percentage points fewer on lecture exams if they are in the nonvolunteer CURE, compared with the nonvolunteer traditional lab (p = 0.001). This negative interaction with the CURE lab remains significant in both Experiments #1 and #2 when we repeat the linear regression using the natural log of the odds of exam score to avoid having the exam score dependent variable trapped between 0 and 1 (Supplemental Materials). It is also notable that the CURE seems to disproportionately negatively impact lecture scores for students with lower concurrent GPAs in both experiments (p < 0.001) and female students in Experiment #2 (nonvolunteer CURE; p = 0.009; Table 5).
Figure 3. Replacement of the traditional lab with a CURE resulted in lower scores on exams in the associated lecture class among students who are both BIPOC and Pell eligible. (A) Raw data, average total lecture exam score by lab type. Embedded in each violin plot is a box plot, with the horizontal bar representing the median. (B) and (C) Regression model-predicted lecture exam scores by student subgroup, controlling for sex, and concurrent GPA; bars indicate 95% confidence interval.
| Experiment #1 (Volunteer) | Experiment #2 (Nonvolunteer) | |||||
|---|---|---|---|---|---|---|
| Percent of lecture exam points | Percent of lecture exam points | |||||
| Predictors | Estimates | CI | p | Estimates | CI | p |
| Intercept | 84.17 | 81.68 – 86.66 | <0.001 | 84.77 | 81.29 – 88.25 | <0.001 |
| CURE (ref = Trad) | 3.38 | –0.67 – 7.43 | 0.102 | 0.48 | –3.29 – 4.24 | 0.804 |
| Pell only (ref = neither Pell elig nor BIPOC) | 0.61 | –2.52 – 3.74 | 0.700 | –1.17 | –5.21 – 2.86 | 0.568 |
| BIPOC only | 0.58 | –3.62 – 4.78 | 0.786 | –0.40 | –5.08 – 4.28 | 0.867 |
| Both Pell elig and BIPOC | –3.34 | –6.92 – 0.24 | 0.067 | –2.25 | –6.81 – 2.31 | 0.332 |
| Female sex (ref = male) | –1.77 | –4.09 – 0.55 | 0.133 | –3.69 | –6.46 – –0.92 | 0.009 |
| ln(5–concurrent GPA) | –20.02 | –22.76 – –17.28 | <0.001 | –21.33 | –24.51 – –18.14 | <0.001 |
| CURE * Pell only | –1.90 | –8.76 – 4.96 | 0.586 | –4.47 | –11.16 – 2.21 | 0.189 |
| CURE * BIPOC only | –4.58 | –13.60 – 4.44 | 0.318 | –4.57 | –12.10 – 2.95 | 0.233 |
| CURE * Both Pell elig and BIPOC | –7.51 | –14.37 – –0.65 | 0.032 | –12.41 | –19.77 – –5.06 | 0.001 |
| Observations | 432 | 406 | ||||
| R2/R2 adjusted | 0.400/0.387 | 0.401/0.388 | ||||
DISCUSSION
CUREs have been widely promoted as a mechanism to democratize the benefits of apprentice-style undergraduate research, particularly for students in under-represented groups in the sciences. This study is the first to analyze a CURE at a highly diverse regional institution and evaluate its effect in contexts both with and without volunteer bias with a large number of students.
The Tiny Earth Introductory Biology Lab at EMU is a High-Quality CURE
The Tiny Earth lab at EMU was intentionally designed to meet the key features of Collaboration, Iteration, and Discovery/Relevance in CUREs, and our analysis of the LCAS data indicates that students who volunteer for the Tiny Earth lab at EMU perceive significantly greater Collaboration, Iteration, and Discovery/Relevance compared with students in the traditional lab (Figure 1; Table 3). In addition, high Discovery/Relevance is materially supported by our recent publication of a new antibiotic discovered by students in the Tiny Earth CURE at EMU (Mohamed et al., 2021). Based on the structure of implementation of the Tiny Earth lab at EMU and the LCAS survey data, the Tiny Earth lab class as it is taught at EMU fully meets the definition of a high-quality CURE. Thus, it is appropriate to further study the impact of this CURE at EMU. Our findings are relevant to the 700+ instructors teaching the Tiny Earth CURE worldwide (https://tinyearth.wisc.edu/) and are also broadly applicable to those interested in pedagogical impacts of CUREs, particularly at institutions with high student diversity.
Volunteer Bias Affects Students’ Perception of Discovery/Relevance of the Tiny Earth CURE at EMU
There is a distinct difference in students’ perception of Discovery/Relevance from the LCAS between Experiment #1 (volunteer CURE) and Experiment #2 (nonvolunteer CURE), in which only students who chose to enroll in the CURE reported a significantly greater sense of Discovery/Relevance compared with students in the traditional lab (Figure 1; Table 3). Yet, the content and structure of both the CURE and the traditional labs was unchanged between experiments #1 and #2. Thus, volunteer bias clearly affects students’ perception of Discovery/Relevance in their laboratory class. Drawing from SDT, we suspect that this enhanced perception of Discovery/Relevance may result from greater intrinsic motivation among students who volunteered for the CURE. Students who chose to enroll in the CURE may have done so because they brought a prior interest in discovery through authentic research or because the description of the class resonated with a prior personal relevance to research on finding new antibiotics. Further research is needed to evaluate how volunteer bias influences students’ perceptions of Discovery/Relevance in CUREs and whether intrinsic motivation is important for sensing a high level of Discovery/Relevance in the course.
Students’ Sense of Research Project Ownership is Higher in the CURE than the Traditional Lab, Irrespective of Volunteer Bias
Several studies have reported that students in CUREs have higher cognitive and emotional project ownership compared with students in traditional labs (Hanauer et al., 2017; Cooper et al., 2019, 2020; Lo and Le, 2021; Killpack and Popolizio, 2023). In the Tiny Earth CURE at EMU, both enjoyment and surprise (subfactors of emotional project ownership) are significantly greater among CURE students than traditional lab students, whether or not volunteer bias is present (Figure 2; Table 4). In the volunteer CURE we also observed an additional positive effect in emotional ownership among students who are Pell eligible that was not observed in the nonvolunteer CURE. We are aware of one other study that reports a significant positive effect specifically for Pell-eligible students: Martin et al. (2021) report that their CURE curricula may bolster the confidence of Pell-eligible students who start the semester with lower confidence. We did not observe any differential effect of the CURE on emotional ownership among students who are BIPOC (Table 4), but our small survey sample size may have prevented us from detecting such an effect.
It is proposed that the academic emotion of enjoyment depends on both positive appraisal of competence and positive appraisal of the intrinsic value of the action, and that cooperation with other students makes working on tasks more enjoyable (Pekrun and Linnenbrink-Garcia, 2012). Based on SDT, we had anticipated greater intrinsic motivation among students who volunteered for the CURE would result in greater enjoyment of the CURE among volunteer students. It is possible that the perception of greater collaboration in the CURE than the traditional lab (Figure 1) may contribute to the lack of volunteer bias in students’ sense of enjoyment. We also suspect the CURE lab at EMU presents tasks to students in such a way that whether or not they volunteered for the course, they feel that they are able to accomplish the research goals of the lab and that the topic of the lab itself has value, thus supporting a high level of enjoyment for all students. Further research is needed to evaluate the relevance of intrinsic motivation for students’ perception of academic enjoyment.
Greater research project ownership, both cognitive and emotional, have been proposed to mediate enhanced persistence in biology (Hanauer and Dolan, 2014; Corwin et al., 2018). Our finding that all students, regardless of whether or not they volunteered for the CURE, experience increased emotional project ownership, suggests that the Tiny Earth course at EMU has the potential to increase student retention in the sciences. Further research is needed to evaluate the long-term effects of the CURE at EMU on student persistence.
Lecture Course Student Performance is Negatively Impacted by Replacing the Traditional Lab with a CURE
Because CUREs do not reinforce the weekly topics of a lecture class, it is possible that replacing a traditional lab with a CURE could negatively affect the success of under-prepared students in the associated lecture course. Like many regional institutions, EMU has a high percentage of academically under-prepared students and when the Tiny Earth CURE was first implemented in the Introductory Biology lab at EMU, departmental faculty were concerned about how our change of lab could impact student success in the associated lecture course because the lab would no longer explicitly support student understanding of the lecture content. In this study, we find that students who are both Pell eligible and BIPOC have significantly lower lecture exams scores if they were in the CURE lab (whether volunteer or nonvolunteer; Figure 3; Table 5). This contrasts with data from other institutions, where CUREs were not observed to have negative effects on student success in the associated lecture course (Olimpo et al., 2016; DeFeo et al., 2020; Ing et al., 2021; Freeman et al., 2023), however, none of these previously published studies evaluated the effects on students with intersectionality.
At EMU, ∼30% of students are both Pell eligible and BIPOC, thus we are concerned about how our change of laboratory instruction has negatively affected students with students with more than one underserved, underrepresented, or minoritized identity. We hypothesize that EMU students who are both Pell eligible and BIPOC are more likely to work longer hours, attend college part-time, and be present on campus only when they are in class (Tinto, 2012). Thus, it may be of benefit to our students to combine aspects of the Traditional and CURE labs; for example, it is possible that the addition of a short supplemental instruction session at the start of the lab time, similar to that incorporated by Olimpo et al. (2016), may be an effective support in the associated lecture course for EMU students. Future research is needed to evaluate whether the weekly 1-h recitation that was included with our Traditional lab, or the Traditional lab activities, or both, are important for supporting our students’ success in our associated lecture course, and whether combining aspects from both the Traditional lab and the CURE would benefit EMU students.
Limitations
Because our analyses are for a single institution, more research in nonvolunteer contexts and among diverse student bodies is needed to make broad claims about the effects of replacing traditional labs with CUREs. Additionally, because postcourse survey response rates were low, our survey data may be confounded by volunteer bias; students who had particularly strong positive or strong negative feelings about their lab class experiences may have been more likely to respond. However, the survey data do not impact our analyses of lecture course success. We also acknowledge that we cannot avoid the potential confounding of postcovid effects on students in our analyses of the CURE from Experiment #2 (nonvolunteer CURE), however, the effect of the CURE on exam scores for students who are both BIPOC and Pell eligible was observed in both Experiment #1 and Experiment #2.
CONCLUSION
In our study of a CURE at a regional comprehensive university, we find that volunteer bias can have significant impacts on student perceptions of the course, and that our replacement of the traditional lab with the CURE had unintentional negative effects on the lecture course success of students with intersectionality. Further research is needed to better understand the impacts of CUREs in differing university contexts, using experimental designs that are not confounded by volunteer bias.
ACKNOWLEDGMENTS
Thanks to Don Lund in the EMU IRIM office for assistance with institutional data. Thanks to Sara Brownell, Sarah Eddy, Scott Freeman, Mary Pat Wenderoth, and Bob Winning for assistance with research planning and analysis and constructive comments on drafts of the manuscript. Thank you to the editor, Lisa Corwin, and two anonymous reviewers whose thoughtful comments and suggestions greatly improved this article. The research was conducted with approval by the EMU Human Subjects Review Committee, protocols UHSRC-FY19-20-266 and UHSRC-F20-21-123. This research was supported by funding to A.M.C. from the National Science Foundation, award number 2024020.
