Passing Rates, Course Grades, and Research Instruction: An Eight-Year Study of Community College Students

Stephanie C. Gillespie*

This study investigates the impact of targeted one-shot research instruction (RI) on course grades and passing rates of 50,901 community college students across various research-based courses over eight years. It examines RI’s impact on different student status types, whether RI is associated with higher outcomes of both college-ready (CR) and not-college-ready (NCR) students, if RI is still correlated with higher outcomes when chronically absent students are removed from all groups, and whether factoring in RI attendance significantly clarifies results. In each case, the findings indicate that RI is correlated with higher grades and passing rates. Taking attendance during RI is a crucial factor in accurately measuring results.

Introduction

Instruction librarians face an impasse: the prevailing model of offering information literacy (IL) instruction once per semester to a class (i.e., one-shot sessions), although widely practiced, has not consistently demonstrated a positive impact on student success outcomes. Nevertheless, structural constraints, such as course instructors having limited time to devote to IL, impede abandoning the one-shot model. However, ample anecdotal evidence suggests librarians make a meaningful difference, as students frequently express gratitude, and instructors routinely request their return. This creates a tension between the perceived effectiveness of IL instruction and the difficulty in capturing empirical evidence of its impact on student success. This study seeks to (1) examine why some studies succeeded in demonstrating a positive correlation between IL instruction and passing rates and/or grades while others failed, (2) to contribute a large dataset to the literature, and (3) to document our methodology and evaluation strategies for future exploration and improvement.

Pellissippi State Community College is the largest community college in Tennessee, with an average full-time equivalent of 6,000 students in the fall. Pellissippi State offers 79 university parallel programs and career pathways. Almost half of the students (47%) are enrolled in university parallel programs, followed by undeclared (29%) and career pathways (22%). Undeclared status is often used by students whose major is not offered at the college but plan to transfer their credits. The student body is largely 18 to 24 years old, White, and first-generation, with significant populations of academically underprepared and financially insecure students. Fall-to-spring persistence rates average 77%, and fall-to-fall retention rates average 52%. Pellissippi State is teaching-focused and accepts all students, while providing corequisites for underprepared students, which comprises around 55% of the student body.

In 2017, the Pellissippi Libraries partnered with the college’s department of Institutional Effectiveness, Assessment, and Planning (IEAP) to evaluate the impact of research instruction (RI) on grades and passing rates. Our study focuses specifically on RI rather than all types of instruction because RI lessons address a research assignment that accounts for 10%–15% of the course grade, whereas other IL is not necessarily tied to any graded assignment. Thus, RI could reasonably affect the course grade and passing rates. RI lessons are scheduled at students’ point-of-need (i.e., when the project is assigned or just after), and both live sessions and tutorials use the same hands-on lesson plans and teaching materials (e.g., ENGL 1010 Argument Essay, PSYC 2100 Lifespan Project). While lessons are tailored to the subject area and project, the content still addresses the same IL concepts.

We sought to determine whether RI lessons help students achieve their academic goals, to evaluate our instructional materials and methods, and to perhaps demonstrate our value to our college community. We collaborated with IEAP statisticians from the outset to design the study, analyze results, and obtain IRB approval. They helped us define valid study questions to effectively use institutional data, establish appropriate parameters, align our study to institutional goals, complete the IRB process, learn to use Statistical Package for the Social Sciences (SPSS), and analyze the data properly.

The analyses include an intervention group, two control groups, sample sizes, standard deviations, pairwise comparisons for categorical data, z-tests (three or more groups) or t-tests (two groups) for numerical data, and student demographics as context. All reported IL and RI numbers reflect students who participated either through attending a live session or by completing a tutorial. Additional questions are addressed by examining the student outcomes from various angles, such as by semester, student status, college readiness, and without chronically absent students. Finally, the data is reanalyzed without factoring in RI attendance to demonstrate whether its inclusion is significant to the findings.

IL Program Overview

During the study, our librarian team consisted of eight tenure-track librarians, the library dean, and two part-time librarians. Due to the nature of their positions, the eight full-time librarians (including the author) teach the majority of classes, including live sessions across five physical campuses, various high schools, and online. Classes are assigned based on librarian availability and campus rather than by subject liaison.

As the instructor coordinator, I guide the library instruction program. I send personal emails twice a semester to instructors of research-based classes, indicating how we can help their students complete specific course projects, and offer relevant library resources and tutorials. I schedule instruction, create lesson plans and teaching materials, develop tutorials, train new librarian instructors, coordinate co-teaching and peer-observation for ongoing professional development, and assess our instruction efforts. I also advise instructors on the optimal timing of RI sessions to reflect students’ point-of-need. Upon instructors’ request, I co-create research assignments, gather tutorial submissions, and answer student questions at the end of tutorials.

While some course instructors routinely invite us into their classes or require their students to complete tutorials, the librarians do not consistently provide IL to every section in a course. No course is required to have IL instruction, so students typically only participate at the behest of their instructor, even though tutorials are accessible on the libraries’ website. Table 1 shows the prevalence of RI and IL instruction relative to the total fall headcount. All instruction data has been deduplicated. Because students sometimes participate in both RI and other IL, total IL numbers are lower than RI and other IL combined.

On average, 27% of the total student population participates in RI and other IL instruction (see Figure 1). One in five students typically participate in RI, excluding fall 2020, when it dropped to 10%. This anomaly is likely due to COVID-19, when instructors struggled to keep students on track in fully online classes and might not have had enough room in their course content for RI instruction.

Other IL instruction trends upward during the time studied, likely due to adding tutorials that teach IL skills, such as crediting sources, library orientation, MLA and APA citation, and evaluating sources. Other IL participation averages 12% of the total headcount. These lessons are omitted from RI, because it is not known whether students apply the concepts to a particular assignment as a part of the course grade.

Conversely, RI lessons are tailored to a specific research assignment and teach students only relevant IL skills. In the 50-minute live RI session, a librarian helps students understand how to choose a topic, use different resource types, evaluate credibility, identify keywords, navigate OneSearch (discovery system) or relevant databases, easily read scholarly articles, use citation tools, check those citations with our citation guide, and get help with researching and writing. Librarians lead two class activities for students to practice identifying appropriate resource types for different information needs and evaluating credibility. Otherwise, students can take an interactive RI tutorial that derives from our lesson plans and teaching materials, providing a comparable lesson and offering practice at every step.

The students often have limited library knowledge or experience, so the librarians teach the basics and provide rules of thumb that can be immediately implemented in their current assignment. The librarians’ IL philosophy focuses on teaching a scaffold that students can develop and refine as they continue along their academic path. While students tend to have basic googling skills, they often feel overwhelmed conducting college-level research. They need a game plan for the project at hand, to feel like they grasp basic college-level research, and to know how to seek help if they encounter problems.

Literature Review

The connection between IL and course grades and passing rates seems logical: IL aims to improve student research skills, which should boost student performance on research assignments, and—as these assignments are large portions of student grades—better performance should raise course grades and passing rates. While an early systematic review found IL instruction was associated with only marginally better outcomes compared to no instruction, the authors acknowledged that these studies often suffered from flaws in their methodology, reporting, and/or analysis (Koufogiannakis & Wiebe, 2006). In the 2010 Value of Academic Libraries report, ACRL described earlier studies as “sporadic, disconnected, and focused on limited case studies” and urged librarians to join the broader movement in academia to systematically demonstrate their contributions to student success through outcomes-based data that align with institutional goals (p. 14). Since then, studies focusing on course grades and passing rates have shown inconsistent results.

A persistent issue plaguing studies is the failure to take attendance during IL instruction, which would allow researchers to assess student-level data. Studies that assume full attendance have shown mixed results, either finding a mild benefit (Asher, 2016; ACRL, 2016) or no benefit (Coulter et al., 2007; Lundstrom et al., 2016; Squibb & Mikkelsen, 2016; Krieb, 2018). However, studies that take attendance find more positive results, likely due to removing absent students, who lower the average outcome. When IL was embedded into an English composition course at a community college, Burgoyne and Chuppa-Cornell (2015) found a positive correlation to retention and course completion among 1,273 students. However, the study lacked a control group. Despite their encouraging results, semester-long courses may not be scalable or feasible at many institutions. Chodock (2018) also tracked attendance at a community college and found a positive correlation between IL instruction and higher course grades when the lesson focused on researching rather than general information. Nevertheless, the sample sizes were not provided for this outcome, and the results were weak, leading him to speculate that the slightly higher grades resulted from mere class attendance. Shao and Purpur (2016) found a positive correlation between IL and grades among 398 students, but without a control group, the connection remained unclear. Conversely, Rowe et al. (2021) included a control group in a sample size of 10,000 students and found that students who participated in IL instruction were almost three times as likely to pass than those who did not. However, this study’s control group combined students who were absent with those not offered instruction, merging two distinct populations, which likely impacted the results. Even with some small issues, these studies demonstrate the potential of IL instruction to increase student success and highlight the importance of tracking attendance during the IL intervention.

Another consistent issue in study design is the failure to factor in demographic information when analyzing results. While a couple of studies (Burgoyne & Chuppa-Cornell, 2015; Rowe et al., 2021) offer demographic information, they neither analyze it to establish a key factor for success nor account for it when analyzing student outcomes.

Due to issues in study design and reporting of many IL studies, even those few that showed positive results do little to persuade others of the effectiveness of IL instruction. Robertshaw and Asher (2019) conclude that, “because of the myriad of factors that affect grade, retention, and attainment outcomes, library use or instruction is unlikely to be able to statistically demonstrate a meaningful real-world impact on these outcomes” (pp. 94–95). After more than a decade of concentrated effort to measure IL’s impact on students with little to show for it, Robertshaw and Asher’s skepticism is reasonable. However, while other factors undeniably influence student outcomes, this study seeks to demonstrate how these factors can be accounted for or isolated to provide meaningful results.

One-shot sessions, in particular, have garnered most of the blame for the failure to demonstrate positive results. Much of the literature disavows one-shot sessions as unsustainable and ineffective (Carlozzi, 2018; Pagowsky, 2021; Bowles-Terry & Donovan, 2016; Coulter et al., 2007; Gandhi, 2004; Vossler et al., 2023). Even when demonstrating positive results of one-shot sessions, Rowe et al. (2021) do not endorse them. While their concerns about one-shots are valid and librarians should continue to explore new instruction methods and models, these sessions can be useful to teach IL at students’ point-of-need and can have a measurable influence on student outcomes, as shown by previous studies that factored in attendance during the IL intervention.

Methodology

Study Design

This study focuses on RI rather than all types of library instruction because it directly addresses a research assignment usually worth 10%–15% of the overall course grade. The study excluded instruction that targets academic skills (e.g., citation or crediting sources) because its direct impact on course success is uncertain. Also, focusing on one specific intervention increased the validity of the outcomes. Lesson types were coded in the instruction schedule to easily identify relevant events (e.g., RI for research instruction, CW for citation workshop, RW for research workdays, CA for career research).

The primary objectives of this study are to determine how likely students are to (1) pass their courses and (2) earn a higher course grade. To analyze outcomes, the students (N = 50,901) are divided into three statistically valid groups:

• RI: 14,911 (29%) students participate in one or more live session(s) or tutorial(s)
• Abs: 6,316 (12%) students do not participate in any RI opportunities offered
• NoRI: 29,674 (58%) students are in other sections of these courses not offered RI

These groups apply to a single course per semester, so students could belong to a different group for another course. If a student belongs to different groups in the same course (e.g., absent from the live session but completes a tutorial), they are categorized as an RI participant. If a student was neither absent nor an RI participant for a course, they are considered no research instruction (NoRI). As a point of interest, 51% of absent students failed to take a tutorial rather than missed a class, meaning they have comparable attendance to RI students, whereas 49% missed an in-person session.

The study used various methods to accurately compare groups, including pairwise comparison, z-tests, and t-tests. For categorical variables (e.g., college readiness and passing rates), averages were compared pairwise with the Bonferroni correction to reduce the likelihood of false correlation. For numerical variables (course grade), the study conducted one-way ANOVA tests and performed z-tests (three or more groups) or t-tests (two groups), which helped better compare averages of different-sized groups and to indicate how significantly different they are from each other. Groups with similar results (i.e., p-value is more than 0.000) are indicated by letters in the tables.

Finally, the study includes the standard deviation score for grades. Averages of large groups can hide significant variation in student outcomes. Thus, the standard deviation offers a transparent look into how closely individual students align with their group average. Because grades are often skewed to each end, the standard deviations are higher than if the grades had a normal distribution.

Data Collection

This large-scale study (N = 50,901) assesses the effect of RI on student grades and passing rates over several years. The study collected eight years of attendance and tutorial submissions from students in many different research-based courses. Table 2 shows the course composition of the dataset, which is largely students taking English composition courses (43%). Other disciplines include English literature (13%), communication (12%), history (6%), college success (6%), sociology (5%), biology (5%), psychology (5%), philosophy (2%), engineering (1%), and business (1%) among others.

To ensure the study’s quality and validity, attendance had to be recorded. As the instruction coordinator, I was permitted to access limited student information (e.g., preferred name, college identification number, email) tied to research-based course enrollment (e.g., subject, course, course reference number, section, term code, instructor, total enrollment) via Evisions Argos Enterprise Reporting system. Each week, I sent attendance sheets to librarian instructors and gathered attendance. At the end of the semester, I collected RI tutorial submissions and ensured the student-submitted data matched enrollment records in Argos. Both attendance and tutorial submissions were compiled into one spreadsheet per semester, with additional columns to mark the tutorial and semester. The study compared fall-to-fall data, assuming consistency in student makeup. Comparing fall data is a common IEAP practice. However, this may not be necessary if the key factor is used to help interpret results.

Dataset Creation

IEAP removed student names from the library dataset and added relevant demographic information that may affect student outcomes, including student status (e.g., first-time freshman, freshman, other freshman, sophomore, undergraduate special), ever Pell-eligible, college-ready, and first-generation status.

Pellissippi State designates students as “not college ready” if they score below certain thresholds on the ACT, SAT, ACCUPLACER, SAILS, and/or high school GPA. Not-college-ready students are required to take a corequisite course in the discipline (e.g., English composition or math) that they scored below the threshold. Students are considered “first generation” if they self-report that neither of their parents graduated from a four-year institution.

The study did not use ACT information, because not all students took it, and college readiness accounted for this factor. IEAP also added letter grades and credits earned to the spreadsheet. Then, they created a separate dataset of all students enrolled in these research courses with all relevant demographic and outcome information for each semester to identify NoRI students enrolled in the course but not offered RI. These NoRI students also completed similar research assignments.

The library dataset was combined with the enrollment dataset, creating one file for each semester. The files were sorted by student college identification number, and rows were combined if the student appeared more than once per course in the semester (ascending levels: NoRI, Abs, and RI). Thus, the dataset had one row per student per course per semester. Additional columns totaled the number of tutorials, live sessions, and total RI sessions.

Other columns coded the number of RI sessions (0, 1, 21), RI Type (RI, Abs, and NoRI), student status (first-time freshman, freshman, and other freshman were combined into group 1, sophomore in group 2, and undergraduate special in group 3), and semester/year. All freshmen are degree-seeking but lack enough credit hours to qualify as sophomores. Sophomores have earned at least 27 credit hours. Undergraduate special students are nondegree seeking, dual enrollment, or earning certification.

Initially, IEAP and I analyzed results with two RI groups (1 RI and 21 RI); however, t-tests and pairwise comparisons found no significant difference between them for college readiness rates, passing rates, and course grade. Therefore, in these analyses, these groups are combined into the RI group.

Letter grades were transformed into a numerical grade based on the grade distributions found in the course syllabi, which may have slight differences among academic departments (e.g., A = 90-100 or A = 93-100). Because each is a range, we averaged them (e.g., A = 95 or A = 96.5). To not skew passing rates, 2,787 students who had withdrawn (2,776 with W grade), audited (six with AU grade), or did not complete (five with I grade) the class were excluded. Most who withdrew are a part of the NoRI and Absent groups, likely due to RI instruction being held later in the semester at the students’ point of need. That is, students have likely already withdrawn from the class before RI instruction occurs. Had withdrawn students been included, passing rates for NoRI and Absent groups would be lower, because they earned no credit, having failed to complete the course.

Finally, for each dataset, the study compiled all fall semesters to analyze them simultaneously.

Results

Key Factor Analysis

To properly interpret results, it is necessary to first identify which key factor(s) is most strongly associated with student success and then to use this as a predictor to estimate expected outcomes. Beyond describing the population studied, a key factor analysis of demographics can identify what makes students more likely to be successful. Analyzing this factor among intervention and nonintervention groups provides a valuable benchmark for accurately assessing their results, particularly when groups are not equally likely to succeed. Considering the key factor when reporting outcomes can also help normalize results among studies of different institution types and student populations.

Identifying and considering this factor is particularly important when comparing groups that differ in their likelihood of success. Rather than comparing groups solely to each other—and attributing success just to group membership—the study first compared observed outcomes to their expected outcomes and then assessed the differences between observed and expected results. By comparing these differences across groups, the study could more accurately interpret whether disparities are due to underlying factors or reflect additional, unaccounted-for influences.

Under the guidance of IEAP statisticians, I used SPSS to analyze the data. Based on Schilling and Applegate (2012), students’ demographics were examined as a constellation to identify the key factor for both passing rates and average course grade. Although Pell-eligibility was initially considered, it proved to be insignificant and was therefore excluded from the final key factor analysis to simplify groups. Because only college readiness and first-generation status are influential, students are arranged into one of four groups: Group 1 is not college ready (NCR) and first generation (FG), Group 2 is NCR and not first generation (NFG), Group 3 is college ready (CR) and FG, and Group 4 is CR and NFG.

As depicted in Table 3, the two largest groups are demographically opposite each other. Group 4 (16,064 students) is college ready (CR) and not first-generation (NFG). They have the best outcomes: 88% passing rate, 79% average course grade, and the lowest grade variation (25.1 SD). Group 1 (14,394 students) is not college ready (NCR) and first-generation (FG). They have the worst outcomes: 73% passing rate; 66% average course grade; and the widest variation in grades (31.8 SD). Group 2 (NCR, NFG), with 9,206 students, performs only a little better than Group 1, with a 78% passing rate, 69% average grade, and a little lower grade variation (29.5 SD). Though much better than Group 2, Group 3 (CR, FG) performs slightly worse than Group 4, with an 84% passing rate, 75% average grade, and slightly wider grade variation (28.0 SD). Z-tests and pairwise comparisons confirm that all groups have different outcomes (0.000 sig).

The analysis reveals that, while first-generation students have slightly lower passing rates and course grades, college readiness is the most significant factor affecting these outcomes. CR students consistently have higher passing rates and course grades compared to NCR students. This finding shows that the college readiness rate is a good indicator for expected passing rates and establishes a useful benchmark to properly interpret grades.

Initial Analysis

College Readiness

NoRI students (56% CR) are more college ready than other groups, so they should have the highest passing rates and course grades. Pairwise comparisons confirm that RI participants (50% CR) are as college ready as Abs students (49% CR), so they should perform similarly. Why are RI and Abs students less likely to be college ready? NCR students are required to take corequisite courses, such as college success (COLL 1500) and reading (READ 0900), and many of these classes participate in RI because there is course time allotted to skill-building, such as research instruction. To better compare groups with different college readiness levels, the difference between college readiness and passing rates was calculated.

Passing Rates

Passing rates capture course completion, which is a primary goal for some students rather than high grades. Even though RI and Abs students have similar college readiness rates (as indicated by letters in Table 4) and should have similar outcomes, pairwise comparisons indicate that all groups have distinct passing rates. RI students pass more often than any other group, at 88%. Despite their low college readiness rate, they exceed expectations with a 38% difference from their college readiness rate. NoRI students, who are the most college ready at 56%, pass 82% of the time, a modest 25% difference. Abs students pass 62% of the time, a mere 13% difference.

Though college readiness rates indicate that RI and Abs students should pass at similar rates, RI students are over four times (OR = 4.37) as likely to pass as Abs students. Even when compared to the more college ready NoRI students, they are a little over half again more likely (OR = 1.57) to pass. Therefore, RI is positively correlated with passing rates.

Do instructors of Absent and RI (AbsRI) students grade more leniently than those of NoRI, causing AbsRI students to pass at higher rates? The plurality of the dataset is students taking English composition (43%) or English literature (13%), and these courses are taught by many different English instructors. To promote consistency in assessment, all English instructors adhere to a common set of grading standards and participate in grade-norming sessions (adjuncts annually and full-time faculty semiannually). During these sessions, instructors review an anonymized student paper and engage in collaborative discussion to justify and calibrate their grading decisions to ensure fairness and uniformity. Consequently, both instructors of NoRI and AbsRI grade similarly, and passing rates must be attributed to another factor.

Course Grades

Grades reflect the quality of students’ performance. RI participants earn the highest grades at 79%, which is modestly better than expected, even outperforming the most college ready NoRI students (73%). Accounting for their college readiness, NoRI students perform slightly better than expected. Abs students are similarly college ready to RI students yet earn the lowest grades (56%), which aligns with their expected outcome (see Figure 2). Z-tests indicate each group’s outcomes are significantly different (0.000 sig). Thus, RI is positively correlated with grades: RI students earn the highest grades, even surpassing those of the most college-ready group.

The standard deviation (SD) measures grade variation among individual students in a group. SD scores in our study are high overall, which points to the grade distribution not having a normal bell-curve distribution. RI students earn grades at or near 79% more consistently, indicated by the lowest SD of any group (23.1 SD). NoRI students show wider variation from their 73% average (28.5 SD), but Abs students vary the most (36.9 SD) from their 56% average. Hence, RI is associated with not just a higher group grade but also consistently higher grades for individuals in the group.

Semester Analysis

Admittedly, condensing eight years of data into one average flattens results and may obscure inconsistencies. A semester analysis provides a more detailed view of the data and reveals much the same pattern each semester.

College Readiness

Though college readiness varies slightly each semester, distinct patterns emerge. NoRI students (52%–59% CR) are often the most college ready, followed by RI (42%–57% CR) and Abs (40%–55% CR) students. Pairwise comparisons show groups are similar for NoRI and RI in 2021; RI and Abs in 2017, 2018, 2022, and 2024; NoRI and Abs in 2022; and for all groups in 2020 and 2023. Therefore, NoRI students should generally have the highest outcomes, followed by RI, and then Abs students.

Passing Rates

Each fall, RI participants pass their classes the most of any group (83%–92%), surpassing the most college ready NoRI group (74%–86%). They also consistently outperform what their college readiness forecasts by 32%–41%, the best of any group. NoRI students pass modestly more than their college readiness indicates (19%–29% higher). Abs students (48%–75%) have the lowest passing rates and tend to perform slightly better than expected (5%–22% higher). Aside from 2017, when RI (83%) and NoRI (82%) students passed at similar rates, pairwise comparisons confirm that passing rates are distinct for each semester.

Overall, RI students are three to over seven times as likely as Abs students (OR = 2.99-7.21) to pass. As for NoRI students, except for fall 2017 when NoRI students were almost half again more likely (OR = 1.44) to pass, RI students are up to four times as likely (OR = 1.41-3.97) as NoRI students to pass.

Course Grade

Each semester, RI participants (75%–83%) earn the highest grades, moderately outpacing their college readiness. Z-tests indicate that, in 2017, RI (75%) and NoRI (73%) students earn somewhat similar grades, although the significance is not high at 0.007. Nevertheless, RI students (42% CR) are far less college ready than NoRI students (55% CR), so RI participants still outperform NoRI students. Every semester, NoRI students (67%–77%) are second best, slightly above their college readiness. Abs students (45%–69%) earn the lowest grades, performing either as well as, or a little better, than expected.

Each semester, RI students show the least variation in grades (20.3–25.9 SD). NoRI students’ grades vary more widely than their group average (25.9–33.3 SD). Finally, the grades of Abs students range the most from their group average (32.3–38.4 SD).

Even with more granular semester data, the same pattern emerges: RI students consistently have the highest passing rates and grades, especially given their low college readiness rates, and have the least variability in their grades. Even though NoRI students are much more college ready, their outcomes are still second best, performing modestly better than expected. Also, their grades vary more from the group average than those of RI participants. Abs students have the worst passing rates and grades, aligning with their low-college-readiness rates. Their grades differ the most from their group average. Thus, even in a detailed semester analysis of eight years, RI is confirmed as positively correlated with both passing and grades.

Student Status

The above analyses offer only college readiness for context. However, student status (e.g., freshman, sophomore, and undergraduate special) also influences outcomes. Freshmen (Fresh) are degree-seeking but lack enough credit hours to qualify as sophomores. Sophomores (Soph) have earned at least 27 credit hours. Undergraduate special (US) students are nondegree seeking, dual enrollment, or earning certification. Analyzing the data by student status brings greater clarity to how RI affects each, because variations in demographics and outcomes can obscure results when student status types are in different proportions among groups. For example, sophomores are more likely to be college ready and are more experienced than freshmen; both factors increase their likelihood of success in their classes. Undergraduate special students are often dual-enrollment high school students who are much more college ready than other groups. Do certain student status types benefit from RI more than others?

Proportions

The NoRI group benefits from having the most sophomores (21%) and fewest freshmen (59%), whereas the Abs group is overwhelmingly freshmen (69%). In other words, freshmen are at the highest risk of absenteeism or nonengagement. The RI group is also mostly composed of freshmen (66%) and has the fewest sophomores (15%) of any group. Both RI and NoRI groups have equal proportions of undergraduate special students (19%).

Freshman

Freshman students are the least college ready, pass less often, and earn lower course grades than any other student status. Pairwise comparisons show Abs (41% CR) and RI (39% CR) freshmen are equally college ready, so they should have similar outcomes. NoRI freshmen (47% CR) are much more college ready, highlighting that college-ready freshmen are often not offered RI.

Despite their clear advantage, only 75% of NoRI freshmen pass (27% difference), while 83% of RI freshmen do, greatly exceeding expectations (44% difference). Abs freshmen, who should perform similarly to RI, pass only 52% of the time, a little more than expected (11% difference). RI freshmen are nearly five times as likely to pass as Abs freshmen (OR = 4.70), even though they are equally college ready, and almost twice as likely as NoRI freshmen (OR = 1.69), who are much more college ready.

RI freshmen also earn higher grades (75%) than the NoRI freshmen (67%), despite NoRI students’ college readiness advantage. RI freshmen exceed expectations while NoRI students perform marginally better than expected. Abs freshmen (48%) earn grades as expected. Z-tests indicate each group has different outcomes (0.000 sig). Freshmen have the widest grade variation than other student status types, but RI freshmen (25.3 SD) vary less than either NoRI (31.2 SD) or Abs (37.5 SD) freshmen.

Being the least college ready, freshmen have the most to gain from RI, as evidenced by having the highest differences between college readiness, passing rates, and grades of any student status type. RI freshmen also have the most consistently high grades of any of their peers.

Sophomore

Sophomores have higher college readiness, passing rates, and course grades than freshmen. Pairwise comparisons show sophomore groups (54%–57% CR) are similarly college ready, so they should perform likewise.

Nevertheless, RI sophomores pass the most (94%) and remarkably more than expected (40% difference). NoRI sophomores pass 89% of the time, much more than expected (33% difference). They are followed by Abs sophomores (83%), who pass moderately more than expected (28% difference). Passing rates were different for each group, according to pairwise comparisons. RI sophomores are over three times as likely to pass (OR = 3.31) as Abs sophomores and nearly twice as likely as NoRI sophomores (OR = 1.88).

RI sophomores earn the highest grades (85%), performing much better than expected. Despite being as college ready as RI sophomores, NoRI sophomores average a 79% grade, only moderately better than expected, and Abs sophomores perform slightly above expectations, with an average of 74%. Z-tests show that grades are different for each group (0.000 sig). RI sophomores also earn grades closest to their group average (18.0 SD) compared to NoRI (23.2 SD) or Abs (28.2 SD) sophomores.

While not as pronounced a difference as freshmen, sophomores who participate in RI still have much higher passing rates and grades than nonparticipants. Like freshmen, RI sophomores also have consistently higher grades than their peers.

Undergraduate Special

Undergraduate special students are the most college ready, have the highest passing rates, and earn the highest course grades. Pairwise comparisons indicate they (82%–84% CR) are equally college ready and should perform similarly.

RI undergraduate special students pass more often (97%) than the NoRI (95%) and Abs (87%) groups, although the difference in their expected outcome (14%) is less conspicuous than other student status types due to their high college readiness. NoRI students perform slightly better than expected (10% difference), and Abs students as expected (4% difference). Pairwise comparisons show passing rates are different for each group. RI undergraduate special students are over five times as likely (OR = 5.33) to pass as Abs students and twice as likely (OR = 1.99) as NoRI students.

RI undergraduate special students earn the highest grades, with an average of 87%, followed by NoRI (84%) and Abs (76%) students. Z-tests confirm all outcomes are distinct (0.000 sig). RI and NoRI students perform as expected, whereas Abs students do slightly worse. RI students (13.6 SD) have the least grade variation from their group average, followed by NoRI (18.2 SD) and Abs (27.2 SD) groups.

Undergraduate special students have remarkable passing rates and grades overall, but those who participate in RI have the best. Nearly all RI students pass, and though they earn only a slightly higher grade than NoRI students, their grades cluster more closely around the group average than those of NoRI students.

For all student status types, RI is positively associated with both passing rates and course grades. Freshmen and sophomores particularly benefit, with significant differences in outcomes. Undergraduate special students also benefit, although the differences are less dramatic due to their higher college readiness rates. Among all student status groups, RI participants earn more consistently high grades, closer to their average group grade than non-participants.

College Ready Versus Not College Ready

So far, the findings might seem to suggest that RI benefits NCR students the most, and we could reasonably argue that RI should target them. But do CR students benefit from RI? If so, how much compared to NCR students?

Passing Rates

Among both NCR and CR students, RI groups pass more often than nonparticipants. Pairwise comparisons indicate all results are distinct within their peer groups. Both NCR (53%) and CR (71%) Abs students have the lowest passing rates, followed by NoRI-NCR students (76%). When NCR students participate in RI, they pass slightly less (83%) than CR students in the NoRI group (86%). When CR students participate, 92% pass, the highest rate of any group.

For NCR students, the RI group is over four times as likely (OR = 4.35) to pass as the Abs group and more than half again as likely (OR = 1.56) as the NoRI group. For CR students, the RI group is almost five times as likely (OR = 4.76) to pass as the Abs group and almost twice as likely (OR = 1.84) as the NoRI group. These results indicate that both CR and NCR students benefit from RI.

Course Grade

For both NCR and CR students, RI participants earn higher grades than nonparticipants. Outcomes are distinct for each group, according to z-tests (0.000 sig). Abs students earn the lowest grades, whether NCR (49%) or CR (64%). NoRI-NCR students follow, with an average of 68%. Like with passing rates, the grades of NCR participants (75%) resemble those of CR students in the NoRI group (77%). CR participants earn the highest grades, with an average of 82%.

For NCR groups, RI students (25.1 SD) earn grades closer to the group average more consistently than either the NoRI (30.5 SD) or Abs (37.3 SD). As for CR groups, RI students (20.3 SD) are even more consistent, more so than either of their NoRI (26.1 SD) or Abs (35.1 SD) counterparts.

A 2019 meta-analysis by Robertshaw and Asher concluded that high-performing students (i.e., CR students) are more likely to use library services, potentially skewing results. To address their concerns, this study separated groups by college readiness and found that both CR and NCR participants have higher passing rates and grades compared to their nonparticipant peers. These results invite us to consider RI as a standard pedagogical practice that helps all students, whether college ready or not college ready.

Course Attendance

Course attendance is strongly correlated with course grade (Credé et al., 2010; Gaftandzhieva et al., 2022), and RI student outcomes may just be benefiting from general class attendance, which is what Chodock (2018) concluded from his study. Chronically absent students may be lowering NoRI and Absent groups’ grades and passing rates, because they not only receive no credit from not passing but also a fail to attend (FA) grade, which results in 0%. This FA grade is lower than a failing (F) grade, which is calculated as 29.5% (the mean of 0%–59%). If FA students are removed from all groups, is RI still correlated with higher passing rates and grades?

At Pellissippi State, if students stop attending classes at any point in the semester, do not withdraw, and their instructors mark a last date of attendance, they receive an FA grade at the end of the semester. Removing these chronically absent students, who receive an FA grade, from each group, leaves students who participated similarly. Chronically absent students comprise 23% (1,446 students) of the Abs group, 8% (2,257 students) of the NoRI group, and 3% (407 students) of the RI group. Combining the Absent and RI students (1,853) who are in the same class puts these percentages into perspective relative to NoRI students, as they are at 9% and 8%, respectively. As a point of interest, FA grades are inversely correlated with college readiness. That is, NCR students are almost twice as likely (OR = 1.78) to receive FA grades than CR students.

College Readiness

When chronically absent students are removed, both NoRI students (57% CR) and Abs students (53% CR) are more college ready than RI students (50% CR), which suggests they should have better outcomes than RI students. Pairwise comparisons confirm that all groups have different CR levels. As a point of interest, students in the Abs group are almost twice as likely (OR = 1.71) to have failed to take a tutorial than they are to have skipped a live class. Also, Abs students who skip the tutorial are a little more likely (OR = 1.29) to be college ready.

Passing Rates

Even though Abs (80%) and NoRI (88%) students should have higher passing rates due to their higher college readiness rates, RI students (90%) continue to pass more often than any other group. Pairwise comparisons confirm that these outcomes are distinct. While Abs (27% difference) and NoRI (31% difference) students do much better than expected, RI students (40% difference) perform exceptionally well. In fact, RI students pass more than twice as often (OR = 2.25) as Abs students and a little more often (OR = 1.18) than NoRI students, even though NoRI students are much more CR. Therefore, RI is still positively correlated with higher passing rates.

Course Grades

Though RI students are the least CR, they exceed expectations and earn the highest grades of any group, with an average of 81%. While NoRI students earn higher course grades (79%) than Abs students (73%), their performance is almost identical relative to their college readiness rates. Z-tests indicate each group’s outcomes are significantly different (0.000 sig). RI students earn grades at or near 81% more consistently, indicated by the lowest SD of any group (19.3 SD). NoRI students’ grades vary slightly more from their 79% average (20.0 SD), and Abs students vary the most (23.5 SD) from their 73% average. Thus, RI is correlated with the highest grades and most consistently high grades for individuals in the group.

Even when chronically absent students are removed from each group, and the RI group is the least college ready, RI students still outperform all other groups, including the most college ready NoRI group. Even Abs students should perform better than RI students, given their higher college readiness rate and similar course attendance. However, RI students continue to have the highest passing rate and grades, while Abs students have the lowest. RI students also consistently earn grades closer to the group average. When considering college readiness, RI students earn much higher grades than expected, whereas NoRI and Abs students earn only modestly higher grades. These findings demonstrate a positive correlation between RI, passing rates, and grades, even when controlling for course attendance.

Taking Attendance

”Typical one-shot instruction sessions, as provided for community college introductory-level courses, had not been shown to have a measurable impact on academic success,” noted Chodock (2018), “We wondered what factors might be important to make library instruction more effective” (p. 119). This study shows that one-shot sessions can have a measurable positive impact on both passing rates and course grades of community college students. Why are these results different? Many studies focusing on one-shot sessions may have shown similar positive outcomes if they had accounted for attendance and student demographics. Attendance was not taken, often assuming absent students would not impact the results.

This study found that 30% of students enrolled in classes offering RI are in the absent group. This population is large enough to significantly dampen the RI group’s results, particularly when their outcomes are consistently the lowest of any group. Also, including absent students in the NoRI group artificially lowers the NoRI group results. Therefore, absent students must be considered separately. AbsRI Combined Group (N = 21,227) proportions:

• RI: 14,911 (70%) students have one or more live session(s) or tutorial(s)
• Abs: 6,316 (30%) students do not participate in any RI opportunities offered

College Readiness

As mentioned, most studies did not include demographic information, and those that did overlooked tying it to outcomes. AbsRI students (50% CR) are much less college ready than NoRI students (56% CR). Therefore, we expect NoRI students to outperform AbsRI students.

Passing Rates

AbsRI students’ passing rate (80%) is lower than the NoRI group (82%). These results are statistically distinct according to pairwise comparisons. AbsRI students are slightly less likely (OR = 0.89) to pass than NoRI students. Calculating the difference between college readiness and passing rates, AbsRI students (30% difference) are slightly more successful than NoRI students (25% difference), but these results are still lackluster.

For comparison, this study’s initial analysis found that, when attendance is taken, 62% of Abs students and 88% of RI students pass (see Table 3). Therefore, combining these students flattens the passing rate to 80%, which is lower than the NoRI group at 82%. These results align with studies that assumed full attendance and found lower passing rates for students who have IL instruction.

Course Grade

Grades are similarly disappointing, with NoRI averaging slightly higher (73%) than AbsRI students (72%). T-tests show their outcomes are distinct (0.000 sig). When considering college readiness, AbsRI students (22% difference) are slightly more successful than NoRI students (17% difference). However, the grade variability for AbsRI (29.7 SD) and NoRI (28.5 SD) groups are analogous. These results confirm studies that assumed full attendance and found no benefit from IL instruction.

Again, this study’s initial analysis found that when attendance is taken, Abs students earn a 56% course grade, and RI students earn 79% (see Table 3). Also, absent students had a much wider grade range (36.9 SD) than RI students (23.1 SD). When these groups are combined, their average course grade flattens to 72%, slightly lower than the NoRI group (73%), and their grade variability is slightly higher than the NoRI group.

When the Abs group is combined with the RI group, as if attendance was not taken, the results are identical to many other studies that assumed full attendance and showed IL instruction has no benefit. These results demonstrate that gathering attendance during the intervention is crucial to measuring its impact on academic success. Addressing Chodock’s (2018) observation, targeted one-shot RI instruction is likely already effective, and the critical factor in measuring success is factoring in participation in the intervention (i.e., attendance or tutorial submission).

Discussion

Through multiple analyses, this study repeatedly demonstrates a positive association between targeted RI within a one-shot model and higher passing rates and course grades. Even without considering their college readiness, RI students consistently achieve the highest passing rates and grades. When factoring in college readiness, RI students have the highest difference between expected outcome and actual outcome, whereas nonparticipants tend to perform either as expected from their college readiness rate or marginally better. RI students also consistently earn course grades at or near the group average, suggesting RI fills a critical skill gap and reduces variability in outcomes.

Even when analyzed by semester, these results remain consistent across all student cohorts, with RI participants achieving the highest outcomes despite their low college readiness. Further examination of student status types reveals much the same pattern. While the predominately college ready undergraduate special participants experience slightly higher passing rates and more reliably higher grades, freshmen and sophomores (who are generally less college ready) have remarkably higher outcomes. Overall, each student type has consistently higher passing rates and grades when participating in RI.

From these findings, one could reasonably conclude that RI should target NCR students. However, viewing RI as remediation ignores its benefit to CR students, as the results indicate that both NCR and CR students who participate in RI have the highest outcomes of their respective peers. These findings highlight IL as a gap in students’ skill set, even among CR students. Therefore, RI should be integrated into the pedagogical approach for teaching college-level research to all students.

To account for course attendance generally, this study removed chronically absent students from each group and re-examined results only to find a similar pattern. Despite RI students having the lowest college readiness rate, even lower than Abs students, they still outperform every other group. Therefore, general course attendance cannot be the sole cause of higher passing rates and grades for RI students.

These results appear contradictory to those of previous studies that assumed full attendance to measure the impact of IL on student success. However, combining the Abs and RI groups to simulate forgoing attendance yields outcomes consistent with these prior studies. This finding underscores the importance of attendance tracking. Also, understanding the composition of student cohorts when reporting outcomes is essential for contextualizing success and comparing studies across different populations. Accurately measuring efforts is critical to understanding our role in student success, prioritizing our efforts, and effectively communicating value to stakeholders.

Limitations

This study uses only fall data to help ensure that similar student mixes were compared. However, this method provides only snapshots of students’ progression and does not account for prior semesters or other courses in the same semester when students may have had RI. Consequently, graduation rates were not analyzed, as this would require examining data from all semesters to provide a comprehensive view of RI’s influence. Likewise, GPA analysis would benefit from including data from all semesters, offering greater context for evaluating RI’s impact.

At Pellissippi State, students have numerous opportunities to seek academic support, which are not included in this study. Besides RI, the librarians offer research appointments and chat services for one-on-one help. Students can also use our online guides and academic skills tutorials that address IL topics relevant to their projects. Librarians collaborate with instructors to create research assignments that have step-by-step directions that include relevant IL topics. Additionally, students are encouraged to seek support from the Academic Support Center, which offers tutoring and writing help to all students, or from TRIO, which focuses on tutoring disadvantaged students.

Finally, though this study is controlled, it is not randomized. Instructors decide when to invite librarians or assign tutorials, so students are not randomly assigned RI. While the library’s tutorials are available to any student at any time from our homepage, students do not typically take them unless required by their instructor. Even when an instructor offers extra credit for taking the tutorials, generally, very few students complete them for the extra points.

Further Research

Further research should replicate this large-scale study with attendance tracking and key factor analysis to evaluate if this method is fruitful at other institutions. Another study at the same institution could compile data from all semesters to follow students through their time at Pellissippi State and address RI’s effect on graduation, GPA, persistence, and retention. This type of study could consider RI’s influence over time. This data might also be used to examine how different formats of instruction impact student outcomes. Another study might explore the influence of IL reference help, tutoring, and RI on student outcomes. This study would require gathering accurate student data and class information via chat and in-person, which presents many challenges. Lastly, the ways in which student outcomes are affected by research assignments that integrate IL instruction could be studied. The challenge would be ethically creating a control group.

Conclusion

This study’s results indicate that targeted one-shot RI is positively correlated with higher passing rates and course grades. RI students are over four times as likely to pass as Abs students, even though they are similarly college ready, and more than half again as likely to pass than NoRI students, despite being much less college ready than they are. RI students also earn the highest, most consistent grades of any group, even outperforming the much more college ready NoRI group. Whether a freshman, sophomore, or undergraduate special student, RI participants pass the most often and earn the highest, most consistent grades among their peers. Notably, freshman and sophomore RI participants show the biggest differences between expected and actual outcomes, although undergraduate special participants also experience higher outcomes. Both NCR and CR participants have the highest passing rates and grades among their nonparticipant peers, suggesting that RI should be considered a standard pedagogical practice for all students conducting research rather than a remediation for some. When accounting for general course attendance and removing chronically absent students from all groups, RI students continue to see the highest passing rates and grades despite having the lowest college readiness rates of any group, including Abs students. Finally, combining the Abs and RI groups and comparing them to NoRI students, as done in previous studies that assumed full attendance, confirms that factoring in attendance is likely the decisive factor in demonstrating positive results.

Acknowledgments

I want to express my deepest gratitude for the support of several of my colleagues. My librarian team was instrumental in the success of this study, particularly their excellence in teaching and meticulous attendance gathering. Assessment Director Amy Tankersley offered invaluable feedback on evaluating grades and presenting statistics. Finally, I want to thank the entire IEAP team for their guidance throughout this study, especially Austin Lawson, John Heuer, and my IEAP mentor Olga Ebert-Holberg.

References

Asher, A. (2016). Evaluating the effect of course-specific library instruction on student success. Indiana University Scholarly Works. https://scholarworks.iu.edu/dspace/handle/2022/21277

Association of College and Research Libraries. (2010). Value of academic libraries: A comprehensive research review and report. Prepared by Megan Oakleaf. Association of College and Research Libraries. https://acrl.ala.org/value/?page_id=21

Association of College and Research Libraries. (2016). Documented library contributions to student learning and success: Building evidence with team-based assessment in action campus projects. Prepared by Karen Brown with contributions by Kara J. Malenfant. Association of College and Research Libraries. https://bit.ly/42qbelO

Bowles-Terry, M., & Donovan, C. (2016). Serving notice on the one-shot: Changing roles for instruction librarians. International Information & Library Review, 48(2), 137–142. https://doi.org/10.1080/10572317.2016.1176457

Burgoyne, M. B., & Chuppa-Cornell, K. (2015). Beyond embedded: Creating an online-learning community integrating information literacy and composition courses. The Journal of Academic Librarianship, 41(4), 416–421. https://doi.org/10.1016/j.acalib.2015.05.005

Carlozzi, M. (2018). They found it–now do they bother? An analysis of first-year synthesis. College & Research Libraries, 79(5), 659–670. https://doi.org/10.5860/crl.79.5.659

Chodock, T. (2018). Library instruction and academic success. In M. Regalado, & M. A. Smale (Eds.), Academic libraries for commuter students: Research-based strategies. ALA Editions.

Coulter, P., Clarke, S., & Scamman, C. (2007). Course grade as a measure of the effectiveness of one-shot information literacy instruction. Public Services Quarterly, 3(1–2), 147–163. https://doi.org/10.1300/J295v03n01_08

Credé, M., Roch, S. G., & Kieszczynka, U. M. (2010). Class attendance in college: A meta-analytic review of the relationship of class attendance with grades and student characteristics. Review of Educational Research, 80(2), 272–295. https://doi.org/10.3102/0034654310362998

Gaftandzhieva, S., Talukder, A., Gohain, N., Hussain, S., Theodorou, P., Salal, Y. K., & Doneva, R. (2022). Exploring online activities to predict the final grade of student. Mathematics, 10(20), 3758. https://doi.org/10.3390/math10203758

Gandhi, S. (2004). Faculty-librarian collaboration to assess the effectiveness of a five-session library instruction model. Community and Junior College Libraries, 12, 15–48. https://doi.org/10.1300/J107v12n04_05

Koufogiannakis, D., & Wiebe, N. (2006). Effective methods for teaching information literacy skills to undergraduate students: A systematic review and meta-analysis. Evidence Based Library and Information Practice, 1(3), 3–43. https://doi.org/10.18438/B8MS3D

Krieb, D. (2018). Assessing the impact of reference assistance and library instruction on retention and grades using student tracking technology. Evidence Based Library and Information Practice, 13(2), 2–12. https://doi.org/10.18438/eblip29402

Lundstrom, K., Martin, P., & Cochran, D. (2016). Making strategic decisions: Conducting and using research on the impact of sequenced library instruction. College & Research Libraries, 77(2), 212–226. https://doi.org/10.5860/crl.77.2.212

Pagowsky, N. (2021). The contested one-shot: Deconstructing power structures to imagine new futures. College & Research Libraries, 82(3), 300. https://doi.org/10.5860/crl.82.3.300

Robertshaw, M. B., & Asher, A. (2019). Unethical numbers? A meta-analysis of library learning analytics studies. Library Trends, 68(1), 76–101. https://doi.org/10.1353/lib.2019.0031

Rowe, J., Leuzinger, J., Hargis, C., & Harker, K. (2021). The impact of library instruction on undergraduate student success: A four-year study. College & Research Libraries, 82(1), 7–17. https://doi.org/10.5860/crl.82.1.7

Schilling, K., & Applegate, R. (2012). Best methods for evaluating educational impact: A comparison of the efficacy of commonly used measures of library instruction. Journal of the Medical Library Association, 100(4), 258–269. https://doi.org/10.3163/1536-5050.100.4.007

Shao, X., & Purpur, G. (2016). Effects of information literacy skills on student writing and course performance. The Journal of Academic Librarianship, 42(6), 670–678. https://doi.org/10.1016/j.acalib.2016.08.006

Squibb, S. D., & Mikkelsen, S. (2016). Assessing the value of course-embedded information literacy on student learning and achievement. College & Research Libraries, 77(2), 164–183. https://doi.org/10.5860/crl.77.2.164

Vossler, J., Horton, J., & Heady, C. (2023). The questionable efficacy of one-shot instruction for first-year students: A scoping review. Academic College & Research Libraries, 446–454.