Assessing the Information Literacy Skills of First-Generation College Students

Sarah LeMire, Zhihong Xu, Valerie Balester, Leroy G. Dorsey, and Douglas Hahn*

Academic libraries, together with their colleges and universities, are increasingly identifying first-generation college students as an underserved population that is likely to experience barriers to library access and usage. Less is known, however, about the information literacy skills of first-generation students, particularly in comparison with their continuing-generation counterparts. This study assessed the information literacy skills of first-generation college students in general education courses at Texas A&M University to inform information literacy instructional efforts and to inform advocacy efforts for developing substantial and sustained information literacy support for first-generation students at that campus. Study results indicate that first-generation students experience significant information literacy gaps in comparison with continuing-generation students at the same institution and in the same courses.

Introduction

As colleges and universities strive to increase retention and graduation rates on their campuses, increased attention has been paid to underserved student populations such as first-generation college students. First-generation students, defined for the purposes of this study as students whose parents did not graduate from a four-year college, are more likely to experience barriers and are less likely to graduate than their continuing-generation peers.1 As campuses develop learning communities, courses, and other programs for first-generation students, librarians have begun to get involved. While some librarians have fully embedded in first-generation programs on their campuses,2 others are still working to develop collaborative relationships with program leaders or are limited to resource awareness-focused one-shots. Though embedding information literacy instruction is more impactful for first-generation students,3 advocating for time and resources to provide sustained information literacy instruction can meet with resistance. Library administrators and first-generation program coordinators may want evidence to support the need for these increased resources. To date, the library literature has provided limited evidence of the unique information literacy needs of first-generation college students.4

In this study, researchers from several campus departments partnered to assess the information literacy skills of first-generation college students on their campus. This assessment was intended to help librarians and their collaborators across campus better understand the information literacy skills and needs of first-generation college students. The goal of the study was to inform future information literacy collaborations and programming aimed at serving this important but underserved population.

Research questions included:

1. Are there differences in the overall information literacy test scores between first-generation students and continuing-generation students?
2. Are there differences in information literacy outcomes between first-generation students and continuing-generation students?
3. Are there differences in information literacy dispositions between first-generation students and continuing-generation students?
4. Are there differences in information literacy performance indicators between first-generation students and continuing-generation students?

Literature Review

Assessment represents an integral part of information literacy program design, advocacy, and instruction. Oakleaf and Kaske identify three reasons for assessment: to “increase student learning,” “respond to calls for accountability,” and “improve library instruction programs.”5 Assessment results can help librarians design appropriate and tailored information literacy instruction and can also be used to help librarians advocate for instructional time, administrative support, or other resources to increase or sustain information literacy in a target area.

Librarians have long recognized that information literacy skills may differ from one group to another. Accordingly, librarians have conducted assessments to better understand the specific information literacy skills of a variety of unique groups. For example, studies have explored how information literacy skills may vary based on class year, studying first-year college students,6 graduating seniors,7 and incoming graduate students.8 Other studies have investigated information literacy skills of students in specific majors, such as teacher education students,9 international graduate business students,10 and graduate education students.11 Librarians have also explored student information literacy skills based on admission type, such as transfer students,12 or participation in a specific program, such as an educational opportunity program.13

First-generation college students constitute another specific group that has become increasingly of interest in academic libraries. One common thread in the literature focuses on identifying and reducing barriers to library services for first-generation students. Researchers have found that the labyrinthine nature of academic library buildings and services is a problem for this group of students. Brinkman et al. found that first-generation students can experience the library as “a confusing world.”14 Parker found that “many described the physical library in terms such as ‘intimidating,’ ‘hard to navigate,’ and ‘scary.’”15 Accordingly, researchers have recommended alterations to library services to better meet the needs of first-generation students. Tyckoson recommended changes such as adjustments to service hours to better accommodate student schedules and other life responsibilities.16 Arch and Gilman echo many of these recommendations, including peer mentoring and personal librarian programs, and add suggestions for co-locating academic success services and lending programs for textbooks and technology.17

In addition to reducing barriers to library access and services, the literature also explores the information literacy skills of first-generation students. Ilett noted that the research shows “first-generation students improve their information literacy knowledge and skills over the course of their college careers and come to understand the role of the library and librarians in their academic success.”18 Indeed, Logan and Pickard interviewed first-year, first-generation college students and found that, contrary to the expectations of librarians and faculty members, all had experience with research papers from high school. They noted, “These students clearly knew to look for quality information.”19 A follow-up study by Pickard and Logan that compared first-generation seniors to first-year, first-generation students found that first-generation seniors demonstrated a growth in information literacy knowledge and skills during the course of their undergraduate education.20

However, the literature reveals little research that would help them answer a question commonly posed to librarians by administrators. When librarians advocate for library involvement in programming aimed at first-generation students, they may be asked how first-generation students’ information literacy skills differ from those of their continuing-generation peers. Initial research by Graves et al. suggested that first-generation students may demonstrate different information literacy skills related to selecting and documenting sources.21 This study contributes toward filling a gap in the literature by further exploring if, and to what extent, first-generation students demonstrate differences in their information literacy skills in comparison with their continuing-generation counterparts.

Methodology

The primary goal of this study involved establishing a baseline understanding of undergraduate information literacy skills to inform the University Libraries’ instruction program. To do so, the researchers planned to collect a large dataset that they could use to explore a number of research questions. The researchers applied for campus funding to implement a standardized information literacy test. There are benefits and disadvantages to using an information literacy test to measure information literacy skills, as Oakleaf describes.22 Tests have limited utility in measuring student behavior or execution of information literacy skills; instead, they measure students’ ability to recognize information. On the plus side, tests can have high accuracy, particularly if they are extended in length, and they scale well. Finally, Oakleaf notes that a benefit to fixed-choice tests is that “people believe in them.”23 For this reason, fixed-choice tests can be an effective advocacy tool in conversations with campus stakeholders.

The researchers were familiar with a previous campus project24 using Carrick Enterprises’ Project SAILS (Standardized Assessment of Information Literacy Skills) test, which is based on the Association of College and Research Libraries’ (ACRL) Information Literacy Standards for Higher Education25 and which has been sunset. The researchers wanted to build upon this previous work using Carrick Enterprises’ newly validated test, TATIL (Threshold Achievement Test of Information Literacy). Unlike Project SAILS, TATIL was designed in response to the ACRL Framework for Information Literacy for Higher Education.26

Threshold Achievement Test of Information Literacy (TATIL)

To gain the broadest assessment of student information literacy skills, researchers opted to implement all four modules of the TATIL test. Table 1 includes module names and descriptions from Carrick Enterprises.27

Each module of the TATIL test can take approximately 30–50 minutes to complete, according to Carrick Enterprises, so the researchers opted not to assign all four modules to each student. Instead, students would be randomly assigned to a single information literacy module. Though not every student who was assigned a module completed it, this method ensured that a similar number of students completed each module of the TATIL test.

Each module of the TATIL test is composed of four components: outcomes, dispositions, and performance indicators, and an overall score.28 Outcomes are the information literacy skill categories that TATIL is testing; for example, Outcome 1.1 is “Apply knowledge of source creation processes and context to evaluate the authority of a source.”29 Each outcome is measured through completion of several individual questions, which are called performance indicators. Scores on individual performance indicators are mapped to a larger outcome score, and all of the outcome scores for a single module comprise the overall test score for that module. In the TATIL test, dispositions are a series of questions that are intended to measure strategies, attitudes, or behavior rather than knowledge. Disposition scores are kept separate from outcome scores and performance indicator scores and do not factor into overall scores. In the TATIL test, students can score highly on a particular performance indicator or outcome, indicating knowledge of a specific information literacy skill, but score much lower on a related disposition score if the strategies they choose do not reflect their knowledge. For example, a student may be able to identify a scholarly source from a list, but their search strategies may not yield any scholarly sources to use in their paper. Finally, each student who completes a TATIL module is given an overall score for that specific module. The overall score is a composite of that module’s outcome and performance indicator (but not disposition) scores. This study evaluates students’ results in all four aspects of the TATIL modules: outcomes, performance indicators, dispositions, and overall scores.

Recruitment

After receiving institutional review board approval, the researchers contacted instructors of Texas A&M University core curriculum, or general education, courses to request their help in recruiting participants. During the four semesters of this study, 52 unique instructors volunteered to share the study with the students in their general education classes. Instructors had the option to directly share the study invitation to participate with their students, or they could invite the researchers into the classroom to share the study directly with the students. Students who opted to complete the survey could follow the link embedded in the invitation to participate, which prompted them to use their Single Sign On (SSO) authentication credentials to log into a demographic questionnaire, after which students were randomly assigned one of the four modules. To incentivize participation, instructors were permitted to offer extra credit for participation, though not all chose to do so; all participants were entered into a drawing for gift cards at the end of the semester.

Demographics

This study included all testing of the four TATIL modules, spread out over four academic semesters beginning in fall 2018 and ending in fall 2019. To get a valid dataset, the researchers had to eliminate a number of participant responses. In particular, the researchers were concerned about the amount of time participants spent completing the modules, which ranged from 20 to 30 questions. Though the TATIL system requires participants to spend a minimal amount of time on each question page, the dataset included responses from participants who spent less than three minutes on an entire module. In contrast, the median amount of time spent on the modules ranged from 19 to 32 minutes. To ensure that the dataset was focused on participants who had substantively engaged with the module, the researchers dropped all of the participants (134) whose total time of finishing a module was less than 10 minutes.

A total of 631 participants finished Module 1 (EP&A). Among them, 12 participants’ information were dropped because their total participation time was less than 10 minutes. Therefore, 619 participants’ data were analyzed for this module. Among them, there were 466 continuing-generation students and 153 first-generation students. Similarly, 653 participants completed Module 2 (SS), and 53 participants’ information were dropped due to participation time (<10 minutes). Of the remaining 600, 436 continuing-generation students and 164 first-generation students’ data were analyzed for Module 2. A total of 611 participants completed the third module. Among them, we dropped 27 participants’ information because of insufficient participation time (<10 minutes). Thus, 584 participants’ data were analyzed for Module 3 (R&S): 433 continuing-generation students and 151 first-generation students. For Module 4 (VoI), there were 634 participants finishing the survey. The completion time of 42 participants was less than 10 minutes. Therefore, 421 continuing-generation students and 171 first-generation students were included in the analysis for Module 4, with 592 in total. Detailed information about participants were included in table 2.

Data Analysis

To analyze the data, we imported the dataset into STATA and ran two types of analyses: t-tests and multivariate multiple regressions. For both types of analysis, a threshold of p < 0.05 was used to determine statistical significance.

For research question 1, we ran four t-tests to investigate the differences in information literacy and overall scores between first-generation students and continuing-generation students across four modules.

Research question 2 examined the differences in information literacy outcomes between first-generation students and continuing-generation students across four modules. For this research question, four multivariate multiple regressions were analyzed in STATA with the outcome scores as the dependent variable and the group condition (first generation or continuing-generation students) as the independent variable, with library experience as covariates. We chose multivariate multiple regression because the outcome scores are correlated. The least-squares estimation was used as the parameter estimation method. We used weighted scores instead of raw scores because difficulty level is an important component of TATIL scoring.30

To answer the third research question, four multivariate multiple regressions were employed to investigate the differences in information literacy dispositions between first-generation students and continuing-generation students. The group condition and the library experience were used as independent variables, while the disposition scores were used as dependent variables. The least-squares estimation was used as the parameter estimation method. 

Four multivariate multiple regressions were conducted to investigate the differences in information literacy performance indicators between first-generation students and continuing-generation students (research question 4). We used the group condition and the library experience as independent variables, while the performance indicator scores were used as the dependent variables. The least-squares estimation was used as the parameter estimation method. 

For all of the multivariate multiple regression analyses, we first included library experience as the control variable. Since there was no statistical difference existing in the library experience, we dropped the covariate in the final model. 

Results

Overall Scores

To gain an overall understanding of the TATIL test performance of first-generation and continuing-generation students, we first calculated how frequently each group reached each of TATIL’s performance levels. TATIL uses three performance levels—conditionally ready, college ready, and research ready—to describe student achievement on the knowledge section of the test. Each test module uses the same three performance levels, though the description of the knowledge and skills associated with each performance level varies from one module to another. Conditionally ready is the lowest level, while college ready is the intermediate level and research ready is the highest level. TATIL has established for each module specific cutoff scores between each performance level. Table 3 represents participants’ knowledge levels in the overall score category for all four modules.

As a whole, results in all four modules revealed that the vast majority of students scored at the college ready level or higher, regardless of first-generation status. Students in both groups performed most highly on modules 3 (R&S) and 4 (VoI). Only six (3.51%) first-generation students and nine (2.14%) continuing-generation students scored in the lowest category, conditionally ready, on module 4 (VoI). Similarly, 10 (6.62%) first-generation students and 24 (5.54%) continuing-generation students were identified as conditionally ready in module 3 (R&S). Though the differences were small, first-generation students were more likely to be placed in the conditionally ready category. First-generation students were also less likely to be placed in the highest category. Only 23 (15.23%) first-generation students were research ready in module 3, while 126 (29.10%) continuing-generation students were research ready. Similarly, 12 (7.02%) first-generation students were categorized as research ready for module 4, while 54 (12.83%) continuing-generation students were placed in that category.

Overall, students scored lower on modules 1 (EP&A) and 2 (SS), regardless of first-generation status. First-generation students were again more likely to be placed in the conditionally ready category, with 33 (21.57%) first-generation students marked as conditionally ready for module 1 (EP&A), compared to 33 (7.08%) continuing-generation students. Similarly, 24 (14.63%) first-generation students were marked as conditionally ready for module 2 (SS), compared to 46 (10.55%) continuing-generation students. At the other end of the spectrum, neither first-generation students nor continuing-generation students were likely to be categorized as research ready. In module 2, only seven (4.27%) first-generation students were categorized as research ready and a similar percentage (4.59%, or 20) of continuing-generation students were research ready for that module. Interestingly, neither group had a single student score in the research ready category for module 1 (EP&A).

To determine whether there were statistically significant differences between first-generation students and continuing-generation students across modules in the overall scores, we ran four t-tests. We found that there were significant differences between the two groups in each module. In module 1 (EP&A), first-generation students’ overall scores (M = 441.45, SD = 137.30) were statistically lower than continuing-generation students’ scores (M = 510.34, SD = 132.74) (p < 0.001). In module 2 (SS), first-generation students’ overall scores (M = 479.39, SD = 138.75) were statistically lower than continuing-generation students’ scores (M = 518.09, SD = 142.79) (p < 0.01). In module 3 (R&S), first-generation students’ overall scores (M = 459.97, SD = 149.11) were statistically lower than continuing-generation students’ scores (M = 520.79, SD = 155.80) (p < 0.001). In module 4 (VoI), first-generation students’ overall scores (M = 452.85, SD = 134.64) were again statistically lower than continuing-generation students’ scores (M = 491.41, SD = 138.48) (p < 0.01).

Outcomes

In addition to exploring differences in overall TATIL module scores, we analyzed the data for potential differences between first-generation students and continuing-generation students on TATIL test outcomes. Each of the four TATIL modules (EP&A, SS, R&S, VoI) contains two outcomes along with their Carrick Enterprises descriptions in appendix A, along with the code associated with each outcome for analysis purposes.31 TATIL also breaks down outcome scores according to knowledge performance levels, using the same categories as with the overall scores: conditionally ready, college ready, and research ready. Knowledge performance level results for outcomes closely aligned with those of overall scores, which is unsurprising as outcome scores contribute to overall scores. For all eight outcomes, first-generation students were more likely to be placed in the conditionally ready category.

Students performed most highly on modules 3 (R&S) and 4 (VoI), regardless of first-generation status. For module 4 (VoI), only eight (4.68%) first-generation students and 14 (3.33%) continuing-generation students scored as conditionally ready for Outcome 4.1 (O41), while 16 (9.36%) first-generation students and 25 (9.54%) continuing-generation students scored as conditionally ready for Outcome 4.2 (O42). For module 3 (R&S), nine (5.96%) first-generation students and 17 (3.93%) continuing-generation students were conditionally ready for Outcome 3.2 (O32), while 29 (19.21%) first-generation students and 45 (10.39%) continuing-generation students were conditionally ready for Outcome 3.1 (O31).

Similar to the overall scores, the outcome scores for modules 1 (EP&A) and 2 (SS) suggested all students had more difficulty in these areas. For module 1 (EP&A), 23 (15.03%) first-generation students were conditionally ready for Outcome 1.1 (O11), as opposed to 29 (6.22%) continuing-generation students. For Outcome 1.2 (O12), 50 (32.68%) first-generation students were conditionally ready, compared with 78 (16.74%) continuing-generation students. For module 2 (SS), 40 (24.39%) first-generation students and 77 (17.66%) continuing-generation students were conditionally ready on Outcome 2.1 (O21). For Outcome 2.2 (O22), 34 (20.73%) first-generation students and 73 (16.74%) continuing-generation students were conditionally ready.

While first-generation students were more likely to perform as conditionally ready across the board, the results for the research ready performance level were more mixed. Continuing-generation students were more likely to have scores at this level for four of the eight outcomes. However, first-generation students were more likely to score as research ready for the other four, though the differences are small. For Outcome 1.1 (O11), two (1.31%) first-generation students scored as research ready compared to three (0.64%) continuing-generation students. Similarly, Outcome 1.2 (O12) had one (0.65%) first-generation student score as research ready, while no continuing-generation students scored in that category. For Outcome 2.2 (O22), 19 (11.59%) first-generation students and 46 (10.55%) continuing-generation students were research ready. Finally, for Outcome 4.2 (O42), 51 (29.82%) first-generation students were research ready, compared to 123 (29.21%) continuing-generation students. Table 4 provides detailed information about students’ percentage rate of knowledge performance levels for each outcome.

Multivariate multiple regression analysis showed that there were statistically significant differences between first-generation students and continuing-generation students in all eight outcomes. In module 1 (EP&A), first-generation students had lower scores (M = 424.35, SD = 146.99) than their continuing-generation peers (M = 496.86, SD = 144.67) for outcome O11 [t (618) = –5.36, p < 0.001]. Similarly, first-generation students had statistically lower O12 (M = 462.99, SD = 168.69) scores [t (618) = –4.24, p < 0.001] than the continuing-generation students (M= 527.21, SD = 160.26) in module 1. In module 2 (SS), first-generation students’ O21 score (M = 477.36, SD = 152.91) and O22 (M = 483.09, SD = 204.49) were significantly lower [O21 t (599) = –2.74, p < 0.01; O22 t (599) = –2.01, p < 0.05 respectively] than the continuing-generation students’ O21 scores (M = 516.35, SD = 155.88) and O22 scores (M = 520.93, SD = 206.38). There were also statistically significant differences between the first-generation students and continuing-generation students in the outcome score O31 [t (583) = –4.39, p < 0.001] and O32 [t (583) = –3.16, p < 0.01] in module 3 (R&S), and O41 [t (591) = –2.02, p < 0.05] and O42 [t (591) = –2.96, p < 0.01] in module 4 (VoI). Detailed descriptive statistics about the information literacy outcome scores and results from the multivariate multiple regression analysis were included in tables 5 and 6. 

Dispositions

The TATIL test also measures students’ information literacy dispositions. Disposition scores are, according to Carrick Enterprises, “based on a student’s judgments regarding strategies. Students earn high scores on these items if they judge behaviors associated with the disposition to be useful and behaviors not associated with the disposition to be not useful.”32 Each module of the TATIL test has one to three dispositions associated with it. A list of dispositions, along with their Carrick Enterprises descriptions, is available in appendix B.33 Results from multivariate multiple regression analysis showed that there were statistically significant differences between the first-generation students and continuing-generation students in two (22%) of the dispositions. Specifically, there were differences between the two groups in the disposition score D21 (Productive persistence) [t (599) = –2.39, p < 0.05] and D33 (Responsibility to community) [t (583) = –3.15, p < 0.01]. The first-generation students had lower D21 scores (M = 63.16, SD = 9.68) and D33 scores (M = 50.45, SD = 9.39) than the continuing-generation students (M = 65.19, SD = 9.13) and D33 (M = 53.29, SD = 9.58). The descriptive statistics and detailed results from the multivariate multiple regression analysis were reported in tables 7 and 8. 

Performance Indicators

Performance indicators represent the most granular scores on the TATIL test. They measure student proficiency on specific skills areas and are, therefore, valuable as they can indicate specific problem areas that need to be addressed. There are 80 performance indicators across the four TATIL modules: 24 in module 1 (EP&A), 17 in module 2 (SS), 24 in module 3 (R&S), and 15 in module 4 (VoI). We also included in this category the individual disposition questions, as they are similar in granularity to the performance indicators. There are 21 individual disposition questions: six in module 1 (EP&A), three in module 2 (SS), six in module 3 (R&S), and six in module 4 (VoI). Altogether, there are 101 items in this category.

First-generation students scored lower than continuing-generation students on 90 (89%) of the performance indicators and individual disposition questions. There were statistically significant differences between first-generation and continuing-generation students in 36 (36%) of the performance indicators and individual disposition questions. Though there were no statistically significant performance indicators in which first-generation students outperformed their continuing-generation peers, first-generation students did score higher on seven performance indicators and four individual disposition questions. For the sake of brevity, only those performance indicators and individual disposition questions with significant results are included in appendix C.34

Four multivariate multiple regressions were employed to identify whether there were differences between first-generation and continuing-generation students on performance indicators and individual disposition questions across the four modules. In module 1 (EP&A), statistically significant differences between first-generation students and continuing-generation students were found in performance indicators p1211 [t(618) = –3.82, p < 0.001], p116 [t(618) = –2.97, p < 0.01], p126 [t(618) = –3.71, p < 0.001], p119 [t(618) = –3.06, p < 0.01], p129 [t(618) = –2.37, p < 0.05], p113 [t(618) = –3.06, p < 0.01], p1111 [t(618) = –3.46, p < 0.001], p118 [t(618) = –2.55, p < 0.05], p117 [t(618) = –2.11, p < 0.05], p127 [t(618) = –4.00, p < 0.001], p124 [t(618) = –2.20, p < 0.05], D11a [t(618) = –2.22, p < 0.05], D13b [t(618) = –2.23, p < 0.05] and D12b [t(618) = –2.56, p < 0.05]. Results showed that there were statistically significant differences in performance indicators p216 [t(599) = –2.67, p < 0.01], p212 [t(599) = –3.42, p < 0.001], p2111 [t(599) = –2.49, p < 0.05], p222 [t(599) = –2.19, p < 0.05], p221 [t(599) = –2.21, p < 0.05], and D21c [t(599) = –2.38, p < 0.05] in module 2 (SS). In module 3 (R&S), performance indicators p312 [t(583) = –2.56, p < 0.05], p314 [t(583) = –2.18, p < 0.05], p3113 [t(583) = –2.29, p < 0.05], p325 [t(583) = –3.51, p < 0.001], p3112 [t(583) = –2.80, p < 0.01], p326 [t(583) = –2.37, p < 0.05], p315 [t(583) = –2.49, p < 0.05], p3212 [t(583) = –3.24, p < 0.001], p318 [t(583) = –2.66, p < 0.01], p319 [t(583) = –3.14, p < 0.01], p3114 [t(583) = –2.06, p < 0.05], p316 [t(583) = –2.89, p < 0.01], D31b [t(583) = –2.10, p < 0.05], and D33a [t(583) = –2.70, p < 0.01] were found statistically different between groups. There were statistically significant differences in performance indicators p425 [t(591) = –2.29, p < 0.05] and p416 [t(591) = –3.13, p < 0.01] in module 4 (VoI). Among all of the statistically significant performance indicators and individual disposition questions, first-generation students had lower scores than continuing-generation students. Due to the number of indicators and disposition questions, detailed information about performance indicator and individual disposition question scores and the associated multivariate multiple regression analysis is provided in appendix D. 

Discussion

Analysis of the multivariate multiple regression and t-test results reveals a few consistent themes in the results related to the information literacy knowledge, skills, and behaviors of first-generation students as well as all students, regardless of their first-generation status. These results both support and extend the conclusions and recommendations from earlier studies.

Shared Strengths and Weaknesses

One reason that the researchers opted to include all four modules of the TATIL test in this study was to gain a clearer understanding of the specific information literacy strengths and weaknesses of students at Texas A&M University. The overall test scores reveal that the majority of Texas A&M University students score at the college ready level or above, which suggests that most students come to college with at least a moderate level of information literacy preparedness, regardless of first-generation or continuing-generation status. This finding suggests that librarians should not assume that students in either group are information literacy novices; instead, they should consider preassessing student information literacy skills ahead of instruction sessions to determine the specific information literacy learning needs of their students.

Although most students scored in the college ready category or above, reviewing the overall and outcome-specific knowledge performance scores reveals that students’ performance was not consistent across all four modules. There were some differences in the percentages of students scoring in the lower, conditionally ready, category and the higher, research ready, category. Students scored most highly in module 3 (R&S), which Carrick Enterprises notes “combines elements from the Research as Inquiry and Scholarship as a Conversation frames,” and module 4 (VoI), which focuses on the Information Has Value frame.35 This result was somewhat surprising, as these modules include concepts students might have been expected to struggle with, such as identifying gaps in the literature, developing research questions, and plagiarism.

On the other end of the results is module 1 (EP&A), which focuses on the frames Authority Is Constructed and Contextual and Information Creation as a Process, and module 2 (SS), which focuses on Searching as Strategic Exploration.36 Notably, no first-generation or continuing-generation students breached the research ready threshold in the overall scores on module 1 (EP&A). Both first-generation and continuing-generation students had a larger number of students who scored as conditionally ready in these two modules, although both search strategies and source evaluation are frequently taught as part of the Texas A&M University Libraries’ information literacy program.

These findings support librarians’ continued efforts to develop and reinforce information literacy knowledge and skills for all students, regardless of first-generation status, related to search strategies and source evaluation as part of information literacy instructional efforts. Though conducting preassessments with classes is a good way to inform instructional efforts in general, in particular librarians may wish to conduct preassessments related to Information Has Value, Research as Inquiry, and Scholarship as Conversation to ensure that instructional time devoted to those frames is carefully focused on the aspects with which students are less familiar.

Tailored Information Literacy Instruction

Analysis of TATIL test outcomes and performance indicators suggests that there is indeed a gap between the information literacy knowledge and skills of first-generation and continuing-generation students. Indeed, study results suggest that the gap between first-generation and continuing-generation students is wider than anticipated. Study results found statistically significant gaps in all four test modules and across all eight outcomes. They also revealed that first-generation students scored lower than their continuing-generation counterparts on 89 percent of the performance indicators and individual disposition questions, with statistically significant gaps on 36 (36%) of them.

The statistically significant gaps were most concentrated in modules 1 (EP&A) and 3 (R&S). For TATIL test outcomes, the gap between first-generation students and continuing-generation students were the largest for three outcomes:

1. O11: Apply knowledge of source creation processes and context to evaluate the authority of a source.
2. O31: Understand the processes of scholarly communication and knowledge building.
3. O12: Apply knowledge of authority to analyze others’ claims and to support one’s own claims

At the performance indicator level, large gaps between first-generation students and continuing-generation students were seen in all four modules. The five largest gaps were seen in the following performance indicators:

1. P1211: Determine the reason why a quote is used in a given passage (examples: show significance, give authoritative support, provide context, emphasize, summarize).
2. P126: Identify relevant questions to ask about the suitability of a source when considering it as support for a claim.
3. P416: Given a list, select the purposes of citation.
4. P113: Match the elements of a source record to what they reveal about the process used to create the source (for example, publisher name, authors’ names, date, subject terms, source type).
5. P3212: Classify descriptions of specific actions taken during the research process by the stage in the research process when they are most likely to happen.

These findings suggest that offering information literacy instruction specifically for first-generation students is appropriate given consistent information literacy knowledge and skill gaps between first-generation students and their continuing-generation counterparts. Many colleges and universities, including Texas A&M University, offer learning communities and other courses specifically aimed at supporting first-generation college students in their transition to college. Arch and Gilman recommend “integrating information literacy instruction into existing first-generation student programs, including orientations and summer bridge programs.”37 The findings from this study support this recommendation.

Knowledge vs. Dispositions

Though the largest number of statistically significant gaps between first-generation and continuing-generation students are found in the areas of outcomes and performance indicators, there were differences in the disposition category of the TATIL test. The TATIL test handles dispositions differently from the other question categories; unlike performance indicators, outcomes, and overall scores, which are interrelated, disposition scores are kept separate. Dispositions, according to the ACRL Framework, are “ways in which to address the affective, attitudinal, or valuing dimension of learning.”38 Students’ ability to recognize information and their attitude toward or the value they ascribe to that information may be different. For example, students may be able to identify the elements of a book citation from a list of possibilities; but, if they do not value citation, they may be less likely to implement their knowledge when creating a bibliography.

There was a statistically significant difference between first-generation students and continuing-generation students in only two (22%) dispositions. The small number of differences in the disposition category may be due to differences in scoring, as there are fewer points associated with dispositions on the TATIL tests. However, it could suggest that, while there is a difference between first-generation and continuing-generation students in terms of information literacy knowledge and skills, there is a smaller difference in attitudes and behaviors. Additional research is needed to better understand if and how information literacy gaps between first-generation and continuing-generation students exist outside a testing environment.

Though the reason remains unclear for the comparatively small amount of difference between continuing-generation and first-generation students, it is clear that those differences are located in two different modules: module 2 (SS) and module 3 (R&S). It is noteworthy that none of the three dispositions in module 1 (EP&A) had statistically significant differences even though that module showed differences in both the outcomes and performance indicators categories. The two statistically significant dispositions, along with their descriptions from Carrick Enterprises,39 are listed below:

• D21 (Productive persistence): “Learners who are disposed to demonstrate productive persistence during their searches for information approach searching as iterative and not linear by employing alternative strategies and learning from mistakes.”
• D33 (Responsibility to community): “Learners who are disposed to demonstrate a sense of responsibility to the scholarly community recognize and conform to academic norms of knowledge building.”

The differences in these specific dispositions may suggest that first-generation students are less likely to identify as part of the scholarly community and recognize some of the norms related to scholarly searching. Librarians should strive to explicitly welcome first-generation students into the scholarly research community and to make that community’s norms more transparent and inclusive for first-generation students.

Limitations

There are inherent limitations in a standardized testing approach to measuring information literacy skills. Participant behavior may not correlate with their responses on a multiple-choice test. In addition, because tests were administered out of class and for extra credit, the validity of participant responses is uncertain. Tests were administered during multiple semesters and aggregated into a single dataset. It is possible that responses may have differed from one semester to another due to factors external to the study. It is also possible that there are other factors that contributed to differences between the two study populations that were not controlled for in the analysis. Finally, this is a study from a single university, and findings cannot be generalized to other universities.

Conclusion

It is commonly understood by librarians that information literacy skills are critical to academic success for all students. It is also well-established that some student populations are underserved, may experience barriers, and may have different information literacy skills. Both continuing-generation and first-generation students demonstrate moderate to high information literacy skills as a whole, and librarians should take care not to assume that either group consists of information literacy novices. However, this study supports earlier research that suggested first-generation students’ information literacy skills differ from those of continuing-generation students and that therefore targeted and sustained information literacy support for first-generation students is appropriate. Identifying these differences in information literacy skills is an important first step toward advocating for the increased and dedicated resources necessary to develop targeted and sustained information literacy instructional support for first-generation college students. By establishing the need, librarians can open up new avenues for collaboration with first-generation programs and classes on their campus. In this way, librarians can help first-generation students feel welcome and supported by the library and also help first-generation students develop the information literacy skills to excel in higher education and beyond.

APPENDIX A. TATIL Outcomes

APPENDIX B. TATIL Dispositions

APPENDIX C. TATIL Performance Indicators and Individual Disposition Descriptions

APPENDIX D. Performance Indicator and Individual Disposition Question Scores

Notes

1. Xan Arch and Isaac Gilman, “First Principles: Designing Services for First-Generation Students,” College & Research Libraries 80, no. 7 (2019): 996–1012; Emily Forrest Cataldi, Christopher T. Bennett, and Xianglei Chen, “First-Generation Students: College Access, Persistence, and Postbachelor’s Outcomes,” NCES 2018-421, Statistics in Brief (Washington, DC: US Department of Education, 2018).

2. Adriana Parker, “Academic Libraries and Vulnerable Student Populations: A New Model of Embedded Librarianship for First-Generation University Students,” Political Librarian 3 no. 1, (2017): 26–31.

3. Parker, “Academic Libraries and Vulnerable Student Populations.”

4. Darren Ilett, “A Critical Review of LIS Literature on First-Generation Students,” portal: Libraries and the Academy 19, no. 1 (2019): 177–96; Firouzeh Logan and Elizabeth Pickard, “First-Generation College Students: A Sketch of Their Research Process,” in College Libraries and Student Culture: What We Now Know, eds. Lynda M. Duke and Andrew D. Asher (Chicago, IL: American Library Association, 2012), 109–25; Elizabeth Pickard and Firouzeh Logan, “The Research Process and the Library: First-Generation College Seniors vs. Freshmen,” College & Research Libraries 74, no. 4 (2013): 399–415.

5. Megan Oakleaf and Neal Kaske, “Guiding Questions for Assessing Information Literacy in Higher Education,” portal: Libraries and the Academy 9, no. 2 (2009): 273.

6. Melissa Gross and Don Latham, “What’s Skill Got to Do with It? Information Literacy Skills and Self‐Views of Ability among First‐Year College Students,” Journal of the American Society for Information Science and Technology 63, no. 3 (2012): 574–83.

7. Patricia Davitt Maughan, “Assessing Information Literacy among Undergraduates: A Discussion of the Literature and the University of California-Berkeley Assessment Experience,” College & Research Libraries 62, no. 1 (2001): 71–85; Brandy Whitlock and Nassim Ebrahimi, “Beyond the Library: Using Multiple, Mixed Measures Simultaneously in a College-Wide Assessment of Information Literacy,” College & Research Libraries 77, no. 2 (2016): 236–62.

8. Amalia Monroe-Gulick and Julie Petr, “Incoming Graduate Students in the Social Sciences: How Much Do They Really Know about Library Research?” portal: Libraries and the Academy 12, no. 3 (2012): 315–35.

9. Samantha Godbey and Alexandra Dema, “Assessment and Perception of Information Literacy Skills among Teacher Education Students,” Behavioral & Social Sciences Librarian 36, no. 1 (2017): 1–15.

10. Russell Michalak and Monica D.T. Rysavy, “Information Literacy in 2015: International Graduate Business Students’ Perceptions of Information Literacy Skills Compared to Test-Assessed Skills,” Journal of Business & Finance Librarianship 21, no. 2 (2016): 152–74.

11. Amy Jo Catalano, “Using ACRL Standards to Assess the Information Literacy of Graduate Students in an Education Program,” Evidence Based Library & Information Practice 5, no. 4 (2010): 21–38.

12. Min Tong and Carrie Moran, “Are Transfer Students Lagging behind in Information Literacy?” Reference Services Review 45, no. 2 (2017): 286–97.

13. Ma Lei Hsieh, Susan McManimon, and Sharon Yang, “Faculty-Librarian Collaboration in Improving Information Literacy of Educational Opportunity Program Students,” Reference Services Review 41, no. 2 (2013): 313–35.

14. Stacy Brinkman, Katie Gibson, and Jenny Presnell, “When the Helicopters Are Silent: The Information Seeking Strategies of First-Generation College Students,” in Imagine, Innovate, Inspire: Proceedings of the ACRL 2013 Conference, ed. Dawn M. Mueller (Chicago, IL: Association of College and Research Libraries, 2013), 647.

15. Parker, “Academic Libraries and Vulnerable Student Populations,” 29.

16. David A. Tyckoson, “Library Service for the First-Generation College Student,” in Teaching the New Library to Today’s Users: Reaching International, Minority, Senior Citizens, Gay/Lesbian, First-Generation, At-Risk, Graduate and Returning Students, and Distance Learners, eds. Trudi E. Jacobson and Helene C. Williams (New York, NY: Neal-Schuman, 2000), 100–04.

17. Arch and Gilman, “First Principles,” 1002–08.

18. Ilett, “A Critical Review of LIS Literature on First-Generation Students,” 118.

19. Logan and Pickard, “First-Generation College Students,” 113.

20. Pickard and Logan, “The Research Process and the Library,” 411.

21. Stephanie J. Graves, Sarah LeMire, and Kathy Christie Anders, “Uncovering the Information Literacy Skills of First-Generation and Provisionally Admitted Students,” Journal of Academic Librarianship 47, no. 1 (2021): 102260.

22. Megan Oakleaf, “Dangers and Opportunities: A Conceptual Map of Information Literacy Assessment Approaches,” portal: Libraries and the Academy 8, no. 3 (2008): 234–40.

23. Oakleaf, “Dangers and Opportunities,” 235.

24. Graves, LeMire, and Anders, “Uncovering the Information Literacy Skills of First-Generation and Provisionally Admitted Students.”

25. “The Tests,” Project SAILS, accessed June 5, 2020, https://www.projectsails.org/site/background/.

26. “How the Test Was Developed,” Threshold Achievement Test for Information Literacy, accessed June 5, 2020, https://thresholdachievement.com/the-test/background.

27. “Test Modules,” Threshold Achievement Test for Information Literacy (TATIL), accessed June 5, 2020, https://thresholdachievement.com/the-test/test-modules.

28. “Test Modules,” TATIL.

29. “Module Descriptions,” Threshold Achievement Test for Information Literacy (TATIL), accessed June 5, 2020, https://thresholdachievement.com/files/Module-Descriptions.pdf.

30. “Module Descriptions,” TATIL.

31. “How the Test Was Developed,” Threshold Achievement Test for Information Literacy (TATIL).

32. How the Test Was Developed,” TATIL.

33. “Module Descriptions,” TATIL.

34. “Module Descriptions,” TATIL.

35. “Module Descriptions,” TATIL.

36. “Module Descriptions,” TATIL.

37. Arch and Gilman, “First Principles,” 1002.

38. “Framework for Information Literacy for Higher Education,” Association of College and Research Libraries, accessed June 5, 2020, www.ala.org/acrl/standards/ilframework.

39. “Module Descriptions,” TATIL.

40. “Module Descriptions,” TATIL.

41. “Module Descriptions,” TATIL.

42. “Module Descriptions,” TATIL.