Library Usage, Instruction, and Student Success across Disciplines: A Multilevel Model Approach
To better understand the library’s role in student success, this research used student survey data to explore students’ reported library building use, library resource use, library instruction, as well as student perceptions of the library’s role in their success, and how these factors may vary by academic discipline. Cumulative GPAs were also matched with respondents. Quantifying the relationship between library usage and student success is one of the six areas recommended for further research by ACRL. This study focuses on the variation and the uncertainty of the measurement of this relationship across disciplines, using Bayesian multilevel regression methods. Levels of library resource usage and percentages of respondents believing the library contributes very much to the respondents’ academic success vary quite a bit by discipline. In this study, cumulative GPA is higher on average for students who use library resources more frequently, but not for students who receive library instruction. Although, for undergraduates, higher frequency of building usage predicted higher probability of believing the library contributes very much to academic success, it did not predict higher GPA.
Introduction
Libraries have long been interested in analyzing user behaviors and attitudes toward library use. The idea of measuring library service impact as a means of communicating library value to stakeholders gained significant ground in 2010 through the release of ACRL’s Value of Academic Libraries report.1 That report recommended that academic libraries seek to track “library influences on increased student achievement,” and specifically to seek ways to link ostensibly objective indicators of success, such as student grade point averages and test scores, with their library usage experiences. In addition, the report noted heavy investments from academic libraries in library instruction to students but a general lack of programwide and longitudinal assessments that measured real learning resulting from those instruction sessions that might collectively in turn communicate library value more concretely to external stakeholders.2 An additional area of interest from the report focused on seeking to measure students’ experiences with the library, including their “attitudes, and perceptions of quality” through the collection of data.
Quantifying the relationship between library usage and student success is one of the six areas recommended for further research in the 2017 ACRL report, Academic Library Impact: Improving Practice and Essential Areas to Research.3 While many studies have focused on estimating an average relationship across disciplines, while adjusting for disciplinary differences, this study focuses instead on the variation and the uncertainty of the measurement of this relationship across disciplines, providing a fuller set of information than averages, and one that reflects reality better. This approach should help replication when applied in studies at multiple universities. For example, averages at universities focused on science and technology may differ a great deal from averages at universities focused on humanities and social sciences. Describing the relationship as it varies based on group affiliation is accomplished by multilevel regression modeling, which combines a model for the individuals with a model for the groups.
The grouplevel information derived from this study can be used for two purposes: mapping the correlation between library instruction, library resource use and positive attitudes and cumulative GPA; and communicating the value of library instruction and resource use to university administration and faculty, who have observed the relationship between the library and their students within their own discipline, but perhaps not so much for other disciplines. It may be more convincing to see results that reflect their own experience, and it is more informative to study the variations than to limit the results to an average.
Although the results from most published work about the impact of library usage and instruction center around statistical significance, this paper follows the recommendations of the American Statistical Association4 to present estimates with measures of uncertainty and of McShane, Gal, Gelman, Robert, and Tackett5 to “abandon statistical significance.”
Literature Review
Subsequent to the publication of the Value of Academic Libraries Report, an increasing number of library service impact studies began to emerge. In one early study, Wong and Webb6 sought empirical evidence of the positive impact of library service usage on student GPA. The researchers conducted a massive study of more than 8,500 students at Hong Kong Baptist University that analyzed students’ graduation GPA and the number of times these students had checked out material from the library. Results were organized by academic discipline and student rank (undergraduate or graduate), with the finding that 65 percent of the academic subject areas considered showed positive correlations with library checkouts and higher graduation GPA. Subsequent research7 has also concluded that book checkout, library database usage, and ejournal access/usage have positive correlations with student GPA and other forms of academic achievement. It is noteworthy that these libraryuse indicators are all student selfdirected library research activities that require engagement, activity, and often some sort of procedural followthrough from the student.
Other library value studies have often focused on the relationship of library instruction with student success, but with varying results. Wong and Cmor8 investigated the relationship of attending disciplinerelevant library instruction workshops and student graduation GPA, using data from a previous study9 of 8,500 students at Hong Kong Baptist University. Overall, this research found that results varied by student academic disciplines and by number of instruction workshops attended. Data from students in only 24.4 percent of the subject area groups considered showed positive correlations (at varying levels of significance) with a higher graduation GPA. The researchers also found that attending more sessions of disciplinerelevant library instruction increased the likelihood of a stronger positive correlation with higher GPA at graduation.
In a study analyzing 4,489 transcripts, BowlesTerry10 found a very small yet statistically significant positive difference in GPA among students who had been enrolled in an upperlevel course that received courserelated library instruction (CRI) and whose attendance at that instruction session was inferred, while Soria, Fransen, and Nackerud11 found that probable CRI attendance by 5,368 firstyear undergraduates was correlated with a lower GPA; little explanation was given for this outcome other than noting the difficulties of assessing instruction effectiveness. However, a subsequent study by Soria et al.12 used different analysis methods and found opposite results, that “students who engaged in libraries instruction” by attending library workshops, CRI, or a library class had a significantly higher GPA than did those students who had not been involved in library instruction.
Gaha, Hinnefeld, and Pellegrino13 sought to normalize data to account for probable disciplinary grading variations (a multilevel model would make this step unnecessary, even with a small data set). While they found some positive correlations between higher normalized GPA among students who had taken at least one course with library instruction, again there was no control for whether a student had actually attended that instruction session, so conclusions are somewhat speculative. Krieb14 analyzed the impact of CRI and reference desk usage on student retention and GPA at a community college, finding some positive impact of CRI completion on retention but minimal impact on GPA.
Statistical Approaches in Previous Studies
Many big data library value studies have relied heavily on identifying correlations and significance to show which library activities, behaviors, resources, or service usage seem to matter most in terms of supposedly objective student success variables such as GPA. Most of the analyses accounted for the average differences in GPA across majors or colleges. Soria et al.15 did so by allowing varying intercepts for each college but did not allow for varying relationships with the library (different regression slopes for the library usage variables).
Two early library value studies recognized that not only the average GPA varies by major, but that different majors have different relationships with the library;16 for that reason, these studies conducted analyses (Pearson correlation and chisquare tests, respectively) for each discipline group separately. Because some discipline groups had few students, these groups were left out of the analysis. A multilevel regression could have combined both of these analyses into one, while allowing use of data from all discipline groups and using the number of library sessions per discipline as a grouplevel predictor.
Another challenge faced in a number of bigdata instructionvalue studies is how to determine whether a student has indeed received library instruction. One common method has been to crosscheck the institution’s student course enrollment data with library instruction program records of CRI sessions for that course,17 while other studies18 may be able to verify completion of an initial mandatory library orientation session but not attendance at subsequent and potentially more relevant CRI sessions. These methods do not guarantee that the student was actually present at a CRI session. In addition, some researchers state that instruction program records may not be complete and are particularly problematic for highenrollment, multisection courses for which CRI is offered for some but not all sections. Krieb19 described an alternate method of asking students after CRI sessions for their verbal consent to be tracked for research purposes. This method allowed solid identification of both the course and CRI completion, as well as close tracking of individual student retention and GPA success measures. While this method ensures that a student did receive CRI, larger college and university libraries may find it difficult to replicate the individual student consent process for reasons of scale.
Methods
After IRB approval, a survey including undergraduate and graduate students was conducted in 2016 for the purpose of service improvement. Information about the development of this survey, based on the previous 2012 survey, are available in the reports.20 The survey sample was stratified on college and student type (undergraduate or graduate) combination, with higher sampling rates specified for smaller strata. Within each stratum, a systematic sample was drawn, sorting by gender and residency (Iowa, nonIowa US, International) and using a random start, selecting each nth student to reach the specified percentage for that stratum. All regression models included the college and student type sampling strata. Survey sample size and response rates are given in table 1. Details of the statistical analyses are in appendix A.
TABLE 1 

Survey Sample Size and Response Rates 

Undergraduate Students 
Graduate Students 
Postdocs 

Eligible Sample 
2,952 
3,451 
318 
Completed Surveys 
494 
1,116 
117 
RESPONSE RATE 
16.7% 
32.3% 
36.8% 
Three outcome variables were modeled:
 Students’ library resource usage index (LRU) calculated from responses to survey questions about frequency of use of various library resources and services
 Students’ perception of library contribution to academic success (LCS)
 Students’ cumulative GPA
Library Resource Usage Index (LRU)
The LRU index was constructed from survey questions (see appendix B) about the frequency of use of the library website, the library search, article indexes and databases, electronic journals and articles, ebooks, physical items (examples: books, DVDs, CDs, maps, microforms), and of materials from Special Collections and University Archives.
Each of these survey questions had response categories of daily, weekly, monthly, once a semester, less often, and never. The LRU, simulating an estimate of a total average monthly library usage count was created by equating survey responses to estimated counts, as depicted in table 2. These estimated counts were summed over the set of questions, with types of library resources weighted equally. If a student did not answer at least five of these questions, the case was deleted. Sixteen cases were deleted, leaving 1,595 cases for analysis.
TABLE 2 

Conversion of Survey Responses to Estimated Usage per Month 

How often do you do… 
Estimated count 
Daily 
16 times per month 
Weekly 
4 times per month 
Monthly 
1 time per month 
Once a semester 
0.3 times per month 
Less often 
0.2 times per month 
Never 
0 
Evidence that this is a fairly reliable way to estimate the frequency of usage comes from a previous analysis where an estimate of graduate and undergraduate website usage derived in this way was similar to an estimate of undergraduate and graduate website usage derived from a time series regression of web log data.21 The LRU should be regarded as an index or artificial construct. Summing the estimates from the individual resource questions may inflate an estimate of total resource usage; for instance, the respondent may use the library website to find a database, which is then used to find an article, counting the use three times.
Perception of Library Contribution to Success (LCS)
The LCS variable is the binary outcome variable constructed from the response to this question: “To what extent have the library’s resources and services contributed to your academic success?” A response of “very much” was set to 1, and missing or other responses (“some,” “very little,” or “not at all”) were set to 0. Most responses were either “very much” or “some.” Making it a binary variable was a way to simplify the analysis. Since not many answered “very little” or “not at all,” we focused on the respondents who answered “very much.”
Cumulative GPA
There are fewer observations (1,551) for the cumulative GPA analyses22 than for the other analyses because the GPA data was requested retroactively in 2018 and not all cases were matched. The average of the GPAs of the undergraduate survey respondents is 3.05, while the mean for all undergraduates at this institution is 3.03.
Because the range of the GPAs is bounded by 0 and 4 and the distribution is extremely leftskewed, a beta regression with a logit link was used to model the GPAs.23 A beta distribution is a distribution of probabilities between 0 and 1 and can take many shapes, from symmetrical to skewed. The cumulative GPA variable was divided by 4 and transformed so that values were between 0 and 1 and not exactly 0 or 1. The beta regression fits much better and does not predict impossibly high values (see figure 1). The logit link means that, instead of the GPA itself being modeled, a function of the GPA (the logit function) is modeled instead.
Figure 1 

Comparison of Linear Regression to Beta Regression for Modeling Cumulative GPA 





The upper right histogram displays the extreme leftskewness of the cumulative GPA distribution, which is bounded at the upper end at 4.0. The two plots on the left compare the densities of replications (light blue lines) from a linear regression and a beta regression with the distribution of the observations (dark blue line). The replications from the linear regression (top) follow a symmetrical (normal distribution) distribution matching the observations very poorly, while the replications from the beta regression match the observations fairly well. 
Multilevel Regression
In a linear regression, each regression coefficient (slope) may be interpreted as the predicted difference in the outcome variable when the predictor variable is one unit greater (or, for categorical variables, present or not present) while the value of all other predictors remains the same. Predictive does not imply causal, but simply compares the averages of the outcomes for different groups, given different conditions.24
Respondents from various demographic, discipline, classlevel, and graduate students with and without thesis/dissertation requirements may respond differently to questions about the library and have varying relationships with the library. The variation arising from these groups needs to be accounted for in the model so that the importance of other predictors may be compared more directly.25
A common method is to include a dummy variable equal to 1 if a respondent is a member of a particular group and equal to 0 if not. This allows estimates of a different mean or regression intercept for each group. The intercept for each group is estimated separately, so, for a group with few members, the estimate may be unreliable. The result is a set of parallel regression lines (same slope but different intercepts).
An alternative method is a multilevel model, where separate intercepts can be estimated using all the data. Where there are fewer members, the group mean will be closer to the overall mean. This is called partial pooling; it offers a conservative estimate of the differences between groups, alleviating the noisiness and unreliability of small group sample sizes. In addition to differing means of the outcome variable for different groups, there may be different slopes for predictor variables within each group. Although these different slopes for different groups (called interactions) can also be estimated with the dummy variable method, again there is the advantage of partial pooling offering a more conservative and reliable estimate.26 McElreath27 argues that multilevel regression should become the default method of regression, to account for differences between individuals and groups that can’t be measured.
Three possible sets of groups were available for use in the multilevel models. The first, college and graduate/undergraduate type combination, were the sampling strata for the survey and are included to account for the varying probability of selection.
The second group, discipline and graduate/undergraduate type combination, overlaps with college and type. Some colleges at the authors’ institution are represented by one discipline, other colleges contain multiple disciplines whose library usage and attitudes toward library use may differ significantly, and some disciplines are found in multiple colleges.
Disciplines may be a more precise grouping to describe this varying relationship with the library than are colleges. In addition, disciplines make it easier to compare our results to results from other universities, where the college organization may be quite different. The number of respondents in each discipline/type group are given in table 3.
TABLE 3 

Number of Respondents per Discipline 

Number of Respondents 

Discipline 
Graduate 
Undergraduate 
Agriculture & Life Sciences 
123 
121 
Arts & Design 
58 
63 
Business & Economics 
77 
50 
Humanities 
30 
21 
Math & Computer Science 
54 
12 
Physical Science & Engineering 
393 
117 
Social Sciences 
155 
99 
Interdisciplinary 
7 
11 
Veterinary Medicine 
219 
0 
Total 
1,116 
494 
The level of the student (classification 1, 2, 3, and 4 for undergraduates, and master’s degree with no thesis, master’s degree with thesis, and PhD for graduate students) is the third overlapping grouping.
Bayesian methods were used in this study as an alternative to the conventional null hypothesis significance testing and to present ranges of values for parameter estimates. The ranges, or credible intervals, can be used to estimate the probability that an effect is greater or less than zero, or greater or less than the effect for another group, given the data.
Statistical significance calculations are based on a null hypothesis; in reallife social science, treatment effects or differences between groups are highly unlikely to be zero. They are likely to be small and varying among individuals, groups, and situations. Dividing results into categories of significant or nonsignificant by the arbitrary .05 pvalue level often leads to a misunderstanding that a pvalue greater than .05 is proof that there is no treatment effect. On the other hand, too much confidence is placed in low pvalues, as studies with low pvalue results often fail to replicate.28 Recommendations include presenting estimates with Bayesian uncertainty or credible intervals, which present the probability that the value is really within the interval, given the data, with the uncertainty of the estimate.29 Bayesian estimation starts with a “prior” belief about the values of the parameters to be estimated. These priors may come from previous research. The data are then used to update that prior estimate. The result is a distribution of plausible estimates, given the data, called a posterior distribution. In this study, we assume we know nothing and do not use previous research to set priors. The priors used in this analysis are distributions centered at zero, called weakly informative priors,30 which allow the data to have most of the influence while restraining unrealistic estimates and stabilizing the computation. Again, this produces conservative estimates since the starting point is the area near zero, while leaving room to be a reasonable distance from zero as well. The most plausible values of the parameters we are trying to estimate, given the data, are the most frequent values in the posterior distribution.
Newer advanced software, such as the rstanarm R package31 used in this study, and more computing power allow more computationally stable estimation of these complex models than were possible using nonBayesian methods. Muth, Oravecz, and Gabry32 provide a tutorial for using rstanarm and a brief review of Bayesian data analysis, with a more complete explanation of Bayesian methods from Gelman and Hill,33 Kruschke,34 and McElreath.35
To estimate average levels of library resource use (LRU) and of library contribution to success (LCS) within each discipline and type, multilevel regression and poststratification (MRP) was used to adjust for differential nonresponse rates by gender and international status. The multilevel model estimates the average for each college, discipline, type, and demographic combination, which is then weighted by the population for each combination.36 Females made up 43% of the university student population and 54% of the response, while international students made up 12% of the university student population and 31% of the response.
Average Predictive Comparisons
Each of our three outcome variables were modeled with nonlinear models and interactions between variables, making the direct interpretation of regression coefficients difficult, unlike in linear regression. In these situations, average predictive comparisons allow intuitive comparisons to be made.
To make average predictive comparisons, the values at which to compare an input are chosen, the input variable is set to the first value over all observations, predictions made, then the input value is set to the second value overall and predictions made again. The average difference between the set of predictions at the two different values is the average predictive comparison. For twoway interactions, the values of both inputs can be changed to clearly show the predicted differences (Gelman and Hill,37 Gelman and Pardoe38).
Results of the models are communicated primarily in graphical form. This allows the easiest comparison of the size of the effects of the predictors between each other and across groups; it also communicates the amount of uncertainty in the estimates at a glance. Graphs are constructed in the R package ggplot2.39
Results
Levels of library resource usage (LRU) and percentages of those believing the library contributes very much to the respondents’ academic success (LCS) varied quite a bit by discipline. Both were quite a bit higher for graduate students than for undergraduates. LRU and LCS tended to be high in the same groups, with Humanities the highest in usage and nearly the highest in success (see figures 2a and 2b).
Figure 2a 
Average Library Resource Usage Index (LRU) by Discipline 
Average levels of the LRU (sum of estimated uses per month) by discipline for graduate and undergraduate students. Wider bars indicate more uncertainty or a wider range of plausible values for the estimate, caused by fewer respondents in the group or a wider spread in responses. The thicker segments represent the middle 50 percent of the plausible values. Partial pooling shrinks the estimates toward the mean, especially for groups with few respondents. 
Figure 2B 
Library Contributed to Your Success? (by Discipline) 
Average levels by discipline for graduate and undergraduate students for the proportion who believe that the library’s resources and usage contribute “very much” to their academic success (LCS) (right). Wider bars indicate more uncertainty or a wider range of plausible values for the estimate, caused by fewer respondents in the group or a wider spread in responses. The thicker segments represent the middle 50 percent of the plausible values. Partial pooling shrinks the estimates toward the mean, especially for groups with few respondents. 
Our study defined library instruction as courserelated instruction (CRI) or, as the survey put it, a librarian coming to one of their classes. Reporting library instruction in a class predicted higher library resource use (LRU) (see figure 3). International students use the library more than US residents, with or without CRI. Receiving CRI predicts higher usage for both international students and US residents, but more so for international students.
Figure 3 
Average Predicted Difference for Library Resource Usage Index with 
CRI predicts a larger increase in LRU for international graduate students than for US residents. Without CRI, LRU of international students is higher than for US residents (shown as baseline at 0%). CRI also predicts a larger increase in LRU for undergraduates than for graduate students. 
For graduate students, LRU was predicted to be 15 percent higher for US residents reporting courserelated instruction (CRI) than for those without CRI, and 49 percent higher for international students. The difference in LRU for undergraduates reporting CRI was larger than for graduate students, with estimates of 64 percent for US residents and 56 percent for international students (subtracting the two dots on the plot). The estimates for undergraduate international students are very uncertain since there are few of them, so it is not clear that there is a difference in LRU between the two conditions, since the 50% credible intervals denoted by the thick bars overlap.
Gender and perception of library contribution to success (LCS) are held constant in these calculations, and the differences do not vary much by discipline. Female graduate students use the library about 5 percent less than male graduate students, while female undergraduates have about 5 percent higher LRU than males.
Average predictive comparison plots, such as figures 3 and 4, show the size of the predicted difference in the outcome as the value of one predictor is changed, for each predictor listed on the yaxis. The median of the plausible values for the predicted difference is presented as a white dot. The thick gray bars represent the 50 percent credible interval (the middle 50% of the plausible values from the posterior distribution). There is a 50 percent chance that the unobserved parameter is within that range, given the data. The thin lines represent the 90 percent credible interval. The longer the bars and thin lines, the greater the uncertainty of the estimate.
In figure 4, the average difference in LRU between those who believe the library contributes very much to success and those who do not varies quite a bit by discipline, but it is always positive. Veterinary Medicine students with LCS average 200 percent higher LRU than those without, while Physical Sciences & Engineering, Math & Computer Science, and Humanities graduate students with LCS have about 75 percent higher LRU than those without. Among undergraduates, Physical Sciences & Engineering is one of the disciplines with the largest differences (150% higher LRU with LCS), and Business & Economics among the lowest (around 40% higher LRU).
Figure 4 
Difference in Library Resource Usage Index Predicted by “Library Contributes Very Much to Success” (by Discipline) 
Average differences in library resource usage (LRU) when the perception of library contribution to success (LCS) is “very much.” The ranking of disciplines by differences does not match between graduate and undergraduates. For example, LCS predicts 77 percent higher LRU for graduate students in physical sciences and engineering, in the lower end, and 148 percent higher LRU for undergraduates, the highest among the undergraduates. 
International students tended to report both higher LRU than US residents and were also more likely to say the library contributes “very much” to their academic success (LCS), for both undergraduate and graduate students (8% for undergraduates and 10% for graduates, if no CRI is reported). (See figure 5.) Female students were more likely than male students (7% for undergraduates and 9% for graduates) to say the library contributes “very much” to their success.
Figure 5 
Predicted Differences in Probability of Responding “Library Contributes 
Comparing the average predicted probability of saying the library contributes “very much” to the respondents’ academic success (LCS), at 4 times per month to once per month for library resource usage index (LRU) and building usage; for international students compared to US residents, without CRI; and for receiving CRI compared to no CRI, averaged over respondents’ LRU, for both international and US residents. 
While receiving courserelated instruction predicted higher probability of LCS for US residents, this was not true for international students with LRU held constant at four library resource uses per month. International undergraduate students who reported no CRI were more likely to respond that the library contributes “very much” to their success than those who reported CRI. There was no difference for international graduate students (see figure 5). In the line in figure 5 labeled “International with CRI,” undergraduate column, the thick bar is on the left side of the red zero line, indicating that there is 75 percent probability that international undergraduates who receive CRI are less likely to report LCS than international undergraduates who do receive CRI.
Higher frequency of building usage predicts higher probability of LCS for undergraduates (by 11%), but not for graduate students, comparing 4 times per month to 1 time per month. For undergraduates, higher building usage predicts more of a difference in LCS than does LRU. For graduate students, building usage predicted only a slight difference in LCS (see figure 5).
For most undergraduate disciplines, both higher LRU and higher building usage predicted a greater probability of LCS, a little more than 10 percent for building usage and a little less than 10 percent for LRU. Humanities had a larger difference, about 12 percent for LRU and 17 percent for building usage, comparing 4 times per month to 1 time per month (see figure 6). Building usage seemed to be slightly more important for international undergraduates than for US residents.
Figure 6 
Average Predicted Comparisons for “Library Contributes Very Much to My Academic Success” 
For each discipline, the predicted probability of saying that the library contributes “very much” to the respondents’ success, compared at lower and higher levels of LRU and building usage of once per month and 4 times per month. Squares indicate US residents and triangles indicate international students, which shows a little higher LCS with higher LRU for international graduate students. Otherwise, the two groups are very close. Building use seems to be slightly more important for international undergraduates than US resident undergraduates, and for international graduate students in the Arts & Design, Business & Economics, and Veterinary Medicine disciplines. 
For graduate students, the predicted difference in probability for LCS with higher LRU ranged from about 20 to 25 percent for Interdisciplinary, Humanities, and Arts & Design down to 10 percent for Business & Economics and Math & Computer Science graduate students (see figure 6). The predicted difference was a bit higher for international students than US residents, by about 2 to 3 percent in most disciplines.
Model of Cumulative GPA
Cumulative GPA was higher on average for students who used library resources more frequently, but not for undergraduate students who received library instruction. An undergraduate with a library resource usage index (LRU) of 4 times per month averaged a higher GPA by 0.1 than an undergraduate with an LRU of 1, if they had not received courserelated instruction (CRI). The difference was greater than 0 with at least 95 percent probability, given the data. This is seen in figure 7, as the entire 90 percent credible interval (thin bar) is to the right of 0, together with the other 5 percent to the right of the bar. Little to no difference in GPA was predicted for higher LRU for those undergraduates who reported CRI.
Figure 7 
Cumulative GPA Compared at Different Values of Each Predictor 
Comparing the average predicted cumulative GPA at different levels of library resource usage (LRU, once per month vs. 4 times) with courserelated instruction (CRI) vs. none; building use (once per month vs. 4 times); and received CRI compared to no CRI for international students and for US residents. Increased LRU predicts increased GPA for both graduate students and undergraduates with at least 95 percent probability (90 percent credible interval is above zero), but with less than 75 percent probability for undergraduates who received CRI, while increased building usage predicts lower GPA. Receiving CRI predicts lower GPA for both international and US residents by the same amount. 
The average predicted graduate student GPA was higher by about 0.05 at an LRU of 4 times per month vs. 1 time per month. The difference was greater than 0 with at least 95 percent probability, whether or not the student reported receiving library instruction.
More frequent building usage predicted lower GPA, comparing 4 times per month to 1 time per month, for both undergraduates and graduate students (see second line of figure 7). On average, both graduates and undergraduates who had received library instruction in class had the same or slightly lower GPAs than those who had not. Receiving CRI predicted lower GPAs by 0.03 for US undergraduates, although this was estimated with a great deal of uncertainty. There was little to no difference between international students and US residents (see third and fourth lines of figure 7).
In the left column of figure 8, the predicted difference in GPA between those with an LRU of 4 and those with an LRU of 1, averaged over the conditions of receiving CRI or not, varied by discipline. Veterinary Medicine students’ difference of 0.11 was the highest for graduate students. Veterinary Medicine students had a wider range of GPA than other graduate students. Other graduate student disciplines ranged from .03 higher GPA (Business & Economics and Interdisciplinary) to 0.05 higher GPA. There was more variation for undergraduates, ranging from 0 for Math & Computer Sciences to 0.09 for Physical Sciences & Engineering.
Figure 8 
Cumulative GPA Compared at Lower and Higher Levels of Library Resource Usage 
Average difference in predicted cumulative GPA, comparing an LRU of 1 time per month (red baseline at zero) to 4 times per month (left column), and comparing CRI to no CRI, while averaging over range of LRU. Thick segments show the middle 50 percent of plausible values, while thin segments show the middle 90 percent. If the entire 50 percent interval is greater than zero, then there is at least 75 percent probability that the difference is positive (higher GPA). If the entire 90 percent interval is greater than zero, then there is at least 95 percent probability that the difference is positive. 
Because exploratory plots showed a distinct grouping with 0 usage, and because the average of the GPAs of that group seemed to be higher than those with a small amount of usage, a dummy variable for 0 usage was included in the model, which could vary by discipline. For this reason, the lower level of LRU in the comparison in figures 7 and 8 was 1 rather than 0 usage.
The differences in figures 7 and 8 adjusted for gender (females average higher GPA); international status (difference varied by discipline, but was mostly neutral to negative); class level for undergraduates and whether a graduate student was in a nothesis, thesis, or dissertation program; the sampling strata; building use; and average cumulative GPA for class level and college.
A simplified model was used for the scatter plots in figure 9, not adjusting for CRI, building usage, residency, gender, and class level, and using only the zero use indicator and log2 of LRU as predictors, with varying slopes for both in the discipline/student type groups, and including the college/type sampling strata (see appendix A for details).
Figure 9a 
Regression of Cumulative GPA on Library Resource Usage (LRU), 95% Uncertainty Intervals of the Predictor (Graduate Students) 
Scatter plot of cumulative GPA vs. LRU on a logarithmic xaxis for graduate students, with points representing each respondent’s combination. Lines are from the predictions from the model, drawn over the range of data for each discipline. Different intercepts for the college sampling strata result in multiple parallel lines in some disciplines. A zerouse indicator results in a higher prediction of GPA for students who report zero library resource usage and a steeper slope for students who do use library resources. Gray bands represent uncertainty in the estimate of the slope (95% credible intervals). Disciplines are ordered by slope, with the largest increase in GPA predicted by doubling of LRU at the end. 
Figure 9B 
Regression of Cumulative GPA on Library Resource Usage (LRU), 95% Uncertainty Intervals of the Predictor (Undergraduate Students) 
Scatter plot of cumulative GPA vs. LRU on a logarithmic xaxis for undergraduate students, with points representing each respondent’s combination. Lines are from the predictions from the model, drawn over the range of data for each discipline. Different intercepts for the college sampling strata result in multiple parallel lines in some disciplines. A zerouse indicator results in a higher prediction of GPA for students who report zero library resource usage and a steeper slope for students who do use library resources. Gray bands represent uncertainty in the estimate of the slope (95% credible intervals). Disciplines are ordered by slope, with the largest increase in GPA predicted by doubling of LRU at the end. 
The points in figure 9 indicate the combination of the cumulative GPA (yaxis) and the library resource usage index (LRU) (on a logarithmic scale xaxis) for each individual respondent. The regression lines show how the GPA on average increased as the LRU doubled, with the slope of the line varying by discipline. The gray band indicates the 95 percent credible interval for the estimate of the slope of the line. The wide scatter of the points above and below the line at every level of LRU demonstrate that library resource usage did not explain very much of cumulative GPA and that there was a rather weak correlation.
The angle of the line upward to the left at 0 for the zerouse indicator varied by discipline, showing that there were students who reported no library resource usage yet still achieved high grades, more so in undergraduate disciplines, but also in the graduate disciplines of Business & Economics, Math & Computer Science, and Physical Sciences & Engineering.
The Arts & Design and Humanities disciplines were the only graduate disciplines, along with Humanities undergraduates, in which all respondents reported at least some library resource usage.
The slope of the regression line (upward to the right as LRU increases, in most disciplines) tended to be steeper for most undergraduate disciplines and Vet Med students than for other graduate disciplines, as graduate students tend to have a restricted range of cumulative GPA, 3.0–4.0 in most cases. Undergraduates in Math & Computer Science and Business & Economics were the only groups that showed no increase in GPA at higher levels of LRU.
Discussion
Librarians would not expect the correlation between students’ success and their use of the library to be uniform across disciplines; multilevel modeling provides a way to estimate the varying strength of those correlations and levels. Rather than trying to reject a onesizefitsall null hypothesis, this analysis was descriptive, providing estimates along with the uncertainty of those estimates.
The descriptive approach allows us to recognize the possibility that some students don’t use the library and yet do just fine in terms of their GPA. Putting this aside, when students do use library resources, the cumulative GPA is higher on average as library resource use doubles, although this is not so for building usage. For undergraduates, this correlation between LRU and GPA is true only if they did not report receiving CRI in the last year.
As much as we would like to believe that use of library resources leads to higher GPAs, the widely scattered points in the scatterplots (see figure 9) in this study make clear that LRU does not explain very much of GPA. The results are very similar to other analyses such as those from Soria, Fransen, and Nackerud,40 which show a small association between use of library resources and GPA (an increase of .02 in R2). This is not surprising. As Meehl41 put it, the “crud factor” means that, in the social sciences, “everything correlates to some extent with everything else.”
As expected, results varied by discipline. In all disciplines, if students believed the library contributed to their success (LCS), they tended to use resources (LRU) more. In some disciplines, this tendency was more pronounced; for example, LCS predicted 200 percent higher LRU for graduate students in Veterinary Medicine and 150 percent higher LRU for Agriculture & Life Sciences, compared to approximately 75 percent higher LRU for Physical Sciences & Engineering, Humanities, and Math & Computer Science. Results for undergraduate students also varied by discipline, again with LCS predicting more of a difference in LRU in some subject areas. The highest difference among undergraduates was 150 percent higher LRU in Physical Sciences & Engineering. We might posit what these results might mean, but that is beyond the scope of what the survey can tell us. Finally, predictions of CRI’s effect for LRU did not vary much across disciplines.
The survey respondents who said that LRU contributed “very much” to their academic success [LCS] tended to be in the higher end of both LRU and GPA. This study found that frequency of library building usage is predictive of undergraduates believing that the library contributes “very much” to their success, more than LRU, yet data results indicate building use is not predictive of higher GPA. How do students define success and why do undergraduates think that library building use contributes to it? Similarly, why do results from graduate students disagree with these findings?
Educational research has found compelling linkages between students’ sense of belonging with their retention and success;42 this is particularly so with students of color and other minority communities. For undergraduates, might it be that repeated library building use leads to their greater familiarity and comfort, contributing to a sense of “belonging” at the library where one can “fit in” and engage in various scholarly behaviors and activities likely to lead to success? Our study found that international students were 6 to 8 percent more likely than other student groups to agree that the library’s resources and services contribute “very much” to their success, and they also reported higher LRU.
Instruction (CRI) Contributed to Success (LCS) and Library Resource Use (LRU)
Many library value studies have sought to examine the impact of library instruction on student GPA and other success measures. Again, our study defined library instruction as courserelated instruction (CRI) or, as the survey put it, a librarian coming to one of their classes.43 The results of this study are similar to others in that they show modest, if hard to explain, positive correlations after adjusting for LRU between library instruction and student success, defined as student beliefs in whether they felt the library contributed very much to their success (LCS), and negative correlations with GPA, again after adjusting for LRU.
In other words, students who had attended a CRI session were more likely to have a favorable attitude about the library’s role in their success, even though their GPA may not have been similarly impacted, although this was only true for U.S. residents.
Are classes that receive library instruction more likely to be harder or easier for students to receive a good grade? Most disciplines showed a slightly negative or neutral result; however, if there was a positive effect of library instruction on GPA, it may be diluted by the effect of all other college experiences the student receives.
Receiving CRI did strongly predict higher LRU for many student groups, especially international students. These results may be explained by a strong likelihood for courses receiving CRI to have assignments or expectations that require LRU for student assignments or papers. This may also suggest that students learn through CRI about relevant resources as well as the databases and strategies to explore for finding additional resources on their own.
While these results are intriguing and open to interpretation, are these types of library value studies that emphasize bigpicture quantitative data the most productive ways of looking at the value of library instruction? For example, it is difficult for quantitative studies to get at the quality issues that matter in learning and teaching, such as the quality of the collaboration between librarian and course instructor; the type of learning strategies employed; the level of instruction offered and how well it aligns with student and course needs; the expertise of the librarian as teacher; and so on.
Recommendations
It must be recognized that cumulative GPA may be a blunt instrument to measure the effect of library usage or instruction on student success. In what proportion of classes is it plausible that library usage or instruction would help to increase the student’s grade? The effect could be a drop in the pond in a cumulative GPA.
To more fully understand and measure the relationship between library building and resource use and student success, it would be helpful to examine how students define success and how the library assists them to achieve it. A longitudinal study such as a diary study or panel survey could gather such qualitative information, while allowing students to be their own controls, using multilevel models with multiple observations nested in each student. GPAs could also be collected for a more standardized and objective measure to go along with the students’ goals.
The study from Whitmire44 of library usage is an excellent example of a much fuller data set than ours. It has a range of variables, including background characteristics, college experiences, and library activities; one point that is important to note is that it covered three years with the same students. A multilevel model could combine all the data with the three years’ observations nested within each student and students nested within institutions.
Normand45 recommends increasing the number of observations per person over increasing the number of people in the study. With students as their own control, all aspects of their background, personality, social/job life, study habits, and abilities can be controlled for. Using outcomes linked more tightly to the treatments and focusing on variations over time in individual performance rather than averages, such a study could provide much richer information about how and why the library can help student success.
Conclusion
Student attitudes and beliefs about their own library use matter, whether or not their use can objectively be shown to have been positively linked to success outcomes such as higher GPA. This is particularly so with undergraduates in our study. GPA may not be the best measure of whether the library is an aid to student success, although there is a correlation between LRU and GPA. Higher LRU, building use, and CRI does predict student attribution of academic success to the library; by selfreport, it helps some students. It may be of more help to some students than to others either because it is disciplinedependent or because availability of alternatives varies. It’s important also to recognize that students are diverse and do not represent a homogeneous block, as seen with the differences between US residents and international students in this study. Including more studentcentered measures of specific ways the library can aid the student, including studentdefined measures of success, and including more observations per student might bring the issue of library impact on student success into sharper focus.
APPENDIX A. Statistical Analysis
Library Resource Usage Index (LRU)
The Library Resource Usage Index (LRU) emulates count data (nonnegative integers). Either Poisson regression or negative binomial regression is usually used to model count data. Since the variance of the LRU was much larger than the mean, a multilevel negative binomial regression was used.
Variable name: sumnobldg2
Perception of Library Contribution to Success (LCS)
The LCS variable is the binary outcome variable constructed from the response to the question of “To what extent have the library’s resources and services contributed to your academic success?” A response of “very much” was set to 1, and missing or other responses (“some,” “very little,” or “not at all”) were set to 0. It was modeled by a multilevel logistic regression.
Variable name: succ2
Cumulative GPA
The cumulative GPA was modeled with the LCU as one of the predictor variables. For the graduate students, the cumulative GPA did not include spring 2016, while spring 2016 was included for the undergraduates. Because the range of the GPAs is bounded by 0 and 4 and the distribution is extremely leftskewed, a multilevel beta regression with a logit link was used to model the GPAs (Smithson and Verkuilen).46 A beta distribution is a distribution of probabilities between 0 and 1 and can take many shapes, from symmetrical to skewed. The cumulative GPA variable was divided by 4 and transformed so that values were between 0 and 1 and not exactly 0 or 1.
Variable name: GPA.std.beta
Predictor Variables
Female = 1 if female, 0 if male
International = 1 if international student, 0 if US resident
instruct = 1 if “Yes” to Q18 (In the past year, have you … received instruction on using library resources from a librarian in one of your classes), 0 otherwise
log2.sumnobldg: base 2 logarithm of LRU
log2.bldguse: base 2 logarithm of estimate of count of building uses per month
Grouplevel Variables
College.Type: Combination of college and undergraduate or graduate student type. These were the sampling strata.
Discipline.Type: Combination of discipline and undergraduate or graduate student type. Discipline groups could be found in multiple colleges or only one college. Some colleges contained multiple disciplines.
TypeLev: If the type is “Undergraduate” then TypeLev is the undergraduate classification (1, 2, 3, or 4); if the type is “Graduate” then for a nothesis master’s student it is 5; for master’s with thesis it is 6, and for a PhD student it is 7.
Auxiliary Variables
Auxiliary information at the group level can be added to increase the precision of the estimates. This is something that can be done with multilevel modeling but not with a model that uses dummy variables.
The average cumulative GPA for all students in that college and classification (obtained from the university): CollAvg_GPA.std or GrpAvg_GPA.std (averages at the class level for undergraduates). Transformed to range from 0 to 1.
The average library book/material checkout per student in that discipline: circper (values in table A1).
Percentage of courserelated instruction sessions in courses in a discipline: cripct (values in table A2).
TABLE A1 

Values of Auxiliary Variable Circper (number of items checked out per student by discipline and type for FY17) 

Type 
Discipline 
Count Of Items Checked Out 
Enrollment 
Average Circulation Per Student (Circper) 
Graduate 
Ag & Life Sciences 
933 
801 
1.16 
Graduate 
Arts & Design 
1,082 
162 
6.68 
Graduate 
Business & Econ 
575 
467 
1.23 
Graduate 
Humanities 
1,777 
182 
9.76 
Graduate 
Interdisciplinary 
59 
238 
0.25 
Graduate 
Math & Comp Sci 
867 
478.5 
1.81 
Graduate 
Phys Sci & Engineering 
3,385 
1656 
2.04 
Graduate 
Social Sciences 
1,392 
844 
1.65 
Graduate 
Vet Med 
889 
747 
1.19 
Undergraduate 
Ag & Life Sciences 
3261 
5,536.5 
0.59 
Undergraduate 
Arts & Design 
3,243 
1,946.5 
1.67 
Undergraduate 
Business & Econ 
2,058 
4964 
0.41 
Undergraduate 
Humanities 
2,988 
1,828.5 
1.63 
Undergraduate 
Interdisciplinary 
402 
1,299.5 
0.31 
Undergraduate 
Math & Comp Sci 
908 
1,398.5 
0.65 
Undergraduate 
Phys Sci & Engineering 
4,577 
8,404.5 
0.54 
Undergraduate 
Social Sciences 
2,484 
4,124.5 
0.60 
TABLE A2 

Values of Auxiliary Variable Cripct (percentage of course sections with CRI by type and discipline for FY16) 

Type 
Discipline 
CRI Sessions 
Course Section Count 
Percentage Of Course Sections With CRI (Cripct) 
Graduate 
Ag & Life Sciences 
3 
1,153 
0.26 
Graduate 
Arts & Design 
1 
303 
0.33 
Graduate 
Business & Econ 
1 
272 
0.37 
Graduate 
Humanities 
2 
181 
1.10 
Graduate 
Interdisciplinary 
13 
176 
7.39 
Graduate 
Math & Comp Sci 
2 
342 
0.58 
Graduate 
Phys Sci & Engineering 
8 
1,055 
0.76 
Graduate 
Social Sciences 
5 
668 
0.75 
Graduate 
Vet Med 
0 
83 
0.00 
Undergraduate 
Ag & Life Sciences 
30 
1,773 
1.69 
Undergraduate 
Arts & Design 
1 
1,126 
0.09 
Undergraduate 
Business & Econ 
6 
952 
0.63 
Undergraduate 
Humanities 
24 
1,448 
1.66 
Undergraduate 
Interdisciplinary 
2 
939 
0.21 
Undergraduate 
Math & Comp Sci 
0 
921 
0.00 
Undergraduate 
Phys Sci & Engineering 
1 
2,752 
0.04 
Undergraduate 
Social Sciences 
29 
1,583 
1.83 
Undergraduate 
Vet Med 
0 
135 
0.00 
Appendix B: Survey Instrument. ISU Library User Survey 2016
Variable Sample Information
CaseID Respondent’s Case ID#
Date Survey Completion Date
Status Survey Completion Status
Type Graduate Student, Undergraduate Student, or Postdoc
G = Graduate Student
UG = Undergraduate Student
PD = Postdoc
College Respondent’s College
A = Agriculture
ADMIN = Administration
C = Design
E = Engineering
H = Human Sciences
M = Business
S = Liberal Arts and Sciences
U = Graduate
V = Veterinary Medicine
Department Respondent’s Department
Gender Respondent’s Gender
F = Female
M = Male
Residency Respondent’s Residency Status
Age Respondent’s Age (Not available for postdocs)
RaceEthnicity Respondent’s Race / Ethnicity (Not available for postdocs)
FullPart Respondent’s Full or Parttime Status (Students only)
Degree Respondent’s Degree Program (Graduate students only)
Classification Respondent’s Classification Year (Undergraduate students only)
1 = Freshman
2 = Sophomore
3 = Junior
4 = Senior
AdmitType Respondent’s Admission Type (Undergraduate students only)
1 = Direct from High School
2 = Transfer
VARIABLE SURVEY
Q1. Which library or reading room do you use most?
1 = Parks Library
2 = Veterinary Medical Library
3 = Design Reading Room
4 = I don’t use any library spaces or reading rooms
Q2 – Q4. Please indicate how often you do the following activities.
1 = Daily
2 = Weekly
3 = Monthly
4 = Once a semester
5 = Less often
6 = Never
Q2. Go to the library building or reading room that you use most
Q3. Use the Library website
Q4. Use an external website (e.g., Google, Google Scholar, Bing, YouTube, etc.) for your work
Q5 – Q7. Please indicate how often you use the following types of materials in your coursework or research.
1 = Frequently
2 = Occasionally
3 = Never
4 = No Coursework
Q5. IF UG or G/P (not PD): Online materials provided by your instructor
Q6. Online materials discovered by yourself
Q7. Physical items (books, DVDs, print journals) provided by the ISU library
IF UNDERGRADUATE STUDENT (UG), SKIP Q812, GO TO Q13.
IF GRAD/PROFESSIONAL STUDENTS/POSTDOCS (G/P/PD), ASK Q812.
Q8. Do you teach undergraduate students?
1 = Yes (Go to Q912)
2 = No (Go to Q13)
IF Q8 = YES:
Q9 – Q11. Please indicate your overall satisfaction with your students’ performance in the following areas.
1 = Low
2 = Medium
3 = High
4 = Not sure
Q9. Undergraduate students’ ability to find and use appropriate information for assigned activities
Q10. Undergraduate students’ ability to ethically use information for assigned activities
Q11. Undergraduate students’ ability to create a standard citation or bibliography
Q12 – Q16. Please indicate your satisfaction with your own skills listed below.
1 = Low
2 = Medium
3 = High
4 = Not sure
Q12. IF G/P/PD & Q4 = YES (Teach): Your ability to find and use appropriate information for your teaching
Q13. IF G/P/PD: Your ability to find and use appropriate information for your research
Q14. IF UG or G/P (not PD): Your ability to identify and find appropriate information for assigned activities or coursework
Q15. ASK ALL: Your understanding of plagiarism
Q16. ASK ALL: Your ability to create a standard citation or bibliography
Q17 – Q19. In the past year, have you participated in the following tasks related to your ISU coursework or research?
1 = Yes
2 = No
3 = Not sure
Q17. ASK ALL: Asked a library staff member for assistance
Q18. IF UG or G/P (not PD): Received instruction on using library resources from a librarian in one of your classes
Q19. IF G/P/PD: Used bibliographic management software provided by the library or other resources to manage your sources (e.g., EndNote Web, cite features in databases, Zotero, etc.)
Q20. How often have you used Quick Search, the main search bar on the Library website home page, to find library materials?
1 = Daily
2 = Weekly
3 = Monthly
4 = Once a semester
5 = Less often
6 = Never
IF Q20 = 15, GO TO Q22
IF Q20 = 6 (NEVER), GO TO Q21, SKIP Q2224, AND GO TO Q25.
Q21. IF Q20 = NEVER: How do you usually find the information that you need?
[Open text response]
Q22. IF Q20 = 1, 2, 3, 4, or 5: What is your primary reason for using Quick Search?
1 = Search for a specific title or author
2 = Search for topics
3 = Both of these equally
Q23. IF Q20 = 1, 2, 3, 4, or 5: Overall, how satisfied are you with Quick Search results when you look for a specific title or author?
1 = Low satisfaction
2 = Medium satisfaction
3 = High satisfaction
Q24. IF Q20 = 1, 2, 3, 4, or 5: Overall, how satisfied are you with Quick Search results when you are exploring general topics or subjects?
1 = Low satisfaction
2 = Medium satisfaction
3 = High satisfaction
Q25 – Q29. How often do you use the Library website for the following purposes?
1 = Daily
2 = Weekly
3 = Monthly
4 = Once a semester
5 = Less often
6 = Never
Q25. Use article indexes and databases (e.g., Academic Search Premier (EBSCO), Web of Science, JSTOR, etc.)
Q26. Find and use electronic journals and/or articles
Q27. Find and use ebooks
Q28. Find physical items (e.g., books, DVDs, CDs, maps, microforms, etc.)
Q29. Find materials in Special Collections and University Archives (e.g., manuscripts, rare books, etc.)
Q30 – Q36. How satisfied are you with the following ISU Library resources and services?
1 = Low
2 = Medium
3 = High
4 = Don’t use
Q30. Remote (offcampus) access to library materials and services through the Library website
Q31. Assistance or help from library staff through Chat or email
Q32. Reserve a library space (e.g., group study room, multimedia production studio, or to practice a presentation)
Q33. Research and Course Guides
Q34. Interlibrary Loan and document delivery
Q35. IF G/P/UG: Course Reserves
Q36. IF G/P/PD: Library workshops and seminars supporting your research (e.g., bibliographic management tools such as EndNote, funding resources, using the Digital Repository, etc.)
IF UG, GO TO Q40.
IF G/P/PD, ASK Q3739:
Q37. Digital Repository @ Iowa State University provides free and open access to scholarly and creative works, research, publication, and reports by Iowa State’s faculty, students, staff and administrative units.
Are you familiar with the Library’s Digital Repository?
1 = Yes
2 = No
Q38. IF Q37 = YES: Have you used the Digital Repository to search?
1 = Yes
2 = No
Q39. IF Q37 = YES: Have you added your publications, presentations, research, or thesis to the Digital Repository?
1 = Yes
2 = No
Q40. What suggestions do you have to improve the Library’s website, including Quick Search, functionality, ease of use, services, online collections, etc.?
[Open text response]
IF P (VET MED), SKIP Q4161 AND GO TO Q6268.
IF UG/G/PD, ASK Q4154:
Q41 – Q54. Please indicate your overall satisfaction with each of the following resources provided by Parks Library.
1 = Low
2 = Medium
3 = High
4 = Don’t use
Q41. Library staff assistance at the Circulation/Research Help Desk
Q42. Computers and printers available for public use
Q43. Scanners available for public use
Q44. Quality of lighting
Q45. Access to electrical outlets and chargers
Q46. Availability of space to work collaboratively
Q47. IF UG: Reserved group study rooms on third floor
Q48. Availability of space to work individually
Q49. IF G: Assigned library research study rooms for private research
Q50. Multimedia production studios and equipment (Learning Connections Center)
Q51. Learning Connections Center Help Desk for technology assistance
Q52. Parks Library Bookends Café and Fireplace Reading Room
Q53. Special Collections and Archives
Q54. Media Center
IF DESIGN COLLEGE, ASK Q5561:
Q55 – Q61. Please indicate your overall satisfaction with each of the following resources provided by the Design Reading Room.
1 = Low
2 = Medium
3 = High
4 = Don’t use
Q55. Library staff assistance at the front desk
Q56. Computers and printers available for public use
Q57. Scanners available for public use
Q58. Quality of lighting
Q59. Access to electrical outlets and chargers
Q60. Availability of space to work collaboratively
Q61. Availability of space to work individually
IF VET MED, ASK Q6268:
Q62 – Q68. Please indicate your overall satisfaction with each of the following resources provided by the Veterinary Medical Library.
1 = Low
2 = Medium
3 = High
4 = Don’t use
Q62. Library staff assistance at the front desk
Q63. Computers and printers available for public use
Q64. Scanners available for public use
Q65. Quality of lighting
Q66. Access to electrical outlets and chargers
Q67. Availability of space to work collaboratively
Q68. Availability of space to work individually
Q69. What suggestions do you have to improve library services, spaces, or resources?
[Open text response]
IF G/P/PD, ASK Q70 & 71:
Q70 – Q71. The Library wants to ensure that its services support graduate/professional student and postdoc success. How would you rate the importance of the following future initiatives for graduate students, professional students, and postdocs?
1 = Low importance
2 = Medium importance
3 = High importance
4 = Need more information
Q70. Develop a trusted online repository to store and provide continuing access to data created by graduate and professional students and postdocs
Q71. Provide advisory services for research data management
Q72. To what extent have the Library’s resources and services contributed to your academic success?
1 = Very much
2 = Some
3 = Very little
4 = Not at all
IF POSTDOC, ASK Q7374:
*Q73. What is your gender?
1 = Male
2 = Female
*Q74. Are you from…
1 = Iowa
2 = Another U.S. state
3 = Another country (International)
Q75. Please record any additional comments or suggestions about ISU Library resources or services in the space below.
[Open text response]
Q76. Would you like to be entered into the drawing for a $20 gift card for any ISU oncampus dining location? (50 gift cards will be awarded.)
1 = Yes (Go to Q77)
2 = No
**Q77 IF Q76 = Yes: Drawing winners will be notified electronically. Please enter your preferred email address:
[Open text response]
Notes
The data set “use2016” contained all student responses from the survey (1,595 responses), except those with missing values for the LRU. The data set “GPAuse” was somewhat smaller (1,551 responses), as not all respondents could be matched with their cumulative GPA.
Default values of priors in rstanarm were used. The notation (1GroupName) indicates that varying intercepts were estimated for each group, while the notation (VariableGroupName) indicates varying intercepts and varying slopes for the variable.
R Code for Models Used for Each Figure
library(rstanarm)
FIGURE 2 (left)
SColType.sumno.disc.MRP.3 < stan_glmer(sumnobldg2 ~ Female.International + circper + cripct + (1College.Type) + (1discipline.Type), family = neg_binomial_2, adapt_delta=0.95, data=use2016[,c(“sumnobldg2”,”Female”,”International”,”Female.International”,”circper”,”cripct”,”College.Type”,”discipline.Type”)], seed = SEED, QR=TRUE)
FIGURE 2 (right)
SColType.succ.disc.MRP.2 <stan_glmer(succ2 ~ Female.International + cripct + circper + (1College.Type) + (1discipline.Type), family = binomial(link=“logit”), adapt_delta=0.95, data=use2016[,c(“succ2”,”Female”,”International”,”Female.International”,”cripct”,”circper”,”College.Type”,”discipline.Type”)], seed = SEED, QR=TRUE)
FIGURE 3
Average Predictive Comparisons for LRU
SColType.sumno.disc.simple.2 < stan_glmer(sumnobldg2 ~ succ2*Type + Female*Type + instruct*International*Type + (1TypeLev) + (1College.Type) + (1discipline.Type),
family = neg_binomial_2, adapt_delta=0.95, data=use2016[,c(“Type”,”sumnobldg2”,”succ2”,”Female”,”instruct”,”International”,”circper”,”TypeLev”,”College.Type”,”discipline.Type”)], seed = SEED, QR=TRUE)
FIGURE 4
LCS and Received CRI Effect on Library Resource Usage
SColType.sumno.disc.simple.3 < stan_glmer(sumnobldg2 ~ succ2 + Female + instruct*International +
circper + (1TypeLev) + (1College.Type) + (succ2 + instruct*Internationaldiscipline.Type), family = neg_binomial_2, adapt_delta=0.99, data=use2016[,c(“sumnobldg2”,”succ2”,”Female”,”instruct”,”International”,”circper”,”TypeLev”,”College.Type”,”discipline.Type”)], seed = SEED, QR=TRUE)
FIGURE 5 and FIGURE 6
Average Predictive Comparisons for “Library Contributes Very Much to Academic Success”
SColType.succ.disc.Intl.All.44 <stan_glmer(succ2 ~ log2.bldguse*International + Female + instruct*International + instruct*log2.sumnobldg + International*log2.sumnobldg + cripct + circper + (log2.bldguse*International + instruct*log2.sumnobldgTypeLev) + (1College.Type)+ (instruct*log2.sumnobldg + International*log2.sumnobldgdiscipline.Type), family = binomial(link=“logit”), adapt_delta=0.999, data=use2016, seed = SEED, QR=TRUE)
FIGURE 7 and FIGURE 8
Average Predictive Comparisons for Cumulative GPA
SColType.GPA.Beta.Zero.Type.34 < stan_glmer(GPA.std.beta ~ sumnobldg.0 + International*log2.bldguse + Female + instruct*log2.sumnobldg + instruct*International + International*log2.sumnobldg + GrpAvg_GPA.std + (1TypeLev) + (1College.Type) + (sumnobldg.0 + International*log2.bldguse + instruct*International + instruct*log2.sumnobldg + International*log2.sumnobldgdiscipline.Type), family=mgcv::betar(link=“logit”), adapt_delta=0.99, data=GPAuse, seed = SEED, QR=TRUE)
FIGURE 9
Model for GPA Scatter Plot
SColType.GPA.Beta.Zero.disc.13 < stan_glmer(GPA.std.beta ~ sumnobldg.0 + log2.sumnobldg + CollAvg_GPA.std + (1College.Type) + (sumnobldg.0 + log2.sumnobldgdiscipline.Type), family=mgcv::betar(link=“logit”), adapt_delta=0.95, data=GPAuse, seed = SEED, QR=TRUE)
Function for Creating Average Predictive Comparisons
Used for the analyses of the LCS and GPA outcome variables, both of which modeled a logit function of the outcome variable. The rstanarm function posterior_linpred with transform=TRUE transforms the outcome variable back to probabilities for LCS and for GPA, back to a 0 to 1 scale (afterwards multiplied by 4). The function compares the outcome at two different values of one predictor, resulting in two posterior distributions which are differenced (ME). The predicted differences are averaged over the values of the other predictors present in the survey responses (AME). Credible intervals are formed by taking the quantiles of the posterior distribution (QME).47
 nd: data set used to make predictions
 type: graduate or undergraduate
 d: a group such as discipline, or “All”
 mfit: model fit used to predict
 v: variable for average predictive comparison, such as “instruct” or “LRU”
 lo and hi: comparison levels of variable
 subgrp: To compare interactions of two predictors, first all cases are set to the first subgroup in a set, the function is applied, then all cases are set to the second subgroup. Subgroup sets include “International” and “US Resident”, or “Received CRI” and “No CRI”.
effconplot.fun < function(nd,type,grp,mfit,v,lo,hi,subgrp){
nd[,v] < lo
### transform=TRUE returns probabilities
p0 < posterior_linpred(mfit, newdata=nd, transform=TRUE)
nd[,v] < hi
p1 < posterior_linpred(mfit, newdata=nd, transform=TRUE)
ME < p1  p0
AME < rowMeans(ME)
QME < quantile(AME, c(.025,.25,.5,.75,.975))
MADME < mad(ME)
SDME < sd(ME)
eff < data.frame(grp.label=grp, grp.type = type, L95=QME[1],L50=QME[2],Med=QME[3],U50=QME[4],U95=QME[5],MADME=MADME,SDME=SDME,treat=v,subgrp)
return(eff)
}
Notes
1. Association of College and Research Libraries (ACRL), The Value of Academic Libraries: A Comprehensive Research Review and Report, prepared by Megan Oakleaf (Chicago, IL: ACRL, 2010).
2. ACRL, The Value of Academic Libraries, 14.
3. Association of College and Research Libraries (ACRL), Academic Library Impact: Improving Practice and Essential Areas to Research, prepared by Lynn Silipigni Connaway, William Harvey, Vanessa Kitzie, and Stephanie Mikitish of OCLC Research (Chicago, IL: ACRL, 2017), 1.
4. Ronald L. Wasserstein and Nicole A. Lazar, “The ASA’s Statement on Pvalues: Context, Process, and Purpose,” American Statistician70, no. 2 (2016): 129–33.
5. Blakeley B. McShane et al., “Abandon Statistical Significance,”American Statistician73, no. sup1 (2019): 235–45.
6. Shun Han Rebekah Wong and T.D. Webb, “Uncovering Meaningful Correlation between Student Academic Performance and Library Material Usage,”College & Research Libraries72, no. 4 (2011): 361–70.
7. Krista M. Soria, Jan Fransen, and Shane Nackerud, “Library Use and Undergraduate Student Outcomes: New Evidence for Students’ Retention and Academic Success,”portal: Libraries and the Academy13, no. 2 (2013): 147–64; Graham Stone and Bryony Ramsden, “Library Impact Data Project: Looking for the Link between Library Usage and Student Attainment,”College & Research Libraries74, no. 6 (2013): 546–59.
8. Shun Han Rebekah Wong and Dianne Cmor, “Measuring Association between Library Instruction and Graduation GPA,”College and Research Libraries72, no. 5 (2011): 464.
9. Wong and Webb, “Uncovering Meaningful Correlation between Student Academic Performance and Library Material Usage,” 361–70.
10. Melissa BowlesTerry, “Library Instruction and Academic Success: A MixedMethods Assessment of a Library Instruction Program,”Evidence Based Library and Information Practice7, no. 1 (2012): 82.
11. Soria, Fransen, and Nackerud, “Library Use and Undergraduate Student Outcomes.”
12. Krista M. Soria, Jan Fransen, and Shane Nackerud, “Beyond Books: The Extended Academic Benefits of Library Use for FirstYear College Students,”College & Research Libraries78, no. 1 (2017).
13. Ula Gaha, Suzanne Hinnefeld, and Catherine Pellegrino, “The Academic Library’s Contribution to Student Success: Library Instruction and GPA,”College & Research Libraries79, no. 6 (2018): 737.
14. Dennis Krieb, “Assessing the Impact of Reference Assistance and Library Instruction on Retention and Grades Using Student Tracking Technology,”Evidence Based Library and Information Practice13, no. 2 (2018): 2–12.
15. Soria, Fransen, and Nackerud, “Beyond Books,” 8–22.
16. Wong and Webb, “Uncovering Meaningful Correlation between Student Academic Performance and Library Material Usage,” 361–70; Wong and Cmor, “Measuring Association between Library Instruction and Graduation GPA,”464.
17. BowlesTerry, “Library Instruction and Academic Success,”82–95; Gaha, Hinnefeld, and Pellegrino, “The Academic Library’s Contribution to Student Success,”737; Soria, Fransen, and Nackerud, “Library Use and Undergraduate Student Outcomes,”147–64.
18. Wong and Cmor, “Measuring Association between Library Instruction and Graduation GPA,”464.
19. Krieb, “Assessing the Impact of Reference Assistance and Library Instruction on Retention and Grades Using Student Tracking Technology,” 2–12.
20. Sarah Passoneau, 2012 Library User Survey Report (Ames: Iowa State University Library, 2012), available online at https://assess.lib.iastate.edu/sites/default/files/Survey%20Uploads/Final%202012%20Library%20User%20Survey%20Report.pdf [accessed 15 January 2020]; Linda Anderson et al., 2016 Library Student and Postdoc User Survey (Ames: Iowa State University Library, 2016), available online at https://assess.lib.iastate.edu/sites/default/files/Survey%20Uploads/2016%20Library%20Student%20and%20Postdoc%20User%20Survey%20Report%20%20December%202016%20%20Final%20(4517).pdf [accessed 15 January 2020].
21. Linda Anderson, “Library Website Visits and Enrollment Trends,”Evidence Based Library and Information Practice11, no. 1 (2016): 4.
22. For the graduate student respondents, the cumulative GPA did not include Spring 2016, while Spring 2016 was included for the undergraduates.
23. Michael Smithson and Jay Verkuilen, “A Better Lemon Squeezer? MaximumLikelihood Regression with Betadistributed Dependent Variables,”Psychological methods11, no. 1 (2006): 54.
24. Andrew Gelman and Jennifer Hill,Data Analysis Using Regression and Multilevel/Hierarchical Models (Cambridge, UK: Cambridge University Press, 2006).
25. Richard McElreath, Statistical Rethinking: A Bayesian Course with Examples in R and Stan (Boca Raton, FL: Chapman and Hall/CRC, 2018).
26. Gelman and Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models.
27. McElreath, Statistical Rethinking.
28. McShane et al., “Abandon Statistical Significance,”235–45.
29. Wasserstein and Lazar, “The ASA’s Statement on Pvalues,”129–33.
30. Gelman and Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models.
31. Jonah Gabry and Ben Goodrich, “rstanarm: Bayesian Applied Regression Modeling via Stan, R Package Version 2.18. 2” (2018), https://mcstan.org/rstanarm [accessed 7 March 2020].
32. Chelsea Muth, Zita Oravecz, and Jonah Gabry, “Userfriendly Bayesian Regression Modeling: A Tutorial with rstanarm and shinystan,” Quantitative Methods for Psychology 14, no. 2 (2018): 99–119.
33. Gelman and Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models.
34. John Kruschke, Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan (Cambridge, MA: Academic Press, 2014).
35. McElreath, Statistical Rethinking.
36. Gelman and Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models.
37. Gelman and Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models.
38. Andrew Gelman and Iain Pardoe, “2. Average Predictive Comparisons for Models with Nonlinearity, Interactions, and Variance Components,”Sociological Methodology 37, no. 1 (2007): 23–51.
39. Hadley Wickham and Winston Chang, “Package ‘ggplot2’,” Create Elegant Data Visualisations Using the Grammar of Graphics, Version 2, no. 1 (2016): 1–189.
40. Soria, Fransen, and Nackerud, “Beyond Books,”8–22.
41. Paul E. Meehl, “Why Summaries of Research on Psychological Theories Are Often Uninterpretable,”Psychological Reports 66, no. 1 (1990): 195–244.
42. Terrell L. Strayhorn, College Students’ Sense of Belonging: A Key to Educational Success for All Students (AbingdononThames, UK: Routledge, 2018); Silas M. Oliveira, “The Academic Library’s Role in Student Retention: A Review of the Literature,”Library Review 66, no. 4/5 (2017): 310–29.
43. Our study asked students themselves to indicate whether they received library instruction. While this approach is not problemfree, as it does not delve into quality or level of instruction and relies as well on student memory, it avoids the problems of inferred CRI attendance.
44. Ethelene Whitmire, “The Relationship between Undergraduates’ Background Characteristics and College Experiences and Their Academic Library Use,”College & Research Libraries 62, no. 6 (2001): 528–40.
45. Matthew P. Normand, “Less Is More: Psychologists Can Learn More by Studying Fewer People,”Frontiers in Psychology 7 (2016): 934.
46. Michael Smithson and Jay Verkuilen, “A Better Lemon Squeezer? MaximumLikelihood Regression with BetaDistributed Dependent Variables,”Psychological Methods 11, no. 1 (2006): 54, https://doi.org/10.1037/1082989X.11.1.54.
47. Ben Goodrich, “Calculating Marginal Effects in Binomial Logit Using rstanarm,” Stackoverflow.com, available online at https://stackoverflow.com/questions/45037485/calculatingmarginaleffectsinbinomiallogitusingrstanarm/45042387 [accessed 24 October 2019]; Ben Goodrich, “Calculating Credible Intervals for Marginal Effects in Binomial Logit Using rstanarm,” Stackoverflow.com, available online at https://stackoverflow.com/questions/48431893/calculatingcredibleintervalsformarginaleffectsinbinomiallogitusingrsta [accessed 24 October 2019].
This work is licensed under a Creative Commons AttributionNonCommercial 4.0 International License.
Article Views (Last 12 Months)
Contact ACRL for article usage statistics from 2010April 2017.
Article Views (By Year/Month)
2022 
January: 83 
February: 128 
March: 74 
April: 70 
May: 56 
2021 
January: 42 
February: 42 
March: 56 
April: 52 
May: 29 
June: 52 
July: 75 
August: 38 
September: 50 
October: 63 
November: 67 
December: 47 
2020 
January: 0 
February: 0 
March: 0 
April: 250 
May: 44 
June: 45 
July: 28 
August: 46 
September: 34 
October: 35 
November: 43 
December: 36 