A Three-Year Mixed Methods Study of Undergraduates’ Information Literacy Development: Knowing, Doing, and Feeling

Ellen Nierenberg, Mariann Solberg, Torstein Låg, and Tove Irene Dahl*

This article reports results of a mixed-methods study following the development of undergraduates’ information literacy over three years. Information literacy knowledge and skills in this sample (n = 116) increased with time, as did information literacy attitudes when measured by interest and information literacy’s perceived usefulness and importance. Correlations among students’ information literacy knowledge, skills, and attitudes also increased with time, implying a progressively stronger integration of the three. Complementary interviews with 13 students revealed that they became more interested in being information literate. Some experienced an identity change as a result of this development, indicating that transformative information literacy learning can occur.

Introduction

The term post-truth era is often used to describe modern society, in which “objective facts are less influential in shaping public opinion than appeals to emotion and personal belief.”1 The term post-truth became common after the 2016 US presidential election and Brexit referendum in the UK, when misinformation spread by public figures became more prevalent. Of course, misinformation arises from and is distributed by other sources as well, including artificial intelligence-based chatbots. These major challenges in our information environment make it difficult to distinguish between fact and fiction and underscore the importance of information literacy (IL) skills, such as the ability to critically evaluate information.

Information literacy is defined in this study as encompassing, “the knowledge, skills, and attitudes needed to be able to discover, evaluate, and use information sources effectively and appropriately in order to answer questions, solve problems, create knowledge, and learn.”2 As expressed in the Association of College & Research Libraries’ Framework for information literacy in higher education3 [ACRL Framework], these competencies are composed of cognitive, behavioral, and affective elements. They are essential in education, the workplace, and daily life, for navigating the information landscape in the post-truth world, for critically evaluating information, and for becoming responsible citizens and reflective, lifelong learners.

In higher education, academic librarians and teaching faculty have long contributed to students’ acquisition of critical thinking skills and other necessary information competencies through information literacy instruction and assessment. We believe it is important to facilitate students’ development beyond simply learning to perform information literacy skills and on toward becoming information literate adults. The aim of this study is to explore how students’ information literacy competencies develop, as well as how their perceptions of themselves as information literate people evolve during their college years. Are there glimmers of transformative information literacy learning or notable changes in students’ identities as information literate people?

There are two main research questions in this study:

1. How do undergraduates’ information literacy knowledge (know), skills (do), and interest in being or becoming information literate (feel)—and their interaction—change during the first three years of higher education?
2. Is there any evidence of transformative information literacy learning in students? If so, what does it look like, and how do students experience it in relation to their identity as information literate people?

We used a pragmatic, emergent mixed methods approach to examine the development of information literacy knowledge, skills, and feelings—referred to as knowing, doing, and feeling—in undergraduates over three years. Mixed methods research involves employing and integrating quantitative and qualitative methods to reach deeper understandings of research questions.4 Mixing occurs both as each strand’s data, results, and inferences informs the development of the next strand, and in the integrated analyses of final findings. We created new questions in both surveys and interviews if the need arose after analyzing previously collected data, for example in asking interviewees to explain unexpected survey results.

Theoretical Grounding

Important concepts in our exploration of information literacy development are grounded in interest theory and transformative learning theory. This section describes the study’s theoretical grounding and provides a brief overview of relevant literature.

Information Literacy Development

Studies over time, with or without specific interventions, often focus on one facet of information literacy, such as Rosman et al., who found an increase in students’ information-seeking knowledge over three semesters.5 Broader studies, encompassing several aspects of information literacy development over time, are more challenging to undertake and less prevalent in the literature. There is a need for more research to fill this gap and better understand students’ longer-term information literacy learning. Due to our belief in the importance of a more personally integrated information literacy, we find value in studies that assess information literacy knowing, doing, and feeling, and especially those that explore relationships among the three, over time.

Pinto and Fernández-Pascual explored short-term learning in their quantitative study of Library and Information Science students, where they measured changes in knowledge, skills, motivation (defined as self-efficacy), and attitudes (defined as belief in importance) for a set of information literacy competencies. When measuring these components before and after a one-semester intervention, they found positive, significant relationships among some, but not all, of the competencies. Scores for information literacy knowledge and skills were lower than for self-efficacy, showing a miscalibration between students’ actual and believed skills.6 We found, however, no longitudinal research following information literacy knowing, doing, and feeling over a longer period. To assess information literacy knowledge, skills, and interest, we developed four quantitative measures (described in “Materials”).

Interest and Meta-awareness

Information literacy instructors strive to teach in the best possible manner to help students become information literate individuals. To explore its relationship to information literacy learning, the measurement of feeling in this study is based mainly on interest. Interest is conceptualized as an exploration-related feeling linked to both the psychological state of an individual engaged in certain content, as well as their motivation to continue that engagement. Interest influences learning by increasing both attention, expended effort, cognitive functioning, and the realization of goals.7

Our operationalization of interest is informed by Hidi and Renninger’s four-phase model of interest development. Their model describes ways in which interest changes over time, and thereby has implications for both learning motivation and teaching methods.8 The model describes four qualitatively distinct phases of interest, from triggered situational interest (the most fleeting and situation-dependent) to well-developed individual interest (the most stable and independently pursued). Each phase has a unique psychological architecture described by the relationship among four variables: 1. Situation dependence, 2. Positive affect, 3. Competence, and 4. Meaningfulness.9

The four-phase model was first operationalized in a reliable and conceptually valid way by Dahl and Nierenberg, who tested it both with (i) self-chosen objects of interest, such as playing the bassoon, and (ii) a specific interest, namely interest in being or becoming information literate.10 We use the Tromsø Interest Questionnaire (called TRIQ) to measure interest in being or becoming information literate in the current study.11

Transformative Learning

Transformative learning theory is used as a lens to explore information literacy learning. Transformative learning was introduced in the 1970’s to describe perspective transformations in adults that can occur after critical reflection of previous life experiences, assumptions, and beliefs.12 Mezirow believed that transformative learning represents a paradigm shift, as it induces more comprehensive behavioral changes than other kinds of learning.13 Mezirow’s theory has received criticism for being too cognitively oriented and lacking social and emotional elements of learning, as well as for not adequately capturing what it is that gets transformed in transformative learning.14 Pedagogue Knud Illeris suggests that it is a person’s identity that gets transformed, and proposes a revised definition of transformative learning, namely: “all learning which implies changes in the identity of the learner.”15 Illeris adds that transformative learning, “implies a qualitatively new formation [in a learner] … something more than the acquisition of new knowledge and skills… [which may] include changes and transformations in the learner’s general experience and behavior.”16

Personal identity, according to psychoanalyst Erik Erikson, consists of both our perceptions of ourselves and how others judge us; it forms mainly during youth and remains stable throughout life. Illeris and others, including sociologists Bauman and Giddens, maintain, however, that personal identity can change throughout life, thus supporting Illeris’ supposition that identity is something that can be transformed.17

Transformative learning can be differentiated from other types of learning in several ways. It refers specifically to learning in adults, after critical self-reflection, that leads to profound changes in both our perspectives and behavior.18 In addition, transformative learning is learning that changes our identity,19 and that is irreversible.20

In their literature review examining transformative learning within the context of academic libraries, Hooper and Scharf explore how transformative learning theory can inform librarians’ information literacy teaching. They note that the ACRL Framework for information literacy has clear references to transformative learning, including threshold concepts, defined as, “core or foundational concepts that, once grasped by the learner, create new perspectives and ways of understanding… [and] produce transformation within the learner.”21 Other studies from the perspective of the information literacy educator include Chatterjee and Nichols Hess, who discuss how transformative learning can inform information literacy instructional practice in higher education. Kenney studied transformative learning in public library users, none of whom were students. Hucks et al. combined the perspective of the student and the instructor by exploring how teacher education students displayed evidence of transformative learning when teaching information literacy in practice.22 Other than Hucks et al., we have found no studies that investigate transformative information literacy learning from the student perspective.

Our research thereby fills gaps in the information literacy literature by: 1. studying the development of relationships among information literacy knowledge, practice, and feelings over time; 2. exploring transformational information literacy learning in students (as opposed to educators, who are more commonly studied), what it looks like and how students experience it in relation to their identity as informational literate people; and 3. reestablishing the construct of interest, an important motivator for learning, into the literature.23

The Current Study

We studied the development of information literacy in undergraduates over time, using mixed methods. We employed quantitative methods during the first half of the study. It gradually became clear, however, that these methods alone were not sufficient to answer the research questions. We therefore employed qualitative methods during the second half of the study, enabling us to explore students’ perceptions of their information literacy development more deeply.

This article is divided into three sections: first the quantitative and then the qualitative studies, including methods and results, and lastly the mixed methods section, where the quantitative and qualitative findings are integrated and discussed.

Quantitative Changes in Knowing, Doing, and Feeling Scores

Methods

Study Design

This study measures development in information literacy knowing, doing, and feeling in psychology undergraduates at the world’s northernmost university, The Arctic University of Norway (UiT). We employed four measures, described in “Materials,” to quantitatively examine students’ information literacy development over their first three years. These tools measure their: 1. knowledge of core facets of information literacy (know), 2. skills in evaluating sources (do), 3. skills in using sources (do), and 4. interest in being or becoming information literate (feel).

Using these tools, we collected data cross-sectionally during students’ first and sixth semesters. For some of the sixth-semester respondents, we had also collected data three years previously, at the start of their first semester. Although we did not include these matched-sample, first-semester data in the cross-sectional study, where data from the same individual cannot be repeated, we did use them in a complementary longitudinal, matched-sample study together with sixth-semester data taken from the cross-sectional study for these respondents. The cross-sectional study thereby measured the development of different students at the same point in their educational trajectory, while the longitudinal study followed the same students over time to examine in more detail what they know, do, and feel related to information literacy. In this exploratory research, we were able to identify trends and validate results by comparing results of the cross-sectional and longitudinal studies.24

Materials

The four measures employed to assess information literacy—knowing, doing, and feeling—make up the Tromsø Information Literacy Suite (TROILS) and are freely available for others to use.25 Additional details about the development and testing of the measures, including evidence of their reliability and validity, are described in Nierenberg et al. and Dahl and Nierenberg.26

Knowing. The Tromsø Information Literacy Test (TILT) is a 21-item, multiple-choice, psychometrically evaluated test designed to assess undergraduates’ knowledge of three core, source-based facets of information literacy: finding, evaluating, and using sources.27 There are seven items in each facet, each with four alternative answers, one of which is most correct. Maximum test score is 21 points.

Doing. Students’ information literacy skills in practice are assessed, using mandatory assignments, for source evaluation and source use. These two measures arguably have inherent validity, as they are designed by teachers partly to assess these abilities.

Source evaluation scores are based on mandatory assignments in the first and sixth semesters. Students find scholarly sources for their first-semester final papers, as well as their sixth-semester bachelor’s theses, and describe why they consider each to be a reliable, scholarly source. In this research, three randomly chosen sources from these assignments are analyzed based on three criteria:

1. Quality: how scholarly the source is, on a scale of 0 (not scholarly) to 3 (scholarly).
2. Variety: number of unique source-evaluation criteria in the annotation, for example authority, relevancy, or accuracy. Each specific criterion is worth one point.
3. Frequency: total number of instances source-evaluation criteria appear in the annotation. Each criterion is worth one point.

The total score is the sum of the quality, variety, and frequency scores for the students’ three sources. Scoring was performed by three raters in the first assignment, then averaged. Interrater reliability was found sufficient (intraclass correlation coefficient [ICC] = .89).28 A qualified rater from the first-semester assignment scored the sixth-semester assignment, so all ratings are comparable.

Source use scores, with a maximum of 5 points, are based on students’ first-semester final papers and sixth-semester bachelor’s theses. For both assignments, students were required to incorporate scholarly sources and cite them in APA style.29 The overall use of sources was assessed using five criteria, each worth 0-1 point:

1. Are scholarly sources used to support arguments?
2. Are sources cited in the text when necessary?
3. Are in-text citations written in correct APA style?
4. Is the reference list written in correct APA style?
5. Are all in-text citations listed in the reference list, and vice versa?

The assessment of source use in the first semester was performed by three raters. Raters calibrated their scoring to increase reliability, and interrater reliability was found satisfactory (ICC = .92).30 In the sixth semester, two raters assessed assignments and interrater reliability was again found sufficient (ICC = .93).

Feeling. Defined in terms of interest in being or becoming an information literate person and measured by the Tromsø Interest Questionnaire (TRIQ).31 TRIQ consists of a self-assessed phase of interest and six theoretically based subscales: 1. General interest, 2. Situation dependence, 3. Positive affect, 4. Competence level, 5. Competence aspiration, and 6. Meaningfulness. The subscales are derived from key variables that underly the psychological architecture of each phase of Hidi and Renninger’s four-phase model of interest development.32

Participants

When research began in fall 2019, we recruited psychology students from two programs, a three-year bachelor’s program (n = 75) and a six-year professional studies program (n = 52). All students had the same information literacy instruction, starting with an academic writing course with embedded information literacy in their first semester, and all had instruction in literature search in their third year.

Cross-sectional. We collected data for the cross-sectional study from cohorts that began in 2017-2019. For the 2019-cohort, we analyzed data from those students who participated in the survey (including TILT and TRIQ) in their first or sixth semester. For those who completed the survey in both semesters, we only included sixth semester data in the cross-sectional study, so no data from the same student were duplicated. The 2019-cohort had two mandatory assignments in their first and sixth semesters that we used to measure practical skills in source evaluation and source use (do). Bachelor’s students in cohorts that began in 2017 and 2018 (n = 80) had one comparable source-use assignment at the end of their sixth semester, namely their bachelor’s theses. Since these cohorts had the same information literacy instruction and assessment criteria as the 2019-cohort, we included their theses in the cross-sectional, source-use data. Students from the 2017 and 2018 cohorts, however, had no source-evaluation assignment in their sixth semester, which explains why there are fewer participants than in the source-use assignment for that semester (see table 1).

Matched sample. Matched data in the longitudinal study is from the 2019-cohort, who we followed over three years. Thirty-three of these students completed the survey (know, feel) in their first and sixth semesters. For the assignment-based measures (do), there is matched data only from students in the bachelor’s program: source-evaluation data for sixteen students and source-use data for eighteen students.

Procedure

We collected TILT (know) and TRIQ (feel) data from the 2019-cohort at the start of the first semester and end of the sixth semester.33 We recruited students via their Learning Management System (LMS) and email and we offered prizes for participation.

With students’ informed consent,34 we linked source-evaluation and source-use data (do) from mandatory, written assignments in the first and sixth semesters together with TILT and TRIQ scores. We assessed the use of sources in the first six pages of students’ final exams in semester one and bachelor’s theses in semester six, where the nature of the writing is the same.

We performed data analyses in IBM SPSS Statistics. For the cross-sectional study, independent sample we employed t-tests to compare first- and sixth-semester data from the four measures, as the assumptions for parametric tests were fulfilled in most cases.

For the longitudinal study, the non-parametric Wilcoxon signed rank test with exact statistic for matched samples enabled us to do the simplest, conservative analysis and still determine whether there were changes of note between the first and sixth semesters.35

Results

Cross-sectional Analyses

Semester one and six scores on the know (TILT), do (Source evaluation and Source use), and feel (TRIQ) measures in the cross-sectional study are provided in table 1.

Scores on TILT and the source-evaluation and source-use measures range from two thirds to almost one and a half standard deviations higher in semester six compared to semester one. Independent sample t-tests showed significant increases over time for scores on TILT and the source-evaluation and source-use measures, as expected. Interest phase remained the same over time, while TRIQ subscales show varied results. Significant results include growth in “Competence level” and “Meaningfulness,” which could be expected, and a decline in “Competence aspiration.”

Matched Sample Analyses

Development Over Time

Results of the Wilcoxon signed rank test for matched samples, to examine know, do, and feel over time in the longitudinal group, are found in table 2.

As shown by mean scores and ranks, there were significant improvements for TILT (know) in the longitudinal group. Source-evaluation and source-use scores (do) show growth as well, but this growth is not statistically significant.36 Finally, in terms of what students feel, results from TRIQ subscales indicate similar trends as in the cross-sectional group, with the same subscales increasing or decreasing over time. “Meaningfulness” increased slightly and students reported that their “Competence level” improved significantly. However, their “General interest” decreased significantly, along with their feelings of “Positive affect” and “Competence aspirations,” while, as one would expect with these decreases, their “Situation dependence” significantly increased.

Data from cross-sectional and matched-sample groups are comparable, regarding both values and trends for each variable. The matched-sample results thereby corroborate findings from the cross-sectional sample.

Comparisons Between Knowing, Doing, and Feeling Over Time

To examine how relationships between knowing, doing, and feeling developed between semesters one and six in the cross-sectional sample, we analyzed scores on TILT, the two do-measures, and TRIQ’s “Meaningfulness” subscale. Since each TRIQ subscale measures a different aspect of interest, it gives no meaning to average them, as with the do-measures. We chose the “Meaningfulness” subscale to represent overall interest because: 1. it has positive, significant correlations to students’ interest phase in both semesters; 2. it increases most with interest phase, both in theory37 and in practice;38 3. it contains items that are relevant to students’ interest in being or becoming information literate, as well as to its usefulness and importance; and 4. it is generally recognized in the field of educational psychology that both interest and the motivation to learn increase when activities are found meaningful.

Figure 1 is a qualitative illustration of information literacy growth, from semester one to six, in the cross-sectional data (in tables at the bottom of the figure). All scores were transformed to percentage correct in relation to the highest score achieved in this semester. Circle size represents mean percentage scores for know (TILT), do (average of source-evaluation and source-use percentage scores), and feel (TRIQ Meaningfulness subscale) measures. All three mean scores increased significantly over time. Overlapping areas between circles represent Pearson’s r correlations. These illustrate areas where transformative learning may occur (see “Discussion”). Correlations also increase, though only the do-feel correlational change is statistically significant.39

Qualitative Reflections on Information Literacy Change

Methods

Participants

We collected qualitative data in focus groups and individual interviews. Participants were sixth-semester, undergraduate psychology students from UiT, recruited via their LMS and email. Thirteen students participated in interviews—nine in focus groups and four in individual interviews.

During spring 2021, we conducted focus groups with students who began the psychology bachelor’s program in 2018, who were then in the final semester of the three-year program, writing their bachelor’s theses. There were three students in each of the three focus groups. Seven of the nine participants were women; the two men were in different groups. These students had the same curriculum and information literacy instruction as the cohort that began in 2019.

During spring 2022, we conducted individual interviews with four students who began studying psychology in 2019. Two were enrolled in the three-year bachelor’s program and currently writing a bachelor’s thesis, and two were in a six-year professional study program.40

Materials

Interview guides for focus groups and individual interviews included questions about students’ perceptions of their information literacy development, the importance of information literacy for them, and their interest in being or becoming information literate people. Interview guides for the individual interviews included additional questions related to transformative learning and identity.

We also used an “interest-o-meter” during individual interviews.41 This is an empty graph onto which participants plotted their levels of interest in being, or becoming, an information literate person during the past six semesters. The y-axis shows the intensity of the interest, and the x-axis is the timeline, in semesters. This graph, together with participants’ simultaneous commentary and responses to other interest-related questions, provided us with a richer account of their interest development.

Procedure

Thirteen students volunteered and consented to participate in interviews. At the start of each interview, we presented the procedure, the definition of information literacy used in the project, and information about the participants’ privacy. The lead author conducted focus groups digitally in spring 2021, and we provided all participants the opportunity to answer each question. We conducted individual interviews face-to-face in spring 2022. Interviews were semi-structured, providing the opportunity to pose follow-up questions and allowing students to introduce new topics. We audio recorded and transcribed interviews verbatim, without emphasizing dialect or emotional expressions.

To detect any transformative information literacy learning, we chose to analyze the data thematically. This would help us identity and organize patterns of meaning, thus revealing the semantic content of students’ responses to interview questions regarding their development. Two authors of this article performed the analysis in several stages using Braun and Clarke’s42 six-phase model for thematic analysis,43 one of the most highly cited models for thematically analyzing interview material in the qualitative literature.

After familiarizing ourselves with the data, we began coding the transcripts in NVivo. In the first stage of coding, we employed an inductive approach, creating codes to sort and describe the empirical data. In the next stage, we reorganized codes, splitting some and merging others. This stage was more abductive, informed by theories of transformative learning and identity, and focused on the study’s research questions. The subsequent creation of themes was based on these codes, with the intention of making meaningful patterns in the dataset that would contribute to answering research questions.

When confirming whether these themes were supported by enough data, however, we found that more data pertinent to transformative learning were necessary. We therefore conducted follow-up individual interviews with three of the four participants from the first round, those who showed signs of transformative learning. In this round, this article’s second author interviewed students to see if new questions and a different interviewer’s style could stimulate more detailed responses regarding any transformative learning that may have occurred. The interview began by asking participants to plot their interest levels, per semester, on the interest-o-meter while commenting on their interest development and answering impromptu questions based on their visual representations. Subsequently, the interviewer posed a question based on an unexpected survey finding: “We have seen in the quantitative study that interest in being/becoming information literate decreased over time. Why do you think this might be the case?”

After this last round, we performed a thematic analysis based on codes from all interviews, identifying major themes in the data relevant to our research questions. Results of the interest-o-meter and thematic analysis are presented below.

Interest-o-meter

Use of the interest-o-meter combined graphical, quantitative depictions of three students’ interest in being, or becoming, information literate people over six semesters, with simultaneous oral accounts of this development. Figure 2 shows the three drawings in the same graph. To give a brief snapshot of how students described their experiences, Kari (yellow) explained that her interest in the first semester was medium-low but rose steadily to medium-high by the end of her sixth semester. Magnus (red) drew a gradually tapering upward curve depicting rapid initial growth and slower subsequent growth, with interest levels increasing from low-medium to high. He explained that he was more interested in being information literate than in information literacy in general. Amy’s growth (blue), from low to high interest, was more cyclical, with jumps to higher plateaus each successive academic year. She commented that information literacy is interesting when important, but not as a subject in itself.

Results of Thematic Analysis

Based on transcripts of focus groups and individual interviews, we created sixty-two inductively surmised codes, with up to thirty references per code. We then combined and semantically systematized similar codes into hierarchical categories, yielding twenty-eight main codes and forty subcodes. Based on these, as well as the research questions and the theoretical framework, we deductively identified five major themes. We described these themes below and illustrate them with (translated) quotes from interviews. The themes are presented in the order of the research questions. Names are changed to protect the anonymity of participants.

1. Development in Information Literacy Knowing

When inquiring about their information literacy knowledge development, all participants expressed considerable growth. Magnus began intentionally learning more about information literacy after realizing its usefulness and importance. He also mentioned how becoming more knowledgeable can lead to a change in identity, stating, “when we had information literacy instruction, I suddenly realized how much I didn’t know and have since been trying to learn more…. Obtaining more knowledge in general contributes to a change in one’s identity.”

Terry described how her knowledge regarding the use of sources developed during her time at the university as follows:

Starting with my first course at the university, we’ve been taught how to cite sources academically and where to cite them. This information has been repeated in almost all courses I’ve had … so [my knowledge] has developed.

Vance exemplified an additional risk of not being information literate and pointed to the difficulty of knowing which sources to trust. He said:

It’s interesting to look at the consequences of not being information literate, how dangerous it can be to simply consume information without being critical, both in extremist and other political environments.

2. Development in Information Literacy Doing

When asked how their abilities to find, evaluate, and use sources had developed, most students expressed substantial growth in all three areas. Source evaluation was where their greatest information literacy growth had occurred, followed by searching for sources. Those planning to continue their education expressed the need to learn even more about finding and evaluating sources because of their instrumental value; they still had use for these skills.

Learning to search more effectively for reliable sources represented an important breakthrough for several students. Ingrid was not alone in learning how to search for information other places than Google or Wikipedia. She shared, “In high school I thought that an information literate person searched in Wikipedia … It was an a-ha experience for me when we learned about PsycINFO.”44 Similarly, Thor’s abilities to find and evaluate information sources developed considerably during higher education. He said:

I didn’t know what it was to be critical to sources before I started college. I’ve really made a huge leap. Everything—from how I read, to where I choose to find information, to how I evaluate it—has changed, evolved.

Several participants told of great strides in learning why, when, and how to cite information sources. The following quote shows Ingrid’s development in reference technique in her written work:

Compared to my first year at the university, [citing sources] has become almost automatic. It’s embarrassing to look at my reference lists then compared to now. APA style has become a good friend.

Plagiarism and its consequences were often mentioned as risks of not being able to cite sources correctly. Several participants, including Vera, were afraid of plagiarizing unintentionally, especially when writing in a non-native language. She said:

I hope I haven’t plagiarized!… It’s hard to know where to draw the line between plagiarism and paraphrasing… There’s a gray zone there, especially in English, with my limited vocabulary. It’s hard to find equally precise words as in the source.

3. Interest in Being or Becoming Information Literate

In TRIQ and interviews, instead of inquiring about students’ interest in information literacy, we asked about their interest in being or becoming information literate people. The distinction between the two is significant, as the latter is more relevant for one’s identity. Although we expected student interest in information literacy to be fairly low, we thought that they might still recognize the value and importance of being or becoming information literate, and thereby have a potential interest in that aspect of their identity. This was the case for the students quoted below. For them, information literacy is mainly instrumental, something they need to write papers and pass their courses. They are not necessarily interested in becoming information literate, but recognize that by being information literate, they can achieve their goals. Amy said, “I don’t know if interest is a word I’d use to describe it, but I think it’s important to be information literate,” while Fred responded, “I’m quite interested in being an information literate person… The main goal is to succeed as a student.”

Eli’s feedback is an example of how interest in being information literate can be triggered by a certain situation where information literacy skills are required, how need can drive interest. He said:

I thought much less about [being information literate] before I started studying…I felt that I didn’t need to learn more about information literacy before we had [database search] instruction this semester. I benefited greatly by learning to search in databases.

4. Development in Information Literacy Attitudes

Participants felt that their attitudes about being information literate had developed during their undergraduate study. This evolution took place as a result of acquiring and using information literacy skills, including critical thinking, both in their studies and in daily life. Kari, who was writing her bachelor’s thesis, expressed this change in attitude after illustrating her development as an information literate person on the interest-o-meter. Compared to her first year, she now cares more about the quality of her work and about complying with the conventions of academic writing. She shared, “[I’m now] more involved and feel more ownership of what I write. I care much more about the product and will make sure that it’s good quality.” Similarly, Arthur reflected on his growth as a critical thinker, regarding both the consumption and creation of information, saying, “You learn a lot in just three years as a student, especially about becoming more critical of the information you receive….[I now] think critically about things I post, for example in social media.”

The need for information literacy skills, for example when writing papers, often triggered interest in being or becoming information literate. For example, Ingrid’s attitude about the importance of citing sources has evolved as she herself is becoming an information producer. She said, “the more papers I write, the more I realize how important it is to that I [cite sources] correctly.”

And when asked which values and characteristics she generally associates with information literacy, Amy mentioned curiosity and an “open-minded critical approach.” She felt that these characterized her as an information literate person.

5. Changes in Identity

Several students revealed that their perceptions of themselves (personal) and how others view them (social)—which is our understanding of identity in this study—had changed as a result of becoming more information literate. The following quote reveals that Amy’s perception of herself has shifted; what she calls an identity change, and what Illeris would consider evidence of transformative learning.45 Amy said,

I must have become [a more information literate person] without noticing it! When we talk about it now, absolutely, it has become a part of me, I have become very critical of what people say and where they get it from… It’s the situation [university] and practice [colloquia] that have played the biggest roles in my identity change…I’m very happy that I’ve become an information literate person.

While Amy described her transition in an academic setting, Ruth and Heidi referred to daily life. Becoming critical of what people say and the sources they use represents a change toward a more critical way of thinking. From Mezirow’s perspective, this would be considered a change in “frames of reference”46 and from Illeris’s perspective, this would be considered an identity change, both of which are indicative of transformative learning.47 Ruth shared, “After three years at university, I feel that I’m somewhat more competent than my parents…Becoming more information literate has made me a stronger person in a discussion context because I know how to express my opinions in an academic way,” and Heidi said, “[I now] have less confidence in what I hear and see, and I have a greater need to verify… so I know what basis others have for what they say.”

Other students also spoke about personal changes as a result of becoming more information literate. Several felt more capable of arguing effectively, both in written work and in discussions with family and friends. They experienced that other people’s perceptions of them had therefore changed, and that they were now being taken more seriously. This can be considered a change in social identity. It should be kept in mind, however, that changes in identity varied in degree from glimmers to stronger indications.

Critical self-reflection—questioning the validity of one’s beliefs—is another indication of an identity change, and an important characteristic of transformative learning. In the following quote, we see how Amy critically reflects on her previous behavior: “I now think twice before making assertions. I’ve always blurted out claims quickly… but now I’m better at waiting a little.”

Mixed Methods

Integrating Findings

Table 3 integrates results from the quantitative and qualitative studies. Columns contain: 1. quantitative results from know (TILT), do (Source evaluation and use), and feel (TRIQ subscales) measures in the cross-sectional study; 2. corresponding qualitative results; and 3. quotes from interviews.

Though we saw the changes described in table 3, not all students followed the same trajectory, as indicated by the interest-o-meter (figure 2); nor did every student express their growth in the same way, as indicated in the focus groups and interviews; and not all of these reflections were expressed by all interviewees. As such, there is substantial evidence of growth, though notable variation in the details of how and when students grow as information literate adults.

Discussion

Research Questions

The first research question in this study asks how know, do, and feel, and their interaction, change over the first three years of an undergraduate education for psychology students at a Norwegian university. Regarding information literacy knowledge and skills, we found—as expected—that both increased significantly with time in the quantitative data. Pinto and Fernández-Pascual, Rosman et al., and Scharf found similar results, although their studies followed students over only one to three semesters.48 We also found analogous results in interviews, where all informants experienced substantial growth in their information literacy knowledge and skills, articulating having grown mainly through doing. Assignments fueled their information literacy interest and the development of further knowledge and skills.

In addition to interviews, we assessed feel using the interest measure TRIQ. Tables 1 and 2 show some TRIQ subscales increasing and others decreasing with time. The quantitative increases in “Competence level” and “Meaningfulness,” which could be expected, were also expressed in interviews, where students felt that their information literacy knowledge and skill levels increased simultaneously with information literacy’s meaningfulness for them. Students expressed interest in acquiring information literacy knowledge and skills when these were necessary for specific tasks; the need for skills fueled their interest.

Other subscale changes were more unexpected, such as declines in “General interest” (in information literacy and in being/becoming information literate) and “Competence aspiration” (desire to learn and do more with information literacy). This may be due to students’ lesser-felt need over time to acquire additional information literacy skills, beyond those already learned. Students’ changing interest phases can explain other unexpected subscale variations (see tables 1 and 2). For example, in the four-phase model of interest, lower interest phase signifies lower “Positive affect” and higher “Situation dependence,” which we observed.49 Paradoxically, while interest phase declined or remained stable when measured quantitatively, interviewees expressed increased interest over time, also in interest-o-meters. This discrepancy may be due in part to interviewees being more motivated than others, as indicated both by their participation in interviews, their felt social expectations to communicate positive growth, or the fact that several intended to continue with graduate studies, and thereby had a further need for their information literacy skills.

Interactions between knowing, doing, and feeling over time are shown by correlations in figure 1, as well as in interview material. When using the Meaningfulness subscale as a quantitative measure for feel, all correlations increased with time. The concurrent acquisition of information literacy knowledge, skills, and interest—and their interaction—together with results from the qualitative study, supports our assertion that transformative learning regarding information literacy may have taken place, as discussed below.

As for the second research question, students’ perceptions of themselves as information literate people have clearly evolved during three years in higher education. This is evident both in TRIQ responses, where students reported substantial growth in their perceived information literacy abilities and in information literacy’s meaningfulness for them, and in interview responses (see table 3), where students spoke of feeling more competent and confident in their abilities to find, evaluate, and use sources, and in their ability to argue effectively. But it this learning transformative?

We believe that there are glimmers of transformative learning and identity shifts in this study and base this claim on the defining characteristics of transformative learning stated earlier in the article. Transformative learning refers specifically to learning:

1. …in adults. Third-year undergraduates are young adults with responsibility for their own learning.
2. …after critical self-reflection. Students shared that, after three years of higher education, they now question the validity of their previous beliefs and values. They have become less prone to believing and spreading false information and more aware that their previous methods of searching for information were ineffective. They better understand the importance of citing sources properly and have more integrity as knowledge creators.
3. …that leads to profound changes in perspectives and behavior. Several students told of behavioral changes in how they find, consume, and communicate information, and that they have become critical thinkers—basing their judgements and actions on reliable sources and deliberate, sound reasoning rather than emotional responses.
4. …that leads to changes in identity. In interviews, some students’ words described identity changes—both how they perceive themselves and their perceptions of how others judge them.50 They feel more competent and confident in their information literacy abilities, use them in their education and daily life, and observe that others take them more seriously now that they base their arguments on reliable information.
5. …that is irreversible. When asked in interviews whether they could forget what they had learned about information literacy, students replied that they would always remember the major tenets of information literacy, such as the importance of critically evaluating information sources. (Future research would be required to determine whether this indeed is the case).
6. … implies a qualitative change in the learner, beyond the acquisition of new knowledge and skills.51 Several respondents expressed precisely this type of change, as exemplified by the selected quotes.

These illustrations offer examples of what transformative information literacy learning and identity change might look like. Teachers can use this when planning and monitoring their instruction to encourage transformative change. In terms of scholarship, such evidence can be valuable fodder for future research.

By employing mixed methods, where quantitative and qualitative findings are integrated and reinforce each other, the evidence for transformative learning is strengthened. Quantitatively, we believe that the increase in correlations between knowing, doing, and feeling results (see circle overlaps in figure 1) indicates that these become more tightly integrated with time, and that this can be interpreted within a framework of transformative learning. We imagine transformations taking place as this integrated whole of cognitive, behavioral, and affective dimensions gradually becomes part of one’s identity.

Qualitative results strengthen this supposition. Some interviewees described identity changes (i.e., transformations) as they became information literate people, including changes in their attitudes and behavior. This was the case both when our questions included the word ‘identity’ and when identity was operationalized in other terms rather than named explicitly. An example of an operationalized interview question concerning identity as an information literate person is: Has your development and learning of information literacy changed your opinions or how you fundamentally think?

Participants indicated that they used their skills in settings beyond academia— evidence of transfer—when they spoke of broadly applying their information literacy abilities. Information literacy instructors may benefit from taking information literacy’s transferability into consideration when teaching.52

Limitations

A limitation of the quantitative study is the relatively small sample size for the source-evaluation measure (do) in the sixth semester. In the cross-sectional sample, only some students—those in the bachelor’s program—had a source-evaluation assignment that semester. In the matched sample, few of the original students continued studying for three consecutive years, likely because of the pandemic.

Although qualitative research in general does not strive to be representative, it is worth mentioning, as a limitation, that interviewees were possibly more interested in information literacy than others in their cohort, given that they volunteered for interviews. Secondly, a limitation with interviews that require retrospective assessments of development is that respondents may not accurately recollect past experience. Had we interviewed students at separate points along their trajectories, instead of just their sixth semester, we may have obtained different results. Thirdly, by including the word ‘identity’ in some interview questions, we may have prompted interviewees to use this term when they otherwise may not have. However, since we also posed questions based on our operationalization of identity, without using the term, we heard different perspectives from which to detect any changes in identity and thereby obtained a broader understanding of students’ development.

Future Research

Ideas for future research include: 1. comparing our know, do, and feel findings with other institutions of higher education; 2. comparing evidence of information literacy growth and transformation from students who have information literacy instruction embedded in their studies, to those who have one-shot instruction; 3. comparing results from information literacy measures with students’ grades or completion rates to determine how information literacy levels are associated with outcome measures; and 4. studying growth with qualitative interviews along the entire student trajectory rather than just at the end.

Conclusion

This article contributes to the body of information literacy research in several ways: 1. by following the development of undergraduates over three years; 2. by its use of mixed methods; 3. by assessing three aspects of information literacy development in parallel: knowledge, skills, and feelings (as measured by students’ interest in being or becoming information literate people); and 4. by looking for signs of transformative learning in students.

Information literacy is a heterogeneous construct encompassing cognitive, behavioral, and affective elements, and research aimed at understanding information literacy development should therefore strive to touch upon all of these. Our mixed methods design, with its pragmatic, emergent nature, proved valuable in answering our complex research questions. The integration of quantitative and qualitative data enabled a more comprehensive description of students’ information literacy development than either method on its own would have achieved.

For those of us who teach information literacy, this research shows that our efforts are worthwhile. Findings indicate firstly that students’ information literacy knowledge, skills, and interest—when measured by the meaningfulness of being or becoming information literate—tend to increase with time, as do the associations among them. Secondly, students become more critical to information and self-reflective, and may undergo a change in identity as a result of becoming more information literate. Such changes are indicative of transformative learning, a perspective transformation where individuals re-evaluate previous beliefs and assumptions to create new insight and meaning, a result that many educators strive for. Thirdly, when designing information literacy instruction and assessment in the context of higher education, we should keep in mind that relevant tasks in which information literacy skills are incorporated and assessed fuel students’ interest and thereby their motivation to learn these skills.

Information literate adults are essential for a healthy, thriving society, and information literacy instruction is therefore urgent and important work for helping students become better able at managing information in our post-truth era. As one student astutely commented, this can sometimes even be a matter of life and death.

Notes

1. Oxford Dictionary, “Post-truth,” accessed June 6, 2021, https://www.lexico.com/definition/post-truth

2. Ellen Nierenberg, Torstein Låg, and Tove I. Dahl, “Knowing and doing: The development of information literacy measures to assess knowledge and practice,” Journal of Information Literacy 15, no. 2 (2021): 79, https://doi.org/10.11645/15.2.2795.

3. “Framework for information literacy for higher education,” 2015, https://www.ala.org/acrl/standards/ilframework.

4. Abbas Tashakkori, Burke Johnson, and Charles Teddlie, Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences, 2nd ed. (Los Angeles, CA, USA: SAGE, 2020).

5. Tom Rosman, Anne-Kathrin Mayer, and Günter Krampen, “A longitudinal study on information-seeking knowledge in psychology undergraduates: Exploring the role of information literacy instruction and working memory capacity,” Computers & Education 96 (2016), https://doi.org/10.1016/j.compedu.2016.02.011.

6. Maria Pinto and Rosaura Fernández-Pascual, “How a cycle of information literacy assessment and instruction stimulates attitudes and motivations of LIS students: A competency-based case study,” Journal of Librarianship and Information Science 51, no. 2 (2019), https://doi.org/10.1177/0961000617742447.

7. K. Ann Renninger and Suzanne E. Hidi, “Interest development and learning,” in The Cambridge Handbook of Motivation and Learning, ed. K. Ann Renninger and Suzanne E. Hidi, Cambridge Handbooks in Psychology (Cambridge: Cambridge University Press, 2019); Judith M. Harackiewicz et al., “Predicting success in college: A Longitudinal study of achievement goals and ability measures as predictors of interest and performance from freshman year through graduation,” Journal of Educational Psychology 94, no. 3 (2002), https://doi.org/10.1037/0022-0663.94.3.562.

8. Taehee Kim and Diane L. Schallert, “Mediating effects of teacher enthusiasm and peer enthusiasm on students’ interest in the college classroom,” Contemporary Educational Psychology 39, no. 2 (2014), https://doi.org/10.1016/j.cedpsych.2014.03.002; K. Ann Renninger and Suzanne Hidi, The power of interest for motivation and engagement (New York: Routledge, 2016).

9. Suzanne Hidi and K. Ann Renninger, “The four-phase model of interest development,” Educational Psychologist 41, no. 2 (Spring 2006), https://doi.org/10.1207/s15326985ep4102_4.

10. Tove I. Dahl and Ellen Nierenberg, “Here’s the TRIQ: The Tromsø Interest Questionnaire based on the four-phase model of interest development,” Frontiers in Education 6, no. 402 (2021), https://doi.org/10.3389/feduc.2021.716543.

11. Available on https://site.uit.no/troils.

12. Jack Mezirow, “Perspective transformation,” Adult Education 28, no. 2 (1978), https://doi.org/10.1177/074171367802800202.

13. Jack Mezirow, Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning (San Francisco, CA: Jossey-Bass, 1990).

14. Knud Illeris, “Transformative learning and identity,” Journal of Transformative Education 12, no. 2 (2014), https://doi.org/10.1177/1541344614548423; “Transformative learning as change and development of identity,” in Transformative learning meets bildung, ed. Anna Laros, Thomas Fuhr, and Edward W. Taylor, International Issues in adult Education (Rotterdam: SensePublishers, 2017).

15. Illeris, “Transformative learning and identity,” 40.

16. Illeris, “Transformative learning as change,” 179-80.

17. Erik H. Erikson, Identity: Youth and crisis (London: Faber & Faber, 1968); Illeris, “Transformative learning as change and development of identity.”; Zygmunt Bauman, Liquid modernity (Cambridge: Polity Press, 2000); Anthony Giddens, Modernity and self-identity: Self and society in the late modern age (Cambridge: Polity Press, 1991).

18. Mezirow, Fostering critical reflection.

19. Illeris, “Transformative learning and identity;” Illeris, “Transformative learning as change.”

20. Chad D. Hoggan, “Transformative learning as a metatheory: Definition, criteria, and typology,” Adult Education Quarterly 66, no. 1 (2016), https://doi.org/10.1177/0741713615611216; Edmund O’Sullivan, Amish Morrell, and Mary Ann O’Connor, eds., Expanding the boundaries of transformative learning: Essays on theory and praxis (New York: Palgrave, 2002).

21. Michaela D. Willi Hooper and Emily Scharf, “Connecting and reflecting: Transformative learning in academic libraries,” Journal of Transformative Education 15, no. 1 (2017), https://doi.org/10.1177/1541344616670033; Association of College and Research Libraries, “Framework for information literacy for higher education,” 9.

22. Aneliia I. Chatterjee, “Using transformative learning theory to explore adjunct faculty’s experiences with teaching information literacy: A case study of community colleges in New Jersey.” Doctor of Education, Rowan University, 2022; Amanda Nichols Hess, “Academic librarians’ teaching identities and work experiences: Exploring relationships to support perspective transformation in information literacy instruction,” Journal of Library Administration 60, no. 4 (2020), https://doi.org/10.1080/01930826.2020.1721939; Brian Kenney, “The transformative library: A narrative inquiry into the outcomes of information use.” Ph.D., University of North Texas, 2008; Darrell Hucks, Patrick Hickey, and Matthew Ragan, “Collaborations for transformative learning experiences: Technology integration and information literacy in teacher education,” International Journal of Cyber Ethics in Education 4, no. 1 (2016), https://doi.org/10.4018/IJCEE.2016010102.

23. Kim and Schallert, “Mediating effects of teacher enthusiasm and peer enthusiasm on students’ interest in the college classroom;” Renninger and Hidi, The power of interest for motivation and engagement.

24. All data are available at https://doi.org/10.18710/SK0R1N.

25. The tools are freely available on https://site.uit.no/troils and in UiT Open Research Data: https://doi.org/doi:10.18710/L60VDI.

26. Nierenberg, Låg, and Dahl, “Knowing and doing;” Dahl and Nierenberg, “Tromsø Interest Questionnaire.”

27. We do not imply that information literacy consists exclusively of these facets, only that they are crucial to the construct, as shown in the framework analysis of information literacy definitions, standards, and frameworks. See Nierenberg, Låg, and Dahl, “Knowing and doing.”

28. Nierenberg, Låg, and Dahl, “Knowing and doing,” 88.

29. Although this assignment requires APA style, any reference style can be used with this measure.

30. Nierenberg, Låg, and Dahl, “Knowing and doing,” 88.

31. TRIQ, and evidence of its reliability and validity, are found in Dahl and Nierenberg, “Tromsø Interest Questionnaire.”

32. Hidi and Renninger, “The four-phase model.”

33. TILT and TRIQ were included in a Qualtrics survey.

34. The Norwegian Centre for Research Data (NSD) approved the consent form sent to all participants. Only those students who consented to participation after reading this form were included in the study.

35. Usman, M. “Power Efficiency of Sign Test and Wilcoxon Signed Rank Test Relative to T-Test.” Mathematical Theory and Modeling 5, no. 12 (2015): 53-59.

36. This is likely a result of the smaller sample size.

37. Hidi and Renninger, “The four-phase model.”

38. See figures 1 and 4 in Dahl and Nierenberg, “Tromsø Interest Questionnaire.”

39. Fisher’s z = -1.962, p = 0.049. None of the six correlations are significantly different from zero, hence we cannot claim that these patterns will necessarily hold for the population without further testing.

40. One male and one female from each of the two programs participated in individual interviews.

41. The interest-o-meter was modeled after Audun Hetland and Joar Vittersø, “The feelings of extreme risk: Exploring emotional quality and variability in skydiving and BASE jumping,” Journal of Sport Behavior 35, no. 2 (2012).

42. Virginia Braun and Victoria Clarke, “Thematic analysis,” ed. H. Cooper et al., vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological, APA handbooks in psychology: APA handbook of research methods in psychology (Washington, DC: American Psychological Association, 2012).

43. The six phases in this model are: 1. Familiarizing yourself with your data, 2. Generating initial codes, 3. Searching for themes, 4. Reviewing themes, 5. Defining and naming themes, and 6. Producing the report.

44. PsycINFO is a database for the discipline of psychology.

45. Erikson, Identity: Youth and crisis; Illeris, “Transformative learning and identity,” 40.

46. Merizow describes “frames of reference” as the assumptions and values that affect how we view the world. Jack Mezirow, Learning as transformation: Critical perspectives on a theory in progress (San Francisco, CA, USA: Jossey-Bass, 2000, 5).

47. Harvey Siegel, “Educating reason: Critical thinking, informal logic, and the philosophy of education,” Informal Logic 7, no. 2 (1985), https://doi.org/10.22329/il.v7i2.2706; Illeris, “Transformative learning and identity.”

48. Pinto and Fernández-Pascual, “How a cycle of information literacy;” Rosman, Mayer, and Krampen, “A longitudinal study;” Davida Scharf, “An intervention and assessment to improve information literacy.” Ph.D., Rutgers University, 2013.

49. Hidi and Renninger, “The four-phase model of interest development.”

50. cf. Erikson, Identity: Youth and crisis.

51. Illeris, “Transformative learning as change.”