Teaching Source Evaluation to Undergraduate Nursing Students: Comparing the CRAAP Test and the Source + Beyond the Source Framework

Caitlin Ratcliffe*

Given the drawbacks of checklist approaches to source evaluation, this study compared a networked method, the Source + Beyond the Source framework, with the CRAAP Test in a one-shot library class for undergraduate nursing students. In one pre-test and two post-tests, participants evaluated websites promoting misinformation and disinformation, reliable consumer health websites, and peer-reviewed articles. The results showed no significant differences between the groups, although both groups improved following training. The open-ended responses indicated that few participants retained specific details from either evaluation tool. These findings highlight how the one-shot format limits the efficacy of academic librarians’ information literacy instruction.

Introduction

In the contemporary landscape of information literacy, source evaluation remains a cornerstone. The modern information ecosystem is composed of an astonishing wealth of information—causing a subsequent “poverty of attention” (Simon, 1971)—and is further shaped by the pervasive effects of algorithmic personalization (Bull et al., 2021). From this environment, a global “infodemic” of health-related misinformation has emerged (Zarocostas, 2020). In the face of these challenges, undergraduate students struggle to critically evaluate online information, despite claims they are “digital natives” (Breakstone et al., 2021; Hargittai, 2010; Kirschner & van Merriënboer, 2013; McGrew et al., 2018; Wineburg et al., 2016). It is necessary for undergraduate students to cultivate an ability to identify health-related misinformation and disinformation; this is especially true for nursing students as they develop their professional identity and practice. Academic librarians are encouraged by the Association of College and Research Libraries’ (ACRL) Framework for Information Literacy for Higher Education (ACRL Framework) (2016) to meet this need by teaching source evaluation at the undergraduate level.

This study compared two methods for teaching source evaluation to undergraduate nursing students: the CRAAP (Currency, Relevance, Authority, Accuracy, and Purpose) Test and the Source + Beyond the Source framework. CRAAP is a popular acronym used by academic librarians to introduce source evaluation to undergraduate students. As a checklist approach to source evaluation, there are many inherent limitations to the CRAAP Test, including the inefficiency, ineffectiveness, and impracticality of its use in real-world scenarios (Benjes-Small et al., 2013; Bull et al., 2021; Faix, 2021; Fielding, 2019; Kimmel, 2005; Lenker, 2017; Meola, 2004; Ostenson, 2014; Ziv & Bene, 2022). An alternative to checklists is networked approaches, which are more suitable for the present-day algorithm-driven information ecosystem (Ziv & Bene, 2022). The Source + Beyond the Source framework is a networked approach recently developed by librarians at DePaul University Library (Rismiller et al., 2023). There are few studies that quantify the effectiveness of networked approaches when directly compared with checklist approaches, and no studies examining the Source + Beyond the Source framework. This study seeks to fill this gap by assessing the effectiveness of the Source + Beyond the Source framework compared to the CRAAP Test in the context of one-shot library instruction teaching first-semester Bachelor of Science in Nursing (BScN) students to identify websites promoting misinformation and disinformation, reliable consumer health websites, and peer-reviewed articles. This study finds that, although students show a significant improvement following instruction in both methods, there are few differences between the efficacy of the two methods, and neither method produces ideal results. These findings emphasize the limitations of the one-shot format for academic librarians teaching source evaluation.

Background

The CRAAP Test

CRAAP is an acronym developed by Sarah Blakeslee (2004) at California State University, Chico, as a mnemonic tool for source evaluation. In the original articulation of the CRAAP Test, Blakeslee notes that these criteria—currency, relevance, authority, accuracy, purpose—were already widely used to teach source evaluation: only the mnemonic acronym itself was new. Caulfield (2018) further traced the origin of the CRAAP Test criteria to 1978, when it was originally developed as a tool for collection development. The five criteria of the CRAAP Test have been used by librarians to teach source evaluation for nearly two decades.

The effectiveness of the CRAAP Test is a topic of much interest to library and information studies (LIS) scholars. Several studies have found the CRAAP Test to be an effective method for teaching source evaluation when compared with no intervention (Esparrago-Kalidas, 2021; Myhre, 2012; Muis et al., 2022). However, the impact of the CRAAP Test is often inconclusive when compared with other source evaluation techniques, such as “prebunking,” and/or the six journalistic question words (6QW). Caramancion (2022) compared the CRAAP Test with the prebunking method—a technique that pre-emptively teaches participants about rhetorical techniques and information manipulation—to identify misinformation and disinformation and found that the CRAAP Test was slightly more accurate, but also slower, than prebunking; notably, neither method was significantly more effective than the control group. Lowe et al. (2021) compared the CRAAP Test with the 6QW (Who, What, When, Where, Why, and How) method and found both groups scored about equally well on evaluation skill quizzes, although students who received the 6QW training produced papers with higher quality sources. The efficacy and limitations of the CRAAP Test often stem from its format as a checklist.

Checklist Approaches

The CRAAP Test is a checklist approach to source evaluation. Checklists are, at their core, a carryover from the strategies used to analyze analog resources (Caulfield, 2018). There are dozens of alternative checklists for source evaluation, including but not limited to the ABCs (Authority, Bias, Citations) of web evaluation (Schrock, 1998, as cited in Sye & Statton Thompson, 2023); RADAR (Relevance, Authority, Date, Appearance, Reason) (Mandalios, 2013); the Five Ws (Who, What, When, Where, Why, and How) (Radom & Gammons, 2014); ACT UP (Author, Currency, Truth, Unbiased, Privilege) (Stahura, 2018); DIG (Digital Image Guide) (Thompson, 2019); and CARS (Credibility, Accuracy, Reasonableness, Support) (Harris, 2020). Checklists are often employed by librarians when teaching source evaluation. In a study of college and university websites, Ziv and Bene (2022) found that nearly every institutional website in the random sample featured checklist approaches. One reason for the popularity of checklist approaches is that they are easy to teach compared to other methods (Faix & Daniels, 2023) and can be useful for students who are new to research (Ostenson, 2014; Wichowski & Kohl, 2013).

Despite their ubiquity, checklist approaches also garner many critiques. Even in the early days of the internet, it was clear checklists do not serve learners in practice. There is an unrealistic number of criteria to remember and consider (Caramancion, 2022; Lenker, 2017; Meola, 2004; Radom & Gammons, 2014), and the mechanical application of checklist criteria does not promote critical thinking (Meola, 2004) or encourage meaningful learning (Lenker, 2017). Although checklists can provide useful scaffolding for students in the early stages of learning about source evaluation (Ostenson, 2014), they oversimplify the evaluation process (Kimmel, 2005; Ostenson, 2014) and tend to promote dualistic thinking of sources as “good” or “bad” (Benjes-Small et al., 2013; Lenker, 2017; Meola, 2004).

The emergence of Web 2.0 and the evolution of the online information environment have revealed more significant issues with checklist approaches to source evaluation. Although Wichowski and Kohl (2013) argue that the CRAAP Test can be effectively applied to blogs and microblogs, it is easy for misinformation and disinformation to meet a checklist’s criteria (Breakstone et al., 2018; Caulfield, 2018). In a survey of 263 post-secondary students, Wineburg et al. (2020) found that an alarming majority of students could not identify a satirical news site as such or that a “nonpartisan” group was funded by industry lobbyists, even when they used strategies like the CRAAP Test. Because checklists focus on a website’s internal features, they do not capture important contextual elements (Fielding, 2019; Ziv & Bene, 2022). Additionally, checklists do not factor in the impact of personalized search results (Bull et al., 2021; Faix, 2021; Fielding, 2019). Bull et al. (2021) observe that “CRAAP developed in a time when you found information, before the dramatic shift to information finding you” (Section 2; emphasis in original). Bull et al. argue that reactive approaches to source evaluation, including but not limited to checklists, do not address the pervasive effects of algorithmic personalization. Overall, checklist approaches appear increasingly inadequate in an algorithm-driven information ecosystem.

Networked Approaches

In contrast to the CRAAP Test, networked approaches to source evaluation are designed for the modern information environment. Ziv and Bene (2022) describe networked approaches as follows:

Each source is part of a network of information. To understand a single node in the network, one must place it in the context of other networked sources. On the internet, an individual website—a node—is best understood in relation to what other internet sources have to say about it. (p. 907)

Examples of networked approaches to source evaluation include SIFT (Stop, Investigate, Find, Trace) (Caulfield, 2019); proactive evaluation (Bull et al., 2021); and CCOW (Credentials, Claims, Objectives, Worldview) (Tardiff, 2022). The best-known networked approach is SIFT, a method developed by Mike Caulfield. It consists of four “moves,” which Caulfield (2019) defines as “things to do when looking at a source” (emphasis in original). SIFT emphasizes lateral reading to contextualize the information found in a source. Recent studies have found that lateral reading is an effective technique to improve college students’ fact-checking strategies (Breakstone et al., 2021; Brodsky, Brooks, Scimeca, Galati, et al., 2021; Brodsky, Brooks, Scimeca, Todorova, et al., 2021; Fielding, 2019; McGrew et al., 2019). These findings demonstrate that networked approaches, such as SIFT, have the potential to significantly improve students’ source evaluation skills.

Despite these benefits, networked approaches are difficult for many academic librarians to implement. Although Faix and Fyn (2023) argue that SIFT is compatible with the ACRL Framework, there are several practical barriers to adopting networked approaches in information literacy instruction. Faix and Daniels (2023) compared SIFT with the CRAAP Test in two asynchronous information literacy courses; they found that, although SIFT improved the quality of source selection, students required practice to apply SIFT correctly. Kaufman (2023) also compared SIFT with the CRAAP Test in an information literacy course and found that SIFT was more effective when used correctly; however, there were notable barriers for students to apply it correctly, including the time required for users to implement strategies such as lateral reading, as well as the amount and complexity of instruction required. Given the necessary investment of instructional time and practice, existing networked approaches are not well-suited to the context of one-shot information literacy instruction for undergraduate students in their first year.

The Source + Beyond the Source Framework

The Source + Beyond the Source framework is an alternative networked approach to source evaluation. It was developed by a committee of instruction librarians at DePaul University Library as part of the information literacy program for a required first-year writing course (Rismiller et al., 2023). Like many other educators (including the author), Rismiller et al. observed that “the CRAAP Test wasn’t meeting students’ needs in our increasingly complex information landscape” (p. 286). Rismiller et al. reviewed other approaches to source evaluation, including SIFT, ACT UP, and proactive evaluation; they noted that, while each method had useful elements, none met their needs for a one-shot library instruction session at a first-year level. Rismiller et al. developed an alternative source evaluation framework that incorporates the strengths of other networked approaches—such as lateral reading and introspective reflection—that could be used in a one-shot session with first-year students. The Source + Beyond the Source framework consists of two layers of questions: the first layer examines the source itself, and the second layer requires users to conduct additional research and reflect on the context of how the information was created and shared (see Figures 1 and 2 for the full framework).

As an academic librarian with many years of experience conducting one-shot source evaluation sessions with first-year nursing students, the author of this paper posited that the visual design of the two circles (see Figure 1) would be more memorable than an acronym. The author also hypothesized that this visualization would be an accessible representation for first-year undergraduate students of sources as nodes within a larger information ecosystem, addressing the issues with complexity reported with other networked approaches.

Methods

To assess the effectiveness of the Source + Beyond the Source framework compared to the CRAAP Test, the author collected data through one pre-test and two post-tests. These tests were administered throughout the author’s course-integrated information literacy instruction in a mandatory first-term course for BScN students. As the Health Sciences Librarian, the author teaches two labs in this course. The first, “Evidence Lab #1,” takes place on week six and focuses on source evaluation; the second, “Evidence Lab #2,” occurs in week 10 and introduces search strategies and the CINAHL database. As the information literacy instruction takes place in the lab component (rather than the lecture component) of the course, the sessions are 110 minutes and include a significant amount of active learning. There were 144 students enrolled in the course in the 2023–24 academic year—96 in Fall 2023 and an additional 48 in Winter 2024—split between 12 sections with approximately 12 students in each section.

At the start of Evidence Lab #1, the author introduced the study and verbally reviewed the informed consent information. Students were then given a shortened link to the pre-test prior to any other instruction from the librarian. The pre-test included three sources (see Appendix A) that students were prompted to evaluate for reliability. After completion of the pre-test, the author introduced the evaluation tool with a brief lecture. Half the sections received the Source + Beyond the Source training, and half received the CRAAP Test training. To explore the evaluation tool, students were divided into small groups of two or three and asked to analyze the same three sources from the pre-test using a handout with a series of prompts based on the evaluation tool (see Appendices B and C for handouts). Following the small group discussion, the author facilitated a discussion with the whole class about each of the sources using the evaluation tool’s prompts to guide this discussion. After the large group discussion, students had the opportunity to reflect, via an anonymous Kahoot poll, on the extent to which they trusted each source after examining it with the tool. At the conclusion of the activity, the author revealed (if it had not become apparent in the large group discussion) that the first source in the pre-test was actively promoting misinformation. This revelation prompted a short discussion about the prevalence of health-related misinformation and its impact on nursing practice, as well as an introduction to the peer-review process. At the end of Evidence Lab #1, students completed the first post-test, which included seven sources. Five weeks later, at the Start of Evidence Lab #2, students completed the second post-test, which included six sources, to assess retention.

The tests were created as three “quizzes” in LibWizard, a Springshare product for creating surveys, forms, quizzes, and tutorials. All three quizzes had the same structure but included different sources for evaluation (see Appendix A). In each LibWizard quiz, the first page contained the informed consent letter, and students used radio buttons to indicate their ongoing consent. Students who did not consent could still choose to interact with the quiz, including the links to explore each source, but their results were not included in the study’s analysis. As participation in the study was optional, the second page of each quiz required students to select whether they wanted their responses to be included in the study’s analysis. Consequently, the study’s response rate is lower than the total number of potential participants and varies among the three tests. Although inclusion in the study was optional, every student was asked to complete the pre-test as it was part of the in-class active learning exercise in Evidence Lab #1.

On the subsequent pages of the LibWizard quiz, participants were guided through a series of sources (see Appendix A), with the same questions about each source. Each source was represented on a separate page, which included a link to the source, radio buttons to rank reliability, a text box for open-ended responses, and a reminder to spend approximately three minutes evaluating each source. Each source opened in a new tab to encourage participants to move between tabs and engage in lateral reading, rather than remaining within the LibWizard quiz. For each source, participants were asked to rank its reliability on a three-point scale: “(1) Not reliable. Contains false information;” “(2) Somewhat reliable. Contains some true and some false information;” “(3) Very reliable. Contains true information.” Participants could also select “I’m not sure” for each source. Participants had the option to write a short description of their reasoning.

There were 16 sources in total: six reliable consumer health websites; six websites promoting health-related misinformation and disinformation (e.g., leaky gut syndrome, colloidal silver, Wilson’s Temperature Syndrome, etc.); three peer-reviewed articles; and one withdrawn article (see Appendix A for the complete list of sources). For the six websites promoting health-related misinformation and disinformation, lateral reading reveals multiple sources debunking the health claims. The pre-test contained only three sources: a website promoting misinformation, a reliable consumer health website, and a peer-reviewed article. These sources, as well as the shorter length of the pre-test, were selected to facilitate the active learning exercise in Evidence Lab #1.

Results

Of 144 potential participants, 115 students opted in for their results to be included in the pre-test, 101 in the first post-test, and 89 in the second post-test (see Table 1 for complete participant numbers).

Quantitative Scores

Participants were asked to rank the reliability of each source as either: “(1) Not reliable. Contains false information;” “(2) Somewhat reliable. Contains some true and some false information;” “(3) Very reliable. Contains true information;” or “I’m not sure.” For this analysis, websites promoting misinformation and disinformation were considered “(1) Not reliable. Contains false information.” The reliable consumer health websites and peer-reviewed articles were considered “(3) Very reliable. Contains true information.” Sources were carefully selected to unequivocally match the description “Contains false information” or “Contains true information,” so every source should have been ranked as (1) or (3). Responses with the ranking “(2) Somewhat reliable. Contains some true and some false information,” “I’m not sure,” and responses with no ranking were analyzed to illustrate overall confidence. T-tests—a type of statistical analysis used to determine if the difference between two groups along a continuous outcome variable is statistically significant (Brown & Melamed, 1990)—were performed to compare the percentage of average correct responses between the CRAAP Test group and the Source + Beyond the Source group, as well as within each group over time (see Figure 3 and Table 2).

For the CRAAP Test group, mean scores improved from 41% in the pre-test to 52% in post-test 1 and 59% in post-test 2. The difference between the pre-test and post-test 2 is statistically significant (p < .001), demonstrating that the participants improved following instruction. Although the difference between the pre-test and post-test 1 is statistically significant (p = .04), the difference between post-test 1 and post-test 2 is not statistically significant (p = .11). These scores show that participants in the CRAAP Test group significantly improved immediately following training; however, their continued improvement, although slight, after five weeks was not statistically significant.

For the Source + Beyond the Source group, mean scores improved from 48% in the pre-test to 49% in post-test 1 and 58% in post-test 2. The difference between the pre-test and post-test 2 is statistically significant (p = .01), indicating these participants also improved following training. Although the difference between the pre-test and post-test 1 is not significant (p = .79), the difference between post-test 1 and post-test 2 is statistically significant (p = .006). These scores show that participants in the Source + Beyond the Source group did not improve significantly immediately following training; however, their continued improvement after five weeks was statistically significant.

When examining the scores from the CRAAP Test and Source + Beyond the Source groups, the difference between the groups is not statistically significant in the pre-test (p = .16), post-test 1 (p = .49), or post-test 2 (p = .77). This demonstrates there is no significant difference between the effectiveness of the two methods. However, there is a significant difference between the pre-test and the second post-test for both groups. These scores demonstrate that both the CRAAP Test and Source + Beyond the Source training are more effective than no training. This difference is especially clear when considering the sources promoting misinformation (see Figure 4).

Uncertain responses—including “(2) Somewhat reliable. Contains some true and some false information,” “I’m not sure,” and responses with no ranking—can be used to illustrate the group’s overall confidence in their assessments. T-tests were performed to compare the percentage of uncertain responses between the CRAAP Test group and the Source + Beyond the Source group, as well as within each group over time (see Figure 5 and Table 3).

For the CRAAP Test group, the average number of uncertain responses was 40% in the pre-test, 44% in post-test 1, and 31% in post-test 2. While the difference between post-test 1 and 2 (p=.007) is significant, the difference between the pre-test and post-test 1 (p=.53) and the overall difference between the pre-test and post-test 2 (p = .08) is not statistically significant.

These scores show that participants in the CRAAP Test group experienced a small increase in uncertainty between the pre-test and post-test 1, but their uncertainty decreased again five weeks later in post-test 2. Overall, the CRAAP Test group did not experience a significant change in their uncertainty following training.

The Source + Beyond the Source group demonstrates a similar pattern of uncertainty. For this group, the average number of uncertain responses was 33% in the pre-test, 43% in post-test 1, and 28% in post-test 2 (see Figure 5). While the difference between the pre-test and post-test 1 (p = .01) and post-test 1 and 2 (p < .001) are significant, the overall difference between the pre-test and post-test 2 (p = .17) is not statistically significant. These scores show that participants in the Source + Beyond the Source group experienced a small significant increase in uncertainty between the pre-test and first post-test, but their uncertainty significantly decreased again five weeks later in post-test 2. Overall, the Source + Beyond the Source group also did not experience a significant change in their uncertainty following training.

There is no significant difference in the number of uncertain responses between the groups in the pre-test (p = .16), post-test 1 (p = .87), or post-test 2 (p = .40), as noted in Table 3. These scores demonstrate that there was no notable difference in overall confidence between the two groups. The increased percentage of uncertain responses in post-test 1 for both groups is likely due to this test’s close proximity to the large group discussion of the pre-test sources; as illustrated by Figure 4, 92% of all participants did not correctly identify Source #1 as misinformation, and it is likely that the in-class discussion of their earlier mistake temporarily decreased their confidence. However, the results from post-test 2 reveals that training did not significantly affect either group’s confidence in the long term.

Qualitative Responses

For each source, participants had the option to write a short description of the reasoning for their score. These open-ended responses offer insights into the strategies participants used to evaluate the sources. The responses were coded inductively by the author to compare the use of specific strategies. To account for the different number of sources in each test, the frequency of the strategy used in the open-ended responses is divided by number of sources. For example, the authority strategy (that the source is reliable/not reliable because of the authority of the author/website/database) was described 255 times in the post-tests: Source + Beyond the Source; divided between 13 sources in the post-tests, this strategy was used 19.62 times per source (see Figure 6 and Table 4). While there are few differences between the CRAAP Test and Source + Beyond the Source groups, there are noteworthy changes between the pre-test and the post-test. It is worth noting that the frequency of most strategies decreased slightly in the post-tests as many participants wrote shorter, less detailed responses, despite reminders to spend approximately three minutes per source.

The open-ended responses show that, overall, students utilized few specific strategies from either the CRAAP Test or the Source + Beyond the Source framework in the post-tests. A notable exception is currency; participants highlighted the currency of sources more often after training, particularly participants who received the CRAAP Test training with “currency” included in the “CRAAP” acronym. However, lateral reading strategies were not described in the Source + Beyond the Source group’s post-test responses, even through the Source + Beyond the Source training emphasized lateral reading. Although the CRAAP Test did not discuss lateral reading, one participant from the CRAAP Test group described using a lateral reading strategy by looking up other information about the health condition described in a source. Three participants in the pre-test described lateral reading strategies, such as “using other backup sources,” when considering Source #1, a website promoting misinformation. Interestingly, these three participants selected “(2) Somewhat Reliable. Contains some true and some false information” and “I’m not sure” for this source, indicating that, prior to training, lateral reading was only one strategy influencing their score, with other factors counterbalancing their opinions.

The open-ended responses also reveal which strategies participants used to inaccurately evaluate sources promoting misinformation and disinformation. In the pre-test, only 8% of participants (four participants in the CRAAP Test group and five participants in the Source + Beyond the Source group) correctly identified that Source #1 was promoting misinformation (see Figure 4). Participants who scored Source #1 as “(3) Very reliable. Contains true information” primarily justified this score using the authority, celebrity, and heuristic (i.e., design, domain, readability, length, and disclaimer) strategies. One participant, who rated Source #1 as “(3) Very reliable. Contains true information,” succinctly captures these strategies in a single comment: “There weren’t any constant [advertisement] pop-ups, there was a medical information disclaimer at the bottom of the page, and the doctor’s profile seemed legit, and he seems well know [sic]. Information was also consice [sic] and clearly explained.” As seen in Figure 6 and Table 4, the use of these strategies decreased following training with either tool.

Prior to training, participants struggled to accurately identify reliable authority. Fifty-one participants wrote that the author of Source #1 (a website promoting misinformation), Dr. Josh Axe, was a credible authority:

• “I trust the resource as it’s a doctor”
• “the website includes the word Dr [which] means the person talking about the topic has a high level of education and will provide correct information”
• “What made me trust this source is that it is written by a medical professional and contained relavent [sic] information such as how it is caused and how to fix it.”

Three participants highlighted that Dr. Axe was a reliable authority due to his celebrity, or high follower count on social media. For example, one respondent shared their reasoning: “Because of how credible the author of the article is, considering at the very bottom. He has quite a number of followers in his social media page.” Following training, most participants could more accurately gauge authority. Only three participants described a similar author in Source #12, Dr. Lauren Love, as a credible authority. While 12 participants noted the author referenced in Source #9 (a website promoting misinformation), Dr. Nina Pierpont, might be a credible authority, 11 of these participants qualified that observation with other considerations. For example, one participant who scored Source #9 as “(2) Somewhat Reliable. Contains some true and some false information” wrote:

The Dr/ author is a true MD and PhD are a real university, so the knowelge [sic] is there, it just seems that the ‘wind turbine syndrome’ isn’t even a real thing, and alot [sic] of the links take you to her book, and her ‘studies’ seems [sic] like its more of an opinion from a political view that [sic] a legit issue medically.

This skepticism highlights how participants’ scrutiny of authority increased after receiving training in either method.

Similarly, participants’ use of heuristics decreased after the training. In the pre-test, many participants relied on heuristics such as design, domain, readability, length, and disclaimer. Participants who scored Source #1, a source promoting misinformation (see Figure 7), as “(3) Very reliable. Contains true information,” emphasized the professional design of the website, including the clear organization, the “Evidence Based” icon at the top of the page, and the presence of multimedia elements such as short videos:

• “I chose this rating as the source appears to be very reliable. There is a green box in the top lefthand corner that says evidence based, and the source has valid information.”
• “This source was very well organized and put together, to me it is clear that a lot of time and effort was put into this source and so I trust the information that was given.”
• “There are videos and pictures to backup what is said within the source.”
• “broken down sections, there are pictures & or diagrams that help show what they are talking about”

Several participants highlighted the readability of Source #1 in their rationale. For example, one participant wrote: “This source did a good job of writing all the symptoms and causes, and also what people can do to prevent leaky gut syndrome. It is also easy to read and understand.” Participants who scored Source #3, a peer-reviewed article on PubMed (see Figure 8), as “(1) Not reliable. Contains false information” also noted the design of the website:

• “the website doesn’t look official there is not much information and little description of the author”
• “doesn’t have big headlines, doesn’t show any pictures & there is not a lot of information posted about the concept.”
• “It’s a sketchy website, barely has any information that a person searching this topic would find useful.”

However, four participants scored Source #3 as more reliable due to the “.gov” domain. After training, participants used these heuristics less often but still referenced these strategies ­occasionally.

Additionally, participants struggled to accurately identify journal articles prior to training. In the pre-test, 39 comments included the word “article” when describing Sources #1 and #2, both of which are consumer health websites. Conversely, 36 participants expressed skepticism about Source #3, a peer-reviewed article on PubMed (see Figure 8). Although the full text of the article was available to the participants, most participants did not see or open the full text. Limited to only the abstract and other preview information on PubMed (such as “Similar Articles” and “Cited By”), 13 participants who scored Source #3 as “(1) Not reliable. Contains false information” criticized the length and readability of this source.

• “Too short and brief. I would not bother reading it”
• “Lots of big terms used however there is barely any information given”
• “Does not provide enough information about leaky gut and in terms the general public will understand.”
• “The article does not use language that the average person would understand, therefore it does not educate the individual on the information present in the source.”

Participants’ tendencies to conflate accessible text (including the text’s length and readability) and reliability noticeably reduced in the post-tests, and participants could more consistently identify peer-reviewed articles after training. However, no participants identified that Source #16 was a withdrawn article.

One notable trend from the open-ended responses was the use of personal beliefs as a strategy, that is, that the source was/was not reliable because it aligned/did not align with the participants’ personal beliefs or existing knowledge. In the pre-test, several participants reported using personal beliefs to justify their scores. For example, one participant scored Source #1, a source promoting misinformation, as “(2) Somewhat reliable. Contains some true and some false information” and wrote:

Because I just personally agree with some of the food choices in regard to eating whole food, the basics (meat, vegetable, carbs) and food that are not GMO. So i [sic] trust a few of the choices just from doing [my] own research on whole foods and not eating refined sugars and dyes. But there wasn’t any really [sic] sources or citing from books or proven research that [I’ve] seen.

Participants reported less reliance on personal beliefs after receiving training, which suggests the training helped participants articulate judgments they were already making.

Discussion

Findings

There were no significant differences between the participants who received the CRAAP Test training and the Source + Beyond the Source training. In the context of this study, the CRAAP Test and the networked method were similarly effective at enabling students to correctly identify reliable sources of information. Other studies comparing the CRAAP Test with an alternate method for source evaluation produced similar results (Caramancion, 2022; Kaufman, 2023; Lowe et al., 2021). Despite the widely recognized drawbacks of checklist approaches to source evaluation, the CRAAP Test consistently demonstrates efficacy on par with the other available methods when compared directly. These findings reveal a significant difference between the pre-test and the post-tests. Following training in either method, participants were able to identify reliable sources of information compared to misinformation more accurately. This is consistent with other studies of the CRAAP Test, which found that the CRAAP Test was more effective than no intervention (Esparrago-Kalidas, 2021; Muis et al., 2022; Myhre, 2012). Regardless of the method taught, undergraduate students’ source evaluation skills significantly improve with training.

Following training in either method, participants evaluated sources of authority more closely and demonstrated less use of heuristics. Both groups demonstrated a more critical understanding of authority after training. Previous research has shown that, without intervention, students rarely spend time to critically evaluate sources, including scrutinizing an author’s credentials (Hargittai et al., 2010; Walraven et al., 2009). Following training in either method, participants demonstrated more skepticism toward the authority of the source. Similarly, participants’ use of heuristics such as such as graphic design, readability, length, and site domain decreased after receiving training in either method. Young adults’ tendencies to use heuristics such as site design to evaluate credibility is well-documented (Agosto, 2002; Barzilai & Zohar, 2012; Coiro et al., 2015; Iding et al., 2008; Julien & Barker, 2009; McGeough & Rudick, 2018; McGrew et al., 2018; Metzger et al., 2010; Olushola, 2021; Walraven et all, 2009). While participants described these heuristics less often following training, it is worth noting that participants still used these strategies occasionally in the post-tests. The continued use of these strategies highlights that one-shot instruction may influence participants’ existing source evaluation habits but does not completely alter them.

One noteworthy observation is that participants were able to recognize journal articles more consistently after instruction. Prior to instruction, approximately half the participants expressed skepticism about a journal article on PubMed; without opening the full text of the article, these participants described the abstract as too short and too complex to be considered reliable. In the post-tests, nearly every participant was able to recognize journal articles on PubMed, Science Direct, and ProQuest. This finding highlights the importance of teaching undergraduate students about peer-review and academic databases. Post-secondary educators should not assume that undergraduate students will recognize or trust journal articles without explicit instruction.

An analysis of the open-ended responses reveals that participants retained few specific details from either evaluation tool, except the currency criteria from the CRAAP Test training. This finding is corroborated by Lowe et al. (2021), who observed that first-year students did not retain the actual language from either the CRAAP Test or the six question words (6QW) methods in the long term. These results contribute to the large body of literature demonstrating that the mechanical application of checklist criteria is not a realistic or practical method for conducting source evaluation. Notably, no participants who received the Source + Beyond the Source training described the use of lateral reading strategies in the post-tests. This contradicted the author’s hypothesis that the visual design of two circles would be an accessible representation of sources as nodes within a larger information ecosystem, consequently promoting lateral reading strategies. It is possible that, at least in the context of the study, participants were unwilling to spend the time required to use strategies like lateral reading. Kaufman (2023) observed that although the SIFT Method enabled students to correctly identify reliable sources of information more often than the CRAAP Test, it is time-intensive for students to conduct evaluation using lateral reading. Alternatively, participants may require more practice with lateral reading techniques than the one-shot instruction session allowed. Faix and Daniels (2023) found that students required practice to effectively use the strategies associated with SIFT. These findings impact how academic librarians plan information literacy instruction.

Implications for Practice

As an academic librarian, it is easy to feel as though we have wasted our time, or even misled our students, by teaching source evaluation using the CRAAP Test. In an oft-cited blog post about the history of the CRAAP Test, Caulfield (2018) wrote: “we taught [the CRAAP Test] and its variations for almost twenty years even though it did not work, and most librarians I’ve talked to realized it didn’t work many years back but didn’t know what else to do” (We brought a can opener to a gunfight section, para. 27). However, the findings from this study might reassure librarians; in the context of a one-shot session with first-year undergraduate students, the CRAAP Test was as effective as a networked method of source evaluation. Furthermore, it is clear that some instruction is more effective than no instruction, regardless of the method taught.

However, neither method produces ideal results; as demonstrated by the percentage of correct responses in the first and second post-tests (see Figure 3), nearly half the participants could not identify sources of accurate information following instruction in either method. It is evident that one-shot instruction in both the CRAAP Test and the Source + Beyond the Source framework falls short in adequately preparing undergraduate students to critically evaluate sources in the present information ecosystem.

This study contributes to the growing body of literature demonstrating that if networked approaches to source evaluation are to be successfully taught to undergraduate students, they require a time investment that one-shot instruction sessions cannot accommodate. Given this study’s findings, it is worth considering how the one-shot format impacts the efficacy of teaching source evaluation. Pagowsky (2021) describes the one-shot—a standalone lecture or workshop taught by a librarian within an existing course—as “the dominating force of how [academic librarians] engage in library instruction” (p. 300). The inherent limitations of one-shot instruction are well-known among academic librarians (Almeida, 2022). The one-shot is temporally compressed, often only superficially related to course content or narrowly targeted toward assignments and emphasizes rote skills rather than critical thinking (Nicholson, 2016; Pagowsky, 2021; Zvyagintseva & Blechinger, 2023). In the context of source evaluation, Caulfield (2017) observes that affective habits are a key component of fact checking. It is not reasonable to expect students to change their habits after a single lecture or workshop; indeed, Nicholson and Seale (2022) argue that “transactional one-shots can never truly be transformational” (p. 765). There is no evaluation tool or model, no matter how effective, that can overcome the confines of one-shot instruction.

Given the constraints of the one-shot format, academic librarians should consider alternate venues to deliver information literacy instruction that may create a greater impact on students’ source evaluation habits. There are several examples of successful interventions in the literature. In an asynchronous college course, Breakstone et al. (2021) found that four one-hour modules embedded in the course resulted in significant improvement to students’ abilities to evaluate credible online sources. Similarly, Faix and Daniels (2023) observed an improvement by incorporating the SIFT Method throughout asynchronous information literacy courses, including formative and summative assessments targeting the source evaluation outcomes. Brodsky, Brooks, Scimeca, Todorova, et al. (2021) saw improvements in the use of lateral reading and accurate source evaluation when using the Digital Polarization Initiative (DPI) curriculum instead of the typical curriculum within a general education civics course. Notably, each of these successful interventions feature greater integration within courses than the one-shot format. These findings align with the calls across librarianship to move away from one-shots and toward more collaborative approaches (Moeller, 2020; Santamaria & Schomberg, 2022; Schlesselman-Tarango & Becerra, 2022) built on relational, rather than transactional, models of instruction (Arellano Douglas & Gadsby, 2022; Pho et al., 2022). Academic librarians must work closely with instructors and with programs to ensure information literacy, including source evaluation, is integrated throughout curriculums.

Limitations

The number of sources (16 in total) in this study was selected to facilitate nuanced comparisons between the groups following the training. However, the number and distribution of the sources may have affected the results. The shorter length of the pre-test (three sources), which was designed to facilitate the active learning exercise in the author’s course-integrated instruction, limits the validity of the comparison between the pre-test and the post-tests. By contrast, the longer length of the post-tests (seven sources in post-test 1 and six sources in post-test 2) may have influenced participants’ engagement. At the start of each test, participants were directed to spend approximately three minutes per source, which was ample time for participants to explore the source and employ basic lateral reading strategies. However, the author observed that most participants spent far less time on each source, completing the tests very quickly. A fewer number of sources may have encouraged participants to spend more time on each source. It is worth noting that although more time for each source may have improved the accuracy of participants’ responses in the context of the study, this is likely an accurate representation of how participants are conducting source evaluation in their daily lives. In addition, the open-ended responses provided limited insights into the participants’ reasoning, as participants wrote shorter, less detailed responses throughout the post-tests. Future research could incorporate usability testing or semi-structured interviews to address this gap. Finally, although the author strove to teach consistently, there may have been slight natural variations in content and delivery between the 12 sections of the course.

Future Research

This study focused on consumer health websites and peer-reviewed articles; it did not include other forms for health information, such as social media. The “agency of information,” when information finds users through algorithmic personalization (Bull et al., 2022), was discussed in the context of the Source + Beyond the Source framework but was not apparent when examining pre-selected websites. Further research should expand the types of sources included for review. Rismiller et al. (2023) used a tweet in their lesson plans when developing the Source + Beyond the Source framework. Given the prominence of medical misinformation on TikTok, this platform, or other short-form video content, is a valuable avenue to consider. Students interact with consumer health information in a variety of forms in their everyday lives, and additional research in this area would provide a more robust understanding of the efficacy of library instruction.

Conclusion

This study assessed the effectiveness of the CRAAP Test compared to a networked approach, the Source + Beyond the Source framework, when training first-term Bachelor of Science in Nursing (BScN) students to identify websites promoting misinformation and disinformation, reliable consumer health websites, and peer-reviewed articles. Given the increase of health-related misinformation and disinformation, it is vital that undergraduate students be able to critically evaluate online sources, including the ability to identify peer-reviewed articles. This study demonstrates that undergraduate students’ source evaluation skills significantly improve following training, regardless of the method taught. Although the limitations of checklist approaches to source evaluation are well-documented, this study found that the CRAAP Test was as effective as the networked method in the context of one-shot library instruction. This study contributes to the growing body of literature illustrating how the one-shot format limits the efficacy of information literacy instruction. Academic librarians should seek opportunities for information literacy, including source evaluation, to be integrated within courses and throughout curriculums.

Acknowledgments

I am grateful to Dr. Feodor Snagovsky for his assistance with the quantitative analysis of this study and for providing feedback on an early iteration of the manuscript. Many thanks to Adam Cohen, Michele Pagliuso, Danielle Deschamps, and the anonymous reviewers for their thoughtful feedback on the manuscript drafts.

References

Agosto, D. E. (2002). A model of young people’s decision-making in using the Web. Library & Information Science Research, 24(4), 311–341. https://doi.org/10.1016/S0740-8188(02)00131-7

Association of College and Research Libraries. (2016). Framework for information literacy for higher education. https://www.ala.org/acrl/standards/ilframework

Barzilai, S., & Zohar, A. (2012). Epistemic thinking in action: Evaluating and integrating online sources. Cognition and Instruction, 30(1), 39–85. https://doi.org/10.1080/07370008.2011.636495

Benjes-Small, C., Archer, A., Tucker, K., & Vassady, L. (2013). Teaching web evaluation: A cognitive development approach. Communications in Information Literacy, 7(1), 39–49. https://doi.org/10.15760/comminfolit.2013.7.1.133

Blakeslee, S. (2004). The CRAAP Test. LOEX Quarterly, 31(3). https://commons.emich.edu/loexquarterly/vol31/iss3/4

Breakstone, J., McGrew, S., Smith, M., Ortega, T., & Wineburg, S. (2018). Why we need a new approach to teaching digital literacy. Phi Delta Kappan, 99(6), 27–32. https://doi.org/10.1177/0031721718762419

Breakstone, J., Smith, M., Connors, P., Ortega, T., Kerr, D., & Wineburg, S. (2021). Lateral reading: College students learn to critically evaluate internet sources in an online course. Harvard Kennedy School Misinformation Review, 2(1). https://doi.org/10.37016/mr-2020-56

Brodsky, J. E., Brooks, P. J., Scimeca, D., Galati, P., Todorova, R., & Caulfield, M. (2021). Associations between online instruction in lateral reading strategies and fact-checking COVID-19 news among college students. openICPSR. https://doi.org/10.3886/E145661V1

Brodsky, J. E., Brooks, P. J., Scimeca, D., Todorova, R., Galati, P., Batson, M., Grosso, R., Matthews, M., Miller, V., & Caulfield, M. (2021). Improving college students’ fact-checking strategies through lateral reading instruction in a general education civics course. Cognitive Research: Principles and Implications, 6(1), 23. https://doi.org/10.1186/s41235-021-00291-4

Brown, S. R. & Melamed, L. E. (1990). Experimental design and analysis. Sage Publications. https://doi.org/10.4135/9781412984218

Bull, A. C., MacMillan, M., & Head, A. (2021). Dismantling the evaluation framework. In the Library with the Lead Pipe. https://www.inthelibrarywiththeleadpipe.org/2021/dismantling-evaluation/

Caramancion, K. M. (2022). Quantification of infographic intervention effect on mis/disinformation vulnerability. 2022 IEEE 17th International Conference on Computer Sciences and Information Technologies, 297–300. https://doi.org/10.1109/CSIT56902.2022.10000489

Caulfield, M. (2017, January 8). Building a fact-checking habit by checking your emotions. Web Literacy for Student Fact-Checkers. https://pressbooks.pub/webliteracy/chapter/building-a-habit-by-checking-your-emotions/

Caulfield, M. (2018, September 15). A short history of CRAAP. Hapgood. https://hapgood.us/2018/09/14/a-short-history-of-craap/

Caulfield, M. (2019, June 19). SIFT (The four moves). Hapgood. https://hapgood.us/2019/06/19/sift-the-four-moves/

Douglas, V. A., & Gadsby, J. (2022). The power of presence: One-shots, relational teaching, and instruction librarianship. College & Research Libraries, 83(5), 807–818. https://doi.org/10.5860/crl.83.5.807

Esparrago-Kalidas, A. J. (2021). The effectiveness of CRAAP Test in evaluating credibility of sources. International Journal of TESOL & Education, 1(2), 1–14.

Faix, A. (2021). Source evaluation strategies for the misinformation age. South Carolina Libraries, 5(2), Article 1. https://doi.org/10.51221/suc.scl.2021.5.2.1

Faix, A., & Daniels, T. (2023). Teaching SIFT for source evaluation in asynchronous one-credit information literacy courses. Portal: Libraries and the Academy, 23(3), 449–459.

Faix, A. I., & Fyn, A. F. (2023). Six frames, four moves, one habit: Finding ACRL’s framework within SIFT. College & Research Libraries News, 84(11), 411–416. https://doi.org/10.5860/crln.84.11.411

Fielding, J. A. (2019). Rethinking CRAAP: Getting students thinking like fact-checkers in evaluating web sources. College & Research Libraries News, 80(11), 620–622. https://doi.org/10.5860/crln.80.11.620

Hargittai, E. (2010). Digital na(t)ives? Variation in Internet skills and uses among members of the “Net Generation” Sociological Inquiry, 80(1), 92–113. https://doi.org/10.1111/j.1475-682X.2009.00317.x

Hargittai, E., Fullerton, L., Menchen-Trevino, E., & Thomas, K. Y. (2010). Trust online: Young adults’ evaluation of web content. International Journal of Communication, 4, 468–494.

Harris, R. (2020, October 19). Evaluating Internet research sources. VirtualSalt. https://www.virtualsalt.com/evaluating-internet-research-sources/

Iding, M. K., Crosby, M. E., Auernheimer, B., & Barbara Klemm, E. (2009). Web site credibility: Why do people believe what they believe? Instructional Science, 37(1), 43–63. https://doi.org/10.1007/s11251-008-9080-7

Julien, H., & Barker, S. (2009). How high-school students find and evaluate scientific information: A basis for information literacy skills development. Library & Information Science Research, 31(1), 12–17. https://doi.org/10.1016/j.lisr.2008.10.008

Kaufman, A. (2023). The influence of the CRAAP tTest and the SIFT Method on university students’ understanding of credibility of information online. Studies in Teaching 2023 Research Digest, 55–60.

Kimmel, J. (2005). Web evaluation in the history classroom: Reconsidering the checklist. Journal of the Association for History and Computing, 8(2). http://hdl.handle.net/2027/spo.3310410.0008.201

Kirschner, P. A., & van Merriënboer, J. J. G. (2013). Do learners really know best? Urban legends in education. Educational Psychologist, 48(3), 169–183. https://doi.org/10.1080/00461520.2013.804395

Lenker, M. (2017). Developmentalism: Learning as the basis for evaluating information. Portal: Libraries and the Academy, 17(4), 721–737.

Lowe, M. S., Macy, K. V., Murphy, E., & Kani, J. (2021). Questioning CRAAP: A comparison of source evaluation methods with first-year undergraduate students. Journal of the Scholarship of Teaching and Learning, 21(3), 33–48. https://doi.org/10.14434/josotl.v21i3.30744

Mandalios, J. (2013). RADAR: An approach for helping students evaluate Internet sources. Journal of Information Science, 39(4), 470–478. https://doi.org/10.1177/0165551513478889

McGeough, R., & Rudick, C. K. (2018). “It was at the library; therefore it must be credible”: Mapping patterns of undergraduate heuristic decision-making. Communication Education, 67(2), 165–184. https://doi.org/10.1080/03634523.2017.1409899

McGrew, S., Breakstone, J., Ortega, T., Smith, M., & Wineburg, S. (2018). Can students evaluate online sources? Learning from assessments of civic online reasoning. Theory & Research in Social Education, 46(2), 165–193. https://doi.org/10.1080/00933104.2017.1416320

McGrew, S., Smith, M., Breakstone, J., Ortega, T., & Wineburg, S. (2019). Improving university students’ web savvy: An intervention study. British Journal of Educational Psychology, 89(3), 485–500. https://doi.org/10.1111/bjep.12279

Meola, M. (2004). Chucking the checklist: A contextual approach to teaching undergraduates web-site evaluation. Portal: Libraries and the Academy, 4(3), 331–344.

Metzger, M. J., Flanagin, A. J., & Medders, R. B. (2010). Social and heuristic approaches to credibility evaluation online. Journal of Communication, 60(3), 413–439. https://doi.org/10.1111/j.1460-2466.2010.01488.x

Muis, K. R., Denton, C., & Dubé, A. (2022). Identifying CRAAP on the Internet: A source evaluation intervention. Advances in Social Sciences Research Journal, 9(7), 239–265. https://doi.org/10.14738/assrj.97.12670

Myhre, S. K. (2012). Using the CRAAP Test to evaluate websites. 17th Annual Technology, Colleges, and Community Worldwide Online Conference. http://hdl.handle.net/10125/22479

Nicholson, K. P. (2016). “Taking back” information literacy: Time and the one-shot in the neoliberal university. In N. Pagowsky, & K. McElroy (Eds.), Critical library pedagogy handbook (pp. 25–39). Association of College & Research Libraries.

Nicholson, K. P., & Seale, M. (2022). Information literacy, diversity, and one-shot “pedagogies of the practical.” College & Research Libraries, 83(5), 765–779. https://doi.org/10.5860/crl.83.5.765

Olushola, A. (2021). Credibility evaluation of online political information: A study of an information literacy intervention [Doctoral dissertation, University of Kansas]. ProQuest Dissertations & Theses.

Ostenson, J. (2014). Reconsidering the checklist in teaching internet source evaluation. Portal: Libraries and the Academy, 14(1), 33–50.

Pagowsky, N. (2021). The contested one-shot: Deconstructing power structures to imagine new futures. College & Research Libraries, 82(3), 300–309. https://doi.org/10.5860/crl.82.3.300

Pho, A., Abumeeiz, S., Bisbee, K. V., Mody, N., Romero, R., Tranfield, W., & Worsham, D. (2022). You only get one shot: Critically exploring and reimagining the one-shot instruction model. College & Research Libraries, 83(5), 726–738. https://doi.org/10.5860/crl.83.5.726

Radom, R., & Gammons, R. W. (2014). Teaching information evaluation with the five Ws. Reference & User Services Quarterly, 53(4), 334–347. https://doi.org/10.5860/rusq.53n4.334

Rismiller, F., Cerney, H., Shultz, S., Spiewak, G., & Stoytcheva, S. (2023). Going beyond the source: A revised curriculum for source evaluation. College & Research Libraries News, 84(8), 286–292. https://doi.org/10.5860/crln.84.8.286

Santamaria, M., & Schomberg, J. (2022). It doesn’t matter how many “doses”: One-shots aren’t cures. College & Research Libraries, 83(5), 851–853. https://doi.org/10.5860/crl.83.5.851

Schlesselman-Tarango, G., & Becerra, M. (2022). The critical information literacy leadership institute as alternative to the one-shot: Q & A with a faculty partner. College & Research Libraries, 83(5), 844–847. https://doi.org/10.5860/crl.83.5.844

Simon, H. A. (1971). Designing organizations for an information-rich world. In M. Greenberger (Ed.), Computers, communication, and the public interest (pp. 37-72). Johns Hopkins Press.

Stahura, D. (2018). ACT UP for evaluating sources: Pushing against privilege. College & Research Libraries News, 79(10), 551–552. https://doi.org/10.5860/crln.79.10.551

Sye, D., & Thompson, D. (2023). Tools, tests, and checklists: The evolution and future of source evaluation frameworks. Journal of New Librarianship, 8(1), 76–100. https://doi.org/10.33011/newlibs/13/9

Tardiff, A. B. (2022). Have a CCOW: A CRAAP alternative for the Internet Age. Journal of Information Literacy, 16(1), 119–130.

Thompson, D. S. (2019). Teaching students to critically read digital images: A visual literacy approach using the DIG method. Journal of Visual Literacy, 38(1), 110–119. https://doi.org/10.1080/1051144X.2018.1564604

Walraven, A., Brand-Gruwel, S., & Boshuizen, H. P. A. (2009). How students evaluate information and sources when searching the World Wide Web for information. Computers & Education, 52(1), 234–246. https://doi.org/10.1016/j.compedu.2008.08.003

Wichowski, D. E., & Kohl, L. E. (2013). Establishing credibility in the information jungle: Blogs, microblogs, and the CRAAP Test. In M. Folk, & S. Apostel (Eds.), Online credibility and digital ethos: Evaluating computer-mediated communication (pp. 229–251). https://doi.org/10.4018/978-1-4666-2663-8

Wineburg, S., Breakstone, J., Ziv, N., & Smith, M. (2020). Educating for misunderstanding: How approaches to teaching digital literacy make students susceptible to scammers, rogues, bad actors, and hate mongers. Stanford Digital Repository. https://purl.stanford.edu/mf412bt5333

Wineburg, S., McGrew, S., Breakstone, J., & Ortega, T. (2016, November 22). Evaluating information: The cornerstone of civic online reasoning. Stanford Digital Repository. https://purl.stanford.edu/fv751yt5934

Ziv, N., & Bene, E. (2022). Preparing college students for a digital age: A survey of instructional approaches to spotting misinformation. College & Research Libraries, 83(6), 905–925. https://doi.org/10.5860/crl.83.6.905

Zvyagintseva, L., & Blechinger, J. (2023). “The perils of library instruction”: Contextualizing EdTech’s encroachment on teaching through labour theory of value in academic libraries. Canadian Journal of Academic Librarianship / Revue Canadienne de Bibliothéconomie Universitaire, 9, 1–25. https://doi.org/10.33137/cjal-rcbu.v9.40958

Appendix A

List of Sources

Pre-Test

Source #1:

Axe, J. (2023, April 6). The leaky gut diet and treatment plan, including top gut foods. Dr. Axe. https://draxe.com/health/leaky-gut-diet-treatment/

Source #2:

Campos, M. (2023, September 12). Leaky gut: What is it, and what does it mean for you? Harvard Health Publishing. https://www.health.harvard.edu/blog/leaky-gut-what-is-it-and-what-does-it-mean-for-you-2017092212451

Source #3:

Quigley, E. M. M. (2016). Leaky gut – Concept or clinical entity? Current Opinion in Gastroenterology, 32(2), 74–79. https://pubmed.ncbi.nlm.nih.gov/26760399/

Post-Test 1

Source #4:

Li, Y., Shan, Z., & Teng, W. (2021). The iodine status and prevalence of thyroid disorders among women of childbearing age in China: National cross-sectional study. Endocrine Practice, 27(10), 1028–1033. https://www.sciencedirect.com/science/article/pii/S1530891X21001245

Source #5:

Wilson, D. (n.d.). WTS overview. Wilson’s Temperature Syndrome. https://www.wilsonssyndrome.com/identify/wts-overview/

Source #6:

IGenX. (n.d.). What are the symptoms of Chronic Lyme disease? https://igenex.com/tick-talk/what-are-the-symptoms-of-chronic-lyme-disease/

Source #7:

National Institute of Arthritis and Musculoskeletal and Skin Diseases. (2021, June). Fibromyalgia. https://www.niams.nih.gov/health-topics/fibromyalgia

Source #8:

University Health Network Newsroom. (2014, October 30). What is white matter disease? It might be the cause of vascular dementia. https://www.uhn.ca/corporate/News/Pages/what_is_white_matter_disease.aspx

Source #9:

Waubra Foundation. (n.d.). Symptoms of wind turbine syndrome. https://waubrafoundation.org.au/information/residents/symptoms/¹

Source #10:

Johns Hopkins Medicine. (n.d.). West Nile virus. https://www.hopkinsmedicine.org/health/conditions-and-diseases/west-nile-virus

Post-Test 2

Source #11:

Cafasso, J. (2018, September 17). Weil’s disease. Healthline. https://www.healthline.com/health/weils-disease

Source #12:

Love, L. (2018, August 30). The benefits and uses of colloidal silver. Love & Chiropractic. https://www.drlaurenlove.com/the-benefits-and-uses-of-colloidal-silver/

Source #13:

Mayo, J. D. (n.d.). Do you suffer from magnetic field deficiency syndrome? Access Possibilities. https://accesspossibilities.com/blogs/news/do-you-suffer-from-magnetic-field-deficiency-syndrome

Source #14:

Berman, J., & Dugdale, D. C. (2023, June 19). Calcium supplements. MedlinePlus. https://medlineplus.gov/ency/article/007477.htm

Source #15:

Di Donato, V., Caruso, G., Petrillo, M., Kontopantelis, E., Palaia, I., Perniola, G., Plotti, F., Angioli, R., Muzii, L., Benedetti Panici, P., & Bogani, G. (2021). Adjuvant HPV vaccination to prevent recurrent cervical dysplasia after surgical treatment: A meta-analysis. Vaccines, 9(5), 410. https://pubmed.ncbi.nlm.nih.gov/33919003/

Source #16:

Han, Y., Yang, Z., Fang, S., Zhang, M., Xie, Z., Fan, Y., & Zhao, T. (2021). A data-mining based analysis of traditional Chinese medicine in diagnosing and treating COVID-19. The Anatomical Record. https://www.proquest.com/docview/2544759745?parentSessionId=LYPHcduO0Qpq%2B6c2H6u2Zqq6rbxkdL%2FQmTGuRTUDSao%3D

Appendix B

Handout: The CRAAP Test

Appendix C

Handout: Source 1 Beyond the Source Framework

Front

Back