04_Barr_etal

Developing a Framework to Improve Information and Digital Literacy in a Bachelor of Paramedic Science Entry-to-Practice Program

The aims of this research were threefold: (1) explore the perceptions of paramedic educators regarding information literacy education (ILE); (2) reveal the information literacy (IL) abilities of paramedic science students; and (3) develop an integrated IL framework for paramedicine. Two key findings arose from this research. First, paramedic educators need to include higher functions of synthesis and creative thinking in ILE. Second, ILE needs to be mapped and scaffolded to incrementally develop these skills across each year of the program. The results informed the development of a programwide ILE framework implemented in 2019 and to be evaluated in 2021.

Introduction

In Australia, New Zealand, and the United Kingdom, paramedic education has transitioned from the vocational education sector to the university sector. Personal communication with nationally accredited educators in the United States has also indicated a strong interest in transitioning their paramedic training to a baccalaureate. Paramedics in Australia are engaged to provide community–based emergency healthcare,1 and the Australian Health Practitioner Regulation Agency (AHPRA) registers paramedics once they graduate from an accredited, usually three-year, university program. To facilitate and sustain the change in education of paramedics, upskilling of clinicians to serve as faculty staff has occurred. In Australia, Domain 3 of the Higher Education Standards Framework (Threshold Standards) 20152 requires faculty staff to have: a) knowledge of the discipline informed by continuing scholarship, research, or advances in practice; b) skills in contemporary teaching, learning, and assessment principles relevant to the discipline; and c) hold a qualification in a relevant discipline at least one level higher than is awarded for the course of study. Accordingly, many paramedics working as faculty staff within universities have obtained postgraduate qualifications in education or research and some have completed a PhD.

During this transition, Australian government interventions during the last decade have increased participation rates in higher education.3 A new demand-driven system with funding incentives has substantially increased the proportion and diversity of students who are successful in applying for university admission, and this has broadened student preparedness for tertiary education.4 In this era where social inclusion policies have changed patterns of participation, universities still assume the responsibility of maintaining academic standards across all participants.5 However, Australian research has shown significant variance in information literacy (IL) skills among commencing university students, and often poor acquisition of IL during their program of study.6

Paramedicine is an evolving profession responding to changing community needs,7 situated in a healthcare setting where evidence rapidly changes and diagnostic uncertainty is part of clinical decision making.8 Both AHPRA and the Australian Industry and Skills Committee (Skills IQ) expect paramedics to analyze and critically evaluate the information they have collected to make clinical judgments.9 Hence, IL is a foundational skill in paramedicine and is a critical component of evidence-based practice (EBP), as it allows a practitioner to remain current in their field.10 The skills developed during the formative undergraduate years must equip graduates with the essential problem-solving and critical-thinking skills required for their continued professional development and concomitant delivery of safe and effective healthcare. As such, staff responsible for curriculum design must consider the needs of students from varying backgrounds and with differing capabilities when embedding information literacy education (ILE) into the curriculum.

The necessity of improving the EBP skills of graduating paramedics in Australia has enabled a productive librarian-faculty collaboration to move forward in this present research project with the aim of genuine curriculum renewal that will graduate clinically effective, information-literate practitioners. This research blends three distinct discipline strands: paramedic education; educational theory; and ILE with a view to effecting real change in the education of Australian paramedics.

Developing a sustainable program of ILE was at the centre of this project.

Literature Review

The urgency of creating an information literate society has long been formally recognized in the United States,11 the UK,12 and Australia.13 The American Library Association14 has defined IL as “the set of integrated abilities encompassing the reflective discovery of information, the understanding of how information is produced and valued, and the use of information in creating new knowledge and participating ethically in communities of learning.” The Library and Information Association in the UK15 has broadened this definition and implied it is necessary to empower a full life, “Information literacy is the ability to think critically and make balanced judgements about any information we find and use. It empowers us as citizens to develop informed views and to engage fully with society.

Biggs’ principle of “constructive alignment” is based on the Constructivist theories of Bloom, Vygotsky, Dewey, and others and is now an accepted, widely implemented guide underpinning curriculum development.16 The concept of “Informed learning”17 proposes that, when students are aware of information use in their learning, they become more aware of how they may use information in their future academic, personal, and professional lives. Informed learning echoes constructive alignment18 as embedding ILE across a program, requiring three considerations: (1) identifying the critical aspects of intended learning, which include both using and communicating information and subject content; (2) designing assessment methods to evaluate a student’s ability to use information and subject content; and (3) designing transformative learning activities that enable students to pass through threshold concepts regarding important aspects of subject content by using information in a critical way. A recent systematic review19 has demonstrated wide support for these concepts, but it is not universally applied across programs; and ILE within programs may range from a single 50-minute session assessed with a simple paper survey to fully course-integrated IL programs with robust assessment.

There have been repeated attempts to reinvigorate the interest in IL as an essential competency in all disciplines,20 including paramedicine,21 and there is a body of research exploring how students access information or how to improve specific IL skills in single courses or units of study.22 In the health sciences, the discipline of nursing has had the most attention in relation to ILE. This could simply be because there are far more nurses being trained, in more places, and at any given time than in any of the other health disciplines. There have been active nursing-related ILE projects for decades, including some curriculum-aligned and evaluated examples.23 However, not all such efforts have been satisfactory,24 indicating that curriculum design to guide “what the learner does”25 is critical to success.

In the health science disciplines, there are few quality studies exploring the integration of IL across whole programs of study. Much previous librarian-led innovation has failed because of a lack of faculty knowledge or commitment26 as successful ILE demands a committed ongoing partnership between library and faculty. The centrality and necessity of incorporating ILE across many disciplines has long been noted,27 and serious attempts have been made to embed ILE into whole programs.28 Notable Australian studies are by Wang,29 which described integration of ILE in an engineering degree; Salisbury and Sheridan,30 which describes ILE renewal across multiple programs; and the decade-long development and implementation of the RSD framework.31 Elsewhere, however, partial efforts, especially involving the first year of programs, have been much more common,32 possibly because they are easier to achieve in the short term and require less high-level institutional commitment.

To facilitate the required curriculum change, this project had three aims: (1) to explore the perceptions of paramedic faculty staff working in Australian university programs regarding IL; (2) to explore the IL abilities of a defined cohort of the University of the Sunshine Coast (USC) paramedic students; and (3) to develop a framework for embedding ILE through the entire Bachelor of Paramedic Science curriculum at USC, informed by aims 1 and 2.

Methods

This project has defined curriculum as a plan for the acquisition of knowledge and skills leading to a credentialed qualification and professional registration. It includes both the official course syllabus, the hidden curricula and extracurricular activities a student is exposed to during a program of study.

Information literacy has multiple definitions, as it is context bound. To suit the Australian context, Wang’s Information Literacy Curricular Integration Model33 was chosen to inform the curriculum changes required to develop students’ ability to understand basic scientific data and to evaluate and synthesize information.

To contextualize the attributes of IL to Australian tertiary programs and students, our project used the six broad standards of the Australian and New Zealand Information Literacy (ANZIL) framework34 to outline the breadth of IL expected within Australian curricula (see table 1). Following a period of debate, the Association of College and Research Libraries rescinded their Information Literacy Competency Standards for Higher Education (2000) and replaced them with their Framework for Information Literacy in Higher Education (2016).35 It can be argued, however, that a Standards-based approach usefully articulates with the necessary requirements of EBP.36

TABLE 1

Council of Australian Librarians (CAUL) Information Literacy Standards and Elements

1. Recognizes the need for information and determines the nature and extent of the information needed

1.1 Defines and articulates the information needed

1.2 Understands the purpose, scope, and appropriateness of a variety of information sources

1.3 Re-evaluates the nature and extent of the information needed

1.4 Uses diverse sources of information to inform decisions

2. Finds needed information effectively and efficiently

2.1 Selects the most appropriate methods or tools for finding information

2.2 Constructs and implements effective search strategies

2.3 Obtains information using appropriate methods

2.4 Keeps up to date with information sources, information technologies, information access tools, and investigative methods

3. Critically evaluates information and the information-seeking process

3.1 Assesses the usefulness and relevance of the information obtained

3.2 Defines and applies criteria for evaluating information

3.3 Reflects on the information-seeking process and revises search strategies as necessary

4. Manages information collected or generated

4.1 Records information and its sources

4.2 Organizes (orders/classifies/stores) information

5. Applies prior and new information to construct new concepts or create new understandings

5.1 Compares and integrates new understandings with prior knowledge to determine the value added, contradictions, or other unique characteristics of the information

5.2 Communicates knowledge and new understandings effectively

6. Uses information with understanding and acknowledges cultural, ethical, economic, legal, and social issues surrounding the use of information

6.1 Acknowledges cultural, ethical, and socioeconomic issues related to access to, and use of, information

6.2 Recognizes that information is underpinned by values and beliefs

6.3 Conforms with conventions and etiquette related to access to, and use of, information

6.4 Legally obtains, stores, and disseminates text, data, images, or sounds

For our purposes, it was judged best to proceed using the standards based approach of ANZIL rather than the newer ACRL Framework approach.

Data Collection

This project used sequential mixed-methods37 to investigate two research questions. This was the first research question: How do members of the Network of Australasian Paramedic Academics (NAPA) rate the importance of information literacy education in paramedic programs? NAPA is a special interest group of Paramedics Australasia (PA), a peak professional body representing Australian and New Zealand paramedics. PA provided in-kind support for this study by sending emails to all members of the NAPA to promote the study. All participants of the study were employed as faculty staff by Australian or New Zealand universities. No other demographic data were collected.

A modified Delphi process, as described by Lindstone and Turoff,38 was used with three iterative rounds. This method was chosen because it could inform future educational design by describing the aspects of ILE that paramedic educators value. The Delphi process was conducted online using the survey program Opinio® to achieve consensus on learning outcomes based on the ANZIL standards. NAPA was chosen as the reliability of data from Delphi studies are improved when subjects are drawn from the specialized area of knowledge related to the target issue.39

The modified Delphi process began with an initial survey (see appendix A) sent to the respondent group. This survey had two sections; the first proposed six short-answer questions concerning the participants’ perceptions of the information literacy standards, as defined by the ANZIL framework. The second section asked the participants to rate the value they placed on learning outcomes related to the different skills that comprise information literacy and to indicate at which chronological point in an undergraduate program that the student should be able to demonstrate attainment of that outcome. The learning outcomes were developed from generic IL learning outcomes publicly available on Australian University web pages and presented in the survey (see appendix A) attached with the appropriate ANZIL standard. We did this for two reasons: (1) because of the relative inexperience of paramedic faculty, it was deemed beneficial to present several learning outcomes based on the standards for the participants to consider; and (2) it may give rise to themes within each standard to help design learning activities attached to individual standards.

The first survey section was presented once only to the participants. The results of the second section were analyzed, and the questionnaire was modified for each iterative round. The items that achieved greater than 80 percent consensus were removed so that each iterative round of the questionnaire contained only the learning outcomes that had not reached consensus on importance or place in a program. With each iteration, the respondent group was given the opportunity to re-evaluate their original answers to questions that did not achieve consensus based upon examination of the group response.

The second research question—How do paramedic students at USC perceive their information literacy abilities?—was explored through an online questionnaire sent to all USC paramedic students (see appendix B). The findings will inform teaching strategies to enable foundation skills and the incremental development of higher order skills across the three years of the program.

The student survey was based on two validated surveys40 developed by CAUL that were linked to the ANZIL framework. The survey was modified for this project based on the recommendations of the project reference group consisting of a librarian, an academic skills advisor, a curriculum designer, and a senior paramedic member of faculty. The survey was piloted with a small convenience sample of four faculty academic staff and 14 students from Bachelor of Nursing and Bachelor of Sport and Exercise Science programs at USC. This enabled the assessment of face validity and refinement of the question stems and response choices to ensure that the questionnaire could be completed within 15 minutes. Construct validity was assessed through discussions with reference group members, email feedback received from pilot participants and their responses to items.

All USC paramedic students (n = 470) were invited to participate by email. Data collection from student participants was conducted online using Opinio®. Participation was anonymous and voluntary. The survey was open for four weeks, with weekly reminders sent by email to potential participants to increase response rate. No incentives were offered.

Data Analysis

For the Delphi study, the categorical data were summarized using descriptive statistics to determine whether consensus was reached. For the student survey, the nontextual data (categorical, ordinal, and Likert scale responses) were summarized using descriptive statistics for variables. Inferential statistics were then computed using the Fisher’s exact test (FET) to test for associations among variables, gender, and year level in the program.

The textual data from the Delphi study and student survey were analyzed through an interactive process of describing, classifying, and connecting information using NVivo Version 10. This process used a combination of collating data around a small number of a priori codes followed by content analyses, where inductive codes were used to capture emergent themes.41 The content analysis was conducted by one researcher and then checked by a second researcher to improve the accuracy of coding.

Ethical clearance was obtained from the University of the Sunshine Coast Human Research and Ethics Committee (A/15/662 and A/17/922).

Results and Discussion

The results and discussion for the two studies will be presented in turn. We will then discuss how the results have informed curriculum renewal in our program.

A wide range of definitions of IL were provided by the Delphi participants. For example, one participant defined IL as “Recognising when information is needed and having the necessary skills to get hold of that information,” and another provided a deeper insight: “identifying an information gap, knowing how and where to find the information required, being able to judge the quality of the information and to use the information appropriately.” Table 2 shows the frequency of the themes that arose from the participant comments.

TABLE 2

Themes Arising from NAPA Participants (n = 29) Responses to Defining the Meaning of Information Literacy as a Skill

Theme

Frequency, Percentage

Find/Collect

n = 22, 84.6%

Critique/Evaluate

n = 14, 53.8%

Integrate

n = 10, 38.5%

Understand

n = 10, 38.5

Identify Knowledge Required

n = 9, 34.6%

Communicate

n = 5, 19.2%

Learn

n = 1, 3.8%

Manage Data

n = 1, 3.8%

All the Delphi participants indicated that students undertaking a paramedic entry-to-practice qualification need to develop information literacy skills. The comments provided by participants indicated that educational design needs to ensure that students learn how to challenge, critique, and evaluate information. For example, one participant commented: “students need to be able to locate, evaluate and critique information on a regular basis. These skills are required throughout their professional life.”

Delphi participants generally agreed with the CAUL statements regarding benefits associated with the development of IL skills (see table 3). Participant comments also described three additional benefits of ILE: empowerment, safe clinical practice, and improved work readiness.

TABLE 3

Level of Agreement (%) of Delphi Participants (n = 19) with Statements Regarding the Benefits to Students Who Have Developed Information Literacy Skills

Statements

Disagree

Neither

Agree

Students who develop information literacy skills can engage in independent learning through constructing new meaning, understanding, and knowledge.

5.3

0.0

94.7

Students who develop information literacy skills derive satisfaction and personal fulfilment from using information wisely.

10.5

21.1

68.4

Students who develop information literacy skills can search for and use information for decision making, and problem-solve to address personal, professional, and societal issues.

5.3

0.0

94.7

Students who develop information literacy skills can demonstrate social responsibility through a commitment to lifelong learning and community participation.

0.0

26.3

73.7

Note: Statements adapted from the ANZIL information literacy framework.

Consensus on ILE Learning Outcomes and Position within a Program

Delphi participants indicated the level of importance (high, moderate, or low) of learning outcomes and place in the program where these outcomes need to be achieved (early, middle, or late). Consensus on the importance of place was achieved if 80 percent of participants voted for the same category of importance low, moderate, or high.

After three rounds of voting, half (n = 33) of the presented learning outcomes were considered to be of high importance in a paramedic program. Another 3 (5%) of the learning outcomes were judged to be of medium importance and consensus was not reached on the rest (n = 28, 42%). The lowest level of consensus occurred with learning outcomes grouped under standard 3 “Critically evaluates information and the information-seeking process,” with only one quarter of these learning outcomes reaching consensus. No consensus was gained for the three learning outcomes on assessing the usefulness and relevance of the information obtained, one out of four achieved consensus on learning outcomes relating to reflecting on the information-seeking process and revising search strategies, and only two from the five learning outcomes on defining and applying criteria for evaluating information reached consensus on importance. To assist designers of paramedic curricula, we have collated the themes under each ANZIL standard (teaching activities or context) arising from learning outcomes that achieved consensus for a moderate to high level of importance in a paramedic program (see appendix C).

In addition, the Delphi participants indicated that many aspects of IL should be taught at the commencement of a program of study (see table 4), such as finding, describing, storing, and managing information. The front loading of foundation IL is a logical position, as these skills support the development of higher order skills such as analysis, evaluation, critique, and synthesis.

TABLE 4

Learning Outcomes That Reached Consensus (>80%) for Importance
(High, Moderate, or Low) and the Position in the Curriculum (Early, Middle, Late, or
No Consensus [NC]) When the Skill Should Be Achieved

Learning Outcome

Importance

Point in Program

1. Recognizes the need for information and determines the nature and extent of the information needed

1.1.1 Explores general information sources to increase familiarity with the topic

High

Early

1.1.4 Identifies key concepts and terms to formulate the focus questions

High

NC

1.2.1 Understands how information is organized and disseminated

High

Early

1.2.2 Recognizes that the context of the topic changes among the disciplines

High

NC

1.2.3 Differentiates between and values the variety of potential sources of information

High

NC

1.2.5 Differentiates between primary and secondary information sources

High

Early

1.3.1 Reviews the initial information to clarify, revise, or refine the question

High

Middle

1.4.2 Uses a range of sources to understand the issues

High

NC

1.4.3 Uses information for decision making and problem solving

High

NC

2. Finds needed information effectively and efficiently

2.1.1 Investigates benefits and applicability of various investigative methods or research questions

High

NC

2.1.3 Consults with librarians and other information professionals to help identify information access tools

High

Early

2.2.1 Develops a search plan appropriate to the research question

High

NC

2.2.2 Identifies keywords, synonyms, and related terms for the information needed

High

Early

2.2.4 Constructs and implements a search strategy using appropriate commands

High

Middle

2.4.2 Uses research update services

Moderate

NC

3. Critically evaluates information and the information-seeking process

3.2.2 Analyzes the structure and logic of supporting arguments or methods

High

Middle

3.2.3 Recognizes and questions prejudice, bias, deception, or manipulation

High

NC

3.3.3 Reviews information access tools used and expands to include others as needed

Moderate

Middle

4. Manages information collected or generated

4.1.2 Differentiates between the types of sources cited and understands the correct citation style for a wide range of resources

High

Early

4.1.3 Is able to access the correct citation style information

High

Early

4.1.4 Records all pertinent citation information for future reference and retrieval

High

Early

4.2.1 Compiles references in the required bibliographic format

High

Early

5. Applies prior and new information to construct new concepts or create new understandings

5.1.1 Determines whether information satisfies the research or other information need and whether the information contradicts or verifies information used from other sources

High

Middle

5.1.2 Recognizes interrelationships between concepts and draws conclusions based upon information gathered

High

Middle

5.1.3 Selects information that provides evidence for the topic and summarizes the main ideas extracted from the information gathered

High

NC

5.1.5 Extends initial synthesis at a higher level of abstraction to construct new hypotheses

Moderate

Late

5.2.2 Uses a range of appropriate information technology applications in creating the product (such as report, essay, journal)

Moderate

Middle

5.2.3 Incorporates principles of design and communication appropriate to the environment

Moderate

Middle

5.2.4 Communicates clearly and in a style to support the purposes of the intended audience

High

Early

6. Uses information with understanding and acknowledges cultural, ethical, economic, legal, and social issues surrounding the use of information

6.1.2 Understands and respects indigenous and multicultural perspectives of using information

High

Early

6.2.2 Applies reasoning to determine whether to incorporate or reject viewpoints encountered

High

Middle

6.2.3 Maintains an internally coherent set of values informed by knowledge and experience

High

NC

6.3.1 Demonstrates an understanding of what constitutes plagiarism and correctly acknowledges the work and ideas of others

High

Early

6.3.2 Participates in online discussions following accepted practices (such as Netiquette)

High

Early

6.4.1 Understands fair dealing in respect of the acquisition and dissemination of educational and research materials

High

Early

6.4.2 Respects the access rights of all users and does not damage information resources

High

Early

6.4.3 Obtains, stores, and disseminates text, data, images, or sounds in a legal manner

High

Early

6.4.4 Demonstrates an understanding of intellectual property, copyright, and fair use of copyrighted material

High

Early

Note: CAUL, Council of Australian University Librarians; IL, information literacy; NC, indicates that no consensus was reached regarding when the skill should be demonstrated.

University graduates must be able to develop IL skills to prepare for a changing practice environment.42 Scope of practice for paramedics has changed substantially over the last decade,43 which is associated with a requirement for higher levels of critical thinking and clinical reasoning in paramedic-led healthcare due to increased complexity in decision making. Yet the Delphi results indicate that lower-order IL skills are more valued than higher-order skills. This was demonstrated by a lack of consensus regarding learning objectives pertaining to Standard Three: critically evaluates information and the information seeking process. Furthermore, the Delphi participant definitions of information literacy emphasized lower-order skills of finding or collecting information rather than evaluation and synthesis. This finding is unusual because IL is considered the basis for evidence-based-practice,44 but it is consistent with previous Australian studies showing that IL is often underdeveloped in university graduates.45

Student Survey

The survey was administered in semester one 2017 and open for four weeks (27 March to 28 April 2017). The survey was sent to 470 students enrolled in the Bachelor of Paramedic Science program at USC from which 111 students started the survey and 77 students fully completed it. Not all participants answered all questions, and the 30 partially completed surveys did not include demographic attributes (see table 5). Respondents reported their sex as males (n = 30), female (n = 47), and missing (n = 30), and no statistically significant associations between sex and response variables were found using = 0.05. The partially completed surveys were included when reporting descriptive statistics but treated as missing when exploring associations between demographic attributes (sex, position in program, age, and level of education) and other variables.

TABLE 5

Demographic Attributes of Student Participants (n = 77, missing = 34),
Who Provided Their Demographic Attributes

Attribute

Detail

Sex

Male

Female

39% (n = 30)

61% (n = 47)

Position in Program

Entry

Middle Year(s)

Final Year

48.1% (n = 37)

26.0% (n = 20)

26.0% (n = 20)

Age

Range

Mean

43 years; 17–60 years

23 years

Level of Education

Year 12

Diploma or Certificate

Bachelor’s Degree

Other

53.9% (n = 41)

26.3% (n = 20)

13.2% (n = 10)

6.6% (n = 5)

Organizing Information

Participants were asked: Briefly explain how you organize the information and sources gained through information searches for an assignment topic (Q10). The 68 comments aligned to three themes: decisions, indexing, and storage. The decisions theme included keeping the information based on: the relevance (n = 9); the quality of information (n = 4); and the date of publication (n = 2). The indexing theme included alphabetical (n = 1), chronological (n = 2), order of relevance (n = 3), reference list (n = 5), making a research book (n = 1), making a themes table (n = 6), and by topic or section of assignment (n = 17 and n = 3 respectively). The storage theme included hard copy (n = 8) or soft copy (n = 7). Of note was that only a minority of students used bibliographic software to assist with the organization and citation of information. The benefits of bibliographic software are beyond simple referencing, as many applications allow linking of articles through memo writing and annotating data that is entered. This process may facilitate metacognition relating to the quality of the information being collected.46

Understanding Scholarly Requirements: Cultural, Sources [Diversity and Type] and Citations

Participant responses describing what cultural sensitivity meant in the context of writing an assignment fell into three groups: (1) awareness, (2) being respectful, and (3) minimizing bias and prejudice. Twenty-five comments were made regarding awareness and included being aware of differences in philosophies, that cultural perspectives influence needs, and that ethical considerations need to be considered. Twenty-three comments were made about being respectful, which included not causing offense, and 10 comments were made about minimizing bias and prejudice.

Students generally reported good scholarly habits when completing written assignments. Strong evidence was found that final-year students were more likely to report increased use of multiple sources of information to reach a conclusion (FET = 23.467, P = 0.002; first-year students 10.8% compared to final-year students 60%). Half of the participants (54.1%, n = 46) indicated that they were required to cite scholarly articles or textbooks to support their argument in a reflective essay, and this perception also increased with each additional year of study (entry year 24.3% compared to final year 95.0%).

Students reported increased use of a diverse range of sources with each year in the program (FET = 9.455, P = 0.027; entry year 24.3% compared to final year 65.0%). Participants were also asked to indicate from a list which sources of information they used to complete an assignment (Q4). The students indicated that they sought information mostly from peer-reviewed journals (97.3%) and textbooks (82.0%) followed by government reports (64.9%) and online reference sites such as Medscape (60.4%). Collaborative encyclopaedias such as Wikipedia and news media were the least used (12.6% and 2.7% respectively). No statistically significant changes in response were found between program years. Only the first-year participants reported using news media for assignments where this was not appropriate for the task. However, the use of collaborative encyclopedias such as Wikipedia remained the same across the program, and the use of textbooks for citing high-level evidence in assignments was still high at the end of the program. This may indicate a lack of understanding of assignment requirements, poor design of assessment tasks, or a lack of skills to access published evidence.

Approximately three-quarters of students indicated that they often or always worked out how to explain a new idea effectively (73.7%), set out the main ideas from each source, and then combined them to generate new knowledge for an assignment (78.5%). No statistically different changes in response pattern was found between participant year level and the items describing checking whether a concept is still current while developing an answer for an assignment, using a systematic process to help judge the relevance of the information, or using bibliographic software to manage collected information.

Most students reported good behaviors of being aware of plagiarism. Three-quarters (74.1%, n = 63) of the survey participants indicated that they have read USC policies regarding plagiarism and there was no statistical evidence of change in response by year level (FET = 7.059, P = 0.112). However, there was a statistical difference (FET = 13.330, P = 0.022) in the reported use of the USC Style Guide to ensure correct citation of information. From first to third year, there was a slight increase in those reporting they often used the style guide (24.3% to 30.0%) and a large reduction in those reporting they always used the style guide (62.2% to 40.0%).

Communicate Scientific Research

Approximately half (52.9%, n = 45) of the participants reported that they could effectively communicate statistical research outcomes to others in writing (Q19). Nearly a third (32.9%, n = 28) of participants were not sure, and the remainder (14.1%, n = 12) indicated that they could not. No statistical association was found between response type and year of study.

The participants were asked to identify the correct definitions of scientific tests and types of data among a range of choices. Table 6 shows the proportion (%) of correct responses for the statistical concepts explored. Original data was grouped into “I’m not sure,” two incorrect choices, and one correct choice. The data was dichotomized for analysis into correct and incorrect. Students were asked to use the “I’m not sure” choice instead of guessing. While most students perceived that the available statistical evidence within an article was important to consider when selecting information for an assignment (see table 6), the intention did not necessarily transfer into behavior. Additionally, the findings indicate that students have a poor understanding of foundational statistical tests that are used to describe the strength of scientific evidence. This is of concern when students value making judgments about statistical constructs such as P values, confidence intervals, odds ratios, limitations of studies, and levels of evidence when selecting information. These unexpected results may explain why a substantial proportion of students use inappropriate data such as online discussions, blogs, or podcasts for assignments that require interpretation and application of high-quality data.

TABLE 6

Student Perceptions and Behaviors toward Using Statistical Information When Selecting Resources, and Their Understanding of Basic Statistical Concepts

Concept

Important InformationΩ (%) n = 74

Actively ConsiderΨ (%) n = 101

Able to Define Concept (%) n = 76

P Value

66.7

28.7

19.5

Confidence Intervals

41.4

23.8

17.1

Odds Ratio

62.7*

18.6*

14.3

Limitations of Study Design

84.1*

50.6*

Level of Evidence

84.5

72.2

Mean

81.6

Median

76.3

Number Needed to Treat

14.5

ΩPerception: proportion who agreed or strongly agreed with the statement “When you select information for a university assignment, it is important to assess the quality of the information based on the following, if provided” (Q20).

ΨBehavior: proportion who reported they often or always considered statistical information when considering to use an article or information within assignments, if supplied (Q5–9).

The proportion of participants who correctly identified the definition of statistical outcomes form a range of choices (Q23–28).

*Indicates that statistically significant changes were found between the years of study. However, these changes were too small to be practical differences.

— Not explored.

There was evidence that students throughout each year of the program did not understand the concept of levels of evidence. Three questions were developed to explore understanding of the National Health and Medical Research Council hierarchy of evidence47 used in the USC paramedic program. These questions asked participants to indicate the study with the higher level of evidence (LOE) from a choice of two in each question. Two-thirds (68.4%, n = 54) correctly identified that systematic reviews of randomised trials had a higher LOE than expert consensus. One-third (37.7%, n = 29) correctly identified that a rigorous qualitative study is considered a higher level of evidence than a case series. Almost half (45.5%, n = 35) correctly identified that a randomized control trial is considered a higher LOE than a systematic review of descriptive or qualitative studies. No statistical evidence for change in responses being associated with year of study in the program is also a concern when students study science research methods in the second year of the program.

Informing Curriculum Design

Curriculum integration of IL is advocated in Australia48 and internationally,49 with an integrative approach to the development of IL skills described as the most effective way of providing ILE.50 This approach involves critical alignment of institutional level policies, program level planning, and course outcomes and activities (see table 7). The pedagogical approach also needs to account for the learners’ previous experience of ILE and must: be inclusive, transformative, encourage reflection and active learning; be grounded and contextualized to disciplines; and allow student collaboration and creativity.

TABLE 7

Integration of Information Literacy

Level of Institution

Details

Institution Level

IL can be integrated into faculty curriculum through faculty teaching and learning policies and guidelines.

Program

Mapping of staged development of IL competencies. Students need to be provided opportunities to interact with information on an ongoing basis: for example, across curricula from week 1 to week 12 horizontally and from entry into a program of study (year 1) to finishing their study (final year).

Course (Unit of Study) Level

IL can be integrated into a course curriculum by contextualizing it into the course learning outcomes, course assignments and assessments, and class, laboratory, and online activities.

Hence, to effectively design curricula and prepare students, faculty staff and librarians must recognize that students in different disciplines use different information-seeking behaviors. For example, paramedicine students (among others) may prefer independent web browsing rather than using library databases to find information.51 By constructively aligning the ILE curricula throughout a program, faculty staff can support learning and teaching activities that engage a diverse range of students and facilitate deeper learning. It allows students to build on previous learning, reflect on their weaknesses and strengths in content knowledge and information use. Curriculum and teaching methods must develop information literacy skills for health science students that include an ability to evaluate the evidence informing practice. The integration of ILE occurs horizontally and vertically across different components and years of study.

The results of the Delphi study lend support in acknowledging important themes to embed within the ILE strategy in the USC paramedic program. The results of the student survey demonstrate that IL must be better scaffolded to ensure learning outcomes are met, particularly in light of low self-efficacy regarding communication of scientific information and poor knowledge acquisition.

In response to the project findings, the academic team at USC held a series of meetings to develop a framework to better integrate ILE within the program. Several mapping tools were developed to determine and document the changes required. Tables 8 and 9 show the template used to map the ILE learning outcomes across the program and provide an exemplar for standards 1 to 3.

TABLE 8

Tool to Map Embedded ILE

Information Literacy definitions

Insert defined by CAUL <<here>>

Insert defined by university <<here>>

Insert defined by program <<here>>

Year 1

Standards

Scaffolds

Target courses*

Year 2

Standards

Scaffolds

Target courses*

Year 3

Standards

Scaffolds

Target courses*

TABLE 9

Definitions of the ANZIL Standards (1 to 3) for Each Year Level and the Required Scaffolds

ANZIL Standard

Year Standard

Scaffolds

1. Recognizes the need for information and determines the nature and extent of the information needed

Year 1: With direction, the student interprets the nature and extent of information the task requires.

Teacher-led deconstruction of task requirements.

Year 2: To work with minimal direction to interpret the nature and extent of information the task requires.

Student-led deconstruction of task requirements.

Year 3: To work independently to interpret the nature and extent of information the task requires.

Independent deconstruction of task requirements.

2. Finds needed information effectively and efficiently

Year 1: With direction, find information required to answer a task using an appropriate search strategy.

Teacher-led session to develop the skills necessary to use multiple strategies to find information.

Year 2: To work with minimal direction to find the information the task requires.

Student-led session to develop effective strategies to find information required.

Year 3: Independently finds needed information using a structured approach.

Independently uses appropriate search tools to efficiently find required information.

3. Critically evaluates information and the information-seeking process

Year 1: With direction, evaluates the relevance of information.

Teacher-led session on assessing the usefulness and relevance of the information.

Year 2: Identifies themes within the information and constructs a logical argument.

Student-led session on defining and applying criteria for evaluating the information.

Year 3: Critically explores themes and critiques the evidence discovered.

Student-led session on critical evaluation of information.  

Each of the ANZIL standards has several elements. To ensure that all elements were being taught and assessed, these were mapped to courses, assessments, and outcomes. Table 10 shows an exemplar of the form used to map the first-year curriculum against ANZIL Standard 1.

TABLE 10

Exemplar of the Tool Used to Map Each Element of the ANZIL Standard 1 to the First-Year Curriculum

ILE Standards

ILE Themes

101

102

103

Assessment Types

Standard 1

       

Courses

Essay

Report

Reflect

Case Study

Poster

Type (Other)

1.1 Defines and articulates the information needed

Explore information sources, identify key concepts and terms

PAR101

PAR102

PAR103

 

1.2 Understands the purpose, scope and appropriateness of a variety of information sources

Organize and disseminate information; topic context is discipline bound; differentiates between a variety of information sources; identifies primary and secondary information sources

PAR101

PAR102

PAR103

 

1.3 Re-evaluates the nature and extent of the information needed

Themes:

Reviews the initial information to clarify, revise, or refine the question

PAR101

PAR102

PAR103

 

1.4 Uses diverse sources of information to inform decisions

Uses a range of sources; uses information for decision making and problem solving

PAR101

PAR102

PAR103

 

The teaching role of librarians has evolved from teaching students how to locate materials in the library to include complex and challenging tasks of how to analyze and evaluate information and think critically.52 As such, the teaching of critical thinking and information evaluation is a shared responsibility facilitated through critical alignment of faculty staff and librarians. The importance of curriculum alignment within any framework was made clear by Schaub et al.53 in finding that some of the IL terms least understood by students were those most frequently found in assignment instructions and covered by librarians during information literacy education sessions. As such, the key to the sustainability of improved ILE is to ensure that the responsibility for its oversight falls within an academic role and is supported by educational design and IL specialists such as librarians.54 Figure 1 shows the model accepted by the USC paramedic faculty designed to be sustainable within USC and did not require additional resources. The model delineates responsibilities and provides structure and assistance to the teaching team.

Figure 1

Suggested Framework to Quality Assure the Embedding Information Literacy Education into Bachelor of Paramedic Science Program at USC

Figure 1. Suggested Framework to Quality Assure the Embedding Information Literacy Education into Bachelor of Paramedic Science Program at USC

Note: The Working Party can be one or more staff.

Conclusion

Due to this research being founded in established IL and educational theory, it has wide applicability to other allied health professional education and nonhealth disciplines, especially those regulated by accrediting bodies such as engineering, education, and others.

The project aimed to develop a sustainable program of ILE within our paramedic degree program. The new mapping tools and quality structure within the program will help to facilitate librarians’, academics’, and support staffs’ understanding of pedagogical interventions to improve IL skills of students. Key to integrating ILE across the program of study is that program coordinators, in conjunction with academic staff, librarians, and curriculum designers, identify appropriate learning outcomes and constructively align these to develop learning activities and assessment tasks that encourage the mastery of a suite of IL skills appropriate to the year level of the unit of study.

The impact of an integrated and constructively aligned ILE is broader than individual student attainment. The immediate impact is that the student body will be the beneficiaries of improved ILE, as this will enhance their professional practice. To find, interpret, analyze, synthesize, and communicate information is an expected graduate attribute by the registering body. It underlies safe and effective practice and builds professional trust within the healthcare team and increases the broader community’s confidence in the delivery of community-based emergency healthcare. For universities, the broader impact will be an opportunity to implement an evidence-based framework for the vertical and horizontal integration of ILE within similar programs.

Evidence for the efficacy of the curriculum renewal will be observable via improved student responses to assignment tasks and captured by further research. We aim to retest the students in the coming years and are confident that the evidence-based approach to curriculum design will achieve the desired outcomes.

APPENDIX A. NAPA Survey

The Council of Australian University Librarians have developed information literacy standards (ANZIL) for use within higher education programs. The purpose of this study is to achieve a consensus view on the important aspects of information literacy education to embed within paramedic education. The ANZIL standards form the basis of the questions and learning outcomes presented in this survey.

The next iteration of this survey will be shorter. The open-ended questions will be removed from further iterations of this survey along with any items in which consensus is reached.

  1. So that we send you the next iteration of the survey, please enter your email address.
  2. Section One—Your thoughts about information literacy

  3. Please describe how you define information literacy?
  4. Please indicate your level of agreement with each of the following statements:
  5. Statement

    Generally disagree

    Neither agree or disagree

    Generally agree

    Students who develop information literacy skills can engage in independent learning through constructing new meaning, understanding, and knowledge.

         

    Students who develop information literacy skills derive satisfaction and personal fulfilment from using information wisely.

         

    Students who develop information literacy skills individually and collectively are able to search for and use information for decision making and problem solving to address personal, professional and societal issues; and

         

    Students who develop information literacy skills can demonstrate social responsibility through a commitment to lifelong learning and community participation.

         

    Please list any additional benefits.

         
  6. Do students undertaking an entry to practice paramedic program need to develop information literacy skills? Yes / No
    1. If you responded yes, briefly describe how students acquire those skills.
    2. If you responded no, briefly describe why you believe this to be the case.
  7. Does the program in which you teach have specific learning objectives for the development of information literacy skills? YES/NO.
    If possible, please add an example of the learning objectives and the year in which they are taught.

In this section of the survey there are 66 examples of learning outcomes relating to the information literacy standards defined by the Australian and New Zealand information literacy (ANZIL) framework.

We are interested in your opinion in two broad areas. The first concerns the value you place on particular learning outcomes related to the different skills that comprise information literacy. The next area concerns the point in an undergraduate entry to practice program of study at which the student should be able to demonstrate attainment of that outcome. Place a tick in the nongreyed-out boxes below to indicate your responses. You may also provide comments.

Standards and learning outcomes

 

Importance

At which time point in an undergraduate entry to practice program of study should the student be able to demonstrate attainment of that outcome

Comments

 

N/A

Low

Moderate

High

Early

Middle

Late

 

1. Recognizes the need for information and determines the nature and extent of the information needed

               

1.1 Defines and articulates the information needed

               

Explores general information sources to increase familiarity with the topic

               

Defines or modifies the information needed to achieve a manageable focus

               

Confers with others to identify a research topic or other information need

               

Identifies key concepts and terms in order to formulate the focus questions

               

Standards and learning outcomes

Importance

At which time point in an undergraduate entry to practice program of study should the student be able to demonstrate attainment of that outcome

Comments

N/A

Low

Moderate

High

Early

Middle

Late

1.2 Understands the purpose, scope, and appropriateness of a variety of information sources

               

Understands how information is organized and disseminated

               

Recognizes that the context of the topic changes among the disciplines

               

Differentiates between, and values the variety of, potential sources of information

               

Identifies the intended purpose and audience of potential resources (such as popular vs scholarly, current vs historical)

               

Differentiates between primary and secondary information sources

               

1.3 Re-evaluates the nature and extent of the information needed

               

Reviews the initial information to clarify, revise, or refine the question

               

Articulates and uses criteria to make information decisions and choices

               

1.4 Uses diverse sources of information to inform decisions

               

Understands that different sources will present different perspectives

               

Uses a range of sources to understand the issues

               

Uses information for decision making and problem solving

               

Standards and learning outcomes

Importance

At which time point in an undergraduate entry to practice program of study should the student be able to demonstrate attainment of that outcome

Comments

N/A

Low

Moderate

High

Early

Middle

Late

2. Finds needed information effectively and efficiently

               

2.1 Selects the most appropriate methods or tools for finding information

               

Investigates benefits and applicability of various investigative method or research question

               

Investigates the scope, content, and organisation of information access tools

               

Consults with librarians and other information professionals to help identify information access tools

               

2.2 Constructs and implements effective search strategies

               

Develops a search plan appropriate to the or research question

               

Identifies keywords, synonyms and related terms for the information needed

               

Selects appropriate vocabulary or classification specific to the discipline or information access tools

               

Constructs and implements a search strategy using appropriate commands

               

Implements the search using investigative methodology appropriate to the discipline

               

2.3 Obtains information using appropriate methods

               

Uses various information access tools to retrieve information in a variety of formats

               

Uses appropriate services to retrieve information needed e.g. document delivery, professional associations, institutional research offices, community resources, experts and practitioners

               

Uses surveys, letters, interviews, and other forms of inquiry to retrieve primary information

               

2.4 Keeps up to date with information sources, information technologies, information access tools and investigative methods

               

Maintains awareness of changes in information and communications technology

               

Uses research update services

               

Purposefully browses print and electronic sources

               

3. Critically evaluates information and the information seeking process

               

3.1 Assesses the usefulness and relevance of the information obtained

               

Assesses the quantity, quality, and relevance of the search results to determine whether alternative information access tools or investigative methods should be utilised

               

Identifies gaps in the information retrieved and determines if the search strategy should be revised

               

Repeats the search using the revised strategy as necessary

               

3.2 Defines and applies criteria for evaluating information

               

Examines and compares information from various sources to evaluate reliability, validity, accuracy, authority, timeliness, and point of view or bias

               

Analyses the structure and logic of supporting arguments or methods

               

Recognises and questions prejudice, bias, deception, or manipulation

               

Recognises the cultural, physical, or other context within which the information was created and understands the impact of context on interpreting the information

               

Recognises and understands own biases and cultural context

               

3.3 Reflects on the information seeking process and revises search strategies as necessary

               

Determines if the original information need has been satisfied or if additional information is needed

               

Reviews the search strategy

               

Reviews information access tools used and expands to include others as needed

               

Recognises that the information search process is evolutionary and nonlinear

               

4. Manages information collected or generated

               

4.1 Records information and its sources

               

Organises the content in a manner that supports the purposes and format of the product e.g. outlines, drafts, poster

               

Differentiates between the types of sources cited and understands the correct citation style for a wide range of resources

               

Is able to access the correct citation style information

               

Records all pertinent citation information for future reference and retrieval

               

4.2 Organises (orders/classifies/stores) information

               

Compiles references in the required bibliographic format

               

Creates a system for organising and managing the information obtained e.g. EndNote, card files

               

5. Applies prior and new information to construct new concepts or create new understandings

               

5.1 Compares and integrates new understandings with prior knowledge to determine the value added, contradictions, or other unique characteristics of the information

               

Determines whether information satisfies the research or other information need and whether the information contradicts or verifies information used from other sources

               

Recognises interrelationships between concepts and draws conclusions based upon information gathered

               

Selects information that provides evidence for the topic and summarises the main ideas extracted from the information gathered

               

Understands that information and knowledge in any discipline is in part a social construction and is subject to change as a result of ongoing dialogue and research

               

Extends initial synthesis at a higher level of abstraction to construct new hypotheses

               

5.2 Communicates knowledge and new understandings effectively

               

Chooses a communication medium and format that best supports the purposes of the product and the intended audience e.g. poster, report, essay

               

Uses a range of appropriate information technology applications in creating the product (e.g. report, essay, journal)

               

Incorporates principles of design and communication appropriate to the environment

               

Communicates clearly and in a style to support the purposes of the intended audience

               

6. Uses information with understanding and acknowledges cultural, ethical, economic, legal, and social issues surrounding the use of information

               

6.1 acknowledges cultural, ethical, and socioeconomic issues related to access to, and use of, information

               

Identifies and can articulate issues related to privacy and security in both the print and electronic environments

               

Understands and respects Indigenous and multicultural perspectives of using information

               

6.2 Recognises that information is underpinned by values and beliefs

               

Identifies whether there are differing values that underpin new information or whether information has implications for personal values and beliefs

               

Applies reasoning to determine whether to incorporate or reject viewpoints encountered

               

Maintains an internally coherent set of values informed by knowledge and experience

               

6.3 Conforms with conventions and etiquette related to access to, and use of, information

               

Demonstrates an understanding of what constitutes plagiarism and correctly acknowledges the work and ideas of others

               

Participates in online discussions following accepted practices e.g. Netiquette

               

6.4 Legally obtains, stores, and disseminates text, data, images, or sounds

               

Understands fair dealing in respect of the acquisition and dissemination of educational and research materials

               

Respects the access rights of all users and does not damage information resources

               

Obtains, stores, and disseminates text, data, images, or sounds in a legal manner

               

Demonstrates an understanding of intellectual property, copyright and fair use of copyrighted material

               

Final Comments

Please make any other comments that you feel are important and not captured earlier.

Thank you for participating. Our research assistant will email you the next iteration of the survey. The items that have reached consensus and the initial open-ended questions will be removed. As such, the survey should be shorter.

APPENDIX B. Student Survey Item Number, Stem, and Response Choices

Question

Item Stem

Response Choice

1

I use diverse sources of information to develop my assignment. (Choose one response only.)

None of the time, Some of the time, Often, Always

2

I check whether a concept is still current while developing an answer for an assignment. (Choose one response only.)

None of the time, Some of the time, Often, Always

3

I use a systematic process to help me judge the relevance of the information. (Choose one response only.)

None of the time, Some of the time, Often, Always

4

Which of the following sources do you use to answer assignments? (Choose all that apply.)

government reports; news media; online collaborative encyclopedias, such as Wikipedia; online reference articles, such as those from WebMD or Mayo Clinic;

peer-reviewed journal articles; textbooks;

other (please list)

Preamble to questions 5 to 9:

Please indicate how often you evaluate the quality of an article using the following information.

5

P values (if present or if provided)

None of the time, Some of the time, Often, Always

6

Confidence intervals (if present or if provided)

None of the time, Some of the time, Often, Always

7

Odds ratio (if present or if provided)

None of the time, Some of the time, Often, Always

8

Limitations

None of the time, Some of the time, Often, Always

9

Levels of evidence

None of the time, Some of the time, Often, Always

10

Briefly explain how you organize the information and sources gained through information searches for an assignment topic.

Comment

11

Do you use bibliographic software such as Endnote to manage collected information? (Choose one response only.)

None of the time, Some of the time, Often, Always

12

When I get a new idea, I work out how to explain it effectively. (Choose one response only.)

None of the time, Some of the time, Often, Always

13

When I write an assignment, I set out the main ideas from each source and then combine them to generate new knowledge for an

assignment. (Choose one response only.)

None of the time, Some of the time, Often, Always

14

When I write an assignment, I rarely need to use multiple sources of information to reach a conclusion. (Choose one response only.)

Strongly Disagree, Disagree, Neither, Agree, Strongly Agree

15

Please tell us what cultural sensitivity means to you in the context of writing an assignment.

Comment

16

I use the USC style guide to ensure that I cite my sources accurately. (Choose one response only.)

None of the time, Some of the time, Often, Always

17

I have read USC’s policies regarding plagiarism. (Choose one response only.)

Yes, No, I’m not sure

18

As a reflective essay is about your personal beliefs, it does not include references to scholarly articles or textbooks. (Choose one response only.)

True, False, I’m not sure

19

I can effectively communicate statistical research outcomes to others in writing. (Choose one response only.)

Yes, No, I’m not sure

Preamble to multiple items in question 20:

When you select information for a university assignment, it is important to assess the quality of the information based on the following, if provided.

20

P values (if present or if provided)

Strongly Disagree, Disagree, Neither, Agree, Strongly Agree

Confidence intervals (if present or if provided)

Strongly Disagree, Disagree, Neither, Agree, Strongly Agree

Odds ratio (if present or if provided)

Strongly Disagree, Disagree, Neither, Agree, Strongly Agree

Levels of evidence

Strongly Disagree, Disagree, Neither, Agree, Strongly Agree

Limitations

Strongly Disagree, Disagree, Neither, Agree, Strongly Agree

Preamble to multiple items in question 21:

In academic research, evidence-based guidelines are presented with a statement of the level of evidence that informed the development. Levels of evidence are ranked. In the following questions, please indicate which form of evidence ranks higher. If you are not sure, please check the “I’m not sure” option.

21

Levels Evidence from (A) a systematic reviews of randomised control trials; or (B) from expert consensus opinion.

A is higher, B is higher, I’m not sure.

Levels Evidence from (A) a qualitative study; or (B) from a case series.

A is higher, B is higher, I’m not sure.

Levels Evidence from (A) a randomized control trial; or (B) from systematic reviews of

descriptive and qualitative studies.

A is higher, B is higher, I’m not sure.

22

At the end of my program of study, I expect that I will have to keep maintaining and updating my knowledge. (Choose one response only.)

Strongly Disagree, Disagree, Neither, Agree, Strongly Agree

23

Which of the following statements about an odds ratio (OR) is correct? (Choose one response only.)

OR=1 Exposure does not affect odds of outcome,

OR>1 Exposure is associated with lower odds of outcome,

OR<1 Exposure is associated with higher odds of outcome,

I am not confident which answer is correct.

24

Which of the following statements about a confidence interval is correct? (Choose one response only.)

A confidence interval of 95% (x, y) is the probability that the true value falling between x and y is 0.95;

The size of the confidence interval is not dependent on the sample size or the standard deviation of the study groups;

A confidence interval of 95% (x-y) is the probability that the true value falling outside x and y is 0.95;

I am not confident which answer is correct.

25

Which of the following statements about P values is correct? (Choose one response only.)

A P-value is the probability that an observed difference is due to random chance when the null hypothesis is true;

A P-value is the probability that an observed difference is due to random chance when the null hypothesis is false;

I am not confident which answer is correct.

26

Which of the following statements is the correct definition for the “mean”? (Choose one response only.)

A mean is the value that occurs most often in a data set;

The mean is the “middle” value in the list of order ranked numbers in a data set;

The average value of numbers in a data set;

I am not confident which answer is correct.

27

Which of the following statements is the correct definition for “median”? (Choose one response only.)

A median is the value that occurs most often in a data set;

The median is the “middle” value in the list of order ranked numbers in a data set;

The median is the average value of numbers in a data set;

I am not confident which answer is correct.

28

Which of the following statements is the correct definition for “number needed to treat” (NNT)? (Choose one response only.)

NNT is the number of patients that must be treated to prevent one additional adverse outcome;

NNT is the number of patients that must be treated to eradicate a disease;

NNT is the number of patients that must be treated to power a study;

I am not confident which answer is correct.

29

What is your current year of study? If you are studying part-time, enter the full-time equivalent year. (Choose one response only.)

Entry year, Middle, Final year

30

What is your gender? (Choose one response only.)

Female, Male

31

What is your age in years?

17–21, 22–26, 27–31, 32–36, 37–41, 42–46, 47–51, 52–56, 57–61

32

What is your highest level of education? (Choose one response only.)

Bachelor level qualification;

Diploma level or certificate;

Year 12 secondary education;

Other (please state).

APPENDIX C. Themes Generated from Learning Outcomes That Reached Consensus Grouped by Each Element of the Council of Australian University Librarians (CAUL) Information Literacy Standards

1. Recognizes the need for information and determines the nature and extent of the information needed

1.1 Defines and articulates the information needed

Themes: Explores information sources, identifies key concepts and terms

1.2 Understands the purpose, scope, and appropriateness of a variety of information sources

Themes: Organizes and disseminates information; topic context is discipline bound; differentiates between a variety of information sources; identifies primary and secondary information sources

Re-evaluates the nature and extent of the information needed

Themes: Reviews the initial information to clarify, revise, or refine the question

Uses diverse sources of information to inform decisions

Themes: Uses a range of sources; uses information for decision making and problem solving

2. Finds needed information effectively and efficiently

2.1 Selects the most appropriate methods or tools for finding information

Themes: Evaluates the applicability of various investigative methods or research question; consults with librarians

2.2 Constructs and implements effective search strategies

Themes: Develops constructs and implements a search strategy; identifies keywords, synonyms, and related terms

2.3 Obtains information using appropriate methods

Themes: nil consensus

2.4 Keeps up to date with information sources, information technologies, information access tools, and investigative methods

Theme: Uses research update services

3. Critically evaluates information and the information-seeking process

3.1 Assesses the usefulness and relevance of the information obtained

Themes: nil

3.2 Defines and applies criteria for evaluating information

Themes: Analyzes the logic of supporting arguments; recognizes and questions prejudice, bias, deception, or manipulation

3.3 Reflects on the information-seeking process and revises search strategies as necessary

Theme: Utilizes a range of information access tools

4. Manages information collected or generated

4.3 Records information and its sources

Themes: Differentiates between the types of sources cited; uses the correct citation style; keeps accurate citation records

4.4 Organizes (orders/classifies/stores) information

Themes: Compiles references in the required bibliographic format

5. Applies prior and new information to construct new concepts or create new understandings

5.1 Compares and integrates new understandings with prior knowledge to determine the value added, contradictions, or other unique characteristics of the information

Themes: Determines whether new information discovered contradicts or verifies information from other sources; recognizes relationships and draws conclusions based upon information gathered; selects information that summarizes the main ideas discovered; constructs new hypotheses

Communicates knowledge and new understandings effectively

Uses the appropriate software to create an artifact; incorporates principles of design and communication in a style to support the purposes of the intended audience

6. Uses information with understanding and acknowledges cultural, ethical, economic, legal, and social issues surrounding the use of information

6.1 Acknowledges cultural, ethical, and socioeconomic issues related to access to, and use of, information

Themes: Respects indigenous and multicultural perspectives of using information

Recognizes that information is underpinned by values and beliefs

Themes: Applies reasoning; maintains internal consistency

6.3 Conforms with conventions and etiquette related to access to, and use of, information

Themes: Understands plagiarism; correctly acknowledges ideas of others; participates appropriately online (netiquette)

6.4 Legally obtains, stores, and disseminates text, data, images, or sounds

Themes: Understands fair dealing and intellectual property and fair use; respects the access rights of all users; obtains, stores, and disseminates text, data, images, or sounds in a legal manner

Notes

1. Paramedicine Board of Australia, “Professional Standards,” Australian Health Practitioner Regulation Agency (AHPRA), available online at https://www.paramedicineboard.gov.au/Professional-standards/Registration-standards.aspx [accessed 30 March 2019]; Paramedics Australasia, “Paramedic Registration,” www.paramedics.org.au [accessed 19 March 2019]; Gail Whiteford, Mahsood Shah, and Chenicheri Sid Nair, “Equity and Excellence Are Not Mutually Exclusive: A Discussion of Academic Standards in an Era of Widening Participation,” Quality Assurance in Education 21, no. 3 (2013): 299–310.

2. Department of Education and Training, “Higher Education Standards Framework (Threshold Standards) 2015” (Canberra: Australian Government, 2015), available online at https://www.teqsa.gov.au/hesf-domain-3 [accessed 6 December 2019].

3. James Griffiths and Marilyn Harrington, “Increasing Participation in Tertiary Education,” in Parliamentary Library Briefing Book: Key Issues for the 45th Parliament (Canberra, Australia: Department of Parliamentary Services, 2016), 58.

4. Daniel Edwards and Eva van der Brugge, “Tracking Demand: An Early Audit of Australia’s New Student Demand Driven System,” Joining the Dots Research Briefings 2, no. 1 (2012): 13; Whiteford, Shah, and Nair, “Equity and Excellence Are Not Mutually Exclusive.”

5. Whiteford, Shah, and Nair, “Equity and Excellence Are Not Mutually Exclusive.”

6. Fiona Salisbury and Sharon Karasmanis, “Are They Ready? Exploring Student Information Literacy Skills in the Transition from Secondary to Tertiary Education,” Australian Academic and Research Libraries 42, no. 1 (2011): 43–58; Fiona Salisbury and Linda Sheridan, “Mapping the Journey: Developing an Information Literacy Strategy as Part of Curriculum Reform,” Journal of Librarianship and Information Science 43 (2011): 185–93.

7. Talal AlShammari, Paul A. Jennings, and Brett Williams, “Emergency Medical Services Core Competencies: A Scoping Review,” Health Professions Education 4, no. 4 (2018): 245–58; Nigel Barr et al., “A Qualitative Exploration of Infection Prevention and Control Guidance for Australian Paramedics,” Australasian Journal of Paramedicine 6, no. 14 (2017): Article 3.

8. Emily Adams et al., “Clinical Reasoning of Junior Doctors in Emergency Medicine: A Grounded Theory Study,” Emergency Medicine Journal 34, no. 2 (2017): 70–75; Annemaree Lloyd, “Informing Practice: Information Experiences of Ambulance Officers in Training and On-Road Practice,” Journal of Documentation 65 no. 3 (2009): 396–419.

9. Paramedicine Board of Australia, “Professional Standards”; SkillsIQ, “Ambulance and Paramedic: Industry Reference Committee Industry Skills Forecast,” SkillsIQ Limited, available online at https://www.skillsiq.com.au/site/DefaultSite/filesystem/documents/Industry-Skills-Forecasts-June2017/2018%20Final%20ISFs/SIQ_SkillsForecast_Ambulance_Paramedics_online.pdf [accessed 30 March 2019].

10. Susan Kaplan Jacobs, Peri Rosenfeld, and Judith Haber, “Information Literacy as the Foundation for Evidence-Based Practice in Graduate Nursing Education: A Curriculum-Integrated Approach,” Journal of Professional Nursing 19, no. 5 (2003): 320–28.

11. American Library Association, Framework for Information Literacy for Higher Education, American Library Association, available online at www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/infolit/Framework_ILHE.pdf [accessed 24 September 2019].

12. Chris Armstrong et al., “CILIP Defines Information Literacy for the UK,” available online at http://eprints.rclis.org/7459/1/Article_Update_25102004.pdf [accessed 24 September 2019]; SCONUL Advisory Committee on Information Literacy, “Information Skills in Higher Education: A Sconul Position Paper,” available online at https://www.sconul.ac.uk/sites/default/files/documents/Seven_pillars2.pdf [accessed 24 September 2019].

13. Australian and New Zealand Information Literacy Framework Principles, Standards and Practice, 2nd ed., Alan Bundy, ed. (Adelaide, South Australia: Library Publications University of South Australia, 2004); John W. Willison, “When Academics Integrate Research Skill Development in the Curriculum,” Higher Education Research & Development 31, no. 6 (2012): 905–19.

14. American Library Association, Framework for Information Literacy for Higher Education.

15. Library and Information Association, “What Is Information Literacy?” available online at https://www.cilip.org.uk/page/informationliteracy [accessed 4 July 2018].

16. John Biggs and Catherine Tang, Teaching for Quality Learning at University: What the Student Does, 4th ed. (Maidenhead, UK: McGraw-Hill, 2011), 95ff.

17. Christine Susan Bruce et al., “Information Literacy and Informed Learning: Conceptual Innovations for Il Research and Practice Futures,” Journal of Information Literacy IL, no. 1 (2017): 4.

18. John Biggs, “Aligning Teaching and Assessing to Course Objectives,” Teaching and Learning in Higher Education: New Trends and Innovations 2, no. April (2003): 13–17.

19. Allison Erlinger, “Outcomes Assessment in Undergraduate Information Literacy Instruction: A Systematic Review,” College & Research Libraries 79, no. 4 (2018): 442–79.

20. Sharon Weiner, “Information Literacy: A Neglected Core Competency,” available online at https://er.educause.edu/articles/2010/3/information-literacy-a-neglected-core-competency [accessed 20 September 2019].

21. Leanne Hamilton, “Embedding Information Literacy into the Prehospital Care Curriculum,” Journal of Emergency Primary Health Care 6, no. 1 (2008): Art 990291; Lloyd, “Informing Practice.”

22. Isaac P. Gilman, “Evidence-Based Information-Seeking Behaviors of Occupational Therapists: A Survey of Recent Graduates,” Journal of the Medical Library Association 99, no. 4 (2011): 307–10; Willison, “When Academics Integrate Research Skill Development in the Curriculum.”

23. Carlos Argüelles, “Curriculum-Integrated Information Literacy (CIIL) in a Community College Nursing Program: A Practical Model,” Community College Journal of Research and Practice 40, no. 11 (2016): 942–53; Lisa Sue Flood, Nanci Gasiewicz, and Terry Delpier, “Integrating Information Literacy across a BSN Curriculum,” Journal of Nursing Education 49, no. 2 (February 2010): 101–04; Susan Franzen and Colleen Bannon, “Merging Information Literacy and Evidence-Based Practice in an Undergraduate Health Sciences Curriculum Map,” Communications in Information Literacy 10, no. 2 (2016), https://doi.org/10.15760/comminfolit.2016.10.2.26; Marilyn Verhey, “Information Literacy in an Undergraduate Nursing Curriculum: Development, Implementation, and Evaluation,” Journal of Nursing Education 38, no. 6 (1999): 252–59.

24. Verhey, “Information Literacy in an Undergraduate Nursing Curriculum.”

25. Biggs and Tang, Teaching for Quality Learning at University.

26. Maryam Derakhshan and Diljit Singh, “Integration of Information Literacy into the Curriculum: A Meta-Synthesis,” Library Review 60, no. 3 (2011): 218–29, https://doi.org/10.1108/00242531111117272.

27. Clive Cochrane, “Embedding Information Literacy in an Undergraduate Management Degree: Lecturers’ and Students’ Perspectives,” Education for Information 24, no. 2/3 (2006): 97–123; Richard Dearden et al., “Aligning Information Literacy with the Faculty Teaching and Learning Agenda,” Australian Academic & Research Libraries 36, no. 4 (2005): 138–52.

28. Dearden, “Aligning Information Literacy with the Faculty Teaching and Learning Agenda”; Helena Ward and Julie Hockey, “Engaging the Learner: Embedding Information Literacy Skills into a Biotechnology Degree,” Biochemistry and Molecular Biology Education 35, no. 5 (2007): 374–80.

29. Xiaoli Wang, “Integrating Information Literacy into Higher Education Curricula and Information Literacy Curricular Integration Model” (doctoral thesis, Queensland University of Technology, 2010), available online at http://eprints.qut.edu.au/41747/.

30. Salisbury and Sheridan, “Mapping the Journey.”

31. University of Adelaide, “Research Skill Development for Curriculum Design and Assessment,” available online at https://www.adelaide.edu.au/rsd/ [accessed 16 March 2019].

32. Pauline J. Ford, N. Foxlee, and Wendy Green, “Developing Information Literacy with First Year Oral Health Students,” European Journal of Dental Education 13, no. 1 (2009): 46–51; Hamilton, “Embedding Information Literacy into the Prehospital Care Curriculum.”

33. Wang, “Integrating Information Literacy.”

34. Bundy, Australian and New Zealand Information Literacy Framework Principles, Standards and Practice.

35. Chartered Institute of Library and Information Professionals (CILIP), “Defining Information Literacy for the UK,” available online at https://www.researchgate.net/publication/28804606_Defining_Information_Literacy_for_the_UK [accessed 10 July 2020]; American Library Association, “Presidential Committee on Information Literacy: Final Report” (1989), available online at www.ala.org/acrl/publications/whitepapers/progressreport [accessed 24 September 2019]; SCONUL Advisory Committee on Information Literacy, “Information Skills in Higher Education: A SCONUL Position Paper.”

36. Franzen and Bannon, “Merging Information Literacy and Evidence-Based Practice.”

37. John W. Creswell, Research Design: Qualitative, Quantitative and Mixed Methods Approaches, 3rd ed. (London, UK: Sage, 2009), 203–25.

38. Harrold Lindstone and Murray Turoff, The Delphi Method: Techniques and Applications (Boston, MA: Addison-Wesley Publishing Company, 1975), 3.

39. Chia-Chien Hsu and Brian A. Sandford, “The Delphi Technique: Making Sense of Consensus,” Practical Assessment, Research and Evaluation 12, no. 10 (2007): 1–8.

40. Ralph Catts, Information Skills Survey Technical Manual (Adelaide, South Australia: Council of Australian University Librarians, 2005).

41. Steve Stemler, “An Overview of Content Analysis,” Practical Assessment, Research and Evaluation 7, no. 17 (2001): 137–46.

42. Paramedicine Board of Australia, “Professional Capabilities for Registered Paramedics,” available online at https://www.paramedicineboard.gov.au/Professional-standards/Professional-capabilities-for-registered-paramedics.aspx [accessed 18 April 2019].

43. Barr et al., “A Qualitative Exploration of Infection Prevention and Control Guidance for Australian Paramedics”; Leo McCann et al., “Where Next for the Paramedic Profession? An Ethnography of Work Culture and Occupational Identity,” Emergency Medicine Journal 32, no. 5 (2015): e1–11.

44. Jacobs, Rosenfeld, and Haber, “Information Literacy as the Foundation for Evidence-Based Practice in Graduate Nursing Education.”

45. Salisbury and Sheridan, “Mapping the Journey.”

46. Merinda Kaye Hensley, “Citation Management Software: Features and Futures,” Reference & User Services Quarterly 50, no. 3 (2011): 204–08.

47. National Health and Medical Research Council, “NHMRC Levels of Evidence and Grades for Recommendations for Developers of Guidelines,” Australian Government, available online at www.nhmrc.gov.au/_files_nhmrc/file/guidelines/evidence_statement_form.pdf [accessed 22 February 2016].

48. Alan Bundy, “Enabling the Knowledge Nation: What Australia Needs in the 21st Century,” Australian Library Journal 51, no. 2 (2002): 103–16; Salisbury and Sheridan, “Mapping the Journey”; Wang, “Integrating Information Literacy.”

49. American Library Association, Framework for Information Literacy for Higher Education.

50. Cochrane, “Embedding Information Literacy in an Undergraduate Management Degree”; Derakhshan and Singh, “Integration of Information Literacy into the Curriculum.”

51. Heather Brodie Perry, “Information Literacy in the Sciences: Faculty Perception of Undergraduate Student Skill,” College & Research Libraries 78, no. 7 (2017): 964–77.

52. Heidi Julien, Melissa Gross, and Don Latham, “Survey of Information Literacy Instructional Practices in US Academic Libraries,” College & Research Libraries.79, no. 2 (2018): 179.

53. Gayle Schaub et al., “The Language of Information Literacy: Do Students Understand?,” College & Research Libraries.78, no. 3 (2017): 283-96.

54. Xiaoli Wang, “Developing Peoples Information Capabilities: Fostering Information Literacy in Educational Workplace and Community Contexts,” Library and Information Science 8 (2013): 31-49.

* Nigel Christopher Barr is Discipline Leader—Paramedicine at the School of Nursing, Midwifery and Paramedicine, email: nbarr@usc.edu.au; Bill Lord is in the Department of Paramedicine at Monash University, email: bill.lord@monash.edu; Belinda Flanagan is Lecturer in Paramedicine at the School of Nursing, Midwifery and Paramedicine, email: Belinda Flanagan bflanaga@usc.edu.au; Roger Carter is Liaison Librarian, Science, Health & Sport Sciences at the University of the Sunshine Coast Library, email: rcarter@usc.edu.au. ©2020 Nigel Christopher Barr, Bill Lord, Belinda Flanagan, and Roger Carter, Attribution-NonCommercial (https://creativecommons.org/licenses/by-nc/4.0/) CC BY-NC.

Copyright Nigel Christopher Barr, Bill Lord, Belinda Flanagan, Roger Carter


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Article Views (Last 12 Months)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

2020
January: 0
February: 0
March: 0
April: 0
May: 0
June: 0
July: 0
August: 0
September: 229