How Equitable, Diverse, and Inclusive Are Open Educational Resources and Other Affordable Course Materials?

Ashley D.R. Sergiadis, Philip Smith, and Mohammad Moin Uddin*

Many factors influence whether Open Educational Resources (OER) and other affordable materials foster equity, diversity, and inclusion (EDI). Cost savings to students and better outcomes are commonly cited as indicators. However, more research is needed to improve EDI outcomes in OER adoption. This article reports on survey responses from instructors and students from fourteen general education courses that used open and affordable materials. The survey explores three main topics: equitable use and access of the materials, diversity of representation within the content of the materials, and the possible change to pedagogy, specifically inclusive teaching. These topics are contextualized by comparing the responses of White students with underrepresented minority (URM) students.

Introduction

Since its inception, one of the main motivating factors for creating and using Open Educational Resources (OER) has been to increase access to educational resources by removing or lessening the barriers of cost to students (UNESCO, 2019). This fosters the conditions for a more equitable education environment by including students who would not otherwise be able to fully participate. This is especially important for historically underserved student populations who struggle the most with textbook costs (Jenkins et al., 2020). There is robust evidence that the general population of students in classes using OER perform just as well or slightly better in outcomes such as final grades and that they drop, fail, or withdraw from OER classes at lower rates (Clinton & Khan, 2019; Colvard et al., 2018; Dempsey, 2021; Griffiths et al., 2022; Hilton, 2020). Studies which focus on these outcomes for underserved or underrepresented students show there is an even greater impact for these populations (Colvard et al., 2018; Dempsey, 2021; Griffiths et al., 2022). Colvard et al. (2018) saw higher rates of improvements for grades and DFW rates for Pell recipients, part-time, and underserved (race and ethnicity) students, while Griffiths et al. (2022) reported that “OER programs generally benefited different groups of students to a similar degree” aside from a few cases in which Pell students may have benefited more (p. 1103). Consequently, textbook affordability initiatives supporting the adoption, adaptation, and creation of OER and other affordable materials are assumed to inherently support equity, diversity, and inclusion (EDI). However, this simple conclusion of OER supporting EDI based on cost and outcomes is not the entire story. Many other factors are involved when discussing OER and EDI, including but not limited to the level of difficulty to use and access online materials, accessibility (specifically to accommodate disabilities), diverse representation and inclusive language within the materials, the digital divide (or internet access), and pedagogical use of OER (such as inclusive teaching). Unlike outcomes, several of these topics have not been researched and/or documented extensively. Given the ubiquity of EDI-related initiatives in higher education institutions and their importance for student success and social justice, the potential for OER to support such initiatives is important to explore and document in more detail.

In order to provide a more holistic view of EDI and OER, we evaluated the implementation of OER in East Tennessee State University (ETSU) general education courses through an EDI lens. We distributed surveys to students and instructors that were part of ETSU’s Open and Affordable Awards Program during the fall 2021 and spring 2022 semesters. Overall, this study involved 14 general education courses (and 51 sections), 21 instructors, and 1,454 students. The data accumulated throughout this project serve as a basis to ask the following research question: To what extent did the implemented OER and other affordable materials foster equity, diversity, and inclusion? In response to this question, we present insights based on the data we collected to support the case that, beyond standard cost saving impacts, OER can foster greater EDI by promoting use and access of the materials, better diversity of representation within the content of the materials, and more inclusive teaching strategies. These topics are contextualized by comparing the responses of White students with underrepresented minority (URM) students in order to understand how their experiences may differ. Informed by these results, the discussion proposes solutions to better support EDI in textbook affordability initiatives and OER implementation.

Literature Review

The Association of College and Research Libraries (ACRL) (2019) recently advocated for research agendas on scholarly communication concerns (including Open Educational Resources) that focus on openness, inclusion, and equity in their Open and Equitable Scholarly Communications: Creating a More Inclusive Future. Scholarly activity has begun to explore how OER can foster equity, diversity, and inclusion (EDI). In this literature review we overview studies covering the main themes within our study: equity of use and access, diversity of content, and Open Educational Practices and inclusive teaching in the context of OER. These themes reflect Kimmel et al.’s (2022) finding that instructors’ perceived benefits of OER include better access and ease of use with OER, customization of content and format that facilitate equitable and inclusive materials, and more transformative and reflective pedagogies.

Equity of Use and Access

“Creating systems that permit more access to more people” as discussed by ACRL (2019) is a crucial factor when ensuring all students can access Open Educational Resources (p. 27). Many factors determine whether the use and access of OER are equitable. Students need an internet connection to access OER. According to the Institute for Higher Education Policy, about a quarter of college students report sometimes having difficulty connecting to course content because of internet issues, and an additional four percent can never or rarely connect (Parker et al., 2021). Beyond internet issues, the same report indicated Black students were more likely to use a tablet or cell phone to complete their course work. Furthermore, Black and Latinx students were more likely to have to share devices with others. These basic issues can affect the ability to access and use OER.

The design of OER itself can also hinder students’ use of the materials. In terms of accessibility for students with disabilities, two recent reviews of studies on accessibility and OER demonstrated that OER still needs improvements (Moon & Park, 2021; Zhang et al., 2020). Zhang et al. (2020) noted that the highest percentage of errors in OER were due to compatibility problems with assistive devices and operating systems. Similarly, Moon & Park (2021) mentioned that learners with disabilities faced challenges with trying to manipulate the OER interfaces, demonstrating a need for adaptive designs. Instructors’ limited familiarity with assistive technologies and lack of guidance can further exacerbate user frustrations (Moon & Park, 2021). It is unclear though how all these issues may affect the frequency of OER usage. When surveying students on how often they use OER in comparison to traditional textbooks, Bliss et al. (2013) found that neither students nor their instructors reported meaningful increase in use. The students’ comparable frequency of use suggests that accessibility and other issues with using and navigating OER are not any better or worse than their usual experience with traditional textbooks.

Diversity of Content

The diversity of representation within OERs is less well studied. The importance of diverse representation in academic library collections and learning materials in general—and the need for more research to explore this—is pointed out by ACRL (2019). This is important especially for underrepresented students because their sense of belonging in a particular class, discipline, or in higher education is strengthened in part by seeing positive portrayals of individuals with their identities reflected and represented in the class, discipline, and course materials(Armstrong, A. L.,2021; Simpson et al., 2021).

One way to foster this sense of belonging is for course materials to represent diverse identities in terms of references, perspectives, illustrations, appropriate terminology, names, and other inclusive language (OpenStax, 2020). Nusbaum (2020) notes that although OER has the potential to better represent diverse identities in these ways, there is little research to support it. Their study had students read either portions of a psychology OER that had been adapted with more inclusive language or a control version. Their findings show that first-generation students overall felt less sense of belonging on the campus, but this effect was ameliorated for those that read the diversified OER.

Meanwhile, Lapum et al. (2022) reported on the results of survey and focus group data of Nursing students’ perception of EDI in a Nursing OER. They identified four major themes: representation, learning, identities, and self. “Representation” and “identities” were how students perceived a diversity of cultures and peoples in the text in the form of illustrations, names, examples, and references, and whether they could see themselves in the materials and feel a sense of belonging. The authors concluded that more culturally responsive and diverse representation is not only well received and appreciated by students but essential in creating inclusive programs of study and educating a more diverse workforce of practitioners.

Open Educational Practices and Inclusive Teaching

Several researchers are calling for the importance of considering the context of educational practices in which OER is deployed (Lambert, 2018; Moon & Park, 2021; Morgan, 2019; Werth & Williams, 2022; Wiley & Hilton, 2018). Bali et al. (2020) offer a useful conceptual framework for understanding educational practices on a continuum of less to more open. They argue that less open practices center on the content and the teacher, whereas more open practices center on the process and the student. Practices can also be more or less explicitly focused on social justice issues along economic, cultural, and political lines. They argue that although OER addresses and ameliorates economic injustice by opening access to more students, it hardly creates an inclusive environment for students to have free access to materials that may alienate them by not adequately representing them or through perpetuating harmful cultural biases or stereotypes. Further, even if they are respectfully represented in the content, they may still be excluded or minoritized by less open content/teacher-centric classes. For example, students could be using a free culturally sensitive OER but still be subjected to inflexible course policies, lecture-heavy classes with little active learning, few high stakes exams with little scaffolding, and other similar typical features of college courses that do little to engage students.

As the framework for Universal Design for Learning reminds us, “Information that is not attended to, that does not engage learners’ cognition, is in fact inaccessible” (CAST Guidelines). Accessibility is not just a technical problem for accommodating students with disabilities but also a pedagogical one that affects all students. Thus, in looking at the inclusivity of OERs it becomes necessary to place the resources in the larger context of the educational practices in which they are embedded and consider how inclusive teaching practices relate and can be adapted to open educational practices.

Both inclusive teaching and open educational practices have been variously defined and operationalized. Sathy & Hogan (2022) characterize inclusive teaching as a means of “embracing student diversity in all forms—race, ethnicity, gender, disability, socioeconomic background, ideology, even personality traits like introversion—as an asset. It means designing and teaching courses in ways that foster talent in all students, but especially those who come from groups traditionally underrepresented in higher education.” Cronin (2017) describes OEP as “a broad descriptor of practices that include the creation, use, and reuse of open educational resources (OER) as well as open pedagogies and open sharing of teaching practices” (p.16). Teaching faculty from her study identify four key dimensions of concern in their practices—balancing privacy and openness, developing digital literacies, valuing social learning, and challenging traditional teaching role expectations—that show how OEP influenced them toward greater inclusivity in the classroom, relating how “most participants in the study described their efforts to move away from a didactic lecturing style and to encourage more student engagement (p. 10).” Clinton-Lisell et al. (2021) define OEP and the closely “OER-enabled pedagogy” (Wiley & Hilton, 2018) as “teaching practices that are only possible with OER licensing.”

Hays & Mallon (2021) take the “5 Rs” of OER (the rights to retain, reuse, revise, remix, and redistribute) and crosswalk them to an inclusive teaching framework to suggest how open resources and practices can be aligned with various inclusive teaching practices. For example, the remix affordability of OER can allow students to work together to create study guides from OERs that can be shared, updated, and remixed again by future students. This shows how an open resource supports an open practice that creates an overall more inclusive environment. However, more research on applying OER-enabled pedagogy such as this is needed. One study that begins to explore this (Tillinghast et al., 2020) compares outcomes of course sections implementing OER with those using OER plus open pedagogy but finds few differences between the sections.

Open and Affordable Awards Program

During 2021–22, East Tennessee State University (ETSU) received a grant from the Tennessee Board of Regents (TBR) to integrate EDI more thoroughly into their established Open and Affordable Awards Program. The program offers stipends to instructors to adopt, adapt, and create Open Educational Resources and other affordable materials into courses using commercial textbooks. Beyond stipends, the program provided one-on-one consultations with a librarian and instructional designer and semesterly meetings with a peer community. The grant allowed the program to offer larger stipends ($3,000) to instructors switching from commercial textbooks to OER and other affordable materials in their general education courses. General education courses were targeted due to the high probability of an available OER for the courses and the likelihood of a higher percentage of underrepresented enrolled students.

In addition to the already established one-on-one consultations and peer support, the program required awardees to attend a series of workshops emphasizing topics such as making materials accessible for all students, using materials to promote diverse representation, and integrating OER into inclusive teaching practices. Over fall 2021 and spring 2022, 21 instructors implemented open and affordable materials in the following 14 courses and 51 sections affecting 1,454 students:

• General Chemistry I & II
• Fundamentals of Communication
• Public Speaking
• Introduction to Dance
• Principles of Macroeconomics
• Critical Reading and Expository Writing
• Literary Heritage
• British Literature to 1785
• Geosciences: Earth through Time
• Developmental Psychology
• World History Since 1500
• Introduction to American Government
• Introduction to Theatre

To assess the impact of integrating open and affordable materials into these courses, we, as part of the grant team, surveyed participating students and instructors.

Methods

The purpose of our study is to explore whether implementing OER and other affordable materials fosters equity, diversity, and inclusion. Focusing on the participating courses of the TBR grant aligned well with our purpose because the courses represented a wide range of subjects in general education with greater opportunities to infuse EDI concepts. These courses also have high enrollments and historically had higher rates of underrepresented minority students enrolled compared to other ETSU courses. Furthermore, the instructors themselves considered OER through an EDI lens during the development and implementation of OER materials. 

We developed two surveys (one for students and one for faculty) as a method to collect students’ and faculty members’ experiences with the OER materials. Conducting surveys was cost effective and was an easy and reliable mechanism to capture a large number of student and faculty perceptions in a short period of time. The awards program previously used surveys adapted from Bliss et al. (2013) (under a Creative Commons license), which focuses on evaluating OER based on cost, outcomes, usage, and perception (COUP). Sergiadis and Smith (2022) published the results of those surveys from the first two years of the awards program. 

We continued to use these surveys but adapted them further to incorporate EDI questions, including questions adapted from Open Oregon Educational Resources and Sergiadis et. al (2022). We added multiple choice and open-ended questions in the surveys that directly addressed the equity of use and access as well as the diversity of content of the course materials. Since the awards program focused on open resources more than open practices, our questions on inclusive teaching were more exploratory in nature with primarily an open-ended question for the instructors. The student survey had 36 questions and the instructor survey had 26 questions. ETSU’s Institutional Review Board approved the study after an expedited review (IRB Number: c1019.23swd-ETSU). Appendixes A and B provide a copy of the survey and distinguish the origin for each question.

Prior to distributing the student survey, we requested that instructors indicate the best mode to distribute the surveys to their individual classes: (1) paper survey distributed in person during class, (2) an online survey distributed on Zoom during class, and/or (3) an online survey forwarded by the instructor through email or course site. If the instructors chose an in-class survey, we distributed the survey in their class(es) during the first three weeks of November for the fall 2021 courses and the first three weeks of April for the spring 2022 classes. After we distributed the survey in-class, instructors could provide a link to the online survey for the students who missed class. If the instructors choose to forward the online survey, we provided the survey information during the first week of November for the fall 2021 courses and the first week of April for the spring 2022 courses. Students could choose to take the online survey until the end of the semester (approximately a month). We distributed all instructors’ surveys online through email during the third week of November to instructors teaching the fall 2021 courses and the third week of April to instructors teaching the spring 2022 classes. They had no deadline to complete the survey, but they were required to complete it in order to receive their grant stipend. All instructors permitted us to use their responses for research. The online surveys for students and instructors were distributed using Research Electronic Data Capture (REDCap). Responses from paper surveys were entered into REDCap by a student assistant.

Analysis of Data

Over the two semesters, 731 out of 1,454 students (50% response rate) and 24 instructors (100% response rate) completed their surveys. Three instructors taught different courses in fall 2021 and spring 2022 and were surveyed twice on their unique experiences. Incomplete student surveys were excluded from this analysis. After collecting all the surveys, we reviewed the responses of the open-ended questions to determine commonalities among the responses.

We coded each response based on these commonalities. Specifically, we coded the open-ended questions in the student survey on the difficulty of use and accessibility as well as the instructor survey on changes in teaching practices. Due to the small number of comments, we all reviewed the comments and agreed on the coding. Our coding categories for these questions are provided in Appendix C and our coding is available in the datasets. Anonymous quotes were chosen to illustrate specifics.

For the purposes of this article, data is presented as a comparison of White students with underrepresented minority (URM) students. White students identified themselves as White only. In the survey, URM students identified as American Indian or Alaska Native, Black or African American, Hispanic or Latino, Native Hawaiian or Other Pacific Islander, or Two or More Races. We determined these categorizations based on the practices of previous studies and their institution (East Tennessee State University) (Colvard et al., 2018; Dempsey, 2021; Griffiths et al., 2022). Students who identified as nonresident aliens or did not state their resident status were not included as advised by our institution’s (and federal) guidelines, in which all other race/ethnicity data is ignored if a student is categorized as “nonresident alien” (National Center for Education Statistics, 2002).

Results

The results of this study were derived from EDI-related questions in the survey data and were categorized in four areas: 1) financial barriers, 2) equity of use and access, 3) diversity of content, and 4) Open Educational Practices and inclusive teaching. The “equity of use and access” and “diversity of content” sections are primarily based on student survey responses, while the “Open Educational Practices and inclusive teaching” section focuses on instructor responses. However, all sections provide data from both surveys. All student responses are reported unless there appeared to be noticeable differences between the White and URM student responses. Results were summarized using descriptive statistics only because a control group was not identified, and no statistical test was conducted. Nonetheless, large sample sizes of White and URM students in this study imply strong conclusions.

Demographics

The race and ethnicity demographics were similar among all ETSU undergraduates (East Tennessee State University, 2021), the general education courses using OER in this study, and the survey respondents (figure 1). Within the courses, 69.9% were White students (n=1016) and 15% were URM students (n=220). Out of the total 731 survey results, 75.4% were White students (n=551) and 16.8% were URM students (n= 123).

The following demographic data of the White and URM students demonstrates their similarities and differences. Based on course enrollment data, URM students, particularly Black or African American students, were more likely to be in Learning Support sections than regular sections (figure 2). Learning Support sections are intended for students whose college entrance test scores indicate an academic need. URM students represented 28.8% (n=75) of students in Learning Support sections, in contrast to 12.1% (n=145) in regular sections. In comparison, White students represented 49.0% (n=128) of students in Learning Sections, in contrast to 74.4% (n=888) in regular sections. Based on the survey results, URM students were more likely to be first generation students (47.2%, n=58 versus 26.2%, n=144) and reported English as their second language (12.3%, n=15 versus 0.4%, n=2) in comparison to White students. URM students had slight differences in numbers regarding financial aid. URM students were more likely to receive loans (47.1%, n=57 versus 32.4%, n=176), grants (57.9%, n=70 versus 36.6%, n=199), and work-study positions (25.6%, n=31 versus 16.5%, n=90) but less likely to receive scholarships (62.0%, n=75 versus 72.1%, n=392) than White students. Despite these differences, several of the demographics remain very similar, such as age (91.6%, n=662 of all students were between the ages of 18 and 22), classification (55.0%, n=301 of White students and 63.1%, n=77 of URM students were undergraduate freshman, and 97.2%, n=669 of all students went full-time), and work status (52.8%, n=381 worked part-time and 36.1%, n=260 did not work). White students had about 10% more females than URM students (68.2%, n =375 versus 59.3%, n=73) based on course and survey data.

Financial Barriers

The awards program saved students $178,023 over the two semesters, or $122 per student. When breaking the savings by semester, fall 2021 students saved $109,944 and spring 2022 students saved $68,079. To determine cost savings, enrollment was multiplied by the textbook cost (if purchased new) that the open and affordable materials replaced. URM students saved $26,703 (by multiplying the total sum with the total percentage of URM students). The survey responses indicated that the URM students were more affected by financial barriers. URM students reported purchasing required textbooks less often than White students (figure 3). Specifically, 58.0% (n=69) of URM students never, rarely, or only half the time purchased their required textbooks in comparison to 35.8% (n=196) of White students. More Importantly, URM students reported that it was the cost of course materials that prevented them (57.4%, n=70) from buying a textbook more often than the White students (48.5%, n= 263). Our results indicate that a large number of students have benefited by removing the cost barrier.

Equity of Use and Access

Students reported how often they used their required textbooks in their typical courses and how often they used the OER materials for the course (figure 4). In their typical non-OER courses, URM students had some differences in usage. Specifically, URM students’ responses (13.1%, n=16) to “Never” using the required textbooks were double of White students’ responses (6.4%, n=35). The differences of usage for URM and White students did not drastically change in the courses using OER and other affordable materials, but the overall usage for all students increased. The answer for all students using their course materials two to three times a week or more jumped from 39.3% (n=285) to 54.7% (n=399) in the courses using the free OER materials.

For access, White and URM students generally used the same devices to access the materials. Most students accessed them on their personal computer (97.5%, n=703) followed by their cell phone (59.5%, n=429). Almost all students (98.6%, n=712) cited having internet at their primary residence to access the materials. Most students (88.5%, n=642) rated the level of difficulty in using the materials (e.g., accessing, using features such as searching) as easy. URM students reported having more difficulty (22.1%, n=27) than White students (8.5%, n=47) (figure 5). About 20% (n=19) of the students who reported having difficulty commented on issues in understanding where to find the materials and how to use them (including assignments), whether it was due to the organization of the Learning Management System (Desire2Learn) and/or other unclear instructions from the instructors. In terms of using the OER themselves, there were a couple of specific comments such as not knowing what page each chapter begins in the PDF and the inability to find information in the textbook. Beyond these comments, some students chose the ratings due to the materials’ content being difficult to comprehend or the course being a difficult subject.

Despite the difference in level of difficulty, 94.4% (n=679) of students did not experience accessibility issues (e.g., screen reader, captions, color contrast). Most of the accessibility issues students experienced could be classified as general technology or internet issues. However, there were instances of no read-to-text options, quick changes of the colors (leading to migraines), no captions when viewing videos in class, hard to see images, and inability to copy and paste information (indicating a possible inability to use assistive technology).

As for the instructor survey, only one instructor indicated that the assigned course materials did not meet the accessibility standards to their satisfaction. A different instructor mentioned that they received the following feedback from students: A few said that hyperlinks and/or ads were distracting, and several printed materials because they found it difficult to read on a screen. This instructor also wrote about the challenges of finding accessible materials: My student population includes a number of students with learning differentials. Some of these may be impacted by an electronic reader; we have had to make some adjustments for accessibility. There will be more in the future. A third instructor admitted that they taught more about how to study with online or electronic sources.

Diversity of Content

Almost all students (98.1%, n=712) rated the quality of the materials the same as or better than the materials in their other courses. The survey asked students three different questions about whether the OER and affordable materials integrated topics around diversity, represented their culture, and included diverse identities. In general, the responses to the three questions were similar when aggregating all 731 student responses. Around 10% (n=46-68) of students disagreed with the statements, 60% agreed (n=418-460), and 30% (n=197–246) were unsure. When separating the responses, URM students had a slightly higher percentage of answers for disagree (~5% difference) and unsure (5–12% difference) (figure 6-8).

More so than race and ethnicity, the discipline or course topic mattered. This becomes evident when comparing the communication and chemistry courses (figure 9). When comparing the responses to the statement “the materials in this course represented diverse identities,” 75.6% (n=239) of communication students agreed, 5.3% (n=17) disagreed, and 19.0% (n=60) were unsure. In contrast, 33.6% (n=60) of Chemistry students agreed, 13.0% (n=25) disagreed, and 52.5% (n=94) were unsure. When reviewing the comments from students, numerous students mentioned that the subject made the difference. For chemistry, students who either disagreed or were unsure stated, “I don’t care/pay attention. It’s chem, not social studies,” “we talk about chemistry not about diverse identities or related topics,” and “chemistry has nothing to do w/ diversity.” For communication, students who agreed stated that “we talk about how communication is diverse throughout different countries,” “This is a comm class where we’re learning about diversity in society,” and “Because the course talks a lot about different cultures which includes many diverse cultures, religions, etc.” In comparison to student responses, 79.2% (n=19) of instructors thought that the topics around diversity were integrated throughout the course materials and 75% (n=18) thought that the materials represented diverse identities.

Open Educational Practices and Inclusive Teaching

For Open Educational Practices and inclusive teaching, most of the data came from the instructor survey. Most of the instructors discussed the experimental nature of the open and affordable materials approach with their students 2–7 times in the semester (75.0%, n=18), which is considered an inclusive teaching practice (transparency). However, the extent of the discussions is unknown. The primary data regarding inclusive teaching came from the open-ended question from the instructor survey: “How did using open/affordable materials change your teaching practices?” There were a few common themes in the responses. 

Customization of Ancillary Materials: Over a quarter of the instructor responses (n=7) discussed customizing support materials such as assessments and lectures. This affordance of open materials can be directed toward greater inclusivity by adapting materials to better reflect the diversity of students and thus enhance the sense of belonging, as Nusbaum (2020) found in their study.

Diverse/Relevant Content: A quarter of the responses (n=6) mentioned that they choose more diverse, relevant, and/or current content based on their students’ needs. While this statement aligns with the “diversity of content” category, the instructors indicated that it also changed their teaching practices. One instructor provided a lengthy explanation on how using OER materials connects to their commitment to their students and teaching:

Using OER materials required me to collect the most ‘up to date’ materials and materials that reflected diversity and inclusion—such as research articles, TED talks, and podcasts. It is so easy to rely on a textbook, but in the past my students often question the information presented in a textbook—so having more recent and more relevant information was important to me. This has been time-consuming—and I have only started a collection. This is something I will need to keep doing as part of my commitment to teaching with OERs but also as a commitment to my students—to be a relevant teacher that is open to their ideas, diverse points of view, and challenges herself (me) to remain in touch with new knowledge.

Referenced in Class: A quarter of instructors (n=6) mentioned that they were able to reference their course materials more frequently in class due to how easily they could access the textbook themselves. This may be a less direct indication of inclusive teaching per se, but it does show easier access can facilitate more interaction between students and learning materials. This same limitation applies to the following category as well.

More Time for Teaching: Three instructors in the comments indicated that their students were more prepared than previous semesters. Two of the instructors elaborated that this led to more discussion and conversation with their students (see comments below). Although only a few instructors mentioned it in their comments, a third of instructors (n=7) responded to a multiple-choice question that their students were more prepared than previous semesters, with only one instructor stating they were less prepared. In addition, most students (77.3%, n=557) noted that the materials on this course helped them study more effectively. Only 8.3% (n=60) stated that it did not help them study more effectively.

1. I do less lecturing, because students actually read the book, because it’s easily accessible on multiple platforms. This lessens the need to go over the basics and allows us to spend more time in conversation.
2. The use of OER materials seemed to provide a lot more flexibility during class time. We were able to have more conversations and the course calendar could be a bit more flexible as well (still abiding within the overall syllabus). I think it also allowed students to access the material in more ways, which led to more engagement overall.

Overall Experience

Most students were likely to register for future courses with zero or low-cost materials. However, these rates were somewhat lower for URM students and even lower for the students in the Learning Support sections (figure 10). Despite these concerns, almost 90% (n=612) of students supported using their student fees to encourage instructors to use zero or low-cost course materials.

Limitations of the Study

There are several limitations of the study. First, results were analyzed based on descriptive statistics since a control group was not identified and no statistical test was conducted. Second, reported student and faculty data are based on general education OER courses funded by the Tennessee Board of Regents grant program and is not representative of all the OER courses offered at ETSU. In most cases, traditional courses (with required commercial textbooks) have been offered for many years and matured. In comparison, OER courses were offered for the first time and students and faculty may have had differentiated experiences when comparing their experiences with those traditional courses. Research on courses whose instructors have used OER for several semesters may show different results. Third, this study supports the idea that both OER as well as Open Educational Practices and/or inclusive teaching are necessary for OER to foster equity, diversity, and inclusion in our classrooms and institutions. Our research on their connection was exploratory and research in this area is relatively in its infancy with many possible avenues to explore. Lastly, the study focused on students of underrepresented races and ethnicities but does not account for students from other underrepresented groups (e.g., students within the LGBTQIA2S+ community). Intersectionality of underrepresented groups is also a consideration. Due to these nuances, future studies may benefit from a more qualitative approach (e.g., focus groups, interviews) to learn more about student experiences with OER and/or affordable materials.

Discussion

Overall, students in the courses had a positive experience with Open Educational Resources and affordable materials. Almost all students had internet and personal computers to access the materials, and most had an easy experience using and accessing the materials. They considered OER as comparable in quality to other course materials. Using free resources also increased the usage of the materials, which is not surprising given that the students did not always purchase their required textbooks. Unfortunately, the surveys indicated some needed improvements regarding the implementation of OER. Overall, the students who could potentially benefit the most from these materials, URM students and Learning Section students, were less likely to register for future courses with zero or low-cost materials. In addition, issues arose in our three categories: equity of use and access, diversity of content, and Open Educational Practices and inclusive teaching. First, URM students had more difficulty using and accessing the materials than White students. Second, diverse representation within the materials seemed to be either lacking or not apparent to a significant degree, specifically in some disciplines. Third, some instructors indicated positive changes to their teaching due to the use of OER, but the comments did not indicate a widespread transformative experience for them. The discussion will discuss these findings and provide some potential solutions.

Equity of Use and Access

In terms of equity of use and access, URM and White students did not have drastically different responses regarding internet, accessibility, and modes of access. However, URM students reported more difficulty accessing and using the materials. Based on students’ and instructors’ comments, some students may need additional help in accessing and using online resources. Although the instructors in the study by Kimmel et al. (2022) thought that students had an easier time using OER materials in their Learning Management System (LMS) over publisher websites, some of the students within this study indicated that their difficulty stemmed from trying to use the LMS system or course website. Dempsey (2021) considered that not using a textbook, but a variety of resources may have a negative effect on student outcomes. This could be the case in some of these courses since it requires students to find and navigate multiple resources. Campus experts on the LMS and instructional design can provide the best, customized advice to instructors on how to organize their sites and should be more involved in textbook affordability initiatives. Beyond ensuring their course site is organized well, instructors could employ more inclusive teaching practices that ensure all students can easily navigate and access the materials well. In terms of accessibility, it usually falls on the instructor to ensure the OER is accessible to all students. While tools exist to help in this process (e.g., OER Accessibility Toolkit by Open UBC), instructors often have limited time and sometimes limited technological capabilities (Moon & Park 2022; Open UBC, 2021; Sergiadis & Smith, 2022;). Textbook affordability initiatives could also explore more substantial support in this area.

Diversity of Content

In terms of diversity of representation, students did not overwhelmingly express that OER and affordable materials in their courses included diverse representation. As with use and access, many resources exist to help instructors evaluate and adapt resources to ensure their OERs are inclusive, diverse, and equitable. The “Framework for Reviewing Inclusion, Diversity, Equity, and Accessibility in Open Educational Resources” (Anderson, 2022) lists eight [ categories to consider when creating or adapting OER for more inclusivity: diverse and inclusive imagery; example names; inclusive language; researchers and references; diverse examples and balanced perspectives; appropriate terminology and inclusive metadata; accessibility, usability, and universal design for learning; anti-racist and inclusive open pedagogy. This resource is a practical guide with many concrete examples. OpenStax (2020) publishes a guide with similar advice and examples. Discipline-specific suggestions are also available. Sathy and Hogan (2022) suggested strategies for instructors in STEM fields, which were courses in our study that students seemed to struggle with the most. Examples include weekly profiles of scientists of color, hosting diverse invited speakers, viewing TED Talks, and tapping into resources such as Project Biodiversity.org, databases like “1,000 Inspiring Black Scientists in America” and “500 Queer Scientists”.

While textbook affordability initiatives should promote EDI in all subject areas, students may not notice the diverse representation unless it is worked into the course objectives. This was evident in our survey responses, in which the course subjects and objectives most often dictated which materials were considered as having diverse and inclusive representation. For example, the chemistry sections were using OpenStax textbooks that have clear diversity and representation development guidelines. Anecdotally, one of the chemistry instructors mentioned actively seeking additional materials from scientists of different races and ethnicities. However, this was not reflected in the student survey responses. They assumed that the topic of chemistry meant that the resources did not have or need diverse or inclusive representation. In another study, we compared the responses from our fall 2021 survey to a fall 2021 survey of students in courses using free library e-textbook versions of commercial textbooks (Sergiadis et al., 2022). The subject matter of the course affected the responses regarding diversity similarly. Therefore, our results may not be specific to OERs but course textbooks in general. Once again, Open Educational Practices and inclusive teaching may make a difference in this perception.

Open Educational Practices and Inclusive Teaching

As mentioned previously in the discussion, some of the challenges with usage, access, and diversity of representation can be addressed by employing more inclusive teaching practices. Instructors may need to provide more structure in terms of transparency in learning and teaching (Wilkelmes, 2023). For instance, one strategy Wilkelmes (2023) suggests is to “discuss assignments’ learning goals and design rationale before students begin each assignment.” In other words, rather than just telling students what to do, also being transparent about how to do it (breaking down the steps and skills they will need, such as navigating to the necessary resources) and why they are doing it (drawing connections to learning outcomes and/or personal goals). This approach could be adapted to address usage and representation issues. Instructors can explicitly guide students to the open or affordable course materials located outside the LMS. They can also more frequently explain the “experimental” nature of the OER course materials, which may be unfamiliar to some students. Students universally appreciate that OER is free, but they may also need to know that it may look and work differently than traditional course materials. Instructors can also demonstrate and model how contributions from a diversity of people and viewpoints are crucial to any field of study, especially in STEM-related classes where this connection is not apparent to many students. Where students report difficulties with usage and access or unsureness about diversity of representation, the remedy may lie as much with the instructional practices as with the resources themselves.

Some of the comments from the instructor survey suggested that using OER-enabled pedagogy helped them to reach a greater diversity of students, refer to the readings in class more, and spend more time actively engaged with students in discussion and group work. Bali et al. (2020) developed a framework that can help us consider how to be more inclusive with both open resources and open practices. In this framework, an education practice is more open and inclusive if it centers more on the process rather than the content, and the student rather than the teacher. Some of the survey items from the present study suggest students perceive the educational practices of the courses in this study as more inclusive, that is as centering more on their learning process. In aggregate, students report using the OERs two to three times a week. Students report that the materials help them study more effectively, and instructors reported more referrals to the text in class and more class discussions. However, these indications of greater inclusion from both instructors and students must be tempered with the other findings presented here on issues with usage, access, and diversity of representation. Overall, the study indicates that implementing OER may encourage but not guarantee pedagogical change. Textbook affordability initiatives could be improved by expanding beyond implementing OER to integrating open pedagogy and inclusive teaching.

Conclusion

To return to and address the research question of this study, the extent to which implemented OERs foster a more equitable, diverse, and inclusive environment cannot be very well measured along a single dimension or set of criteria. Although students and instructors generally had positive experiences with our study’s primary areas of focus (usability/accessibility, diverse representation, and inclusive teaching), improvements can always be made. Reviewing feedback and exploring possible solutions have informed the future of ETSU’s Open and Affordable Course Materials initiatives, specifically the Awards Program. Previously, the initiatives have focused on helping instructors find resources that they can use instead of costly, commercial textbooks. Now, the co-organizers hope to provide a more expansive approach by collaborating with disability services and academic technology staff to address usability/accessibility issues, emphasizing how all disciplines can incorporate content with diverse representation, and providing more support for OER-enabled open pedagogical practices and inclusive teaching.

References

Armstrong, A. L. (2021). The Representation of Social Groups in U.S. Educational Materials and Why It Matters: A Research Overview. New America. https://www.newamerica.org/education-policy/reports/the-representation-of-social-groups-in-u-s-educational-materials-and-why-it-matter/

Anderson, N. (2022). Framework for Reviewing Inclusion, Diversity, Equity, and Accessibility in Open Educational Resources. In Enhancing Inclusion, Diversity, Equity, and Accessibility (IDEA) in Open Educational Resources (OER) (p. 8). University of Southern Queensland. https://open.umn.edu/opentextbooks/formats/3117

Association of College and Research Libraries (ACRL). (2019) Open and Equitable Scholarly Communications: Creating a More Inclusive Future. Prepared by Nancy Maron and Rebecca Kennison with Paul Bracke, Nathan Hall, Isaac Gilman, Kara Malenfant, Charlotte Roh, and Yasmeen Shorish. Association of College and Research Libraries.

Bali, M., Cronin, C., & Jhangiani, R. (2020). Framing Open Educational Practices from Social Justice Practice. Journal of Interactive Media in Education, 2020(1): 1–12. http://doi.org/10.5334/jime.565

Bliss, T., Robinson, T. J., Hilton, J., & Wiley, D. A. (2013). An OER COUP: College Teacher and Student Perceptions of Open Educational Resources. Journal of Interactive Media in Education, (1), Art. 4. http://doi.org/10.5334/2013-04

CAST (2018). Universal Design for Learning Guidelines Version 2.2. Retrieved from http://udlguidelines.cast.org

Clinton, V., & Khan, S. (2019). Efficacy of Open Textbook Adoption on Learning Performance and Course Withdrawal Rates: A Meta-Analysis. AERA Open, 5(3), 1–20. https://doi.org/10.1177/2332858419872212 

Clinton-Lisell, V., Legerski, E., Rhodes, B., & Gilpin, S. (2021). Open Educational Resources as Tools to Foster Equity. In C. Ozaki & L. Parson (eds.), Teaching & Learning for Social Justice and Equity in Higher Education, Volume 2. Palgrave MacMillan.

Colvard, N.B., Watson, C.E., & Park, H. (2018). The impact of Open Educational Resources on Various Student Success Metrics. International Journal of Teaching and Learning in Higher Education, 20(2), 262–276. https://www.isetl.org/ijtlhe/pdf/IJTLHE3386.pdf 

Cronin, C. (2017). Openness and Praxis: Exploring the Use of Open Educational Practices in Higher Education. International Review of Research in Open and Distributed Learning, 18(5): 15–24. https://doi.org/10.19173/irrodl.v18i5.3096 

Dempsey, M. (2021). The impact of Free and Open Educational Resource Adoption on Community College Student Achievement. International Journal of Open Educational Resources, 4(1), 45–59. https://ijoer.scholasticahq.com/article/25024-the-impact-of-free-and-open-educational-resource-adoption-on-community-college-student-achievement 

East Tennessee State University. (2021). Fact Book 2021. https://www.etsu.edu/winsv-ir/fact/factbooks.aspx

Griffiths, R., Mislevy, J., & Wang, S. (2022). Encouraging Impacts of an Open Educational Resource Degree Initiative on College Students’ Progress to Degree. Higher Education: The International Journal of Higher Education Research, 84(5): 1089–1106. https://doi.org/10.1007/s10734-022-00817-9

Hays, L., & Mallon, M.N. (2021). Using OER to Promote Inclusion in Higher Education Institutions. Currents in Teaching and Learning, 12(2), 20–33. https://webcdn.worcester.edu/currents-in-teaching-and-learning/wp-content/uploads/sites/65/2022/05/Currents-Volume-12-Issue-02-Spring-2021.pdf 

Hilton, J. (2020). Open Educational Resources, Student Efficacy, and User Perceptions: A Synthesis of Research Published between 2015 and 2018. Education Tech Research Dev 68, 853–876. https://doi.org/10.1007/s11423-019-09700-4

Jenkins, J.J., Sánchez, L.A., Schraedley, M.A.K., Hannans, J., Navick, N., & Young, J. (2020). Textbook Broke: Textbook Affordability as a Social Justice Issue. Journal of Interactive Media in Education, 2020(1), 1–13. https://doi.org/10.5334/jime.549 

Kimmel, S., Bol, L., Ryan, D., & Esqueda, M. (2022). The Experiences of Community College Faculty Using Open Educational Resources versus Publisher Textbooks. Community College Journal of Research and Practice. https://doi.org/10.1080/10668926.2022.2107118  

Lambert, S.R. (2018). Changing Our Dis(course): A distinctive Social Justice Aligned Definition of Open Education. Journal of Learning for Development, 5(3), 225-244. https://doi.org/10.56059/jl4d.v5i3.290 

Lapum, J., Bailey, A., St-Amand, O., Garmaise-Yee, J., Hughes, M., & Mistry, S. (2022). Equity, Diversity, and Inclusion in Open Educational Resources: An Interpretive Description of Students’ Perspectives. Nurse Education Today, 116. https://doi.org/10.1016/j.nedt.2022.105459

Moon, J., & Park, Y. (2021). A Scoping Review on Open Educational Resources to Support Interactions of Learners with Disabilities. International Review of Research in Open and Distributed Learning, 22(2): 315–41.

Morgan, T. (2019). Getting to Open at a Closed Institution: A Case Study of Evolving and Sustaining Open Educational Practices. Journal of Learning for Development, 6(3). https://doi.org/10.56059/jl4d.v6i3.346 

National Center for Education Statistics. (2002). Development of Concepts and Methods: Defining Race and Ethnicity Data. Statistical Standards Program. https://nces.ed.gov/statprog/2002/std1_5.asp

Nusbaum, A. T. (2020). Who Gets to Wield Academic Mjolnir?: On Worthiness, Knowledge Curation, and Using the Power of the People to Diversify OER. Journal of Interactive Media in Education, 2020(1), 4. http://doi.org/10.5334/jime.559 

Open UBC. (2021). OER Accessibility Toolkit. Open UBC. https://open.ubc.ca/oer-accessibility-toolkit/ 

OpenStax. (2020). Improving Representation and Diversity in OER Materials. Rice University. https://www.dickeyphysics.com/Physics_Readings/OpenStax_Representation_and_Diversity_Development_Guidelines_June2020.pdf 

Parker, A.G., Santos, J., & Dancy, K. (2021). Online Isn’t Optional: Student Polling on Access to Internet and Devices. Institute for Higher Education Policy (IHEP). https://www.ihep.org/wp-content/uploads/2021/05/IHEP_Broadband_brief_rd3_web-1.pdf

Sathy, V., & Hogan, K.A. (2022). How to Make Your Teaching More Inclusive. The Chronicle of Higher Education. https://www.chronicle.com/article/how-to-make-your-teaching-more-inclusive/ 

Sergiadis, A.D.R., Clamon, T., Smith, P., & Young, J. (2022). Two peas in a pod? Support for open and affordable course materials through an awards program and e-textbook reserves. Oral presentation at Charleston Conference. https://chsconf.cadmore.media/Category/2d0c18b3-09a4-4e05-b0b6-daa13d1b3cec?returnautoscroll=%23item2d0c18b3-09a4-4e05-b0b6-daa13d1b3cec 

Sergiadis, A.D.R., & Smith, P. (2022). Is It Worth It? Evaluating an Open Educational Resources Awards Program. Tennessee Library, 72(1). https://www.tnla.org/page/72_1_Sergiadis_Smith 

Simpson, D. Y., Beatty, A. E., & Ballen, C. J. (2021). Teaching between the Lines: Representation in Science Textbooks. Trends in Ecology & Evolution 4–8. https://doi.org/10.1016/j.tree.2020.10.010

Tillinghast, B., Fialkowski, M.K., & Draper, J. (2020). Exploring Aspects of Open Educational Resources through OER-Enabled Pedagogy. Digital Education, 5. https://doi.org/10.3389/feduc.2020.00076 

UNESCO. (2019). Recommendation on Open Educational Resources (OER). UNESCO. https://www.unesco.org/en/legal-affairs/recommendation-open-educational-resources-oer

Werth, E., & Williams, K. (2022). The Why of Open Pedagogy: A Value-First Conceptualization for Enhancing Instructor Praxis. Smart Learning Environments, 9. https://doi.org/10.1186/s40561-022-00191-0 

Wiley, D., & Hilton III, J. L. (2018). Defining OER-Enabled Pedagogy. The International Review of Research in Open and Distributed Learning, 19(4). https://doi.org/10.19173/irrodl.v19i4.3601 

WinkeImes, M-A. (2013). Transparency in Learning and Teaching Project. TILT Higher Ed. https://tilthighered.com/transparency

Zhang, X., Ahmed, T., Nascimbeni, F., Burgos, D., Huang, R., Chang, T-W., Jemni, M., Khribi, M.K. (2020). Accessibility within Open Educational Resources and Practices for Disabled Learners: A Systematic Literature Review. Smart Learning Environments, 7(1). https://doi.org/10.1186/s40561-019-0113-2