Health Sciences and Beyond: An Investigation into Canadian Librarian Participation in Systematic Reviews Across Disciplines

Catherine Boden, Susan Bolton, and Angie Gerrard*

The aim of this survey was to describe academic librarian roles in systematic reviews (SR) in any discipline, as a follow-up to a previous survey of Canadian academic health sciences librarians. A convenience sample of librarians at Canadian universities who support SRs were invited to complete a survey. Respondents were asked about their roles, training, knowledge, and barriers to providing SR services. Ninety-four librarians responded to the survey. The most common roles were in the literature search; time and training were the most frequently reported barriers. Librarians are supporting reviews in multiple, diverse disciplines, primarily as expert searchers.

Introduction

Systematic reviews aim to synthesize scientific literature in a rigorous, transparent, and unbiased manner. With a history in the health sciences and the Evidence Based Medicine movement, systematic review methods were developed to answer questions about the effectiveness of health interventions (Djulbegovic and Guyatt 2017). Systematic review methods have since been adapted and adopted to other types of questions and disciplines, respectively. This has led to a proliferation of review types whose methods are derived from systematic reviews, such as scoping reviews (Arksey and O’Malley 2005; Colquhoun et al. 2014), realist reviews (Pawson et al. 2005), and rapid reviews (Khangura et al. 2012). At the core of all these review types (hereafter referred to collectively as ‘synthesis reviews’) is a comprehensive search for which librarian expertise is ideally suited. In fact, funding bodies (e.g., Canadian Institutes of Health Research 2010) and handbooks (Higgins et al. 2022) recommend the inclusion of librarians on systematic reviews.

Health sciences libraries have provided support for systematic reviews, and subsequently, other related review types (e.g., scoping reviews), for many decades. The librarian’s role has traditionally been guiding researchers in formulating a searchable research question and developing the search (i.e., selecting databases, developing the search strategy, managing references), but librarians can also engage in other roles, such as peer reviewing review articles (Beverley et al. 2003; Spencer and Eldredge 2018) or supporting reviewers. Supporting reviewers can entail reference or consultations, instruction or training, and/or collaboration. These services are referred to using various terminology, including systematic review services, synthesis review services, and knowledge synthesis services. More recently, synthesis reviews have been conducted by researchers in disciplines beyond the health sciences, and library services are beginning to follow suit (Kallaher et al. 2020). The present study is an update of a previous survey describing Canadian university health sciences librarian roles in systematic reviews (Murphy and Boden 2015) but aiming to be inclusive of disciplines beyond the health sciences.

Literature Review

While librarians are traditionally relied upon for their expertise in the literature search, the variety of roles that librarians can play has expanded beyond that of just expert searcher. In 2003, Beverley et al. outlined ten potential roles for librarians in the systematic review process ranging from project leader to literature searcher to disseminator. Subsequent studies have also delineated the various responsibilities librarians contribute to synthesis reviews (Dudden and Protzko 2011; Spencer and Eldredge 2018).

Systematic reviews and other types of rigorous reviews are now being conducted in disciplines beyond the health sciences (Brereton et al. 2007; Kallaher et al. 2020; Miljand 2020; Premji et al. 2022; Riegelman and Kocher 2018; Slebodnik et al. 2018). In response, libraries have expanded their services to meet these needs (Kallaher et al. 2020; Riegelman and Kocher 2018). As the demands for systematic review services have continued to increase, more Canadian libraries have formalized services (McKeown and Ross-White 2019), and researchers across Canada from disciplines beyond the health sciences are now seeking systematic review support. This flourishing of synthesis review services in libraries is challenging the sustainability of the service and demanding innovative ways of reimagining how these services should be offered (Luca and Ulyannikova 2020).

Not only have librarians’ roles and the disciplines they support expanded, but there have also been recent developments in health sciences systematic review methods. Significant updates were made to the Cochrane Handbook for Systematic Reviews of Interventions in 2019 (Higgins et al. 2022) and an extension to the PRISMA statement for reporting literature searches in systematic reviews has been published (Rethlefsen et al. 2021). In the field of conservation and environmental management, the Reporting Standards for Systematic Evidence Syntheses (ROSES) were published in 2018 (Haddaway et al.). Further, in 2019, the Campbell Collaboration MECCIR Conduct and Reporting Standards were updated (The Methods Coordinating Group of the Campbell Collaboration; The Methods Group of the Campbell Collaboration). There has also been a proliferation of new systematic review types and typologies in a variety of disciplines (Munn et al. 2018; Sutton et al. 2019).

Thus, the environment in which librarians are providing systematic review services has been evolving. In spring 2014, a benchmarking survey focusing on Canadian university health sciences librarians was conducted on the roles of librarians in systematic reviews and barriers and facilitators to librarian engagement (Murphy and Boden 2015). Roles were examined based on Beverly et al. (2003). Similar to the recent findings of Schvaneveldt and Stellrecht (2021), the 2014 survey suggested that university health sciences librarians engaged primarily in roles related to the literature search stage of a systematic review and to dissemination (Murphy and Boden 2015). There were relatively few reports of library policies or guidelines on the systematic review services; and time and training were the most common barriers at that time.

The authors felt it was an ideal time to repeat the 2014 survey to: (1) understand how the landscape has changed, and what has remained unchanged, in the past eight years; (2) provide evidence-based information about the current state of involvement of librarians working at Canadian universities in systematic reviews in all disciplines; (3) identify other review types that are also being supported; and (4) describe barriers that need to be addressed.

Methodology

A short (about fifteen minute) online questionnaire with multiple choice, Likert, and dichotomous (yes/no) questions was created and distributed using Survey Monkey software (https://www.surveymonkey.com) (Appendix A). As this was an update of a previous benchmarking study (Murphy and Boden 2015), the questionnaire replicated the original with only minor modifications to align with the expansion of the target population. As previously, the questions addressed the level and nature of librarian support to faculty, students, and administrators conducting systematic reviews, based on librarian roles in systematic reviews identified by Beverly et al. (2003). There were questions about the current state of librarian knowledge and training needs, as well as barriers to providing systematic review services. Information about librarian participation in other kinds of knowledge syntheses was also gathered. Given the growth in requests for systematic review services beyond health sciences disciplines (Kallaher et al. 2020) since the original benchmarking study, the inclusion criteria for participation were expanded to include librarians from any discipline. Therefore, a question was added about the disciplines for which the librarian provides systematic review services to understand the diversity of disciplines and, possibly, any unique characteristics or barriers to the provision of systematic review services in different disciplines. The survey was distributed in English and French, the two official languages in Canada.

A convenience sample of librarians working at Canadian universities who had participated in a systematic review in the twelve months prior to the survey were recruited to participate in the study. Non-librarians, and librarians working at an institution other than a university, working outside Canada, or who had not participated in a systematic review within twelve months of the date of the questionnaire were not eligible to participate. The invitation to participate and a link to the online questionnaire was distributed by email to academic librarians across Canada via librarian listservs for the following associations: Canadian Health Libraries Association; the Association of Faculties of Medicine of Canada; Canadian Association of Professional Academic Librarians; Association of College and Research Libraries: Education and Behavioral Sciences Section, Evidence Synthesis Methods Interest Group, University Libraries Section; and the Medical Library Association. The email invitation also encouraged recipients to forward the invitation to other colleagues who may be eligible to participate in the study but were not subscribed to one of the listservs. The survey was distributed on April 18, 2022. Respondents had three weeks to complete the survey, and reminders were sent out two weeks, one week, and one day prior to questionnaire closure.

According to the 2018 Census of Canadian Academic Libraries (Canadian Association of Professional Academic Librarians CAPAL Advocacy Committee 2019) there were 777 librarians working at universities in Canada. Of those, 54 percent offered reference services and might, therefore, provide systematic review support. Based on these values, the population of interest was estimated to be approximately 419 librarians. A power calculation indicated that 201 participants were needed to achieve a margin of error of 5 percent and a confidence level of 95 percent. This is likely an over-estimate of the target population as not all reference librarians provide systematic review support services. The study received approval from the University of Saskatchewan Behavioural Research Ethics Board on March 3, 2022.

Data were analyzed by calculating descriptive statistics using SPSS software (version 28.0.1.0), and tabulating short answer questions where an ‘Other, please specify’ option was given. To understand any unique characteristics or barriers to the provision of systematic reviews services in different disciplines, respondents were grouped into three broad categories: specifically those that exclusively support health sciences disciplines; exclusively support non-health sciences disciplines; and support a mix of health sciences and non-health sciences disciplines. This categorization was based on a multiple-choice question about the academic disciplines supported by the respondent. The disciplines listed in the survey question were identified by an environmental scan of disciplines listed on Canadian university websites, followed by a process of combining like but differently named disciplines. As this was a comparison to a previous study about university health sciences librarians (Murphy and Boden 2015), categories for this study were based on whether or not health sciences disciplines were being supported, recognizing that other labels or categorization schemes were possible. See Table 1 for the definitions of health sciences and non-health sciences categories and disciplines. If a respondent indicated supporting one or more of the health sciences disciplines and one or more non-health sciences disciplines, the respondent was considered to be supporting “mixed” disciplines. The categories were used specifically in analyzing (1) the librarian roles in systematic reviews during consultation, instruction, and collaboration, and (2) the challenges or barriers to participation in systematic reviews.

Results

There were 126 individuals who chose to participate in the survey. Of those, 107 were eligible to participate but only ninety-four completed questions beyond the two initial inclusion questions. Of the ninety-four respondents, three provided minimal data (two answered the last four questions of the survey; one answered only the questions about policy and other review types). Thus, there was relatively complete data from ninety-one respondents and very limited data from an additional three.

The majority of respondents reported working as librarians for ten years or more (0-3 years, n = 9; 4-6 years, n = 11, 7-9 years, n = 16; 10 or more years, n = 55). This pattern of more experienced librarians responding to the survey was consistent across the disciplines (Health Sciences, Non-Health Sciences, Mixed).

Of the ninety-one respondents who indicated the discipline(s) to which they provide systematic review support, forty-one (45 percent) supported health sciences disciplines exclusively, seventeen (19 percent) supported disciplines other than health sciences, and thirty-three (36 percent) supported a mix of health sciences and non-health sciences disciplines (see Table 1). The top five most frequently reported health sciences disciplines were: public health (n = 38), medicine (n = 36), nursing (n = 29), rehabilitation sciences (n = 20) and kinesiology (n = 19). For the non-health sciences disciplines, the five most frequent disciplines were mostly in the social sciences: psychology (n = 24), sociology, social work (n = 10), education (n = 8), business/commerce/management (n = 7), and engineering (n = 7).

Of the sixty-eight respondents who reported their job title using the list provided in the survey, Liaison Librarian (n = 46) was the most frequently selected; Reference Librarian (n = 8), Public Services Librarian (n = 6), Subject Librarian (n = 5), Branch Head (n = 3), and Knowledge Synthesis Librarian (n = 1) were the least frequently reported job titles from the list. The ‘Other’ responses (n = 22) were grouped into five broad categories (see supplemental material for more detail): Teaching and Learning (functional) (n = 6), Research Support Services (functional) (n = 5), Mixed/Cross-category roles (n = 5), Librarian (unspecified) (n = 3), and Health Sciences (specifically specified) (n = 3). One ‘Other’ response was re-coded to ‘Liaison Librarian.’

Roles on Systematic Reviews

Ninety-one respondents indicated engagement within the past twelve months in one or more of the librarian roles identified by Beverly et al. (2003) when providing reference/research assistance, formal instruction, or as a member of a review team (or collaborator).

Respondents who provide systematic review reference or research assistance, reported assistance more frequently in the more traditional librarian roles of search strategy development (n = 73, 80 percent), database selection (n = 73, 80 percent), research question formulation (n = 67, 74 percent), reference management (n = 62, 68 percent), and document supplier (n = 45, 49 percent). Respondents reported providing research or reference assistance less frequently in the less traditional roles of critical appraiser (n = 24, 26 percent), report writer (n = 17, 18 percent), data extractor (n = 15, 16 percent), project leader (n = 14, 15 percent), project manager (n = 12, 13 percent), disseminator (n = 12, 13 percent), and data synthesizer (n = 9, 10 percent). Two respondents reported engaging in all systematic review roles when providing reference or consultation services.

Respondents reported providing instruction in the more traditional librarian roles of search strategy development (n = 61, 67 percent), database selection (n = 59, 65 percent), research question formulation (n = 55, 60 percent), reference management (n = 45, 49 percent), and document supplier (n = 32, 35 percent). For less traditional librarian roles, respondents reported providing instruction as critical appraiser (n = 16, 18 percent), report writer (n = 15, 18 percent), data extractor (n = 13, 14 percent), data synthesizer (n = 8, 9 percent), disseminator (n = 8, 9 percent), project manager (n = 4, 4 percent), and project leader (n = 2, 2 percent). One respondent reported engaging all systematic reviews roles.

When asked about membership on a systematic review team (collaboration), respondents reported participation primarily in the more traditional librarian roles of search strategy development (n = 66, 73 percent), database selection (n = 64, 70 percent), reference management (n = 55, 60 percent), research question formulation (n = 51, 56 percent), and document supplier (n = 35, 38 percent). Report writer, a non-traditional librarian role, was also frequently reported (n = 46, 51 percent). Participation in other non-traditional librarian roles were reported less frequently: disseminator (n = 7, 8 percent), project leader (n = 5, 5 percent), project manager (n = 4, 4 percent), data extractor (n = 4, 4 percent), critical appraiser (n = 3, 3 percent), and data synthesizer (n = 2, 2 percent). One respondent reported engaging in all roles as a member of a research team.

Comparison of these frequencies by discipline groups (Health Sciences, Non-Health Sciences, Mixed) followed similar patterns as the overall responses. Figure 1 provides a breakdown of the engagement in the Beverly et al. (2003) roles according to discipline (health sciences, non-health sciences, mixed) for the three types of service (reference/research assistance, formal instruction, member of the review team).

Knowledge and Training

Respondents were asked to rank their level of training in, and knowledge of, a variety of systematic review activities and roles, using the scale of 0-3, where zero equals ‘none,’ one indicates ‘some,’ two equals ‘pretty good,’ three indicates ‘extensive.’ Respondents were also able to select ‘Not applicable.’

Of the eighty-nine responses to this question, most ranked their level of knowledge as ‘pretty good’ or ‘extensive’ (combined) in the more traditional librarian systematic review activities of selecting appropriate databases and grey literature sources (n = 85, 96 percent), translating the search strategy (n = 84, 94 percent), documenting the search (n = 80, 90 percent), managing citations (n = 80, 90 percent), and formulating the research question (n = 73, 82 percent). Alternatively, respondents most often ranked their level of knowledge as ‘none’ to ‘some’ (combined) in less traditional librarian systematic review roles of conducting a meta-analysis (n = 79, 89 percent), conducting a risk of bias analysis (n = 74, 83 percent), extracting data from studies (n = 67, 75 percent), and selecting articles (n = 46, 52 percent). There were also five ‘Not applicable’ responses for the four less traditional roles.

The responses to librarian levels of training for specific systematic review roles are consistent with the patterns noted above, where respondents reported ‘pretty good’ or ‘extensive’ (combined) training levels in areas that are typically within librarian purview when participating in systematic review activities. Specifically, of the eighty-nine responses, most reported adequate training to select appropriate databases and grey literature sources (n = 76, 85 percent), translate a search strategy (n = 76, 85 percent), document a search (n = 68, 76 percent), formulate a research question (n = 62, 70 percent), and manage citations (n = 61, 68 percent). Respondents however noted their level of training as ‘none’ to ‘some’ (combined) in the following systematic review roles: conducting a meta-analysis (n = 82, 92 percent), conducting a risk of bias analysis (n = 78, 88 percent), extracting data from studies (n = 74, 83 percent), and selecting articles (n = 58, 65 percent). Six respondents indicated these ‘less traditional’ roles were not applicable.

Barriers

Respondents were asked to select all barriers or challenges that had limited their ability to provide assistance with systematic reviews (in educator, support services/consultant, and/or partner in research roles). Not all respondents indicated barriers. The most common response was the lack of time (n = 59, 63 percent), with the second most frequently noted barrier being insufficient training (n = 28, 30 percent). This pattern was evident regardless of which disciplines were being supported. Less frequently reported challenges included systematic review assistance not being part of one’s assigned duties (n = 11, 12 percent), a low institutional priority (n = 10, 11 percent), a lack of requests (n = 7, 7 percent), not being of interest to the librarian (n = 7, 7 percent), and insufficient access to databases (n = 3, 3 percent). No respondents reported that systematic review support was inappropriate for their rank (See Table 2).

Twenty-two respondents included other barriers or challenges not included in the original list (see supplemental file). Of these, two responses were re-coded into existing categories and one response was simply providing context for their selection of an existing category. Thus, there were nineteen ‘Other’ responses. Four respondents reported lack of staff as the most common ‘Other’ reason that limited their ability to provide assistance with systematic reviews. Requestor’s lack of knowledge, few requests, and no barriers/limitations were each reported by three respondents. Two respondents indicated lack of support from administrators, while one respondent each reported lack of disciplinary knowledge to support article selection, lack of sufficient professional development funding to attend training, and lack of awareness of service and librarian expertise. One respondent reported how the barriers they experienced were causally related: “Because it’s a low priority/sys revs are not supported with enough staff or resources, it thus makes it less enticing to seek out training.”

Other Review Types

Respondents were asked if they had participated in or supported review types other than systematic reviews. The most common other review type to be supported was scoping reviews (n = 81, 89 percent), followed by rapid reviews (n = 40, 44 percent). Fewer responses were noted for mixed methods reviews (n = 26, 29 percent), critical reviews (n = 24, 26 percent), realist reviews (n = 22, 24 percent) and meta-narrative reviews (n = 8, 9 percent) (see Figure 2). Some respondents indicated review types that were not listed as options in the survey (i.e. responses to “Other, please specify”), including integrative reviews (n = 2), literature reviews (n = 2), umbrella reviews (n = 2), evidence gap maps (n = 1), and rapid scoping reviews (n = 1).

Policies

Of the ninety-one responses to the question about whether their institutions had guidelines, policies, or other materials to assist them in clarifying their roles and levels of involvement, sixty-two (68 percent) respondents indicated their institutions had a policy or guideline, eighteen (20 percent) indicated they did not, and eleven (12 percent) were unsure. For institutions with policies or guidelines, the characteristics of the policies or guidelines described by the respondents varied quite widely, including:

• service parameters and types (tiered, consultation vs. collaboration, roles)
• formality (formal or informal)
• means of sharing information (web page, LibGuide)
• audience (librarians or requestors, public facing or internal)
• process (determine eligibility, timelines, waitlists)
• forms (request, eligibility, service level agreements)

Three respondents indicated their institution’s policies/guidelines were either under development or being revised. Three respondents also noted a lack of system-wide documentation and messaging, indicative of a siloed or unit-based approach (e.g., “on our sys rev libguide, NOT part of any central messaging”; “Only for the health science library… ”; “lots of documentation created by other librarians. Not any on a system-level”).

Discussion

The current survey describes systematic review engagement by librarians employed at Canadian universities. The demographics of the respondents for the current survey were similar to the previous survey (Murphy and Boden 2015), with a higher proportion of more experienced librarians (ten plus years of experience) with client focused job titles. A recent survey of business librarian involvement in systematic reviews also had a higher percentage of librarians with ten or more years of professional experience (Premji et al. 2022). Librarians supporting a wide range of disciplines responded to the survey, though a lower proportion of respondents supported only the non-health sciences disciplines.

Librarians are engaging in the full spectrum of potential roles during the production of a review, but to varying extents. Overall, librarians tend to engage most frequently in traditional roles regardless of the type of service being offered (consultation, training/instruction, collaboration) and the disciplines being supported. This echoes previous findings (Murphy and Boden 2015), unsurprisingly as these are the roles most closely aligned with librarian training and expertise. Those traditional roles were also evident in a scoping review of librarian roles in systematic reviews by Spencer and Eldredge (2018) and a survey of business librarians (Premji et al. 2022). Moreover, respondents in this survey ranked both their level of training and level of knowledge higher in more traditional librarian systematic review activities and lower in areas considered less traditional for librarian involvement in systematic reviews. Of special interest is that the ‘Not applicable’ responses to level of training and knowledge were identified only for non-traditional roles, thus indicating that some librarians may not see these roles as applicable to their work.

However, there is a small proportion of librarians reporting engagement in each stage of the review process. Thus, in some cases, librarians’ expertise encompasses stages of the review process beyond the literature search. Both Spencer and Eldredge (2018) and Grossetta et al. (2019) point to roles for librarians outside those required to conduct a review (e.g., peer review). Roles beyond direct engagement in the production of a systematic review are interesting but were not included in the present survey as the focus was on librarian engagement in participating in or supporting systematic reviews.

As early as 2011, roles for health sciences librarians as collaborators on research teams were being mentioned (Dudden and Protzko 2011). The current survey results indicate that librarians supporting health sciences and mixed disciplines more frequently report collaborating on research teams than those that support only non-health sciences disciplines. Furthermore, the report writer role is also more frequently reported when librarians support health sciences and mixed disciplines than the non-health sciences. This may be indicative of maturation of systematic review services for health sciences and librarians supporting mixed disciplines.

Not all respondents indicated barriers to engaging in systematic reviews. A non-response on the barriers question could indicate a lack of barriers or simply that the respondent skipped the question. Of the respondents who did report barriers, the most common reasons were lack of time and insufficient training. Lack of time was clearly the most frequent issue. This is consistent with the previous findings of Murphy and Boden (2015) and Nicholson et al. (2017). Nicholson et al. noted that “Respondents reported difficulty balancing work on systematic reviews with other professional duties as well as an inability to keep up with the demand for support with systematic reviews at their institutions” (2017, 289). This suggests at least two potential interpretations of the ‘lack of time’ barrier: the balancing of multiple responsibilities and unsustainable demands for systematic review services. Saleh et al. noted that “Many information professionals are multi-tasking, such as is the case with academic health science librarians and hospital librarians, and therefore time management is of great interest in order to efficiently integrate systematic review searching into one’s routine responsibility” (2014, 43). While a recent survey of business librarians (Premji et al. 2022) also found that time constraints and lack of training were among the top five barriers, their top two challenges were a lack of engagement in synthesis reviews by faculty and students in their discipline, and a lack of requests. Inter-disciplinary differences in synthesis review services and challenges should be explored in more depth.

Respondents reported ‘Other’ barriers that seemed to fall into three groupings: individual (e.g., lack of disciplinary knowledge), internal to the library (e.g., lack of staff to support systematic reviews), and external to the library (e.g., too few requests, lack of knowledge by the requestors). The latter barrier was also evident in the methods-related questions in Nicolson et al. (2017). Some of the most frequently reported challenges in the Nicholson et al. survey related to a lack of methodological knowledge by librarians and faculty, and lack of student support by supervising faculty.

An apparently paradoxical result is that respondents generally reported ‘pretty good’ or ‘extensive’ training levels, yet insufficient training was the second most frequently reported barrier. Closer examination suggests that confidence in training levels tended to be good for more traditional librarian systematic review roles and lower in less traditional roles. One explanation might be that the concerns about training levels are related to non-traditional roles (e.g., conducting a meta-analysis). Alternatively, respondents may not have felt they had sufficient training for skills that were not asked about in the survey (e.g., time management, communicating with research teams, how to have conversations about authorship (O’Dwyer and Wafford 2021; Thurow et al. 1999). Interpersonal challenges in collaboration on systematic reviews (e.g., dysfunctional team, researchers refusing requests for authorship) are relatively frequent (Nicholson et al. 2017). One could surmise that a librarian may feel knowledgeable and well-trained on the skills-based aspects of engaging in a systematic review but less comfortable with the interpersonal aspects. Another possibility is that respondents are supporting multiple disciplines or new disciplines and, thus, are having to learn new skills for each discipline (e.g., new databases, new terminology). Factors such as “experience level, the librarian’s role, the nature and complexity of the systematic review, the level of engagement with other systematic review team members, and institutional expectations… ” (Bullers et al. 2018, 204), as well as librarians not having “sufficient experience to reduce the amount of time they spend on discrete tasks” (Bullers et al. 2018, 205), may also contribute to the amount of time spent on systematic reviews. This finding deserves further examination to inform continuing education offerings.

There has been a proliferation of review types with their roots in systematic review methods but adapted for other purposes and disciplines. Much has been written to clarify typologies of these reviews (Moher et al. 2015; Munn et al. 2018; Paré et al. 2015; Sutton et al. 2019; Tricco et al. 2016). In the current survey, respondents were asked to indicate review types, other than systematic reviews, that they support. In both the 2015 (Murphy and Boden) and the current survey, scoping reviews were clearly the most common of the other review types supported by librarians. It is likely there are librarians working at Canadian universities exclusively supporting other review types (e.g., scoping reviews) who were not eligible for the present survey. Future research should explore synthesis review support by librarians more wholistically to better capture the roles of librarians.

One might surmise that the increase in respondents reporting institutional policies or guidelines from 2015 (50 percent; Murphy and Boden) to the current survey (68 percent) is related to a need by libraries to manage service requests and improve service sustainability. Structured service models have been suggested as a way of addressing some of the challenges arising in collaborations on systematic reviews (Nicholson et al.2017). The literature suggests that libraries are seeking to improve the sustainability of their systematic review services. Systematic review service models have been developed despite challenges with resources and experience (Lackey et al. 2019); services have been formalized to increase capacity and to address collaboration challenges (e.g., McKeown and Ross-White 2019); libraries have re-imagined their services (Luca and Ulyannikova 2020); and strategies have been identified and implemented to manage demand (Campbell and Dorgan 2015). Questions raised about burnout amongst medical librarians supporting systematic reviews (Demetres et al. 2020) may possibly be related to service (un-)sustainability. Future research should examine the effectiveness of service models (and particular configurations of service models), policies, and guidelines in managing demands on systematic review services and librarian workload.

This current study had some limitations. The sample was skewed toward more experienced librarians, as was the previous survey (Murphy and Boden 2015). This may reflect the reality of those supporting systematic reviews in Canadian universities, but it is not possible to be certain from the convenience sample. Furthermore, the sample is smaller than desired. In the preparatory stages of this study, a recommended sample size was calculated based on an estimate of the number of librarians working in Canadian university libraries who might support systematic reviews using statistics from the 2018 Census of Canadian Academic Libraries (Canadian Association of Professional Academic Librarians CAPAL Advocacy Committee 2019). The sample of ninety-four respondents was just under half of the calculated sample size. It should be noted that the total number of librarians supporting systematic reviews upon which that sample size was calculated may have been an overestimation for two reasons: (1) not all librarians providing reference support will provide systematic review services, and (2) not all librarians supporting synthesis reviews will have supported systematic reviews (one kind of synthesis review) within the twelve-month time-frame of our eligibility criteria. Overall, our results are indicative but difficult to generalize.

Conclusions

Not only are librarians supporting disciplines outside the health sciences, but they are also supporting reviews in multiple, diverse disciplines. This disciplinary porousness is possibly related to the emergence of functional roles and organizational structures now at many Canadian universities which co-occur with or replace liaison models. Librarians’ primary contributions to supporting systematic reviews are, unsurprisingly, in the literature search role. In addition, librarians continue to experience challenges arising from lack of time and the need for additional training.

References

Arksey, Hilary, and Lisa O’Malley. 2005. “Scoping Studies: Towards a Methodological Framework.” International Journal of Social Research Methodology 8 (1): 19–32.

Beverley, C. A., Andrew Booth, and Peter A. Bath. 2003. “The Role of the Information Specialist in the Systematic Review Process: A Health Information Case Study.” Health Information & Libraries Journal 20 (2): 65–74. https://doi.org/10.1046/j.1471-1842.2003.00411.x.

Brereton, Pearl, Barbara A. Kitchenham, David Budgen, Mark Turner, and Mohamed Khalil. 2007. “Lessons from Applying the Systematic Literature Review Process within the Software Engineering Domain.” Journal of Systems and Software, Software Performance, 80 (4): 571–83. https://doi.org/10.1016/j.jss.2006.07.009.

Bullers, Krystal, Allison M. Howard, Ardis Hanson, William D. Kearns, John J. Orriola, Randall L. Polo, and Kristen A. Sakmar. 2018. “It Takes Longer than You Think: Librarian Time Spent on Systematic Review Tasks.” Journal of the Medical Library Association: JMLA 106 (2): 198–207. https://doi.org/10.5195/jmla.2018.323.

Campbell, Sandy, and Marlene Dorgan. 2015. “What to Do When Everyone Wants You to Collaborate: Managing the Demand for Library Support in Systematic Review Searching.” Journal of the Canadian Health Libraries Association / Journal de l’Association Des Bibliothèques de La Santé Du Canada 36 (1): 11–19. https://doi.org/10.29173/jchla/jabsc.v36i1.24353.

Canadian Association of Professional Academic Librarians CAPAL Advocacy Committee. 2019. “2018 Census of Canadian Academic Librarians User Guide and Results Summary.” https://capalibrarians.org/wp/wp-content/uploads/2019/03/2018_Census_March_24_2019.pdf.

Canadian Institutes of Health Research. 2010. “A Guide to Knowledge Synthesis.” March 25, 2010. https://cihr-irsc.gc.ca/e/41382.html.

Colquhoun, Heather L., Danielle Levac, Kelly K. O’Brien, Sharon Straus, Andrea C. Tricco, Laure Perrier, Monika Kastner, and David Moher. 2014. “Scoping Reviews: Time for Clarity in Definition, Methods, and Reporting.” Journal of Clinical Epidemiology 67 (12): 1291–94. https://doi.org/10.1016/j.jclinepi.2014.03.013.

Demetres, Michelle R., Drew N. Wright, and Antonio P. DeRosa. 2020. “Burnout among Medical and Health Sciences Information Professionals Who Support Systematic Reviews: An Exploratory Study.” Journal of the Medical Library Association: JMLA 108 (1): 89–97. https://doi.org/10.5195/jmla.2020.665.

Djulbegovic, Benjamin, and Gordon H Guyatt. 2017. “Progress in Evidence-Based Medicine: A Quarter Century On.” The Lancet 390 (10092): 415–23. https://doi.org/10.1016/S0140-6736(16)31592-6.

Dudden, Rosalind F., and Shandra L. Protzko. 2011. “The Systematic Review Team: Contributions of the Health Sciences Librarian.” Medical Reference Services Quarterly 30 (3): 301–15. https://doi.org/10.1080/02763869.2011.590425.

Grossetta Nardini, Holly K., Janene Batten, Melissa C. Funaro, Rolando Garcia-Milian, Kate Nyhan, Judy M. Spak, Lei Wang, and Janis G. Glover. 2019. “Librarians as Methodological Peer Reviewers for Systematic Reviews: Results of an Online Survey.” Research Integrity and Peer Review 4 (1): 23. https://doi.org/10.1186/s41073-019-0083-5.

Haddaway, Neal R., Biljana Macura, Paul Whaley, and Andrew S. Pullin. 2018. “ROSES RepOrting Standards for Systematic Evidence Syntheses: Pro Forma, Flow-Diagram and Descriptive Summary of the Plan and Conduct of Environmental Systematic Reviews and Systematic Maps.” Environmental Evidence 7 (1): 7. https://doi.org/10.1186/s13750-018-0121-7.

Higgins, Julian, James Thomas, Jacqueline Chandler, Miranda Cumpston, Tianjing Li, Matthew Page, and Vivian Welch. 2022. Cochrane Handbook for Systematic Reviews of Interventions. www.training.cochrane.org/handbook.

Kallaher, Amelia, Erin R. B. Eldermire, Christine T. Fournier, Kate Ghezzi-Kopel, Kelly A. Johnson, Jim Morris-Knower, Sara Scinto-Madonich, and Sarah Young. 2020. “Library Systematic Review Service Supports Evidence-Based Practice Outside of Medicine.” The Journal of Academic Librarianship 46 (6): 102222. https://doi.org/10.1016/j.acalib.2020.102222.

Khangura, Sara, Kristin Konnyu, Rob Cushman, Jeremy Grimshaw, and David Moher. 2012. “Evidence Summaries: The Evolution of a Rapid Review Approach.” Systematic Reviews 1 (1): 10. https://doi.org/10.1186/2046-4053-1-10.

Lackey, Mellanye J., Heidi Greenberg, and Melissa L. Rethlefsen. 2019. “Building the Systematic Review Core in an Academic Health Sciences Library.” Journal of the Medical Library Association: JMLA 107 (4): 588–94. https://doi.org/10.5195/jmla.2019.711.

Luca, Edward, and Yulia Ulyannikova. 2020. “Towards a User-Centred Systematic Review Service: The Transformative Power of Service Design Thinking.” Journal of the Australian Library and Information Association 69 (3): 357–74. https://doi.org/10.1080/24750158.2020.1760506.

McKeown, Sandra, and Amanda Ross-White. 2019. “Building Capacity for Librarian Support and Addressing Collaboration Challenges by Formalizing Library Systematic Review Services.” Journal of the Medical Library Association: JMLA 107 (3): 411–19. https://doi.org/10.5195/jmla.2019.443.

Miljand, Matilda. 2020. “Using Systematic Review Methods to Evaluate Environmental Public Policy: Methodological Challenges and Potential Usefulness.” Environmental Science & Policy 105 (March): 47–55. https://doi.org/10.1016/j.envsci.2019.12.008.

Moher, David, Lesley Stewart, and Paul Shekelle. 2015. “All in the Family: Systematic Reviews, Rapid Reviews, Scoping Reviews, Realist Reviews, and More.” Systematic Reviews 4 (1): 183. https://doi.org/10.1186/s13643-015-0163-7.

Munn, Zachary, Cindy Stern, Edoardo Aromataris, Craig Lockwood, and Zoe Jordan. 2018. “What Kind of Systematic Review Should I Conduct? A Proposed Typology and Guidance for Systematic Reviewers in the Medical and Health Sciences.” BMC Medical Research Methodology 18 (1): 5. https://doi.org/10.1186/s12874-017-0468-4.

Murphy, Susan A., and Catherine Boden. 2015. “Benchmarking Participation of Canadian University Health Sciences Librarians in Systematic Reviews.” Journal of the Medical Library Association: JMLA 103 (2): 73–78. https://doi.org/10.3163/1536-5050.103.2.003.

Nicholson, Joey, Aileen McCrillis, and Jeff D. Williams. 2017. “Collaboration Challenges in Systematic Reviews: A Survey of Health Sciences Librarians.” Journal of the Medical Library Association: JMLA 105 (4): 385–93. https://doi.org/10.5195/jmla.2017.176.

O’Dwyer, Linda C., and Q. Eileen Wafford. 2021. “Addressing Challenges with Systematic Review Teams through Effective Communication: A Case Report.” Journal of the Medical Library Association: JMLA 109 (4): 643–47. https://doi.org/10.5195/jmla.2021.1222.

Paré, Guy, Marie-Claude Trudel, Mirou Jaana, and Spyros Kitsiou. 2015. “Synthesizing Information Systems Knowledge: A Typology of Literature Reviews.” Information & Management 52 (2): 183–99. https://doi.org/10.1016/j.im.2014.08.008.

Pawson, Ray, Trisha Greenhalgh, Gill Harvey, and Kieran Walshe. 2005. “Realist Review - a New Method of Systematic Review Designed for Complex Policy Interventions.” Journal of Health Services Research & Policy 10 (1_suppl): 21–34. https://doi.org/10.1258/1355819054308530.

Premji, Zahra, Ryan Splenda, and Sarah Young. 2022. “An Exploration of Business Librarian Participation in Knowledge Synthesis Reviews.” College & Research Libraries 83 (2): 314–36. https://doi.org/10.5860/crl.83.2.314.

Rethlefsen, Melissa L., Shona Kirtley, Siw Waffenschmidt, Ana Patricia Ayala, David Moher, Matthew J. Page, Jonathan B. Koffel, et al. 2021. “PRISMA-S: An Extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews.” Systematic Reviews 10 (1): 39. https://doi.org/10.1186/s13643-020-01542-z.

Riegelman, Amy, and Megan Kocher. 2018. “A Model for Developing and Implementing a Systematic Review Service for Disciplines Outside of the Health Sciences.” Reference & User Services Quarterly 58 (1): 22–27.

Saleh, Ahlam A., Melissa A. Ratajeski, and Marnie Bertolet. 2014. “Grey Literature Searching for Health Sciences Systematic Reviews: A Prospective Study of Time Spent and Resources Utilized.” Evidence Based Library and Information Practice 9 (3): 28–50.

Schvaneveldt, Nena, and Elizabeth M. Stellrecht. 2021. “Assessing the Roles and Challenges of Librarians in Dental Systematic and Scoping Reviews.” Journal of the Medical Library Association JMLA 109 (1): 52–61. https://doi.org/10.5195/jmla.2021.1031.

Slebodnik, Maribeth, Janice Hermer, and Kevin Pardon. 2018. “Investigating Systematic Reviews Outside Health Sciences.” Presented at the Science & Technology Section Research Poster Session, American Library Association Annual Meeting 2018, June 24, 2018, New Orleans, LA., June. https://repository.arizona.edu/handle/10150/628518.

Spencer, Angela J., and Jonathan D. Eldredge. 2018. “Roles for Librarians in Systematic Reviews: A Scoping.” Journal of the Medical Library Association JMLA 106 (1): 46–56. http://dx.doi.org/10.5195/jmla.2018.82.

Sutton, Anthea, Mark Clowes, Louise Preston, and Andrew Booth. 2019. “Meeting the Review Family: Exploring Review Types and Associated Information Retrieval Requirements.” Health Information & Libraries Journal 36 (3): 202–22. https://doi.org/10.1111/hir.12276.

The Methods Coordinating Group of the Campbell Collaboration. 2019. “Methodological Expectations of Campbell Collaboration Intervention Reviews: Reporting Standards.” Campbell Policies and Guidelines Series No. 4. Https://doi.org/10.4073/cpg.2016.4.

The Methods Group of the Campbell Collaboration. 2019. “Methodological Expectations of Campbell Collaboration Intervention Reviews: Conduct Standards.” Campbell Policies and Guidelines Series No. 3. Https://doi.org/10.4073/cpg.2016.3.

Thurow, Amy Purvis, Charles William Abdalla, Julie Younglove-Webb, and Barbara Gray. 1999. “The Dynamics of Multidisciplinary Research Teams in Academia.” The Review of Higher Education 22 (4): 425–40. https://doi.org/10.1353/rhe.1999.0019.

Tricco, Andrea C., Wasifa Zarin, Jesmin Antony, Brian Hutton, David Moher, Diana Sherifali, and Sharon E. Straus. 2016. “An International Survey and Modified Delphi Approach Revealed Numerous Rapid Review Methods.” Journal of Clinical Epidemiology 70 (February): 61–67. https://doi.org/10.1016/j.jclinepi.2015.08.012.

Appendix A – Survey Instrument (English version)

Benchmarking Synthesis Review Support in Academic Libraries - 2022

We are interested in the current state of librarian support (e.g., reference assistance, instruction) and collaboration (i.e., participating as a research team member) at Canadian university libraries for systematic reviews conducted by faculty, graduate students, and academic administrators.

1. During the past 12 months were you employed at a Canadian University Library?
   • □ Yes
   • □ No
2. Have you participated in a systematic review in the past 12 months? (‘Participated’ is defined as supported, collaborated or initiated an ongoing or new systematic review)
   • □ Yes
   • □ No
3. In which disciplines have you provided systematic review services in the past 12 months? Please select all that apply.

□ Agriculture, Bioresources, Soil Science □ Biology, Botany, Plant Science □ Biomedical/Life Sciences □ Business, Commerce, Management □ Chemistry □ Community Health & Epidemiology □ Computer Science □ Dentistry □ Education □ Engineering □ Environment & Sustainability □ Forestry □ Geography & Planning, Community & Regional Planning □ Geology, Earth Sciences, Ocean Science □ Indigenous Studies □ Kinesiology

□ Land and Food Systems □ Medicine □ Midwifery □ Neuroscience □ Nursing □ Nutrition □ Pharmacy □ Physics, Astronomy, Astrophysics, Atmospheric Science □ Rehabilitation Sciences □ Psychology □ Public Health □ Political Studies, Public Policy □ Sociology, Social Work □ Toxicology □ Veterinary Medicine, Animal Science, Zoology □ Other, please specify:

4. Think specifically about systematic reviews when answering the questions below. Please indicate the role(s) you have engaged with in the past 12 months when delivering formal instruction, providing reference/research assistance, and as a member of a review team. If your manner of engagement falls into a “grey zone” (i.e., your engagement does not fulfill ALL criteria), choose the role(s) that are mostly applicable. Select all that apply.

	Have you provided formal instruction in any of the following roles?	Have you provided reference/research assistance in any of the following roles?	In which roles have you participated as a member of a systematic review team?
Project Leader - writing the initial grant (in conjunction with the rest of the team, if applicable), liaising with the sponsors, chairing project meetings, as well as co-ordinating the writing of the final report and dissemination of the review findings	Yes No	Yes No	Yes No
Project Manager - creating and regularly updating the project timetable in order to ensure that the review was completed both within the required timescale and according to budget	Yes No	Yes No	Yes No
Literature Searcher - Research Question Formulation	Yes No	Yes No	Yes No
Literature Searcher - Database Selection	Yes No	Yes No	Yes No
Literature Searcher - Search Strategy (developing, conducting & documenting)	Yes No	Yes No	Yes No
Reference Manager - managing the references – tracking and storing references identified by the search, de-duplicating	Yes No	Yes No	Yes No
Document Supplier - locating and retrieving the full-text of relevant articles and reports	Yes No	Yes No	Yes No
Critical Appraiser - reviewing articles for inclusion/exclusion based on stringent criteria	Yes No	Yes No	Yes No
Data Extractor - the selective abstracting of key components (population, setting, intervention(s), outcome(s), results, etc.) of the included studies	Yes No	Yes No	Yes No
Data Synthesizer - synthesizing the results as a narrative and/or meta-analysis	Yes No	Yes No	Yes No
Report Writer - assist in the writing of the paper or report	Yes No	Yes No	Yes No
Disseminator - providing current awareness or digest services to clinicians, making resources, particularly evidence-based products, readily available online, etc.	Yes No	Yes No	Yes No

5. Please answer the questions in the table below in the context of a systematic review.

How would you rate your level of training in each area?

0 = none 1= some 2 = pretty good 3 = extensive Not applicable

	How would you rate your level of training in each area?
Formulating the research question	0 1 2 3 n/a
Selecting appropriate databases and grey literature sources	0 1 2 3 n/a
Translating the operational definitions of the concepts in the research question into a comprehensive search strategy	0 1 2 3 n/a
Documenting the literature search	0 1 2 3 n/a
Managing the citations (deduplication, storing citations)	0 1 2 3 n/a
Selecting articles for inclusion	0 1 2 3 n/a
Extracting data from included studies	0 1 2 3 n/a
Conducting a risk of bias analysis	0 1 2 3 n/a
Conducting a meta-analysis (or meta-synthesis)	0 1 2 3 n/a

6. Please answer the questions in the table below in the context of a systematic review.

How would you rate your knowledge in each area?

0 = none 1= some 2 = pretty good 3 = extensive Not applicable

	How would you rate your knowledge in each area?
Formulating the research question	0 1 2 3 n/a
Selecting appropriate databases and grey literature sources	0 1 2 3 n/a
Translating the operational definitions of the concepts in the research question into a comprehensive search strategy	0 1 2 3 n/a
Documenting the literature search	0 1 2 3 n/a
Managing the citations (deduplication, storing citations)	0 1 2 3 n/a
Selecting articles for inclusion	0 1 2 3 n/a
Extracting data from included studies	0 1 2 3 n/a
Conducting a risk of bias analysis	0 1 2 3 n/a
Conducting a meta-analysis (or meta-synthesis)	0 1 2 3 n/a

7. Which of the following have limited your ability to provide assistance with systematic reviews (in educator, support services/consultant and/or partner in research roles)? Please check all that apply.
   • □ I don’t have sufficient training
   • □ It is not part of my assigned duties
   • □ I don’t have enough time
   • □ I am not interested
   • □ It is a low priority service at my institution
   • □ My institution lacks the breadth of database resources required to do a systematic review
   • □ I have not received any requests
   • □ It is not appropriate for my rank
   • □ Other (please specify):
8. Does your institution have policies, guidelines or other materials to assist you in clarifying your potential role(s) and level(s) of involvement to the requestor of a systematic review?
   • □ No
   • □ Not sure
   • □ Yes. Please include a brief description of no more than a sentence or two.
9. Which other types of literature reviews have you been involved with in the last 12 months? Being “involved” can include, but is not limited to, ongoing reviews that were started more than a year ago (e.g., you are just finishing up the final publication details, reading drafts, etc.) and new reviews that have just been established (e.g., the request came in yesterday).

Scoping Reviews [Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature]	Yes No Unsure
Critical Review [Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation]	Yes No Unsure
Mixed Methods Review [Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies]	Yes No Unsure
Rapid Review [Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research]	Yes No Unsure
Meta-Narrative “seeks to illuminate a heterogeneous topic area by highlighting the contrasting and complementary ways in which researchers have studied the same or a similar topic”, using qualitative and mixed methods studies http://www.biomedcentral.com/1741-7015/11/20	Yes No Unsure
Realist Review/Synthesis “This approach offers the potential to expand the knowledge base in policy-relevant areas -for example, by explaining the success, failure or mixed fortunes of complex interventions.” http://www.biomedcentral.com/1741-7015/11/21/abstract	Yes No Unsure
Other (please specify)

10. What is your job title?
    • □ Liaison Librarian
    • □ Subject Librarian
    • □ Reference Librarian
    • □ Public Services Librarian
    • □ Knowledge Synthesis Librarian
    • □ Branch Head
    • □ Other (please specify)
11. How long have you been a librarian?
    • □ 0-3 years
    • □ 4-6 years
    • □ 7-9 years
    • □ 10+ years