Training Needs and Preferences for Librarians Supporting Systematic Reviews in the Sciences, Humanities, and Social Sciences

Mê-Linh Lê, Janice Winkler, and Christine J. Neilson*

Systematic reviews, and other forms of knowledge synthesis, are an increasingly popular research methodology being used in the sciences, humanities, and social sciences. Librarians are being called upon to support this work through consultation, instruction, and/or performing the systematic search on behalf of the research team. Professional development is essential for librarians to develop their skills and to build confidence when it comes to providing SR support to researchers. This article reports on a survey of American and Canadian academic librarians serving the Sciences, Humanities and Social Sciences. Participants’ responses indicate their knowledge of SR support activities and identify potential areas to focus on for future training and professional development.

Background

Knowledge synthesis (KS) reviews, or evidence syntheses, are popular forms of research that systematically gather, evaluate, and synthesize existing literature to answer a research question. While systematic reviews may be the most well-known form of KS, the types of reviews included in the KS “family” continues to expand, and includes scoping reviews, meta-ethnographies, and integrative reviews, among many others. The specific type of KS review used by a research team will vary based on several factors, such as the type of question being asked (e.g., narrow or more broad focused), the types of studies to include (e.g., quantitative, qualitative, mixed-methods, etc.), or amount of time available. Systematic reviews are traditionally very focused questions that include specific study types to compare interventions. Scoping reviews typically ask broader questions that are exploratory in nature and can help to identify gaps in the literature or areas needing more specific focus. Meta-ethnographies, which are considered a form of qualitative systematic review, attempt to provide new interpretations or theories to explain what the research is showing, rather than simply collecting all known evidence on a topic. Integrative reviews summarize existing theoretical or empirical evidence to give a broader understanding of a phenomenon or practice across multiple disciplines and can include a wide array of methodologies.1 These four review types can all take anywhere from six months to years but there are reviews that can be done in shorter time periods as well (e.g., rapid reviews). There are now over forty different types of KS reviews, each suitable for different purposes and contexts.2 For the purposes of simplicity, all forms of KS will be referred to as Systematic Reviews (SR) hereafter.

As SRs are literature-based research, supporting SR researchers is a natural role for the library. Health librarians have filled this role for many years, to the extent that growing demand for health librarian involvement has led to concerns over librarian workload, and publications with titles such as “What to Do When Everyone Wants You to Collaborate: Managing the Demand for Library Support in Systematic Review Searching” and “Burnout Among Medical and Health Sciences Information Professionals Who Support Systematic Reviews: An Exploratory Study.”3 But health is not the only field that uses SR methods. Shelbe established a history of the use of SR methods in a variety of disciplines through a bibliometric study of scholarly publication spanning 1972-2011.4 SRs were established in education, psychology, and business in the 1970s, and were later adopted in various other fields at varying rates. The number of SRs published increased over time in all the fields examined; however, the health/medicine disciplines produced the most SRs during the nearly forty year period examined—with an average of 209.4 SRs per 10,000 articles published—because publications in the field included the continuous use of SR methods. The science disciplines had an average of 66.1 SRs per 10,000 publications, and social science disciplines followed closely with an average of 56.4 SRs per 10,000 publications during the time period examined.5 A thorough inventory of SRs published in various disciplinary areas after 2011 is not available; however, even if the use of SR methods stopped growing and remained constant since then, the number of SRs published per year will have increased over the past twelve years as the number of scholarly articles overall has increased. A recent study by Savage and Olejniczak estimates that the number of articles published by American social scientists increased by over 36 percent between 2011 and 2019, while the number of books published dropped by over 23 percent during the same period.6 Meanwhile White reported that worldwide, the number of peer reviewed publications in the Sciences—including health—increased from 1.9 million publications published in the year 2010 to 2.9 million published in the year 2020.7

With more and more SRs being published, library discourse has begun to take a closer look at SRs completed in the Sciences, Humanities, and Social Sciences (SHSS), and the library’s role in supporting SRs in those disciplines. This includes the examinations of the characteristics of SHSS SRs;8 documented librarian involvement in SHSS SRs;9 resources and techniques for conducting SRs outside of health;10 descriptions of developing SR support services beyond health;11 and the benefits of library support for SHSS SRs.12 Of particular relevance to this paper, Kogut and colleagues’ 2020 case study documented a program to develop SR expertise among education liaison librarians in order to meet the growing demand for SR support from the Texas A&M University’s College of Education and Human Development.13 Demand for SR support in Education may not come as a surprise given the long history of SR method use in the field, but in a complementary article, we found that demand for SR support has increased in SHSS broadly. We reported that 70 percent of survey respondents indicated that a faculty member, researcher, or student from the SHSS had asked them to participate in a systematic review in the past five years, and 55.9 percent of respondents indicated an increase in the frequency of those requests during that time.14 Some respondents were responsible for liaison to a health discipline as well as a SHSS discipline, but requests for support were not attributed to health alone; 1.7 percent of SR support requests were attributed to the Humanities, 18 percent of requests came from the Sciences, and 39.3 percent of support requests came out of the Social Sciences.

As library support services for SRs outside of health start to grow, so too does the need for training librarians to provide that support. Professional learning can take place in many forms: in person, virtually, or independently. Examples include professional reading; learning from colleagues through mentorship15 or communities of practice; learning through experience; attending workshops and courses;16 or combinations of the above.17 Townsend and colleagues developed a set of six competency areas for health librarians that may prove useful to librarians serving other disciplines when creating a plan for continuing education. These six areas are: SR foundations; process management and communication; research methodology; comprehensive searching; data management; and reporting.18 While Townsend and colleagues’ competency framework provides a starting point, Kogut and colleagues have pointed out three key elements that lead to the success of their SR training program for Education librarians: 1. Training must be tailored to the librarians’ liaison area, as the health context will not necessarily translate to other disciplines; 2. Experienced subject librarians possess existing disciplinary knowledge and skills that are important for SR support, but it is important to recognize that even with that head start, becoming proficient in SR support is a process that will take years; 3. Making SR support one of library administration’s service priorities is necessary to ensure that librarians take the time needed to actively develop their knowledge and skills in that area.19

If SR training for SHSS librarians is needed, what should that training look like? This article reports on the systematic review training needs and preferences for librarians who support subject areas in the SHSS disciplines. This data was collected as part of a survey of Association of Research Libraries (ARL) and Canadian Association of Research Libraries (CARL) SHSS librarians’ involvement in SRs, their comfort and competence with systematic review processes, and their perception of library administrators’ level of support for SHSS librarian participation in systematic reviews. Information regarding the demand for SR support and librarian involvement are presented in a separate article.20

Methods

This paper reports additional analysis of a previously reported survey.21 A complete account of the study methods are available there and are summarized for the reader here. This study was approved by the authors’ institutional research ethics board (JFREB J2020:062). Librarians who support subject areas in the sciences, humanities, and social sciences (SHSS) at ARL and CARL institutions were surveyed with an open online survey. Participants were recruited in a convenience sample using twenty-two major listservs with an initial recruitment email and a reminder email three weeks later. Librarians were invited to participate if they were currently employed at an ARL or CARL institution, providing direct library services and support to faculty, staff, or students within the sciences, humanities, or social sciences.

In the survey, respondents were asked which disciplines they support, as well as which disciplines have requested SR support, and for the purposes of analysis these responses were grouped into broader categories of Humanities, Sciences, Social Sciences, and Health Sciences. Each discipline only fell under one category, though respondents often support disciplines in more than one category. The method used for grouping disciplinary areas is discussed in more detail in Lê, Neilson, and Winkler,22 and a list of liaison areas which fell under each category is available via Open Science Framework (OSF) at https://osf.io/mqxf2/.23

During data clean-up, responses were removed from the survey if they did not meet the stated inclusion criteria. Namely, respondents had to be librarians working at a CARL/ARL institution who supported at least one discipline within the SHSS. Respondents were removed if after categorization it was determined that they only supported health sciences disciplines, or if they provided their institution and it was found not to be a CARL/ARL institution. Responses were also removed if respondents did not complete the survey beyond the initial demographic questions. A total of 379 participants responded (360 in English, and nineteen in French). After responses were removed that did not meet the stated inclusion criteria, 161 usable responses remained. The largest number of excluded responses (n=156) were from librarians that did not work at a CARL/ARL institution.

The survey contained a total of twenty-nine possible questions; conditional logic ensured that respondents were only asked questions which were applicable to them. All questions were optional, so that participants were able to skip any question to which they did not wish to provide an answer. A subset of ten questions in the survey asked about their training opportunities and preferences; these responses are reported here. The survey asked what SR related training they had received, and whether they received support from their library administrators in the form of funding for fees (e.g., professional development registration fees for SR training) or time to complete the training. It also asked their preferred format for training and if they had anyone to whom they could ask questions about SR methods.

In order to assess training needs, respondents were asked to indicate their knowledge of several standard areas of SR support. They were then asked about their confidence in supporting patrons in these areas: the review process, and what makes different types of reviews unique; different forms of bias and how to mitigate them in the literature search; established tools and published guidelines for systematic reviews; translating search strategies for use in other databases; searching the grey literature; requirements for record keeping and literature search reporting; using reference management software for de-duplication of large result sets; and using screening software designed for use in systematic reviews. Because database searching is a core skill in librarianship, respondents were not asked about their level of knowledge in this area but were asked how comfortable they were assisting patrons with it. Respondents were also asked how knowledgeable they were about Peer Review of Electronic Search Strategies (PRESS); however, because PRESS is a tool intended to help information professionals review database search strategies designed by other information professionals, respondents were not asked about their comfort level supporting library patrons with its use.

Confidential data was stored in password protected folders to which only members of the research team had access. Data was de-identified for sharing by removing potentially identifying variables such as liaison areas, institution name, and all open-ended responses, before sharing on OSF. For information about how disciplinary areas were defined, see OSF at https://osf.io/mqxf2/. Descriptive analysis was conducted for each close-ended question included in the survey. Responses provided in the free-text questions were coded into broad themes.

A statistical consultant was hired to complete more sophisticated statistical analysis to identify potential relationships between variables. The statistical consultant who conducted the data analysis for this project signed a pledge of confidentiality, as required by the authors’ institutional research ethics board, to ensure the security of participant data. The most appropriate method of statistical analysis for each question was determined by the statistical consultant. Two potential relationships were explored: first, between the types of learning activities participants engaged with and the number of SRs they had supported and, second, between the attitude respondents’ administrators held towards SR support and which types of support were available to them to participate in training. The former relationship was examined using Spearman’s correlation coefficient, and the latter through cross tabulation.

Results

There were a total of 379 responses, 161 of which met inclusion criteria and proceeded to the analysis stage, while the remainder were deleted. Ninety-eight respondents supplied the name of their institution. Forty-two of 108 (38.8 percent) US-based ARL institutions and twenty of thirty-one CARL institutions (64.5 percent) were represented in the survey. Of the forty-two ARL institutions, thirty-eight (90.4 percent) were R1 (Doctoral institutions – very high research activity), three (7 percent) were R2 (Doctoral institutions – high research activity), and one (2 percent) was an M1 (Master’s Colleges and Universities: Larger programs) institution. Thirteen (87 percent) of the CARL institutions represented in the sample were a part of the U15 – Canada’s collective of fifteen research-intensive institutions.24 De-identified study data is freely available online at OSF at https://osf.io/mqxf2/.25

As respondents were able to list multiple disciplines they supported, 521 total areas were reported. These were assigned to broader disciplinary categories as described in the Methods. Seventeen percent were Humanities, 41 percent were Social Sciences, 32 percent were Sciences, and 10 percent were Health Sciences.

Knowledge and Confidence Relating to Aspects of SRs

When asked to indicate their knowledge of, and confidence in supporting patrons with, several aspects of SR support, respondents typically reported similar levels of knowledge and confidence for specific aspects of SR support listed in both the Knowledge and Confidence questions. However, for most aspects of SR support, respondents indicated slightly lower levels of confidence in their ability to help patrons with that element than their reported level of knowledge (see Figure 1).

As Table 1 indicates, respondents were most confident helping library patrons with activities related to database searching, with 117 (80.7 percent) indicating they were confident or very confident helping patrons with advanced database searching techniques, and 108 (74.5 percent) confident or very confident with the aspect of translating search strategies for use in other databases. More than half of respondents indicated that they were knowledgeable or very knowledgeable of the following: the review process, and what makes different types of reviews unique (60.7 percent); translating search strategies for use in other databases (66.9 percent); searching the grey literature (53.8 percent); requirements for record keeping and literature search reporting (51.7 percent); and using reference management software for de-duplication of large result sets (62.7 percent). Respondents indicated that they were least knowledgeable about peer review of electronic search strategies (PRESS), with sixty-two (42.8 percent) indicating they were not at all knowledgeable, and using screening software designed for SRs, with forty-nine (33.8%) indicating they were not at all knowledgeable.

Training

Training can take many forms, whether it is self-directed, as part of informal learning groups, or as part of much more structured professional development. Librarians were asked to identify what, if any, training on SRs they had already completed. Figure 2 below shows that 82.6 percent of respondents (120) had previously participated in at least some form of learning activity related to systematic reviews. These activities included: independent learning through reading relevant professional literature and/or online content (66.5 percent; 107 respondents); participating in continuing education workshops, including webinars (52.2 percent; eighty-four respondents); mentorship from a colleague who has experience supporting SRs (48.4 percent; seventy-eight respondents); and completing a postsecondary course, including MOOCs (5.6 percent; nine respondents). Of the remaining respondents, twenty-three (16 percent) said they had not received any training whatsoever, and four (3 percent) were unsure. Multiple responses were possible for this question.

In addition to indicating whether they had participated in any learning activities related to SRs, respondents provided their preferred way to engage in that type of learning. There were 120 responses for this question. As shown in Figure 3 (below), learning independently through literature and online content was the first choice for 56 percent; second choice for 0.9 percent; third choice for 9 percent; and fourth choice for 34 percent. Being mentored by colleagues was first choice for 23 percent; second choice for 22 percent; third choice for 21 percent and fourth choice for 35 percent. CE workshops was the first choice for 10 percent; second choice for 28 percent; third choice for 38 percent and fourth choice for 24 percent. Post-secondary courses was the first choice for 9 percent; second choice for 48 percent; third choice for 32 percent and fourth choice for 11 percent.

Where to Go For Help

When asked who they would go to if they had questions about SR support, the majority of respondents indicated that they would approach library colleagues who have experience supporting SRs, either working within their institution (79.6 percent) or outside of their institution (34.2 percent) (see Figure 4 below). Nine respondents (5.9 percent) indicated that there was nobody they could turn to for help. Other places respondents would go to when help was needed were: online resources, such as LibGuides, videos, or Google (4.6 percent); listservs (2 percent); publications (2 percent); or recognized authorities on SR methods, such as journal authors or research methods groups (2 percent).

Supports for SR Training

Respondents were asked to indicate what types of support (if any) they had received from their administration for SR training (see Figure 5). The most common type of support was fundings for fees (e.g., registration fees) at 61.9 percent (91); time away from regular duties to participate in training at 45.6 percent (67); funding for travel at 45.6 percent (67); and resources such as software or manuals at 40.1 percent (57). Eleven respondents (7.5 percent) reported never receiving training support, and sixteen (11 percent) indicated that the question was “not applicable.” Seventeen respondents (11.6 percent) selected “Other;” the most frequent response noted in “Other,” at 5 percent, was local training provided by health librarians (seven), followed by local discussion groups at 1 percent (two).

Administration Attitude and Support

Respondents were asked a variety of questions about administration attitude and support towards SHSS librarians’ participating in SR projects. Statistical analysis done on these results show that regardless of administration attitude (discouraging, neutral, supportive) funding for SR training programs and time away from regular duties to attend training is provided for between 50-69 percent of respondents. Librarians who report a discouraging attitude towards SRs from administration also less frequently report funding travel to attend SR training and access to training resources (e.g., manuals, software). Administration with a perceived neutral or supportive attitude provides travel funding 48 percent and 40 percent of the time, and access to training resources 54 percent and 24 percent of the time, respectively. Administration that is seen as supportive or neutral of participation provides time away from regular duties to attend training (58 percent and 30 percent, respectively). Librarians who report their Administration is discouraging of participation also report being provided with time away from regular duties to participate in SR training at the highest rate (80 percent).

Learning Activities Completed and Number of Systematic Reviews Completed

Spearman’s correlation coefficient was calculated to examine the relationship between the types of learning activities respondents had completed and the number of SRs they had completed. A correlation was found between respondents who had completed continuing education (.391), mentorship (.267), and independent learning (.429) with the number of systematic reviews completed; the correlation was significant at the 0.01 level. No significant correlation was found between those who had completed post-secondary courses and the number of systematic reviews they had done in the last five years.

Topics to Learn More About

Free-text answers were provided by 115 respondents when asked about additional topics they would like to learn more about. Some indicated that they would like to receive training on elements of support related to literature searching and liaison work, namely grey literature searching and documenting the search (twenty-five), advanced searching (seventeen), peer-review of search strategies (ten), search strategy translation (seven), automation for things like text mining and search strategy development (four), and deduplication of result sets (three). Thirteen respondents wanted to know more about how to effectively liaise with students and researchers who are interested in conducting SRs, and fourteen wanted to learn about applying for funds to support librarian involvement in SRs. However, the topics most frequently mentioned related to SRs themselves. The most frequent topic noted (thirty-three) was the need for greater awareness of and training on software tools to help in the SR process–- whether they are for deduplication, screening, data extraction, appraisal, or reporting (e.g., Covidence, Rayyan, DistillerSR, MaxQDA, and OpenRefine). Twenty-five respondents indicated a need for additional training on identifying and minimising bias. Other topics of interest included choosing appropriate review types (ten), writing and registering protocols (seven), critical appraisal (six), and data extraction (four). Six respondents noted that they needed general information on the applicability and relevance of SRs outside of health, both as a tool for themselves and in discussions with researchers or students.

Additional Comments

At the end of the survey, respondents were able to provide additional free-text comments if they had anything else they wanted to add. Several respondents here noted that additional training was needed, saying, for example, “I don’t have enough regular experience with doing systematic reviews to feel confident;” “I’d like to shadow experienced SR colleagues as they work through the process;” and “I think that librarians in the social sciences would greatly benefit from having training aimed at their disciplines and also dedicated time at work or relief from other responsibilities to provide good support in this area.” Others advocated for increased mentorship or shadowing between librarians with more SR experience and those without it.

Another theme revolved around the similarities and differences between SRs in SHSS and the health sciences. Some noted concern that while shadowing health librarians might be helpful, it does add to health librarians’ workload or that health librarians might be quite separate (geographically) from their non-health librarian colleagues. Others pointed out that health librarians are not as familiar with SR methods or processes used in non-health disciplines. That being said, a background in health sciences was seen as a benefit to helping non-health researchers by others, with one respondent noting, “I started my career as a health sciences librarian and became familiar with systematic reviews at that time. Since then, I have moved to more SS/Humanities support, but am able to use the systematic review knowledge that I gained in my previous position to help current users.” Relatedly, several respondents indicated they were still unclear on how SRs applied to their work as humanities liaison librarians, with one wondering whether they would be supported in this type of work by their institutions and another noting that the nature of humanities scholarship requires that researchers do this kind of work themselves. Another respondent noted that SR methods need to be more inclusive in general, as SHSS reviews can’t always follow the same process as health SRs.

Respondents also mentioned the need for a deeper understanding among graduate students and faculty members in the SHSS of the role of SRs, as well as the work required to complete them. One respondent noted “It seems that almost all graduate students today feel like they have to author a systematic review for every MS and PhD program; simply not true and many topics don’t warrant it.” Another respondent noted that it was difficult to support researchers doing SRs when the librarians don’t have access to the relevant tools, such as Covidence or DistillerSR, or the training needed to use these tools properly.

Finally, those who had received some form of training, such as at University of Michigan’s Systematic Review Workshop, extolled the benefits of the training and their subsequent ability to then mentor their own institutional colleagues. Three respondents noted that due to the increased demand in SR requests, they had been hired specifically into institution-wide SR librarian roles.

Discussion

SR methods are being used in a variety of Science, Humanities, and Social Science disciplines and demand for library support for these projects has been increasing.26 Individual librarians’ degree of interest in supporting such projects may differ, but a basic understanding of SRs and the types of support they require is useful—after all, one never knows when that first request for SR support will be received. The information gathered through this survey indicates that librarians serving the Sciences, Humanities and Social Sciences have been taking steps to learn about various aspects of SR support, but more training opportunities are needed to increase their confidence in providing support to faculty, researchers, and students. The authors were surprised that respondents who indicated that their administration discouraged librarian involvement in SRs also reported that they were allowed time away from their regular duties to participate in SR training more frequently than respondents whose administrations were neutral or supportive of librarians providing SR support (80 percent versus 30 percent and 58 percent respectively). It is possible that these librarians have more autonomy when it comes to deciding how to spend time that is available for continuing education, but conclusions cannot be drawn from the data collected here. It is also interesting to note that even though the primary mode of support librarians receive for learning about SRs is funding and time away to participate in formal continuing education activities, this was not the first choice for most respondents. This is consistent with Premji and colleagues’ findings that the business librarians participating in their study generally preferred informal learning opportunities, such as self-directed reading and on the job mentoring by other librarians, to formalized training.27 There is no one-size-fits-all approach to learning so a combination of learning options is desirable, but administrators may consider facilitating opportunities for more peer-to-peer mentoring and shadowing for SHSS librarians to gain knowledge and experience. This approach would be consistent with examples of successful training programs documented in the literature.28 While mentoring or shadowing more experienced librarians is a good approach, administrators should be cautious not to over-burden librarians who have an abundance of their own SR work to complete. A train-the-trainer approach, such as used by the Evidence Synthesis Institute29 is likely a more sustainable strategy.

One theme that emerged from participants’ free text responses was whether and how SR methodology, tools, and standards used in the health disciplines translate to other disciplinary contexts. Health librarians have benefited greatly from the availability of accepted SR standards such as the Cochrane Handbook and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, and these types of guidance are needed in other disciplines as well. The Methodological Expectations of Campbell Collaboration Intervention Reviews (MECCIR) for Conduct and Reporting are excellent examples of this in the Social Sciences but are currently primarily relevant for interventional studies.30 Other resources, such as Gough et al.,31 provide an overview, but may be too broad in focus to provide guidance for specific disciplines. Areas like Business tend to draw upon highly cited methodological articles for methods guidance,32 but stand-alone articles cannot act as a substitute for the accountability and methodological rigour of an official guideline. The Collaboration for Environmental Evidence’s (CEE) Guidelines for Knowledge Synthesis Methodologies in Environmental Management33 is a good example of a more standard SR methodology in a specific discipline. The continued development of standards tailored to individual disciplines, and the work necessary to maintain and keep them up to date is significant but need not be done in isolation. SHSS librarians may find opportunities to partner with subject specific research centres and contribute to furthering SR guidelines in SHSS.

Another issue noted by some respondents was how best to navigate challenging conversations in an area in which they have limited experience. This can include how to advise researchers on the best type of review to take on, how to dissuade patrons when an SR is not the appropriate type of research in their situation, how to educate on correct methods and reasonable time frames for project completion, and how to advocate for proper acknowledgement of a librarian’s contribution to the work. Building confidence in negotiating these conversations may take time; in the meantime, a useful approach is for librarians to familiarize themselves with tools and resources to which patrons can be referred. For example, online tools such as Right Review34 can prompt students and researchers to carefully consider their research question and the appropriate approach to take to answer it. In planning timelines, gathering concrete evidence on the time needed,35 or making use of online time estimators36 can be helpful. In terms of acknowledgement, clearly outlined expectations around acknowledgement are beneficial. Many libraries now have acknowledgement expectations listed on their website37 or require researchers to confirm that they have read co-authorship requirements before proceeding.38 Beginning with agreed upon roles, expectations, and timelines will ensure a much smoother and stronger research partnership.

Pointing students and researchers to clearly outlined tiers of services and expectations ahead of a first meeting,39 requiring requestors to upload completed protocols prior to a consultation,40 and negotiation between faculty members and library deans or directors on how to make SR assignments reasonable for both students and librarians41 are all ways to ensure that everyone is on the same page prior to starting a new SR. It is noteworthy that this study showed that just over half of respondents (53.1 percent) reported being less knowledgeable and less confident (“not at all” or “somewhat”) about established tools and published guidelines for SRs. Training time might then be well used in learning about these tools and learning to develop clear policies.

Limitations

This study has several limitations that should be considered. The sample size of 161 responses is small, which limits the conclusions that can be drawn as well as how broadly they can be applied. Participant recruitment efforts relied on the use of listservs to recruit a convenience sample because the authors’ Research Ethics Board would not permit directly contacting individual libraries or librarians. This limits the generalizability of results on a large scale or to other institutions. The survey was also limited to librarians working at ARL or CARL institutions. However, as described above, many participants not from ARL or CARL institutions were interested in completing the survey and were excluded. Future research should include librarians that support SRs in the SHSS, working at any academic institution; this would both increase the sample size, and provide a more comprehensive picture of librarian support for SRs in these disciplines.

Another possible limitation is the categorization of a librarian’s liaison area(s) into the broad categories Humanities, Social Science, and Science for the purposes of maintaining participant anonymity and identifying broader trends. This is an imperfect process that is open to interpretation, particularly in multidisciplinary areas (e.g., Biomedical Computing) or those that may be classified differently at different institutions (e.g., Kinesiology). However, broad categorization was necessary to maintain participant anonymity and facilitate data analysis.

A final limitation of this study is that SHSS librarians who have not been asked to support SRs in their liaison areas, or who serve disciplines where this methodology is rarely used, may not have considered completing a survey on systematic reviews because they did not believe it to be relevant to them. SHSS librarians who are already supporting or thinking about supporting SRs could have been more likely to fill out the survey, which may mean they were overrepresented in the sample, giving that group more of a voice on the issue of SR support in the sciences, humanities, and social sciences.

Conclusion

Demand for and interest in librarian support of SRs in the SHSS is evident. This research has shown that overall, SHSS librarians have some knowledge of the SR research process but may have slightly lower confidence levels in providing support for research teams. To further develop both expertise with and confidence in supporting SR research, additional training is required. Participant responses suggest that there is no single ideal training format that works for everyone, so those planning training efforts should be mindful that a mix of approaches would likely work best. In terms of content, training should focus less on areas of traditional librarian expertise (e.g., advanced searching, search translation, differentiating review types, grey literature searching, and deduplication) and more on areas in which respondents indicated they needed more support (e.g., PRESS, protocol registration, screening software, different forms of bias and how to mitigate them in the literature search, and familiarity with tools and guidelines for systematic reviews). It is also clear that more education is needed on the applicability and relevance of SR methods and standards outside of health. Administrators or educators providing training should be mindful that any training, professional development, or mentorship must be tailored to the group taking part, as different disciplines have different SR needs and requirements.

Supplemental Material

All supplemental material, including survey instrument, listservs contacted, and anonymized data are available on the Open Science Framework at https://osf.io/mqxf2

Acknowledgements

This study was funded by the Canadian Association of Research Libraries’ (CARL) Research in Librarianship grant, and we are extremely grateful for their support. We are also indebted to our colleagues at the University of Manitoba for testing early forms of the survey. We acknowledge the help of Loring Chuchmach from the George & Fay Yee Centre for Healthcare Innovation (CHI) for statistical consulting services.

Contributorship

M.L., C.J.N, & J.W. contributed to the design and implementation of the research, to the analysis of the results and to the writing of the manuscript.

Notes

1. Andrea Tricco et al., “Knowledge synthesis methods for generating or refining theory: a scoping review reveals that little guidance is available,” Journal of Clinical Epidemiology 73 (2016): 36-42. https://doi.org/10.1016/j.jclinepi.2015.11.021; Cronin, Matthew A., and Elizabeth George, “The Why and How of the Integrative Review,” Organizational Research Methods, (2020). https://doi.org/10.1177/1094428120935507.

2. Anthea Sutton et al., “Meeting the Review Family: Exploring Review Types and Associated Information Retrieval Requirements,” Health Information & Libraries Journal 36, no. 3 (September 2019): 202–222. https://doi.org/10.1111/hir.12276; David Gough, James Thomas, and Sandy Oliver, “Clarifying Differences between Review Designs and Methods,” Systematic Reviews 1, no. 1 (December 2012): 28. https://doi.org/10.1186/2046-4053-1-28.

3. Michelle R. Demetres et al., “Burnout among Medical and Health Sciences Information Professionals Who Support Systematic Reviews: An Exploratory Study,” Journal of the Medical Library Association 108, no. 1 (January 2, 2020). https://doi.org/10.5195/jmla.2020.665; Sandy Campbell and Marlene Dorgan, “What to Do When Everyone Wants You to Collaborate: Managing the Demand for Library Support in Systematic Review Searching,” Journal of the Canadian Health Libraries Association 36, no. 1 (April 1, 2015): 11–19. https://doi.org/10.29173/jchla/jabsc.v36i1.24353.

4. Laura Sheble, “Macro‐level Diffusion of a Methodological Knowledge Innovation: Research Synthesis Methods, 1972–2011,” Journal of the Association for Information Science and Technology 68, no. 12 (December 2017): 2693–2708. https://doi.org/10.1002/asi.23864.

5. Sheble, “Macro‐level Diffusion,” 2693–2708.

6. William E. Savage and Anthony J. Olejniczak, “More Journal Articles and Fewer Books: Publication Practices in the Social Sciences in the 2010’s,” Edited by Joshua L. Rosenbloom. PLOS ONE 17, no. 2 (February 3, 2022): e0263410. https://doi.org/10.1371/journal.pone.0263410.

7. Karen White, “Publications Output: U.S. Trends and International Comparisons. Science & Engineering Indicators 2020. NSB-2020-6,” National Science Foundation (2019). Accessed November 14, 2022. https://ncses.nsf.gov/pubs/nsb20214.

8. Karen Chapman, “Characteristics of Systematic Reviews in the Social Sciences,” The Journal of Academic Librarianship 47, no. 5 (September 2021): 102396, https://doi.org/10.1016/j.acalib.2021.102396.

9. Zahra Premji, “Exploring the Involvement of Librarians in Systematic Reviews in Business,” In OLA Superconference. Toronto, ON (2020); Kevin Pardon, Janice Hermer, and Maribeth Slebodnik, “Investigating Systematic Reviews Outside Health Sciences,” Science & Technology Section Research Poster Session, American Library Association Annual Meeting New Orleans, LA. (June 24, 2018). https://keep.lib.asu.edu/items/373. Accessed Nov. 14, 2022.

10. Allison Campbell, Brian Taylor, Jessica Bates, and Una O’Connor-Bones, “Developing and Applying a Protocol for a Systematic Review in the Social Sciences,” New Review of Academic Librarianship 24, no. 1 (January 2, 2018): 1–22. https://doi.org/10.1080/13614533.2017.1281827; Megan Kocher and Amy Riegelman, “Systematic Reviews and Evidence Synthesis: Resources beyond the Health Sciences,” College & Research Libraries News 79, no. 5 (May 4, 2018): 248. https://doi.org/10.5860/crln.79.5.248; Diana Papaioannou et al., “Literature Searching for Social Science Systematic Reviews: Consideration of a Range of Search Techniques: Literature Searching for Social Science Systematic Reviews,” Health Information & Libraries Journal 27, no. 2 (October 11, 2009): 114–122. https://doi.org/10.1111/j.1471-1842.2009.00863.x.

11. Amy Riegelman and Megan Kocher, “For Your Enrichment: A Model for Developing and Implementing a Systematic Review Service for Disciplines Outside of the Health Sciences,” Reference & User Services Quarterly 58, no. 1 (October 10, 2018): 22–27. https://journals.ala.org/index.php/rusq/article/view/6837; Nedelina Tchangalova et al., “Working Across Disciplines and Library Units to Develop a Suite of Systematic Review Services for Researchers” Collaborative Librarianship 11, no. 4 (February 21, 2020). https://digitalcommons.du.edu/collaborativelibrarianship/vol11/iss4/6.

12. Michelle Dalton, “How Individual Consultations with a Librarian Can Support Systematic Reviews in the Social Sciences,” Journal of Information Literacy 13, no. 2 (December 3, 2019): 163. https://doi.org/10.11645/13.2.2621; Zahra Premji, “An Exploration of Business Librarian Participation in Knowledge Synthesis Reviews,” College & Research Libraries 83, no. 2 (2022). https://doi.org/10.5860/crl.83.2.314.

13. Ashlynn Kogut, Diana Ramirez, and Margaret J. Foster, “Systematic Review Training Model for Education Librarians: A Case Study,” New Review of Academic Librarianship 28, no. 2 (April 3, 2022): 205–226. https://doi.org/10.1080/13614533.2020.1784761.

14. Lê, Mê-Linh, Christine J. Neilson, and Janice Winkler. "Benchmarking librarian support of systematic reviews in the sciences, humanities, and social sciences." College & Research Libraries 85, no. 4 (2024): 606.

15. Trina Fyfe and Liz Dennett, “Building Capacity in Systematic Review Searching: A Pilot Program Using Virtual Mentoring,” Journal of the Canadian Health Libraries Association 33, no. 1 (July 21, 2014): 12. https://doi.org/10.5596/c2012-009; Paul Levay, Paul et al., “The National Institute for Health and Care Excellence Information Specialist Development Pathway: Developing the Skills, Knowledge and Confidence to Quality Assure Search Strategies,” Health Information & Libraries Journal, October 20, 2022, hir.12460. https://doi.org/10.1111/hir.12460.

16. Tchangalova et al., “Working Across Disciplines;” Fyfe and Dennett, “Building Capacity;” Barbara L. Folb et al., “Continuing Education for Systematic Reviews: A Prospective Longitudinal Assessment of a Workshop for Librarians,” Journal of the Medical Library Association 108, no. 1 (January 2, 2020). https://doi.org/10.5195/jmla.2020.492.

17. Kogut, Ramirez, and Foster, “Systematic Review;” Riegelman and Kocher, “For Your Enrichment;” Sandra McKeown and Amanda Ross-White, “Building Capacity for Librarian Support and Addressing Collaboration Challenges by Formalizing Library Systematic Review Services,” Journal of the Medical Library Association 107, no. 3 (July 1, 2019). https://doi.org/10.5195/jmla.2019.443.

18. Whitney A. Townsend et al., “A Competency Framework for Librarians Involved in Systematic Reviews,” Journal of the Medical Library Association 105, no. 3 (July 7, 2017). https://doi.org/10.5195/jmla.2017.189.

19. Kogut, Ramirez, and Foster, “Systematic Review Training.”

20. Lê, Neilson, and Winkler, “Benchmarking Librarian Support.”

21. Lê, Neilson, and Winkler, “Benchmarking Librarian Support.”

22. Lê, Neilson, and Winkler, “Benchmarking Librarian Support.”

23. Lê, Neilson, and Winkler, “Benchmarking Librarian Support.”

24. U15, “About Us.” U15 - Group of Canadian Research Universities. https://u15.ca/about-us. Accessed Nov. 14, 2022.

25. Lê, Neilson, and Winkler, “Benchmarking Librarian Support.”

26. Lê, Neilson, and Winkler, “Benchmarking Librarian Support.”

27. Premji, Splenda, and Young, “An Exploration of Business Librarian.”

28. Fyfe and Dennett, “Building Capacity.”

29. Evidence Synthesis Institute, “ About the Institute “ https://www.lib.umn.edu/about/evidence-synthesis-institute.

30. Methods Group of the Campbell Collaboration, “Methodological Expectations of Campbell Collaboration Intervention Reviews: Conduct standards,” Campbell Policies and Guidelines Series No. 3 (2019) https://doi.org/10.4073/cpg.2016.3; Methods Group of the Campbell Collaboration, “Methodological Expectations of Campbell Collaboration Intervention Reviews: Reporting standards,” Campbell Policies and Guidelines Series No. 4 (2019) https://www.campbellcollaboration.org/review/methodological-and-reporting-characteristics-of-campbell-reviews/

31. David Gough, James Thomas, and Sandy Oliver, An Introduction to Systematic Reviews, 2nd ed. (London: Sage Publications, 2017).

32. Premji, Splenda, and Young, “An Exploration of Business.”

33. Centre for Environmental Evidence, Guidelines and Standards for Evidence synthesis in Environmental Management. Version 5.1”(AS Pullin, GK Frampton, B Livoreil & G Petrokofsky, Eds). 2022. www.environmentalevidence.org/information-for-authors

34. Knowledge Translation Project, “Right Review,” https://rightreview.knowledgetranslation.net/.

35. Rohit Borah et al., “Analysis of the Time and Workers Needed to Conduct Systematic Reviews of Medical Interventions Using Data from the PROSPERO Registry,” BMJ Open 7, no. 2 (February 1, 2017): e012545. https://doi.org/10.1136/bmjopen-2016-012545; Neal R. Haddaway and Martin J. Westgate, “Predicting the Time Needed for Environmental Systematic Reviews and Systematic Maps,” Conservation Biology: The Journal of the Society for Conservation Biology 33, no. 2 (April 2019): 434–443. https://doi.org/10.1111/cobi.13231.

36. Neal R. Haddaway and Martin J. Westgate, “Predicter,” https://estech.shinyapps.io/predicter/.

37. University of Ottawa Library, “Knowledge Synthesis Services (Health Sciences and Medicine),” https://biblio.uottawa.ca/en/health-sciences-library/knowledge-synthesis-services-policy.

38. Library Services - St. Vincent’s Hospital Melbourne, “Systematic Reviews: Librarian Co-Authorship,” https://library.svhm.org.au/systematic-reviews/librarian-coauthorship.

39. Brock University Library, “Support for Systematic Reviews & Evidence Syntheses Research,” https://brocku.ca/library/systematic-reviews/.

40. McMaster University Health Sciences, “Research Consultation Request,” https://hsl-mcmaster.libanswers.com/form?queue_id=1978.

41. Campbell and Dorgan, “What to Do When Everyone Wants You to Collaborate.”