Documenting the Conversation: A Systematic Review of Library Discovery Layers

This article describes the results of a systematic review of peer-reviewed, published research articles about discovery layers, user-friendly interfaces or systems that provide single-search box access to library content. Focusing on articles in LISTA published 2009–2013, a set of 80 articles was coded for community of users, journal type, research method, and results. The findings suggest sustained and potentially a converging professional interest in discovery layers over time. They also demonstrate what has not been studied, finding very little research about how discovery layers affect public libraries or children.

Be ready to watch your organizational structure change. We found that our traditional work silos are collapsing. Lines are blurring among public services, technical services, and library systems librarians. We are considering a major reorganization along work group lines and flattening the organization.

—Dee Baldwin, Michael Kucsak, and Alice Eng1


Discovery layers, an overarching term used to describe user-friendly discovery interfaces as well as full-blown discovery systems, have the potential to reshape library work and use.2 Discovery interfaces, also known as next-generation catalogs, preceded discovery systems, providing user-friendly access to library holdings through single-search box interfaces and faceted navigation.3 Discovery interfaces can provide “visually rich displays, virtual shelf browsing, a spell-check function, auto-completion options, and search-term suggestions, as well as social features that encourage patron participation, such as tagging, ranking, and reviews.”4 Taking the idea of discovery one step further, discovery systems are highly interconnected systems that sit on top of and seamlessly bring together the results from disparate library databases such as the library catalog, electronic article databases, and e-book packages.5 Discovery systems were developed in 2009, essentially uniting the user-friendliness of discovery interfaces with the functionality of multidatabase search offered through federated search, simplifying search of library materials through the addition of a central index.6

The current research project examines how library professionals are studying discovery (that is to say, especially the dawn of discovery systems, but also relevant discovery interfaces such as BiblioCommons) in the scholarly literature, and how they are communicating about them. In so doing, we explore the notion of discovery layers as change agents with the potential to unite librarians (and departments) from disparate parts of the library as their work converges through the evidence provided in the library literature. We examine notions of librarywide unity using the concept of convergence.

What does convergence mean? For the purposes of this paper, we explore how library department activities and interests (or intralibrary convergence) may be merging. Extra-institutional convergence as a result of technology, especially of libraries, archives, and museums, has been widely discussed7 and is tangentially related, but out of the scope of this research project. The current study, instead, focuses on a recent technological advancement, discovery, and the conversation surrounding its now ubiquitous use in libraries. Convergence in this context is more closely aligned with research about how communication technologies have changed organizations and their internal structures.

Librarians work within an information ecosystem that is increasingly technological in structure, serving to unite the library’s departments and its employees. Christopher Barth explains: “Organizations that have ‘completed’ the merged process are called ‘merged’ library and technology organizations…technology has become increasingly embedded and integrated, and information support will become more and more technology-based.”8 Jennifer Trant focuses on how the digital collections that memory institutions like libraries create and maintain (or curate) served to merge both institutions and people working in various departments of the individual institutions.9 Library technology thus provides a means and an impetus for cooperation between personnel in otherwise unrelated, highly specialized departments (such as circulation, cataloging, or serials departments, and the like) so that the library can better meet its users’ needs. In this view of convergence, technological (or backroom) concerns become front-of-house or public services concerns, and vice versa. An example of this might be new branches of librarianship that combine technology support with customer service. Librarians with job titles such as User Experience Librarians are concerned with all aspects (technological and otherwise) of the library users’ experience, and Emerging Technologies Librarians focus on integrating new web-based technologies into the library. Because discovery layers change the search experience for librarians and library users, the technology itself becomes an object of inquiry, knitting together departmental concerns and requiring, in some cases, a unified library response to be communicated in the generalist literature.

The current study looks specifically at a set of published, peer-reviewed research papers about discovery layers with the intention of exploring the conversations in this area that has the potential to concern all librarians. The Library, Information Science & Technology Abstracts (LISTA) database was queried using search terms relating to discovery layers (see appendix B for a complete list of search terms). All published research articles that met established criteria for inclusion in the study were analyzed according to:

  • The type of user community being studied (for instance, users in academic, public, and other environments)
  • The kind of journal in which the article was being published or the journal’s primary audience (such as academic librarianship, management, medical, cataloging, reference)
  • The method (like usability testing, survey, log analysis)
  • The nature of the results and discussion (for example, selection/implementation, indexing and metadata, theory, library instruction, workflow changes, branding/marketing).

The two researchers coded together the journals for their type. They separately analyzed and coded the set of 80 research article that met the established criteria, then reexamined the papers together to form a consensus regarding categorizations. The codes were stored in Excel spreadsheets. Cross-tabulated analyses of the codes reveal publication trends and trends between the variables as research on the technologies matured.

Research Questions

Two research questions guided the current study:

RQ1: What is being communicated about library discovery in the scholarly literature, and where is it being published?

RQ2: In what way, if at all, has departmental convergence in library operations (as defined above) been evident in the evolution of the emerging scholarly literature on discovery layers over time?


H1: We hypothesize that, because of their impact on a number of library services, discovery layers and especially discovery systems will become increasingly part of the broader professional conversation over time.

Scholarly communication is a good indicator of institutional or professional concern, and the conversations about any new technology should evolve as the technology is adopted. Librarianship as a whole is interested in research about new technologies that have the potential to improve services or efficiency, and library departments will begin to explore them as they become available in a way that aligns with their workflows.

H2: We hypothesize, based on a cursory scan of the recent discovery layers literature, that most research about discovery layers will focus on academic libraries.

Our H2 is supported by the literature. In 1993, Connie Van Fleet and Joan Durrance found that most LIS research is done in academic settings, primarily because public librarians have little time to devote to research, and few incentives to publish it.10 The predominance of peer-reviewed research articles about academic library concerns is problematic; if librarianship (and library product development) is to be informed by evidence, public library patrons and other users outside academia should be studied as well. The wider population (including, for instance, older adults and children) likely has different needs from academic library patrons. Anecdotally, a number of public libraries were moving to adopt discovery layers during the period covered by this review. The primary public library discovery system product is a social discovery interface, (BiblioCommons), which is different from discovery systems used in academic libraries.11

Review of the Literature

We begin with a discussion of scholarly communications and practice because it helps frame our research method: the systematic review. Most systematic reviews are limited to peer-reviewed articles, which was a limitation to which we adhered. Next, we discuss literature on discovery layers, then literature about library operations and convergence. Finally, we return to the systematic review as a method, which segues to the method section. The literature review, thus, provides an overview of literature on research, communication, discovery layers, library operations, and a method of study, the systematic review.

Scholarly Communications and Practice

There are various motivations for writing about research, but we will start with the premise that research begins with a sense of curiosity.12 The result of the research, though, might take a variety of forms. It might be published as a research article in a scholarly journal, as a white paper, presented at a conference, or circulated internally. The motivation for people who go to the trouble to publish in scholarly journals is likely different from people who simply want to discuss what they found with colleagues, or to make a managerial decision.13 Scholarly journals require the author to conceptualize and frame the research appropriately, whether it be a review of the literature or a more qualitative or quantitative method, then to work through an often arduous peer-review process. People who publish in peer-reviewed journals probably have an external motivation, such as tenure requirements.14

The peer-review process, especially in high-impact journals, is arduous, but it also helps get the research out to a wider audience.15 The audience that the research will find, though, is determined, to some degree, by where it is published or presented. For example, publication requirements for various periodicals also differ; in some journals there is no requirement for a review of the literature, especially if the research is done within one’s own library; but, in other journals, such an approach is unacceptable. Journals have aims and scope, which help the editors identify appropriate reviewers and which also help the editor anticipate the interests of the primary readers.

Within any profession, readership will be defined by the scope of the journal in question. In libraries, we identify two main ways for journals to address their readers: either based on the kind of library in which a reader works or where the research takes place, or based on the kind of work that is done—for instance, by department. In librarianship, each of the specialized subfields have journals dedicated to special professional concerns (such as administration, cataloging, or reference). There is also some division among readership according to the type of library (such as academic, public, law, or medicine). In that vein, readers of College & Research Libraries will likely be different from readers of Public Libraries. While some articles will be shared and read widely, much of the knowledge shared among librarians is written for, and read by, particular groups, such as reference or user services, technical services, or systems or in specific kinds of library environments.16

While there are many academic journals devoted to library concerns, we do know that there is much less being published about public libraries. Julie Hersberger and Christopher Demas studied public librarians’ publication trends in peer-reviewed journals, finding that only 7 percent of peer-reviewed articles published between 1996 and 2000 were oriented toward public libraries.17 While public librarians might benefit from some findings in academic-oriented journals, most is probably of little interest. Van Fleet and Durrance found that public librarians want to read research that is applicable to their own situations or context.18 Regardless of why it occurs, the publication bias does present a problem for evidence-based decision making in public libraries.

While scholarly communication is often thought of in terms of silos (like public services, technical services, or systems as described above), the subject of this study, discovery layers, actually and ironically challenges the siloed nature of information seeking among librarians and other library-based practitioners. In theory, the discovery system might help librarians to search outside their silo to gain a global overview of the problems and opportunities. Furthermore, various self-archiving options (such as sharing articles on their own websites or using social media such as ResearchGate or and institutional repositories make gray literature and peer-reviewed papers equally discoverable for people seeking information through Google Scholar.19 There is still a question of quality control, though, offered by the peer-review process that cannot be overlooked. That is a motivation for librarians to use, teach about, and improve discovery systems as part of the formal, scholarly conversation.

Why is peer-reviewed literature so important? Stevan Harnad, who has since published numerous manuscripts that promote open access, wrote in 1999: “Peer review is a quality-control and certification (QC/C) filter necessitated by the vast scale of learned research today. Without it, no one would know where to start reading in the welter of new work reported every day, nor what was worth reading, and believing, and trying to build one’s own further research upon.”20 The peer-review process still gives some assurance of quality control. Therefore, there is a very good reason to publish one’s research in peer-reviewed journals: it is a vetting process that offers some assurance of quality to the reader.

Discovery Layers in Libraries: An Overview

Marshall Breeding has outlined the necessary terminology for describing discovery interfaces and discovery systems, effectively giving librarians a lingua franca for their discussions. According to Breeding, discovery interfaces “emerged to provide a more modern replacement to online public access catalog (OPAC) modules of integrated library systems (ILS).”21 Discovery interfaces function effectively as user-friendly next-generation catalog interfaces and include products such as BiblioCommons’s BiblioCore (commonly referred to by the company name BiblioCommons), Ex Libris’s Primo, and ProQuest’s AquaBrowser. Open-source discovery interfaces include VuFind and Blacklight. Likewise, Breeding’s definition of discovery systems is the one that we adopt in the current research project: a product with a central index uniting disparate library databases, using a single search box, providing relevancy ranking and faceted narrowing of results.22 Examples of discovery systems include Ex Libris’s Primo and Primo Central, EBSCO Discovery Service from EBSCO Information Services, and ProQuest’s Summon.23

Discovery in libraries has undergone rapid change since the advent of the first federated systems in the early 2000s when library databases were first queried simultaneously in real time, though the process was slow and was fraught overall. Discovery systems improved on the notion of federated search and discovery layers by creating a central index from all relevant databases and providing results, generally with relevancy ranking, and faceted navigation for narrowing results.24 Discovery interfaces like BiblioCommons as well as the more sophisticated discovery systems allow users to query library-supplied MARC data in a way that is more intuitive and less cumbersome, with discovery systems including results from other siloed electronic resources in the results. In all the discovery platforms available, visualizations and facets allow users to control their results as never before. Single-search box discovery layers also return results in ways that can be easily understood and evaluated by users.25

As discovery layers gained traction in libraries and turned to a professional interest in the nascent discovery systems, librarians and researchers began studying the various elements involved in selecting, deploying, evaluating, and adjusting to these new systems. Because of their complexity, discovery layers became a focus of study for systems librarians who were implementing them, technical services librarians who were creating and overseeing the library data loaded into them, and user-services librarians tasked with making sense of the results and teaching the interfaces. Beyond the traditional areas of specialization, though, librarians had to come together to explore the data and metadata in these systems. This is because discovery system content supply chains were prone to providing resources and metadata of inconsistent quality (that is to say, incomplete, with metadata not properly synchronized to library holdings data, and in multiple formats26). The discovery layers themselves needed to be evaluated, and the usability of the systems explored and documented. Reviews of the literature from 201227 and 201328 show that not only were different kinds of problems being researched and discussed, but that different communities of librarians with different professional perspectives were contributing to the conversation. One thing that is unclear from the literature, however, is if there are patterns demonstrating how these different professional communities have been discussing discovery systems and the users for whom these research projects are being undertaken.

Library Operations and Convergence

Discovery layers are complex technological systems that offer users new ways to interact with search results and potentially with each other; they are therefore a kind of communication device. There is a rich body of literature about how communication tools lead to convergence. Iansiti & Levien explain: “In networked industrial environments like the computer industry, the performance of any organization is driven in large part by the characteristics and structure of the network, which influence the combined behavior of its many partners, competitors and customers. This makes an enormous difference in both strategy and operations.”29 Organizations that successfully implement complex technologies coordinate their efforts across departments. As was explained above, there are many examples of how digital technologies lead to convergence.

However, libraries still require extensive specialization. This need manifests itself in the different task-based departments, such as public or user services, technical services, systems, or digital services. Although these departments and others—including access services and serials—may be hierarchically situated depending on a library’s size, mission, or even budget, certain general tasks need to be accomplished in libraries. Because these tasks may be incredibly specialized, it makes sense to employ librarians with the necessary skills for their increasingly focused tasks.30

In an effort to capitalize on strengths in different areas of the library, Ruth Makinen recommended to reference services that they staff the reference desk with members of technical services. 31 As recently as 2014, Teressa Keenan reminded reference services staff of the importance of understanding and even of being cross-trained in work done in cataloging.32 The concept of “cross-training” illustrates that a library is a system in which each part of the library benefits from all members understanding the whole. Margaret Bing summed up the situation best, perhaps, when she wrote: “This growing dichotomy of public service vs. technical service is a very disturbing element pervading libraries. The truth is that there is no division between reference and cataloging in terms of public service. You need both to keep your library functioning as a library.”33

According to Robert Steuart and Barbara Moran, the push toward convergence or for a holistic understanding of the library as a single unit (or system) satisfying a mission and providing information to a community of users has been studied since computers first came into libraries. They explain that technology has been changing staff roles for years, though the inertia inherent in library bureaucracies does tend to decelerate change.34 More specifically relevant to our question, Ping Fu and Moira Fitzgerald studied the effects of ILSs on library systems and technical services staffing models, and especially in how systems work blends workflows between technical and user services.35

Despite evidence of convergence, we believe that librarians continue to focus their research initiatives and publishing on work that will be understood by and will be of direct benefit 1) to other librarians with similar job duties or workflows, or 2) to a much broader, generalist readership in their kind of library (academic, public, school, or other). Scholarly communications, and especially the changes in libraries caused by shifting job duties and descriptions around digital initiatives, support this assertion.36 Although librarians may appreciate the work of others in the library and may be working toward convergence, there is still a division of interests and approaches to scholarly topics such as the adoption of discovery technologies in libraries. We propose that an examination of publications about discovery layers will show how the technology has enabled, changed, or broken down professional library departmental siloes.

Systematic Review as Method

The literature of library and information science is replete with primary studies conducted in individual libraries, often about the same phenomenon. There are distinctions between the studies, though; each library presents a unique set of circumstances that will inevitably impact the results of the study. The goal of a systematic review is to gather all relevant studies of a phenomenon and then evaluate, analyze, interpret, and present the methods and results of the review.37 This aggregation might find statistical strength of, for instance, an intervention or procedure.

Harris Cooper and Larry V. Hedges provide an overview of the development of the systematic review and the more statically focused meta-analysis, and the most important developments are summarized below.38 Cooper and Hedges write that the overarching research synthesis was defined as a scientific method in 1971 when Kenneth Feldman defined a process for the systematic review as one of “sampling topics and studies, developing a scheme for indexing and coding material, integrating the studies, and writing the report.”39 As the systematic review developed as a method, it was found to be useful in eliminating chance and contradictions in studies as well as adding objectivity. In 1976 the term “meta-analysis” was coined; it goes a step beyond research synthesis by combining the data from the studies in systematic review. Behavioral scientists adopted this method to compare both quantitative and qualitative data in the late 1970s. In the early 1980s, standard procedures for research synthesis and meta-analysis were defined in widely used research method books.

Systematic reviews have been used across many scientific domains in both social/psychological research and medical research. The focus of a synthesis of existing literature is defined by its purpose or research question, which might be about methods, theories, or perspective, for instance. Systematic reviews have been used in LIS, though not extensively. Jianhua Xu, Qi Kang, and Zhiqiang Song find, however, that their use in LIS is on the rise; prior to 2003, they found no examples of systematic reviews in LIS literature. Between 2003 and 2009, they find 20 systematic reviews in English-language publications in LIS, and from 2010 to 2013, there were already 10.40

Systematic reviews can be applied to a number of different LIS questions. For example, Julie Hersberger and Christopher Demas used the method to uncover frequency of publication, author profiles, subject, and methodology of research on public libraries.41 Lynn Sillipigni Connaway and Timothy J. Dickey used this method to identify overlapping and contradictory findings from twelve user information–seeking studies.42 Xu, Kang, and Song reiterate the importance of the method for librarians seeking “to base their decision-making on the best available research, [as] systematic reviews provide a useful method to gather all relevant data on a specific topic and to synthesize the results in a manageable report, which is useful for librarians who seek evidence for their practices.”43 As Connaway and Dickey say, their systematic review is not a “definitive work” on an issue, but rather a “synthesized document to make it easier for information professionals to better understand” the phenomenon under study.44 In short, this study is not a meta-analysis; it is a systematic review, which is appropriate because we are not looking at medical interventions or procedures, but for patterns. On the other hand, it is different from a literature review because of the systematic nature of literature selection and coding.


The goal of systematic reviews of literature is to uncover “the studies of a phenomenon, and then evaluate, analyze, interpret, and present the methods and results of the review.”45 This review serves to examine the methods, theories, and perspectives of researchers to provide a synthesis of communication about the phenomenon.

The stages outlined in Cooper’s revised method provides the basis for this project:46

  1. Define the Problem47

    This research sought to discover the nature of communication (specifically published, scholarly studies) about discovery layers:

    • Who is talking about them, and who are the authors speaking to? What professional communities and boundaries are discernible in the literature?
    • How did communication change during the earliest years of the technology’s introduction? Discovery layers are an evolutionary development, with 2009, the year of the release of commercial discovery systems, as a watershed year. Many libraries began to adopt the systems or to compare them to discovery interfaces, and they became a subject of wide inquiry. The years of this study cover 2009 to 2013: the first five years of widespread adoption of discovery systems.
    • What methods and results are being reported? By investigating how librarians were researching the systems, we can trace the evolutionary development, as well as the impact on, and acceptance in, the field across different departments in the library (such as technical services, systems, and user services).
  2. Collect the Research Evidence48

    The researchers searched the LISTA database for peer-reviewed articles using a set of search terms (see appendix B) that adequately allowed for research articles addressing the hypotheses to be collected; names of individual discovery platforms such as BiblioCommons were included in the search terms. Search operators such as quotation marks around phrases and the asterisks along with Boolean operators were used to capture the most comprehensive and accurate group of articles. LISTA returns results in which the search terms appear in the full text of the article.49

    Slightly more than 350 articles were pulled from the LISTA database using the search terms that appear in appendix B. Articles were then reviewed individually for possible inclusion based on the following criteria that lend themselves well to this kind of timely systematic review:

    • Published between 2009 and 2013
    • Explicitly about discovery layers (in other words, discovery systems, but also discovery interfaces)
    • Peer-reviewed research articles
      • Include a review of the literature and contain at least five references
      • Span a length of more than five standard pages

Not all publications in scholarly journals are part of the scholarly conversation that lend themselves to this kind of systematic review; as a result, columns, reports, and keynotes were excluded from the study because they did not meet the criteria established above. Ultimately, 80 articles that met the above criteria were retained for analysis.

  1. Evaluate the Correspondence between Methods and Implementation of Studies and the Desired Synthesis Inferences50

Preliminary categories for each of the variables were established (community of users, journal type, research method, and results/discussion). The two researchers read each article and coded each article separately, periodically meeting to discuss whether codes should be added or whether code definitions needed clarification. Several codes were added during the coding process. The researchers then reread the articles and used an iterative process to confirm agreement on all codes.

The information about the articles was coded using the following categories:

  1. About the article:
    • The user community or type of library studied in the individual article (in cases where the article was theoretical or applied to mixed audiences, that was coded) ;
    • The research method (such as case study, survey, usability, theoretical, and other methods);
    • The nature of the results (such as reporting/discussing results pertaining to implementation, indexing, instruction, and so on).
  2. About the journal in which the article was published:
    • The kind or type of journal (such as technology-oriented, reference, cataloging, or other), taking into consideration the kinds of topics generally addressed, the composition of its editorial board, and other considerations.

For both type of journal and user community, only one code was recorded per article. For methods and results, as many codes as applicable were recorded.51

  1. Analyze (Integrate the Evidence from Individual Studies)52

    Data codes were analyzed using Microsoft Excel. Each of the coded variables was analyzed according to the date of publication of the article to assess trends over time. Additional analysis sought relationships between user communities, journal types, research methods, and the nature of the results.

  2. Interpret the Cumulative Evidence53

    The analysis sought to explore trends in the scholarly conversation over time: what was being studied, by whom, and when certain types of articles were being published.

  3. Present the Synthesis Methods and Results54

    In the current article, the method is followed by the results and an explanation of the limitations of the study and a discussion. The final section is composed of conclusions and recommendations.


What Types of Communities Are Being Studied?

This systematic review found that users or patrons of academic libraries or special libraries associated with higher education (such as medical librarians or law librarians in universities) are the primary communities being studied in the discussion of discovery. Additionally, we know little, from the peer-reviewed literature, about how discovery layers are affecting public libraries and their patrons. Only one of the 80 articles, published in 2012, addressed only public libraries and patrons. None of the articles addressed children as users in any capacity. Table 1 shows the breakdown of community types of peer-reviewed research articles on discovery layers, by year. “Other or Mixed” means that the article either did not specify for a community, was for multiple groups (academic and public), or did not employ a methodology that focused on a specific library environment. Thus, H2 (that most research will be about the concerns of academic librarians) was found to be true.

Table 1: Community Type Being Studied, by Year

Where Is It Being Published?

Given the discussion of scholarly communication above, we evaluated the journals in which articles were being published to understand which groups had a scholarly interest in discovery and also as a way of inferring possible readership. The journals were evaluated separately for the kind of research they published. The types of journals in which research was published evolved over the time period of the study. Table 2 shows the change over time as well as the total number of articles in each kind of journal, and table 3 shows how the journals were coded. Although readership is not restricted for different kinds of journals, journals demonstrate what is of interest to a targeted reader or kind of professional.

Some of the more interesting findings regarding publishing trends revealed in table 2 are:

  • Systems librarians/technologists showed the most sustained interest in discovery.
  • In 2009, six of the published articles (60%) were published in systems librarians’ journals.
  • In 2012, 39 articles were published that met our criteria—almost half of all the articles published for the time period under study. Of those, 20 (51%) were published in academic library journals, journals that are by definition somewhat generalist in nature due to the broad range of professional work in academic libraries; however, another seven (18%) were published in journals specifically for general professional interest, together accounting for 69 percent of all scholarship on discovery that year.
  • There were no articles published in cataloging or technical services journals until 2010, that article that was a theoretically oriented case study, and no articles in reference or public services journals until 2011; this can be compared to 60 percent of articles in 2009 appearing in the systems or technologists journals.

Table 2: Journal Type by Year (N=80)

Table 3: Number of Articles Published in LIS Journals and Their Type of Journal

A total of thirty-three journals published the 80 articles on the topic of discovery layers during the time period under study. Most frequently, journals published one article that met our criteria (63%; n=21). These journals ran the gamut, focusing on articles for generalists, academic libraries, catalogers, information literacy librarians, reference and public services librarians, management, and other kinds of specialized librarianship. Only three journals published six or more articles on discovery layers from 2009 to 2013: College & Undergraduate Libraries (n=16), Library Hi Tech (n=7), and Journal of Web Librarianship (n=6); these journals were coded as Academic, Systems/Technologists, and Systems/Technologists respectively. Conspicuously missing are journals relating to public librarianship and also to children’s librarianship or school libraries.

How Is the Phenomenon Being Studied (Methods)?

Methods employed in the articles were recorded, with as many methods being recorded per article as applicable. Case studies of a single library environment, or methods that called themselves case studies, were popular throughout the entire period of study (65% of articles overall), frequently being combined with or supplemented by other methods such as usability studies or surveys. At the same time, we can also see a shift in research methods over the period of study. In 2009, the majority of studies were case studies; many were completed with usability testing and demonstrate that technologists were struggling to find a suitable implementation of the systems in their libraries. In 2010 there was a larger percentage of theoretical articles, as researchers were grappling with the new, implemented systems and their place in librarianship. Log analyses and surveys appeared in the literature that same year (2010), but reviews of the literature did not appear until 2012. In 2012 the novelty of studying discovery layers, especially discovery systems, seemed to peak, bringing with it an exceptional quantity and variety of research as early adopters reflected on their implementation experiences. Often, these reflections were in the form of a case study or a case study supplemented by another method, and they looked to the future as well. See table 4 for a tabular representation of the frequency of the methods used in the 80 articles under study. The emphasis on usability is an indication of blending of interests across various departments.

Table 4: Methods Used, By Date (N=80)*

Although table 4 clearly shows the change in the methods used, it does not demonstrate visually the percentage each represents over the course of a given year. Figure 1 gives a graphical overview of the relative proportions of each method’s use in a year.

Figure 1

Research Methods by Year

What Are the Results of the Studies?

The results reported in the studies addressed a number of concerns that are prevalent throughout the library; as with the methods codes, as many results codes as necessary were applied, with many articles being coded for multiple results. Relevant discussions were also coded. Results and/or discussions most frequently included substantive sections involving considerations for use and users (51%); implementation of the discovery layer (46%) was the second most common topic addressed. Other technical topics included system evaluation (23%) or describing issues with metadata and indexing (18%).

Specifically to addresses H1, the question of workflow was investigated more closely. Workflow was addressed in 15 percent (n=12) of articles. Most of the articles discussing workflow focused on how implementation affected a single department, such as cataloging, acquisitions, reference, or inter-library loan (ILL). However, two articles discussed cross-departmental change: one focused on the newly introduced, common web-based interface that is used across departments, which affected how reports are run and holds are processed.55 Another said that it changed their workflow because the system actually was an impetus to rethink the library as a service instead of collections.56

Another prevalent topic of interest was information literacy and instruction (14%). Instructional librarians rely on particular methods to teach users how to use databases and the library catalog. Exchanging ideas with colleagues in the scholarly environment allows for communication with other instruction librarians and also systems or technical services librarians who support the functioning of the systems and how to enable and display the content. Discovery layers change the way that users (especially undergraduate students) access library information, and understanding how that use takes place benefits all librarians.

Finally, a surprising finding was that discovery layers seem to impact the very identity of the library. The implementation of a discovery layer often prompted or coincided with a new web presence, and library marketing, another topic that could be said to affect all librarians in some way, was discussed in 8 percent of the articles.

Table 5 summarizes the results presented in the articles analyzed.

Table 5: Summary of Results Reported (N=80)*

Table 6 demonstrates evolution in the results/discussion of the articles over time, with the interest moving from the specifics of selection and implementation, a topic of interest to technologists and technically oriented librarians, to use and usability, a topic of interest to a broad group of librarians, over the course of the study. In that same vein, articles about indexing and metadata concerns were highest in 2009 (during implementation), and marketing and instruction or information literacy were highest in 2012 and 2013 respectively, when public services librarians were grappling with how the systems affected their work. Again, these results seem to imply that the focus of research on discovery layers has shifted over time from the very specific to the more general, with the topic of discovery becoming progressively more interesting to an increasingly broader group of information professionals working in academic libraries over time in ways not limited by their own kind of professional work.

Table 6: Results Explored in the Articles under Analysis (N=80). Note: articles could contain multiple types of results

H1 was that discovery layers and systems will become part of the broader professional conversation over time, demonstrating a converging of professional interests. We were able to demonstrate that this is at least anecdotally true, though not statistically true, given the limitations outlined below. Usability studies accounted for 51 percent of the publications, and usability is a broad concern that affects multiple silos of the library. Some of the articles focusing on workflow stated that the systems caused convergence. Changes discussed in user instruction happened because the new technology was introduced, and participants found that public services librarians needed to delve more deeply into metadata and indexing problems. In other words, the current study provides fodder for discussion, with some of the articles broaching the topic, even though there is no direct statistical proof of convergence in light of the method.


The limitations of this study are primarily associated with its method. The method itself introduces a certain amount of bias because it only investigates research that was designed for, and which underwent, peer review and was reported in the scholarly literature. Because of this, other conversations (including those taking place at conferences, within institutions, on listservs, in webinars, and so forth) are not included in the analysis, and those might be the very studies that are informing public librarians. Those might offer significantly different views on the systems from a variety of complementary viewpoints.

An additional limitation associated with the method involves the nature of the results. Although the data collected can be used to paint a tableau of the scholarly conversation, the data itself is not statistical in nature; instead, it is descriptive. Statistical tests, including tests for correlations, cannot be done using the data collected in this study because the research questions were not statistical. The goal was not, for instance, to aggregate results from multiple studies on the same phenomena; it was an exploratory study of communication about a central phenomenon that has the potential for wide impact across the library and its users. Statistical limitations do not preclude observations from being made or assertions from being explored, but they do limit their ability to be tested. Further research is therefore recommended using a different method, such as a document analysis of organizations or job descriptions.

Finally, in capturing the nature of the scholarly conversation surrounding discovery layers in libraries, this project did not attempt to carry out any kind of sentiment analysis on the way that discovery layers were characterized in the articles. In our analysis, therefore, we do not make judgments or assessments about the overall effectiveness of discovery layers. Instead, we approach this research from the point of view of examining the conversation around a technology that has quickly become ubiquitous in academic libraries, not the strengths and weaknesses of products themselves. The limitations mentioned here could be addressed through future research.


Discovery layers have been of wide interest in libraries, and their appearance in the research literature has been consistent over the time period of the study. A very decided peak took place in 2012 with the publication of 39 articles, almost as many as were published in all of the other years combined.

This research sought to investigate what was being communicated in the scholarly LIS literature on the topic of discovery, and where. To do this, 80 research articles were evaluated based on the type of patron community being studied, the journal in which the article was published, the methods employed, and the results/discussion. The interest in discovery in academic libraries was abundantly evident, with 88 percent focusing specifically on an academic library environment. Additionally, many researchers presented the results of research carried out at their own (academic) institutions, specifically in the form of case studies. Fifty-two articles (65%) were case studies of some kind. Articles published in generalist outlets for nonspecialist readerships accounted for 20 percent of the published articles (n=16) overall; but, when also considering the generalist nature of academic journals (29%; n=23) and management journals (8%; n=6), a full 57 percent of articles (n=45) were for generalist audiences.

Additionally, this project sought to investigate the notion of departmental convergence in libraries through the observation of the publishing patterns. With the scholarly conversation about discovery layers so firmly rooted in the academic and generalist literature during a period of maturity, we infer that librarians from a variety of service departments are preferring to write for a broad range of librarians, including those outside their immediate function-based areas, or to focus on the technology itself. Because of this tendency over time to broaden potential readership beyond those in one’s immediate area and to publish in nonspecific or generalist journals, we infer a convergence of task-based library department members interested in communicating about discovery, specifically in the academic library.

Are (Academic) Library Departments Actually Converging?

A broad interest in discovery systems is noted in the journals for the different kinds of librarianship: systems, public services, technical services, and others. Having a single topic of such overwhelming interest to disparate specializations, librarianship bridges the different areas to provide a common topic for scholarly discussion, in a way that only a technology-related topic can. A holistic view of library operations, as described by Steuart and Moran (2007), is necessary when facing a new and potentially disruptive technology.57 Unsurprisingly, research in discovery was first led by those closest to the technology. The largest percentage of articles published toward the beginning of the period of study appeared in journals for technologists and had technology-related topics of study. Generalists’ and academic generalists’ interest peaked in 2012, with the peak in the research on discovery. The following year, management became more concerned with an even higher-level perspective on discovery layers, with 20 percent of articles in 2013 appearing in management journals. Specialized areas maintained a sustained focus on discovery throughout the time of study, demonstrating a parallel yet focused interest in the same topic among these disparate groups.

With time, librarians are likewise converging as a profession in their discussions around discovery layers, given the building interest in generalist journals—journals that potentially provide an even more direct bridge between different service areas in libraries. This assessment supports the first hypothesis about the increasingly broad interest in discovery layers over time. Further evidence of this convergence of professional interest is the appearance of six literature reviews synthesizing prior research, the first of which was published in a journal for technologists. Three were in journals for academic librarians, one in a journal for technical services/cataloging, and one in the management literature.

Who Is Being Left Out?

Based on the analysis, however, the above discussion principally describes the situation in academic libraries, supporting the second hypothesis that stated research would focus on the academic environment. No research articles appeared in journals focusing on public libraries or on children’s librarianship/school library media librarianship. This is fairly shocking, given the importance of discovery to all kinds of library patrons. Additionally, only 1 percent of the articles identified in this study were written with a stated focus uniquely on public libraries; this is lower than the 7 percent found by Hersberger and Demas in 2001.58 Anecdotally, discovery is a topic covered in non–peer-reviewed venues such as the ones mentioned earlier: conferences without proceedings, blogs, unreviewed papers,and so on. When that research is not reshaped to pass peer review, it nonetheless remains invisible beyond the immediate venue in which it is shared as a source of knowledge for the field.

Academic librarians have motivation to publish,59 and investigating systems relevant to their daily operations and to their professional work is a logical way to approach any publishing requirements. Since the work environments of public and children’s librarians do not historically provide the kinds of external motivations necessary to carry out and publish research, we infer that their efforts at studying their environment as part of their professional work are restricted to those with whom they verbally share their results, such as internal committee members (technology, information literacy, and the like), board members (to secure funding, perhaps), conference attendees, or any readers of professional magazines in which they publish. Regardless of the reason, the exclusion of their research from the research literature is problematic.

One suggestion for increasing the number of research articles in areas beyond academic libraries is to task academic librarians or library and information science (LIS) professors and researchers with carrying out the research. Although the single research study focusing uniquely on public libraries was a collaboration between a public librarian and an LIS professor, this model clearly is not the norm for exploring practical questions relating to daily operations in public libraries or in libraries for children, such as school libraries.

Recommendations and Conclusion

Discovery layers have become nearly ubiquitous in academic libraries. Anecdotally, some librarians seem to love them; even more seem to consider them an inadequate but necessary evil to help patrons find the many different kinds of materials housed across disparate places in the library. No matter the perspective, the scholarly conversation around discovery in libraries has been a lively topic in which a number of librarians have engaged during the time these systems have gained widespread adoption.

This paper has investigated the LIS research literature on discovery layers when they were emerging in libraries. Based on the publication patterns revealed through this systematic review, we conclude that research about discovery layers is of great interest across library functions and represents a kind of professional convergence among librarians from disparate service departments in the (academic) library. Most of the research carried out focuses on work directly related to single institutions, giving outsiders insight into a given library’s workings, findings, and decisions.

Moving forward, as libraries reorganize, this across-the-board focus on technology has the potential to impact how libraries are organized. As technology becomes increasingly pervasive in the library ecosystem, the focus on how technology is being used has the potential to flatten the library structure. Specializations are increasingly necessary as technology advances; but, as technology questions increasingly become the questions that all librarians need to explore, as Ours points out, organizational structures may need to adapt accordingly.60 If all librarians need to be systems and, for example, user-experience experts in some way, there is potential for libraries themselves to reorganize to accommodate these new approaches.

However, given the focus on academic settings, not all library types or users are represented in the literature, which points to a need for further research. This study demonstrates that the scholarly communications surrounding discovery systems or layers are evolving, moving from technologists to a merging or converging of general responsibilities and concerns, but with a pronounced focus on academic institutions. This research also demonstrates that there is little research published uniquely about public library users; we found that none is about children as users. Children are important users of library resources, and this understudied population represents both a shortcoming in knowledge of our users and an opportunity for librarians and library researchers to gather important data to shape the future of discovery. As library operations continue to converge in the age of technology, a targeted, organized approach is needed for the investigation of discovery in a variety of environments for various user populations to support a need for personalized searching or a ubiquitous system that works for everyone.

N.B. Data for this study and the codebook are available via the following URL:

APPENDIX A. Articles Included in the Systematic Review

  1. Allison, DeeAnn. “Information Portals: The Next Generation Catalog.” Journal of Web Librarianship 4, no. 4 (2010): 375–89.
  2. Asher, Andrew D., Lynda M. Duke, and Suzanne Wilson. “Paths of Discovery: Comparing the Search Effectiveness of EBSCO Discovery Service, Summon, Google Scholar, and Conventional Library Resources.” College & Research Libraries 74, no. 5 (2012): 464–88.
  3. Baldwin, Dee, Michael Kucsak, and Alice Eng. “Don’t Touch That String! There Went the Databases.” Information Outlook 16, no. 5 (Sept./Oct. 2012): 24–32.
  4. Ballard, Terry, and Anna Blaine. “User Search-Limiting Behavior in Online Catalogs: Comparing Classic Catalog Use to Search Behavior in Next-Generation Catalogs.” New Library World 112, no. 5/6 (2011): 261–73.
  5. Becher, Melissa, and Kari Schmidt. “Taking Discovery Systems for a Test Drive.” Journal of Web Librarianship 5, no. 3 (2011): 199–219.
  6. Boyer, Ginny M., and Megan Besaw. “A Study of Librarians’ Perceptions and Use of the Summon Discovery Tool.” Journal of Electronic Resources in Medical Libraries 9, no. 3 (2012): 173–83.
  7. Bryant, Sally, and Grace Ye. “Implementing OCLC’s WMS (Web-Scale Management Services) Circulation at Pepperdine University.” Journal of Access Services 9, no. 1 (2012): 1–17.
  8. Buck, Stefanie, and Margaret Mellinger. “The Impact of Serial Solutions’ SummonTM on Information Literacy Instruction: Librarian Perceptions.” Internet Reference Services Quarterly 16, no. 4 (2011): 159–81.
  9. Buck, Stefanie, and Jane Nichols. “Beyond the Search Box.” Reference & User Services Quarterly 51, no. 3 (2012): 235–45.
  10. Buck, Stefanie, and Christina Steffy. “Promising Practices in Instruction of Discovery Tools.” Communications in Information Literacy 7, no. 1 (2013): 66–80.
  11. Cmor, Dianne, and Xin Li. “Beyond Boolean, towards Thinking: Discovery Systems and Information Literacy.” Library Management 33, no. 8/9 (2012): 450–57.
  12. Comeaux, David J. “Usability Testing of a Web-Scale Discovery System at an Academic Library.” College & Undergraduate Libraries 19, no. 2–4 (2012): 189–206.
  13. Daniels, Jeffrey, and Patrick Roth. “Incorporating Millennium Catalog Records into Serials Solutions’ Summon.” Technical Services Quarterly 29, no. 3 (2012): 193–99.
  14. Debonis, Rocco, Edward O’Donnell, and Cynthia Thomes. “(Self-) Discovery Service: Helping Students Help Themselves.” Journal of Library & Information Services in Distance Learning 6, no. 3/4 (2012): 235–50.
  15. Denton, William, and Sarah J. Coysh. “Usability Testing of VuFind at an Academic Library.” Library Hi Tech 29, no. 2 (2011): 301–19.
  16. Diamond, Ted, Susan Price, and Raman Chandrasekar. “Actions Speak Louder than Words: Analyzing Large-Scale Query Logs to Improve the Research Experience.” The code4lib Journal 21 (2013).
  17. Diedrichs, Carol Pitts. “Discovery and Delivery: Making It Work for Users.” The Serials Librarian 56, no. 1–4 (2009): 79–93.
  18. Dolski, Alex A. “Information Discovery Insights Gained from MultiPAC, a Prototype Library Discovery System.” Information Technology & Libraries 28 (Dec. 2009): 172–80.
  19. Durante, Kim, and Zheng Wang. “Creating an Actionable Assessment Framework for Discovery Services in Academic Libraries.” College & Undergraduate Libraries 19, no. 2–4 (2012): 215–28.
  20. Eden, Brad, Birong Ho, Keith Kelley, and Scott Garrison. “Implementing VuFind as an Alternative to Voyager’s WebVoyage Interface: One Library’s Experience.” Library Hi Tech 27, no. 1 (2009): 82–92.
  21. Eden, Brad, and John Houser. “The VuFind Implementation at Villanova University.” Library Hi Tech 27, no. 1 (2009): 93–105.
  22. Ellero, Nadine P. “An Unexpected Discovery: One Library’s Experience with Web-Scale Discovery Service (WSDS) Evaluation and Assessment.” Journal of Library Administration 53, no. 5/6 (2013): 323–43.
  23. ———. “Integration or Disintegration: Where Is Discovery Headed?” Journal of Library Metadata 13, no. 4 (2013): 311–29.
  24. Emanuel, Jennifer. “Usability of the VuFind Next-Generation Online Catalog.” Information Technology and Libraries 30, no. 1 (2011): 44–52.
  25. Fagan, Jody C., Meris A. Mandernach, Carl S. Nelson, Jonathan R. Paulo, and Grover Saunders. “Usability Test Results for a Discovery Tool in an Academic Library,” Information Technology & Libraries 31, no. 1 (2012): 83–112.
  26. Fahey, Sue, Shannon Gordon, and Crystal Rose. “Seeing Double at Memorial University: Two WorldCat Local Usability Studies.” Partnership: The Canadian Journal of Library and Information Practice and Research 6, no. 2 (2011), 1–14.
  27. Fawley, Nancy, and Nikki Krysak. “Information Literacy Opportunities within the Discovery Tool Environment.” College & Undergraduate Libraries 19, no. 2–4 (2012): 207–14.
  28. Featherstone, Robin, and Lei Wang. “Enhancing Subject Access to Electronic Collections with VuFind.” Journal of Electronic Resources in Medical Libraries 6, no. 4 (2009): 294–306.
  29. Foster, Anita K., and Jean B. MacDonald. “A Tale of Two Discoveries: Comparing the Usability of Summon and EBSCO Discovery Service.” Journal of Web Librarianship 7, no. 1 (2013): 1–19.
  30. Gaffney, Megan. “Interlibrary Loan Requests for Locally Available Materials: WorldCat Local’s Impact.” College & Research Libraries 73, no. 1 (2012): 68–77.
  31. Gallaway, Teri Oaks, and Mary Finnan Hines. “Competitive Usability and the Catalogue: A Process for Justification and Selection of a Next-Generation Catalogue or Web-Scale Discovery System.” Library Trends 61, no. 1 (2012): 173–85.
  32. Gross, Julia, and Lutie Sheridan. “Web Scale Discovery: The User Experience.” New Library World 112, no. 5/6 (2011): 236–47.
  33. Grotti, Margaret G., and Karen Sobel. “Worldcat Local and Information Literacy Instruction: An Exploration of Emerging Teaching Practice.” Public Services Quarterly 8, no. 1 (2012): 12–25.
  34. Guthrie, Ana, and Rhonda McCoy. “A Glimpse at Discovery Tools within the HBCU Library Landscape.” College & Undergraduate Libraries 19, no. 2–4 (2012): 297–311.
  35. Hallyburton, Ann, and Elizabeth Marcus. “Maximize Discovery and Minimize Cost.” Journal of Electronic Resources in Medical Libraries 9, no. 1 (2012): 1–12.
  36. Harpel-Burke, Pamela. “Catalog Maintenance in the Discovery System: Opinions of Early Adopters.” College & Undergraduate Libraries 19, no. 2–4 (2012): 246–63.
  37. Hessel, Heather, and Janet Fransen. “Resource Discovery: Comparative Results on Two Catalog Interfaces.” Information Technology and Libraries 31, no. 2 (2012): 21–44.
  38. Jarrett, Kylie. “FindIt@ Flinders: User Experiences of the Primo Discovery Search Solution.” Australian Academic & Research Libraries 43, no. 4 (2012): 278–99.
  39. Johnson, Megan. “Usability Test Results for Encore in an Academic Library.” Information Technology and Libraries 32, no. 3 (2013): 59–85.
  40. Johnston, Nicole, Alicia Salaz, and Rob O’Connell. “Determining Usability of VuFind for Users in the United Arab Emirates.” Code4Lib Journal, no. 19 (2013).
  41. Katz, Demian, Ralph LeVan, and Ya’aqov Ziso. “Using Authority Data in VuFind.” Code4Lib Journal (June 2011).
  42. Kaufmann, Karen, Jeanne Larsen, and Patricia DeSalvo. “Discovering the Discovery Tool: The Introduction and Impact on Research and Instruction at Seminole State College of Florida.” College & Undergraduate Libraries 19, no. 2–4 (2012): 278–96.
  43. Ketchell, Debra S., Ryan Max Steinberg, Charles Yates, and Heidi A. Heilemann. “LaneConnex: An Integrated Biomedical Digital Library Interface.” Information Technology and Libraries 28, no. 1 (2013): 31–40.
  44. Kornblau, Amy I., Jane Strudwick, and William Miller. “How Web-Scale Discovery Changes the Conversation: The Questions Librarians Should Ask Themselves.” College & Undergraduate Libraries 19, no. 2–4 (2012): 144–62.
  45. Leebaw, Danya, Brian Conlan, Kasia Gonnerman, Sarah Johnston, and Christina Sinkler-Miller. “Improving Library Resource Discovery: Exploring the Possibilities of VuFind and Web-Scale Discovery.” Journal of Web Librarianship 7, no. 2 (2013): 154–89.
  46. Mahoney, Jessica, and Susan Leach-Murray. “Implementation of a Discovery Layer: The Franklin College Experience.” College & Undergraduate Libraries 19, no. 2–4 (2012): 327–43.
  47. Majors, Rice. “Comparative User Experiences of Next-Generation Catalogue Interfaces.” Library Trends 61, no. 1 (2012): 186–207.
  48. Meadow, Kelly, and James Meadow. “Search Query Quality and Web-Scale Discovery: A Qualitative and Quantitative Analysis.” College & Undergraduate Libraries 19, no. 2–4 (2012): 163–75.
  49. Meredith, William. “Web-Scale Search and Virtual Reference Service: How Summon Is Impacting Reference Question Complexity and Reference Service Delivery.” Internet Reference Services Quarterly 18, no. 1 (2013): 1–13.
  50. Milberg, Craig I. “A Tale of Two Systems: A Case Study on the Implementation of Two Discovery Systems at Davidson College.” College & Undergraduate Libraries 19, no. 2–4 (2012): 264–77.
  51. Moore, Kate B., and Courtney Greene. “Choosing Discovery: A Literature Review on the Selection and Evaluation of Discovery Layers.” Journal of Web Librarianship 6, no. 3 (2012): 145–63.
  52. Mortimore, Jeffrey M., Leah McGinnis Dunn, Drew A. McNaughton, Elizabeth Novicki, Elizabeth J. Wade, and Christine A. Whittington. “Smoke Gets in Your Eyes: Trials and Triumphs of Implementing the Ex Libris Primo Discovery Service in a Small Regional Consortium.” College & Undergraduate Libraries 19, no. 2–4 (2012): 344–66.
  53. Mussell, Jessica, and Rosie Croft. “Discovery Layers and the Distance Student: Online Search Habits of Students.” Journal of Library & Information Services in Distance Learning 7, no. 1/2 (2013).
  54. Newcomer, Nara L. “The Detail behind Web-Scale: Selecting and Configuring Web-Scale Discovery Tools to Meet Music Information Retrieval Needs.” Music Reference Services Quarterly 14, no. 3 (2011): 131–45.
  55. O’Hara, Lisa. “Collection Usage Pre-and Post-Summon Implementation at the University of Manitoba Libraries.” Evidence Based Library and Information Practice 7, no. 4 (2012): 25–34.
  56. Ours, Michael G. “The Evaluation of Discovery Services at Lynchburg College: 2009– 2010.” College & Undergraduate Libraries 19, no. 2–4 (2012): 387–97.
  57. Pace, Andrew. “21st Century Library Systems.” Journal of Library Administration 49, no. 6 (2009): 641–50.
  58. Yeo Pin Pin. “Impact of a New Discovery Service at Li Ka Shing Library.” Singapore Journal of Library & Information Management 41 (2012): 36–50.
  59. Rochkind, Jonathan. “A Comparison of Article Search APIs via Blinded Experiment and Developer Review.” Code4Lib Journal, no. 19 (2013).
  60. Rosenfeld, Jennifer, Raida Gatten, Amy F. Fyn, Vera Lux, and Robert J. Snyder. “Reflections on Teaching and Tweaking a Discovery Layer.” Reference Services Review 41, no. 1 (2013): 113–24.
  61. Rose-Wiles, Lisa M., and Melissa A. Hofmann. “Still Desperately Seeking Citations: Undergraduate Research in the Age of Web-Scale Discovery.” Journal of Library Administration 53, no. 2/3 (2013): 147–66.
  62. Skinner, Debra G. “A Comparison of Searching Functionality of a VuFind Catalogue Implementation and the Traditional Catalogue.” Library Trends 61, no. 1 (2012): 208–17.
  63. Spiteri, Louise F. “Social Discovery Tools: Extending the Principle of User Convenience.” Journal of Documentation 68, no. 2 (2012): 206–17.
  64. Spiteri, Louise F., and Laurel Tarulli. “Social Discovery Systems in Public Libraries: If We Build Them, Will They Come?” Library Trends 61, no. 1 (2012): 132–47.
  65. Stone, Graham. “Searching Life, the Universe and Everything? The Implementation of Summon at the University of Huddersfield.” Liber Quarterly 20, no. 1 (2010): 25–51.
  66. Thomas, Bob, and Stefanie Buck. “OCLC’s WorldCat Local versus III’s WebPAC: Which Interface Is Better at Supporting Common User Tasks?” Library Hi Tech 28, no. 4 (2010): 648–71.
  67. Thompson, JoLinda L., Kathe S. Obrig, and Laura E. Abate. “Web-Scale Discovery in an Academic Health Sciences Library: Development and Implementation of the EBSCO Discovery Service.” Medical Reference Services Quarterly 32, no. 1 (2013): 26–41.
  68. Thomsett-Scott, Beth, and Patricia E. Reese. “Academic Libraries and Discovery Tools: A Survey of the Literature.” College & Undergraduate Libraries 19, no. 2–4 (2012): 123–43.
  69. Thorpe, Angie, and Heather Bowman. “Promoting Discovery: Creating an In-Depth Library Marketing Campaign.” Journal of Library Administration 53, no. 2/3 (2013): 100–21.
  70. Vaughan, Jason. “Investigations into Library Web Scale Discovery Services.” Information Technology & Libraries 31, no. 1 (Mar. 2012): 32–82.
  71. Walker, Shaundra, and Iyanna L. Sims. “Implementing a Discovery Tool at Two HBCUs.” College & Undergraduate Libraries 19, no. 2–4 (2012): 312–26.
  72. Wang, Yongming, and Jia Mi. “Searchability and Discoverability of Library Resources: Federated Search and Beyond.” College & Undergraduate Libraries 19, no. 2–4 (2012): 229–45.
  73. Williams, Sarah C., and Anita K. Foster. “Promise Fulfilled? An EBSCO Discovery Service Usability Study.” Journal of Web Librarianship 5, no. 3 (2011): 179–98.
  74. Wolpers, Martin, Martin Memmel, Joris Klerkx, Gonzalo Parra, Bram Vandeputte, Erik Duval, Rafael Schirru, and Katja Niemann. “Bridging Repositories to Form the MACE Experience.” New Review of Information Networking 14, no. 2 (2009): 102–16.
  75. Wrosch, Jackie, Karen Rogers-Collins, Michael Barnes, and William Marino. “Search Me: Eastern Michigan University’s Journey through the Highs and Lows of Implementing the Summon Discovery Tool.” College & Undergraduate Libraries 19, no. 2–4 (2012): 367–86.
  76. Wusteman, Judith, Brad Eden, and Alejandro Garza. “From OPAC to CMS: Drupal as an Extensible Library Platform.” Library Hi Tech 27, no. 2 (2009): 252–67.
  77. Wusteman, Judith, Brad Eden, Dean James, Michael Garrett, and Leah Krevit. “Discovering Discovery Tools: Evaluating Vendors and Implementing Web 2.0 Environments.” Library Hi Tech 27, no. 2 (2009): 268–76.
  78. Wynne, Susan C., and Martha J. Hanscom. “The Effect of Next-Generation Catalogs on Catalogers and Cataloging Functions in Academic Libraries.” Cataloging & Classification Quarterly 49, no. 3 (2011): 179–207.
  79. Yang, Sharon Q., and Kurt Wagner. “Evaluating and Comparing Discovery Tools: How Close Are We towards Next Generation Catalog?” Library Hi Tech 28, no. 4 (2010): 690–709.
  80. Zhu, Lihong. “The Role of the Cataloging Department in the Implementation of OCLC WorldCat Local.” Library Collections, Acquisitions, and Technical Services 34, no. 4 (2010): 123–29.

APPENDIX B. Search Terms Used in LISTA

Terms used to search articles from 2009 to 2013

  • “discovery system*”
  • “discovery service*”
  • “discovery product*”
  • VuFind and discovery
  • (“BiblioCore” or BiblioCommons) and discovery
  • “Summon” and discovery
  • WorldCatLocal and discovery
  • “AquaBrowser Library” and discovery
  • “Enterprise” and discovery
  • “Primo” and discovery
  • “Arena” and discovery
  • “Encore” and discovery

Additional terms used to search articles from 2009 to 2011

The terms listed above plus additional terms to adjust for describing discovery layers and next-generation discovery interfaces. These inconsistencies were in part due to the field’s newness during this period:

  • “discovery interface*”
  • “discovery layer*”
  • “next-generation” and discovery


1. Dee Baldwin, Michael Kucsak, and Alice Eng, “Don’t Touch That String! There Went the Databases,” Information Outlook 16, no. 5 (2012): 30.

2. Michael G. Ours, “The Evaluation of Discovery Services at Lynchburg College: 2009–2010,” College & Undergraduate Libraries 19, no. 2/4 (2012): 387–97.

3. Marshall Breeding, “The Future of Library Resource Discovery: A White Paper Commissioned by the NISO Discovery to Delivery (D2D) Topic Committee” (National Information Standards Organization, Feb. 2015), available online at [accessed 1 June 2016].

4. William H. Weare Jr., Sue Toms, and Marshall Breeding, “Moving Forward: The Next-Gen Catalog and the New Discovery Tools,” Library Media Connection 30, no. 3 (2011): 54.

5. Breeding, “The Future of Library Resource Discovery”; Open Discovery Initiative Working Group, “Open Discovery Initiative: Promoting Transparency in Discovery” (National Information Standards Organization, 2014), available online at [accessed 1 June 2016].

6. Open Discovery Initiative Working Group, “Open Discovery Initiative.”

7. W. Boyd Rayward, “Electronic Information and the Functional Integration of Libraries, Museums, and Archives,” in History and Electronic Artefacts, ed. Edward Higgs (Oxford: Clarendon Press, 1998), 207–26; Wendy M. Duff, Jennifer Carter, Joan M. Cherry, Heather MacNeil and Lynne C. Howarth, “From Coexistence to Convergence: Studying Partnerships and Collaboration among Libraries, Archives and Museums,” Information Research 18, no. 3 (2013), available online at [accessed 1 June 2016]; Paul F. Marty, “An Introduction to Digital Convergence: Libraries, Archives, and Museums in the Information Age,” Museum Management and Curatorship 24, no. 4 (2009): 295–98; Paul F. Marty, “Digital Convergence in the Information Profession in Cultural Heritage Organizations: Reconciling Internal and External Demands,” Library Trends 62, no. 3 (2014): 613–27.

8. Christopher Barth, Convergence of Libraries and Technology Organizations: New Information Support Models (Oxford: Chandos Publishing, 2011), 2.

9. Jennifer Trant, “Emerging Convergence? Thoughts on Museums, Archives, Libraries, and Professional Training,” Museum Management and Curatorship 24, no. 4 (Dec. 2009): 369–87, doi:10.1080/09647770903314738. See also: Margaret Bing, “The False Dualism,” Journal of Library Administration 29, no. 2 (2000), 23–28.

10. Connie Van Fleet and Joan C. Durrance, “Public Library Leaders and Research: Mechanisms, Perceptions, and Strategies,” Journal of Education for Library and Information Science 34, no. 2 (Spring 1993): 137–52.

11. Cris Ferguson, “Technology Left Behind: Social Cataloging and the Library OPAC,” Against the Grain 21, no. 1 (2013): 51; Louise F. Spiteri and Laurel Tarulli, “Social Discovery Systems in Public Libraries: If We Build Them, Will They Come?” Library Trends 61, no. 1 (2012): 132–47.

12. Andrew Abbott, Methods of Discovery: Heuristics for the Social Sciences (New York: W.W. Norton & Company, 2004).

13. Van Fleet and Durrance, “Public Library Leaders and Research.”

14. June Garner, Karen Davidson, and Becky Schwartzkopf, “Images of Academic Librarians: How Tenure-Track Librarians Portray Themselves in the Promotion and Tenure Process,” Serials Librarian 56, no. 1–4 (2009): 203–08.

15. Eugene Garfield, “Journal Impact Factor: A Brief Review,” Canadian Medical Association Journal 161, no. 8 (1999): 979–80.

16. Christian Schlögl and Wolfgang Petschnig, “Library and Information Science Journals: An Editor Survey,” Library Collections, Acquisitions, and Technical Services 29, no. 1 (2005): 4–32.

17. Julie A. Hersberger and Christopher Demas, “The Current State of Public Library Research in Select Peer-Reviewed Journals: 1996–2000,” North Carolina Libraries 59, no. 1 (2001): 10–14.

18. Van Fleet and Durrance, “Public Library Leaders and Research.”

19. Stevan Harnad, “The Postgutenberg Open Access Journal (revised),” in The Future of the Academic Journal (2nd Edition), eds. B. Cope and A. Phillips (Chandos, Oxford, UK 2013), available online at [accessed 1 June 2016].

20. Stevan Harnad, “Free at Last: The Future of Peer-Reviewed Journals,” D-Lib Magazine 5, no. 12 (1999), available online at [accessed 1 June 2016], paragraph 8.

21. Breeding, “The Future of Library Resource Discovery, 2.

22. Marshall Breeding, “Major Discovery Product Profiles,” Library Technology Reports 50, no. 1 (Feb. 1, 2014): 33–52.

23. Breeding, “The Future of Library Resource Discovery.”

24. Ibid.

25. Jason Vaughan, “Differentiators and a Final Note,” Library Technology Reports 47, no. 1 (2011): 48–53.

26. Suzanne Saskia Kemperman, Bill Brembeck, Elizabeth W. Brown, Alexandra de Lange-van Oosten, Theodore Fons, Catherine Giffi, Noah Levin, Alistair Morrison, Carlen Ruschoff, Gregg A. Silvis, and Jabin White, Success Strategies for Electronic Content Discovery and Access: A Cross-Industry White Paper (Dublin, Ohio: OCLC, 2014), available online at [accessed 1 June 2016].

27. Kate B. Moore and Courtney Greene, “Choosing Discovery: A Literature Review on the Selection and Evaluation of Discovery Layers,” Journal of Web Librarianship 6, no. 3 (July 1, 2012): 145–63, doi:10.1080/19322909.2012.689602; Beth Thomsett-Scott and Patricia E. Reese, “Academic Libraries and Discovery Tools: A Survey of the Literature,” College & Undergraduate Libraries 19, no. 2–4 (2012): 123–43.

28. Nadine P. Ellero, “Integration or Disintegration: Where Is Discovery Headed?” Journal of Library Metadata 13, no. 4 (2013): 311–29.

29. Marco Iansiti and Roy Levien, “Keystones and Dominators: Framing Operating and Technology Strategy in a Business Ecosystem,” Harvard Business School, Boston (2004):24–25.

30. Robert D. Steuart and Barbara B. Moran, Library and Information Center Management, 7th ed. (Libraries Unlimited, Santa Barbara, CA 2007).

31. Ruth H. Makinen, “Scheduling Technical Services Staff at the Reference Desk,” Reference Librarian 28, no. 59 (1997): 139–46.

32. Teressa M. Keenan, “Resource Description and Access: Cataloging Standards Affect Reference Service,” Reference Services Review 42, no. 3 (2014): 446–66.

33. Bing, “The False Dualism,” 24.

34. Steuart and Moran, Library and Information Center Management.

35. Ping Fu and Moira Fitzgerald, “A Comparative Analysis of the Effect of the Integrated Library System on Staffing Models in Academic Libraries,” Information Technology & Libraries 32, no. 3 (Sept. 2013): 47–58.

36. Tyler O. Walters, “Reinventing the Library: How Repositories Are Causing Librarians to Rethink Their Professional Roles,” portal: Libraries and the Academy 7, no. 2 (2007): 213–25.

37. Harris Cooper and Larry V. Hedges, “Research Synthesis as a Scientific Process,” in The Handbook of Research Synthesis and Meta-Analysis, eds. Harris Cooper, Larry V. Hedges, and Jeffrey C. Valentine (Russell Sage Foundation, New York, NY 2009), 3–19.

38. Deborah L. Finfgeld, “Metasynthesis: The State of the Art—So Far,” Qualitative Health Research 13, no. 7 (2003): 893–904; Cooper and Hedges, “Research Synthesis as a Scientific Process,” 3–18.

39. Cooper and Hedges, “Research Synthesis as a Scientific Process,” 7.

40. Jianhua Xu, Qi Kang, and Zhiqiang Song, “The Current State of Systematic Reviews in Library and Information Studies,” Library & Information Science Research 37, no. 4 (Oct. 2015): 296–310. doi:10.1016/j.lisr.2015.11.003.

41. Julie A. Hersberger and Christopher Demas, “The Current State of Public Library Research in Select Peer-Reviewed Journals: 1996–2000,” North Carolina Libraries 59, no. 1 (2009): 10–14.

42. Lynn Silipigni Connaway and Timothy J. Dickey, The Digital Information Seeker: Report of the Findings from Selected OCLC, RIN, and JISC User Behaviour Projects (June 2010), 61 available online at [accessed 1 June 2016].

43. Xu, Kang, and Song, “The Current State of Systematic Reviews in Library and Information Studies,” 296.

44. Connaway and Dickey, The Digital Information Seeker, 1.

45. Cooper, Hedges, and Valentine, The Handbook of Research Synthesis and Meta-Analysis.

46. Harris Cooper, Evaluating and Interpreting Research Syntheses in Adult Learning and Literacy: NCSALL Occasional Paper (National Center for the Study of Adult Learning and Literacy [NCSALL], Cambridge, MA 2007), available online at [accessed 1 June 2016].

47. Ibid.

48. Ibid.

49. Rhonda Whitouse, personal communication, April 1, 2015.

50. Cooper, Evaluating and Interpreting Research Syntheses.

51. For more information about the coding in this project, see the codebook at MOspace URL.

52. Cooper, Evaluating and Interpreting Research Syntheses.

53. Ibid.

54. Ibid.

55. Sally Bryant and Grace Ye, “Implementing OCLC’s WMS (Web-Scale Management Services) Circulation at Pepperdine University,” Journal of Access Services 9, no. 1 (2012): 1–17.

56. Baldwin, Kucsak, and Eng, “Don’t Touch That String! There Went the Databases.”

57. Steuart and Moran, Library and Information Center Management.

58. Hersberger and Demas, “The Current State of Public Library Research in Select Peer-Reviewed Journals: 1996–2000.”

59. Garner, Davidson, and Schwartzkopf, “Images of Academic Librarians.”

60. Baldwin, Kucsak, and Eng, “Don’t Touch That String! There Went the Databases.”

*Jenny S. Bossaller is Associate Professor and Heather Moulaison Sandy is Assistant Professor at the iSchool at the University of Missouri; e-mail:, ©2017 Jenny S. Bossaller and Heather Moulaison Sandy, Attribution-NonCommercial ( CC BY-NC.

Copyright Jenny S. Bossaller, Heather Moulaison Sandy

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Article Views (Last 12 Months)

No data available

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

January: 34
February: 72
March: 54
April: 68
May: 59
June: 57
July: 40
August: 36
September: 26
January: 76
February: 78
March: 75
April: 54
May: 51
June: 37
July: 42
August: 34
September: 43
October: 68
November: 42
December: 43
January: 69
February: 87
March: 113
April: 110
May: 76
June: 66
July: 59
August: 77
September: 70
October: 119
November: 69
December: 30
January: 94
February: 110
March: 71
April: 116
May: 73
June: 60
July: 44
August: 54
September: 61
October: 117
November: 61
December: 66
January: 63
February: 93
March: 129
April: 95
May: 86
June: 101
July: 99
August: 83
September: 101
October: 76
November: 52
December: 63
January: 117
February: 101
March: 186
April: 199
May: 170
June: 143
July: 99
August: 116
September: 167
October: 154
November: 122
December: 82
April: 0
May: 0
June: 0
July: 704
August: 209
September: 281
October: 312
November: 134
December: 151