Three Discovery Tools: A Comparative Analysis of Retrieval Scope, Ranking Effectiveness, and Topic Diversity

Abstract

Discovery tools facilitate access to large-scale academic collections, yet their retrieval
performance varies. This study presents a comparative analysis of three discovery
tools—EBSCO Discovery Services (EDS), EKUAL Discovery Services (EKUAL DS), and
Piri Discovery Services (Piri DS)—evaluating retrieval scope, ranking quality, and
topical diversity index. The iSearch test collection, derived from arXiv articles, was
used with predefined search queries. To assess coverage, the full arXiv corpus was
queried to identify indexing differences. A total of 63 queries were executed, and
retrieved lists were analyzed for relevance and ranking distribution. Expert-evaluated
relevant articles were used to assess retrieval accuracy. Ranking was measured using
Discounted Cumulative Gain (DCG) and Normalized DCG (NDCG), and topical
diversity was evaluated using the Shannon Diversity Index. EDS and EKUAL DS retrieved
identical results, while Piri DS retrieved fewer