Phrasing in Reproducible Search Methodology: The Consequences of Straight and Curly Quotation Marks

Abstract

In recent years, various disciplines have engaged in efforts to increase research reproducibility including the adoption of replicable search methodologies. With the development of reporting checklists and guidelines for systematic reviews such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement, authors are expected to transparently report search strategies. Replicable search strategies are critical since the included studies will be screened for inclusion in some forms of evidence synthesis, which could have practice and policy implications. In cases where search strategies miss germane literature, studies are open to further criticism and can face difficulties in peer review. In some cases, search strategies that contain nonalphanumeric or special characters may not retrieve pertinent literature due to a search platform’s capacity for handling said characters. In this study, we explore issues with phrasing search strategies containing curly and straight quotation marks tested in 40 search platforms. The discovery platforms were tested using quotation characters and the absence of said characters to investigate platform behavior. Searches were categorized into one of five groups: CI (curly ignored), CA (curly acknowledged), CU (curly unclear), NP (no phrases), and UC (unsupported characters). The study found that 42.5 percent of platforms ignored curly quotation marks and interpreted the test term as a phrase, 30 percent of the bibliographic platforms acknowledged curly quotations and completed the phrase search, and one platform flagged curly quotation marks as an unsupported character.