How Well Does ChatGPT Handle Reference Inquiries? An Analysis Based on Question Types and Question Complexities

Abstract

To explore whether artificial intelligence can be used to enhance library services, this study used ChatGPT to answer reference questions. An assessment rubric was used to evaluate how well ChatGPT handled different question types and difficulty levels. Overall ChatGPT’s performance was fair, but it did poorly in information accuracy. It scored the highest when handling facilities and equipment-related questions but the lowest when dealing with e-resources access problems. ChatGPT was weak in answering advanced research questions, complex inquiries, and known item searches relating to a specific local environment, but it could be adopted to enhance library communication with users.