Harnessing Generative AI for enhanced Archival Metadata: A cross-sector collaboration

Abstract

The National Library Board of Singapore (NLB) is pioneering the use of generative AI to enhance metadata for archival records, improving access to historical materials. This project exemplifies innovation by bridging the gap between archival science and artificial intelligence. The initiative leverages AI tools such as PAIR and Custom GPT to automate metadata generation. PAIR, a generative AI tool for Singapore public officers based on GPT-3.5, was deployed in Phase 1 of the project to analyse archival content such as Singapore's National Day Rally Speeches and Oral History Interviews. In Phase 2, Custom GPT (GPT-4.0) processed an expanded dataset, including Government Records, Press Releases, and News Archives. The AI tools generate Library of Congress Subject Headings (LCSH), FAST (Faceted Application of Subject Terminology) headings, and extract named entities (People, Places, Organizations). Phase 3 will include the integration of HybridRAG (Retrieval-Augmented Generation), combining Text Retrieval and Graph Retrieval tools for enhanced metadata-driven knowledge discovery, harnessing the power of NLB's Knowledge Graph. This cross-sector collaboration optimizes Linked Data API-powered tools, demonstrating how archives can leverage AI and external partnerships to improve metadata access, knowledge discovery, and digital preservation.