Accessibility and Digital Repositories: Describing Digital Collections using LLMs

Abstract

In April 2024, the United States Department of Justice announced web accessibility regulations that require all state and local governments to make their websites, mobile apps, and content accessible as prescribed by U.S. law. This prompted the University of Illinois Urbana-Champaign (U of I) Library to both evaluate the accessibility of our repository systems and the content we steward, and also review how digital assets are created.

Broadly, there are three areas of focus for digital accessibility in our repositories: user interfaces, existing digital assets, and future digital asset creation. The University of Illinois boasts one of the largest physical library collections in the United States and hosts more than more than 3 million digital assets across our repository services, including digitized newspapers, special collections, scholarship, and digitized books. The content creation, ingest pipelines, and homegrown repositories have not been designed with accessibility first, making adherence to the web accessibility requirements a daunting challenge. U of I librarians and staff are investigating whether multimodal large language models (LLMs) can be leveraged to meet the technical accessibility requirements for description of digital assets. The authors designed a pilot project to generate alternative text (i.e., alt text) using a local installation of Meta’s pre-trained Llama 3.2-Vision.

Our preliminary findings suggest that, while it’s a viable tool to describe some types of images,...