Text & data mining - a librarian overview

Abstract

Text and data mining offers exciting research opportunities over a broad range of fields. As large corpora of data accumulate, automated and semi-automated analysis of their contents (and often of many different data sets correlated together) reveals patterns and allows establishment of fact patterns invisible to the naked eye. Libraries and librarians have an exciting opportunity to support this work.

This paper reviews some of the possibilities for such work and outlines the challenges and the way ahead for librarians. One challenge lies in the terms by which data sets are licensed and made available to academic and other users; librarians need to be proactive in ensuring that these terms are favorable for the kind of use researchers will need and that the resources themselves are available in a format that allows innovative mining-based research. Another challenge is the need to support users who wish to engage in text and data mining with limited experience, especially when they approach data sets made available through library resources. Librarians should develop the expertise to support their users by making data resources available to them on favorable terms and supporting their mining efforts .