Cost and security improvement of the Long Time Preservation by the use of open source software and new ISO standard in the National Library repository

Abstract

Long Time Preservation (LTP) Policy besides high-level organizational elements should comprise technological foundation based on available technologies. Omitting this element can undermine rationality of the former.

The LTP System should: • be independent from a single manufacturer and have known data path, • be permanently ready for total disaster (The Doomsday Clock is now as close to midnight as it was in the fifties), • be scalable in capacity and performance, • keep costs within available budget. • The carrier and format themselves should be sufficiently durable to limit the frequency of migration. The analysis of these criteria led to the selection of well-established technologies based on open or well-documented standards, independent from a single manufacturer: • Magnetic tape - decades of experience and predictable shelf life, • Linear Tape Open (LTO) - many manufacturers produce tapes compatible with drives from various manufacturers, • Linear Tape File System (LTFS) - allows exchanging data between compatible systems from different producers using open-source software. In the year 2016 LTFS became adopted as ISO/IEC standard, • Metadata Encoding and Transmission Standard (METS) – XML container of digital objects. Above technologies and some our inventions were used to develop the software of new LTP System. • The System has since worked productively for 10 months and archived almost petabyte of data. • Cost per byte (3 copies) is more than order of magnitude lower than in the former solution. • The System uses standard tape library, is scalable and redundant. • Digital objects saved (each integrally) on every single tape are readable and may be migrated without any knowledge about the LTP System. • The entire data resource is easily transportable