Multiple users have reported problems submitting materials to IDEALS. We are aware of the issue and are working to correct it ASAP. We appreciate your patience!

University of Illinois Urbana-Champaign

A QUESTION OF CHARACTER: How do we automatically recharacterize data at cloud scales?

O’Sullivan, Jack; Clipsham, David; Soni, Divyesh; Smith, Richard; Tilbury, Jonathan

Permalink

https://hdl.handle.net/2142/121087

Description

Title
A QUESTION OF CHARACTER: How do we automatically recharacterize data at cloud scales?
Author(s)
  • O’Sullivan, Jack
  • Clipsham, David
  • Soni, Divyesh
  • Smith, Richard
  • Tilbury, Jonathan
Issue Date
2023
Keyword(s)
  • Scalability
  • Automation
  • Characterization
  • Preservation Actions
Date of Ingest
2023-11-01T18:09:06-05:00
Abstract
Many preservation actions that we undertake on digital content are driven by the format of the content in question. Format information is often determined at the point of ingest and is not regularly updated as our knowledge of file formats improves over time. Periodically re-characterizing all content in a repository would ensure that we get more accurate identifications over time, but a more sustainable approach would be to only re-characterize content that was actually likely to have changed. Preservica’s new Automated Active Digital Preservation feature seeks to do exactly this, but even when considering only subsets of the data in our cloud systems, we are faced with significant challenges of scale. In this paper, we describe those challenges, the approach we have taken to implement the feature, and the testing we have performed to verify the viability of this approach.
Series/Report Name or Number
iPRES 2023
Type of Resource
text
Language
en
Copyright and License Information
Copyright held by the author(s). The text of this paper is published under a CC BY-SA license (https://creativecommons.org/licenses/by/4.0/).

Owning Collections

Long Papers PRIMARY