METS for Transferable Metadata

METS is the Metadata Encoding and Transmission Standard, which is applied to encoding metadata via a standardized XML schema.  METS handles all types of metadata that is relevant to preservation: descriptive, administrative, and technical/structural metadata are all included in the schema, and a METS document will serve as the container for all of this information about a digital object.

The schema was initially developed for the digital library community, and has thus extended to the digital repository and preservation communities.  The fact that METS confines varying types of an object’s metadata to one standard XML-based file type is excellent news for sharing and preserving resources.

Photo by Flickr user SyN+H, CC license

As is evident through the experience of many current digital preservation programs, collaboration among multiple institutions is a very strategic move for a successful digital preservation program.  Using METS as a guideline for creating readable and transferable metadata ensures a more seamless sharing experience.  It also aids in escape strategies should the repository or institution hosting the repository fails and the digital objects need to be transferred to someone else’s care.

History

The beginnings of a standardized metadata scheme for collections of digital objects can be traced back to 1997, when UC Berkeley and the Digital Library Federation (DLF) initiated a project to further the concept of digital libraries sharing resources.  By 2001, the DLF-sponsored METS schema emerged, which is supported by the Library of Congress, and was made a NISO standard in 2004, and was renewed in 2006.

By 2006, it had become clear that METS could not only serve as an answer to the interoperability needs associated with sharing digital objects, but that METS is also valuable for preservation purposes.  Jerome McDonough (2006) states that “the METS standard can be considered one of many efforts to try to determine…how complex sets of data and metadata might best be encoded to support both information exchange and information longevity.”

How it’s Used

The OAIS reference model considers an acceptable digital object as one that includes the original content as well as the metadata required to understand the content, its structure, its rendering needs, and its preservation history.  This information plus the actual content forms a complete “information package,” which comes in the flavors of SIPs, AIPs, and DIPs, depending on the object’s role in a repository, as discussed in the OAIS reference model.  The metadata that comes in each of these flavors is referred to as the Preservation Description Information (PDI). (Note: DIPs do not always have PDIs since they are the distribution versions.)

We know from the OAIS model that a PDI categorizes a digital object’s metadata into reference, provenance, context, and fixity categories.  METS is capable of fulfilling these metadata requirements with corresponding sections in each METS document:

• Descriptive <dmdSec>
• Administrative <amdSec> (covers provenance and rights)
• File Groups <fileGrp> (lists any and all files that comprise the digital object)
• Structural Map <structMap>
• Structural Links <structLink>
• Behavior <behaviorSec>

It is important to realize, however, that according to the METS standard, the only required part of a METS document is the Structural Map.  So in order for METS to be effective when applied to preservation, there must be information in each of these sections (FYI – a truly complete METS file will also include a header <metsHdr>).

So where do we get this information to fill up a METS file?  The answer is PREMIS.

METS and PREMIS – A Perpetual Preservation Honeymoon

You may recall that PREMIS is also an XML schema that has been developed for preservation metadata.  The PREMIS structure is based on entities and semantic units that will harbor information about a digital object that is necessary for supporting and recording digital preservation actions.

What’s important here is that PREMIS will sit inside the METS document.  You can see an example of this here.  All of the preservation information will be present in the PREMIS file, and by nesting the PREMIS data into the METS file, the metadata becomes transferable to other repositories.

The flexibility of both of these schemas implies that there are variations and complications with integrating PREMIS and METS.  The Library of Congress created a working draft of guidelines for this process, which is viewable here (PDF, 25K).

Helpful METS Resources

• METS Primer (Revised 4/2010) (PDF, 1.53MB) – Readable, and has color images and examples.

• PREMIS in METS toolbox, information about the project here.

• METS Creation Tools.

McDonough, J. (2006). METS: Standardized Encoding for Digital Library Objects. International Journal on Digital Libraries, (6)2, 148-158.

Video Digital Preservation Workshop

On Monday, I was thrilled to attend a workshop entitled Digital Preservation for Video, presented by Linda Tadic for Independent Media Art Preservation (IMAP) .  The workshop was held in San Francisco at the Bay Area Video Coalition (BAVC).  The scope of the event was to cover some of the key considerations in digitizing video and creating a digital preservation program at the DIY level (i.e. without a huge IT department backing you up).  A few of the institutions represented by attendees included BAVC, the Pacific Film Archive, the California Institute of the Arts, the California Academy of Science, the Sierra Club, the San Francisco Symphony, and the California Film Institute.

Prior to this workshop, I hadn’t had a great deal of exposure to the digital preservation challenges of moving visual materials.  In fact, I confess that I hardly knew anything about the current physical formats used for video storage, nor much about the hard work that is necessary for digitizing them.  Most of the attendees have done their share of digitizing moving images (or of outsourcing the digitization), and I think that most of us were there to explore the answers to the question of “now what?”

The Move to File-Based Video Storage

Physical moving image storage formats are on death row.  We spent the bulk of the morning going over the characteristics of different physical media and their expiration dates, which served as an effective motivator for digitization and instilling all but panic among the attendees.

Photo by Serena Epstein, CC Attribution-Noncommercial-Share Alike 2.0 Generic license

Unlike paper, the magnetic tapes, reels, and discs that moving images are physically stored on are on a very tight deadline; aside from succumbing to format obsolescence, most of the media is reaching the end of its life expectancy, after which the images on them will simply not exist anymore.  To give some examples of formats that I was more familiar with, the life span of VHS is approximately 15 years, while MiniDV, DVCam, and Video are 5-10 years.  This illustrates a point that in some cases, it isn’t necessary to digitize the oldest things first.

Digitization is arguably the increasingly best preservation option for some of these formats, and it is important that the road to digitization doesn’t result in a dead end.  That is why we need to ensure that once digitization has occurred, there is a digital preservation plan in place to ensure that the video content will continue to survive, especially since the original physical sources of the content will be dead in short while.

Indeed, we are observing a shift from format-based physical video storage to the file-based storage of digital video content.  Preservation will no longer be about making the tapes last as long as possible, but by caring for the digital files representing the content that the tapes once held.

Preservation Concerns for Digital Video

I appreciate how Linda was adamant in reminding us that digital preservation is not a one-time fix for digital video longevity.  She was very clear in telling us that it requires a constant guardianship consisting of a deliberate, scheduled management of the digital files.  To use her phrasing, there is no “store-and-ignore” solution.  Preservation activities involve keeping file formats current so that they can be accessed by the software of the now.  It also involves exercising the hard drives that your files may be stored on and not letting them sit idle for more than 6 months.  It requires diligent updating of the files’ accompanying preservation metadata so that changes to the files can be tracked and managed.

Linda also stated what nobody likes to hear about digital preservation: that there is no one way to do things, and that there is no one set of instructions to follow that will help you save your content.  As with all file types, the preservation decisions you make will depend on your content, your files types, your storage, and your intended access methods.  So, in the case of making storage selections and creating a plan, knowledge is power.  I’ll try to summarize some of the key points covered.