Tools

Archival Management System (AMS)

The Archival Management System (AMS) is a multi-functional tool that supports management of the digitization workflow, especially useful for projects involving multiple departments or organizations. Existing functions in this open source software include: 1) Aggregation and normalization or refinement of collection inventories. Using tools such as Open Refine and MINT, data cleanup can be done analytically and in bulk, though the system also allows individual record-level editing. 2) Prioritization and selection of items for digitization. 3) Scheduling and system alerts to inform users when it is time to begin packing materials for shipping to digitization vendors, shipping dates, when the vendor has completed a batch, and when the materials will be shipped back. 4) Record level search which includes an audio or video player for playback of the digitized item. 5) Bulk ingest of technical and preservation metadata generated by the vendor. 6) Dashboard reporting that tracks project progress, including number and types of items, percentage of project completed, departments/locations, and other pertinent information. AMS was originally developed to support the digitization of 40,000 hours of audiovisual materials from 120 public media stations as part of the Corporation for Public Broadcasting American Archive project, and AVPreserve has also customized an instance for the Flemish Institute for Archiving (VIAA) to manage nationwide digitization from broadcasters, universities, and museums. The source code is available for download from GitHub, and AVPreserve can also provide services to customize it for project particulars such as organizational structure, workflow specifics, language, other material types (such as newspapers), reporting, systems integration, and more. A sandbox instance will be available here soon.

Catalyst Inventory Software

Catalyst is an innovative method of creating item-level inventories of audiovisual collections. The process uses a team of photographers onsite to image each item in a collection, capturing all information-carrying sides of a cassette/reel/disk, its housing, and any paper inserts. The photos are uploaded daily to our central server where they are sorted into item records and fields for Unique ID, Location, and Format are automatically generated. After this the database records are immediately accessible by a team of offsite catalogers who use the images to enter further metadata. Taking advantage of automated processing and minimal datasets, even a small team can work through hundreds or thousands of items a day. Catalyst data can be exported to generate reports for preservation planning and selection, or to become the basis of a finding aid or more complete catalog record. The benefit of the photos is that materials can be searched for and reviewed without the need to pull tapes until correct items are identified, minimizing handling and staff time. Also further descriptive cataloging can take place at a more reasonable pace or after reformatting has been completed. The Catalyst Inventory software is currently only available as part our inventory services, but screenshots are posted below or here and here.

MDQC

MDQC reads the embedded metadata of a file or directory and compares it against a set of rules defined by the user, verifying that the technical and administrative specs of the files are correct. This automates and minimizes the time needed to QC large batches of digitized assets, increasing the efficiency of managing digitization projects. MDQC can be used on any file type supported by ExifTool and MediaInfo. Both ExifTool and MediaInfo will need to be installed on your system in order for MDQC to work.

Fixity

Fixity is a utility for the documentation and regular review of stored files. Fixity scans a folder or directory, creating a manifest of the files including their file paths and their checksums, against which a regular comparative analysis can be run. Fixity monitors file integrity through generation and validation of checksums, and file attendance through monitoring and reporting on new, missing, moved and renamed files. Fixity emails a report to the user documenting flagged items along with the reason for a flag, such as that a file has been moved to a new location in the directory, has been edited, or has failed a checksum comparison for other reasons. Supplementing tools like BagIt that review files at points of exchange, when run regularly Fixity becomes a powerful tool for monitoring digital files in repositories, servers, and other long-term storage locations.

AVCC

AVCC is a set of forms and guidelines developed to enable efficient item-level cataloging of audiovisual collections. Each module (Audio, Video, Film) includes individualized data entry forms and reports that quantify information such as format types, base types, target format sizes, and other data critical to prioritizing and planning preservation work with audiovisual materials.

Based on years of experience with how audiovisual collections are typically labeled and stored, AVCC establishes a minimal set of required and recommended fields for basic intellectual control that are not entirely dependent on playback and labeling, along with deeper descriptive fields that can be enhanced as content becomes accessible. The focus of of AVCC is two-fold: to uncover hidden collections via record creation and to support preservation reformatting in order to enable access to the content itself.

More information and a request form for access to your own module is available through the METRO Keeping Collections website at http://keepingcollections.org/avcc-cataloging-toolkit/. AVCC is currently in Beta form and has been designed in Google Docs. Currently only the Audio and Video modules are available. The development of a more stable web-based database utility is anticipated in early 2014, and your feedback in testing this current version will help. Please direct any questions to AVPreserve Senior Consultant Josh Ranger via the AVCC or avpreserve.com contact forms.

interstitial

interstitial is a tool designed to detect dropped samples in audio digitization processes. These dropped samples are caused by fleeting interruptions in the hardware/software pipeline on a digital audio workstation. The interstitial tool Follows up on our work with the Federal Agencies Digitization Guidelines Initiative (FADGI) to define and study the issue of Audio Interstitial Errors.

interstitial compares two streams of digitized audio captured to a digital audio workstation and a secondary reference device. Irregularities that appear in the workstation stream and not in the other point to issues like Interstitial Errors that relate to samples lost when writing to disc. This utility will greatly decrease post-digitization quality control time and help further research on this problem.

AVI MetaEdit & reVTMD

AVI MetaEdit supports embedding and validating metadata in RIFF-based AudioVisual Interleave format (AVI) video files. AVI is currently the target format for creation of Preservation Masters within the Digitization Services Branch at the National Archives.

reVTMD is an XML schema tailored to include fields that address the creation and long term management of reformatted videos, especially with the cultural heritage community. It is a concise subset of the large array of technical metadata available, structured in a way to make it highly usable for accessing and managing all types of video files beyond AVI.

Both tools were developed by NARA in collaboration with AudioVisual Preservation Solutions. AVI MetaEdit is available for download at NARA’s GitHub site, and reVTMD is available on NARA’s website.

BWF MetaEdit

BWF MetaEdit is a free, open source tool that supports embedding, validating, and exporting of metadata in Broadcast WAVE Format (BWF) files. BWF MetaEdit is available for download at SourceForge and was developed by the Federal Agencies Digitization Guidelines Initiative to support its guideline for embedded metadata in the bext and INFO chunks. The application was developed by AudioVisual Preservation Solutions.

Users of BWF MetaEdit can:

  1. *Import, edit, embed, and export specified metadata elements in WAVE audio files
  2. *Export technical metadata from Format Chunks and minimal metadata from bext and INFO chunks as comma-separated values and/or XML, across a set of files or from individual files
  3. *Evaluate, verify and embed MD5 checksums, as applied to the WAVE file’s data chunk (audio bitstream only)
  4. *Enforce the guideline (above) developed by the Federal Agencies Audio-Visual Working Group, as well as specifications from the European Broadcasting Union (EBU), Microsoft, and IBM
  5. *Generate reports that show errors in the construction of WAVE files
  6. *Choose from command line and GUI, for Windows/PC, Macintosh OS, Linux.

A Study of Embedded Metadata Support in Audio Recording Software

This report presents the findings of an ARSC Technical Committee study, coordinated and authored by AVPS, which evaluates support for embedded metadata within and across a variety of audio recording software applications. This work addresses two primary questions: (1) How well does embedded metadata persist, and is its integrity maintained, as it is handled by various applications, and (2) How well is embedded metadata handled during the process of creating a derivative? The report concludes that persistence and integrity issues are prevalent across the audio software applications studied. In addition to the report, test methods and reference files are provided for download, enabling the reader to perform metadata integrity testing.

PBCore Instantiationizer

PBCore Instantiationizer is part of a toolset for conforming extracted technical metadata to the PBCore 1.2.1 metadata standard instantiation element set. The automated approach to extraction and conformance of this element set allows for consistent application of standards to fields that require a strict level of control for usability while also relieving the burden from the cataloger to document what can be a large dataset that can be human-readable unfriendly. This draft version of the Instantiationizer Toolset contains an XSL stylesheet as well as a Mac drag-and-drop application for an even simpler conformance process.

**New Version 1.2 Available**
The update of the PBCore Instantiationizer tool to version 1.2 presents refinements that improve usability and user control. Follow the links below for downloading elements of the Toolset and further information on the latest version and the development and use of the tool.

DV Analyzer

DV Analyzer is a technical quality control and reporting tool that examines DV streams in order to report errors in the tape-to-file transfer process. DV Analyzer also reports on technical metadata and patterns within DV streams such as changes in DV time code, changes in recording date and time markers, first and last frame markers within individual recordings, and more. To those concerned with preservation and archiving, this means that you now have the ability to automatically monitor integrity during reformatting of DV tapes and extract meaningful metadata from DV files.