Due to the susceptibility and challenges of both digital and analog carriers, data must be periodically moved from one carrier to another within a preservation process. When analog data is migrated from its original carrier to a new digital carrier, the analog data is ultimately transformed through the process of sampling. Challenges are then posed to authenticating the accuracy of such a migration. Despite the perceptual exactness of an analog source to its digital copy, the analog data and the digital data are never exactly the same. However, in the realm of file-based digital-to-digital migration, exactness can be achieved and evaluated. Within the entirely file-based environment, checksums and data comparison tools can verify that two copies are exact matches or reveal their deviation in a way that is not feasible between analog and digital environments.

Of course, living in a Northwest logging town, it wasn’t as if there were regular Russian classes available, so I turned to the only resource I knew — the children’s section at the County Library. Probably not surprisingly (there must have been earlier anchor babies than I), there was a Russian picture-book dictionary in the stacks, which I happily added to my pile of Mad back issues, Ray Bradbury, and Encyclopedia Brown.

When I got home, I found the prospect to be more difficult than I had imagined. Seems that Russian uses a different alphabet, don’t you know. Luckily, there was a handy transliteration chart in the back that showed what the Cyrillic letters were equated to in American. This was obviously the key I was looking for that would allow me to translate English words into Russian. If D=Д, O=О, and G=Г, then ДОГ was DOG.

I tested my new found knowledge by applying it in reverse, taking the Russian words from the picture dictionary and transforming them into English. To my frustration and confusion, it didn’t work. How could ‘собака’ be ‘dog’? I had already figured that word out. It didn’t make sense. After a few more attempts, I put the defective book aside and moved on to something more immediately satisfying, like Pixie Stix or something.

************************************************

Eventually I figured out the difference between translation and transliteration — that or I was highly skilled at buffaloing my foreign language teachers — but it wouldn’t be the last time I would have to deal with overcoming a conceptual shift in order to learn a new skill. I think about this a lot not only because the speed of changing technologies has presented a steady stream of shifts within my lifetime, but also because of the huge conceptual shifts that are a part of media archiving and preservation practices. Of course we all are quite familiar with the differences between best practices within traditional paper archiving and those required for time-based media — though we surprisingly still have to hammer away at those to be acknowledged.

Of great help along with that hammer have been other tools developed over the years to specifically address the needs of media preservation: The AD Strips and other resources developed by Image Permanence Institute, the FACET tool and guidelines out of Indiana University, optical scanning of mechanical audio carriers developed through collaboration of Library of Congress and Lawrence Berkeley National Laboratory, the development of new metadata schemas such as PBCore that better address the variability of how media is produced and used. It seems almost banal to mention them now, like writing an ode to a ratchet set, but isn’t that a whole lot better than trying to make do with a couple pairs of pliers in a space that’s too close to get very much leverage or movement in?

Which now makes me think, maybe it isn’t entirely a reconceptualization of ideology that is important here, but also a initial conceptualization of tools and processes. We are working towards the same basic goal — getting the nut off the bolt; establishing access to and maintaining existence of an object and its content — but the avenues towards that endpoint are quite different. Which now makes me think about the current major conceptual shift in archiving and preservation: the shift in practices dealing with analog media to those dealing with file-based and born digital media. Perhaps the struggle to establish standards and practices is not entirely a problem of thinking about the management of digital objects differently than physical objects, but also an issue of not having a full set of tools for inspection and management.

We make use of a number of tools such as dvgrab, ffmpeg, DATXtract, Live Capture Plus, MediaInfo, and PBCore, but most of these were not developed specifically for archiving needs. As a result, AVPS has made the development of tools and resources a major component of our work. We have been involved in developing DVAnalyzer, which allows a user to inspect and analyze the quality of a DV stream captured over firewire from DVCam; BWF MetaEdit, which allows a user to view and edit the embedded metadata in a Broadcast WAV file; PBCore Instantiationizer, which automates the creation of PBCore instantiation elements based on the embedded metadata in file-based assets; and a number of other internal tools or resources in development that can be used to assess or manage file-based collections.

Of course we aren’t the only ones working on this. For example, METRO’s new book Digitization in the Real World is full of strategies and methodologies for digital collection management, and the Dance Heritage Coalition has been doing innovative work on the development of online cataloging utilities and access. What is needed is a greater allocation of efforts and resources directed towards developing these new tools before too much is lost. The make up of digital media will not allow us to wait 50 years before addressing persistence and continuing access. Мы должны подействовать теперь, comrade.

— Joshua Ranger

After Chris gives a lay of the land of the challenges involved in conceptualizing and preserving these hybrid formats within the the context of audiovisual archiving, David and Richard will present real life case studies where innovative approaches and solutions were developed to manage migration schemes. David’s presentation will include the initial findings from some controlled experiments he has been performing on the capture of miniDV tape using variable hardware and software configurations. Enabled by DVAnalyzer, David has been able to run comparative analytics on the error rates and bit structures in the data streams of multiple captures of the same tape. The early results are fascinating and the planned further testing looks to be equally revelatory and instructive. Keep an eye on avpreserve.com for further details on David’s testing and for future updates to the DVAnalyzer tool.

Richard’s presentation promises to be highly instructive as well, as he discusses three separate tape-to-file migration decisions and outcomes implemented at the BBC. The migration of media-based born-digital content presents a number of scenarios that are dependent on parameters of the source original and of the ultimate goals for the use of the resulting file, and Richard gives an excellent outline for the decision making process. Follow the developments at PrestoPRIME (http://www.prestoprime.org/) for future release of Richard’s report and for essential guidance on audiovisual preservation.

He even says “videos” instead of “movies”! [swoon]

A follow up column on the experience of migrating his MiniDV content (“Moving Taped Past to Hard-Drive Future”) takes a more technical than emotional bent on the process (though it does end with a stirring call-to-arms for everyone to start similar projects). I hope you read the article, but, in short, Pogue ran into some roadblocks with his first plan and had to go back and revise his strategy. I think it’s telling about the challenges involved in audiovisual preservation, especially as we move more into the digital realm, that what seems like a simple process (stick the tape in and let the machines do their work) brought some consternation to a tech expert (and one who gets personal emails from Steve Jobs, none the less). Yes, people should start working on reformatting their personal media collections, but there are a number of avenues, and branches of options off of those avenues, in deciding how to best do it, and not everyone has the resources at hand to help in those decisions.

I guess this is the point, then, where I should bring up some resources for people to consult about the reformatting of DVCam and MiniDV tape. They are touchy formats due to their small size and the makeup of the tape and binder, and it’s true that the formats are trending towards obsolescence, but there’s a lot of unique content out there shot on DV that’s going to need taking care of. David Rice has written a great piece about the ins and outs of migrating DV tapes, expressing the importance of capturing it as a data stream rather than as a video signal (“Digital Tape Preservation Strategy: Preserving Data or Video?”). Additionally, our free and open source DV Analyzer application is a simple tool that anyone can use to review the metadata in the DV data stream that’s carried over during a Firewire migration of DVCam or MiniDV.

(Allow me one geek moment here in response to Pogue’s article: Final Cut Pro can carry over the date and time metadata, but only if the captured stream is not re-transcoded during within the process. This may be accomplished by selecting File>Export or pulling the file from the Capture Scratch directory instead of using the Export>Using Quicktime Conversion option.)

The DV Analyzer tool also identifies and lists error codes in the DV stream that occur during playback. The garbbled video that Pogue mentions is likely a result of error concealment performed by the playback device — most typically misread data in frame being patched up with data from the previous frame — and DV Analyzer would provide the error detection code for those sections that could then be analyzed to see if it can be determined what the cause of the error was. Sometimes this is due to degradation, but often enough these errors are caused by the touchy nature of DV tapes. Many times the same errors will not appear if played through the deck again or run through another deck. Further information can be found at http://www.avpreserve.com/dvanalyzer/what-does-it-analyze/ or under the Case Studies section on the DV Analyzer main page http://www.avpreserve.com/dvanalyzer/.

A final important point from the article is how Pogue’s experience underscores how much we have to monitor and advocate for the tech companies to better understand and maintain the capabilities that enable preservation and access. The idea that professional film and television editors don’t need to know the date of when something was shot is fairly ludicrous. I doubt a news program would feel all right using footage for a story they can’t properly identify, or that film editors wouldn’t want to be able to find content from a certain date of shooting. Outside of this, the date stamp, timecode, and other metadata are absolutely necessary for the authenticity of archival materials, especially in matters of research or, increasingly, in legal matters (see this Times article about metadata as evidence). It’s maybe a tad idealistic to think we can always have an effect on corporate decisions, but a positive point is that a little vocal activity did do some good in getting Firewire ports back after Apple decided to remove them. Sure Firewire dependent devices were severely decreasing in manufacture, but there is so much out there that has been produced on those devices, and the future ability to access or capture that content for preservation extends well beyond the end of manufacturing.

— Joshua Ranger

DV tapes are not a preservation medium. As the support for DV tape technology gradually declines, it is imperative to begin migrating DV content from tapes to files for storage and ongoing preservation of the content. In the migration process, the technical nature of DV tapes frequently causes issues generally described as “glitchy” or “finicky”. DV tapes often perform differently from one play to the next and across playback devices. DV Analyzer provides an automated way to monitor and report on the accuracy of the migration process.

DV Analyzer is a technical quality control and reporting tool that examines DV streams in order to report errors in the tape-to-file transfer process. DV Analyzer also reports on technical metadata and patterns within DV streams such as changes in DV time code, changes in recording date and time markers, first and last frame markers within individual recordings, and more. To those concerned with preservation and archiving, this means that you now have the ability to automatically monitor integrity during reformatting of DV tapes and extract meaningful metadata from DV files.

AVPS is pleased to offer DV Analyzer free of charge to the AV preservation community which we serve. DV Analyzer is licensed by AudioVisual Preservation Solutions under the open source GNU General Public License.

For more information on DV Analyzer, or to download it go to: http://www.avpreserve.com/avpsresources/tools

To request specific information about the product go to: http://www.avpreserve.com/you/

Or e-mail us at info@avpreserve.com

We look forward to hearing your comments and suggestions about DVAnalyzer.

Be sure to catch up with David Rice, Chris Lacinak, or Joshua Ranger of AVPS at the AMIA conference this week. They can also answer your questions about DVAnalyzer.

The three sessions that we will be chairing and presenting at are as follows:

• Harnessing Collective Knowledge: Three Case Studies of New Collaborative Tools

Chris Lacinak (AVPS), Richard Wright (PrestoSpace), Mick Newnham (National Library of Australia)

A discussion and viewing of three new exciting projects – PrestoSpace’s wiki, National Library of Australia’s Mediapedia, and AudioVisual Preservation Solutions’ CEDAR – each of which provides open, collaborative, online resources that harness the expertise within the community through the use of centralized sites.

• Accessioning and Managing File-Based Born Digital Content

Chris Lacinak (AVPS), Grace Lile (WITNESS), Brian Hoffman (NYU), Dirk Van Dall (Broadway Video)

This session brings four experts and two case studies to the table to offer insights into the challenges that born digital file based video brings to your archive and offers strategies for managing it.

• Digitizing 102: Video Digitization Workflows and Challenges

David Rice (AVPS), Angelo Sacerdote (Bay Area Video Coalition), Skip Elsheimer (A/V Geeks LLC)

This session is a primer on the planning process for video digitization projects. It will examine case studies for working with damaged or ‘not-to-spec’ materials, address documentation practices for preservation workflow, and stress how to perform quality control on the process and the results.

Please join us!  We hope to see you there.
For more information about the Annual AMIA Conference:
www.amiaconference.com