Museums are increasingly turning to video as an easy and inexpensive method of documentation and of developing valuable primary resources, but numerous barriers interfere with scholars’ use of the materials. While technical difficulties are diminishing and attitudes and traditions are shifting, issues of intellectual property rights and information retrieval remain. Publishing video content online first demands a comprehensive rights assessment and clearing. Even more challenging is dealing with information retrieval in time-based and visual-aural resources, which makes finding a specific bit of data in mounds of footage a daunting task. Transcription enables free text search but is costly, time consuming and not entirely effective, delivering only the verbal aspect of original content. Descriptive metadata can be tied to time codes, adding broad or granular indexing to support navigation and retrieval, but the same drawbacks apply. Possible solutions include crowdsourcing, software tools and collaborative initiatives, applying metadata standards yet to be established and adopted.

museums
museum informatics
digital video files
indexing
information retrieval
metadata
access to resources
scholars

Bulletin, February/March 2012


A Haystack Full of Needles: Scholarly Access Challenges in Museum Video Archives

by Heather Nodler

When it comes to the production of video content in museums, the current era is, in a Dickensian sense, the best of times and the worst of times. Owing to several factors, video has become an increasingly popular on-the-fly documentation method for many museums. Recording events on video and then presenting them online is a much faster, simpler and less resource intensive process (in some respects) than ever before. Factors influencing this development include the following:

  • Proliferation of “prosumer” and mobile digital video recording devices such as DSLR cameras, HD camcorders and tablets and phones built upon the Android and iOS platforms.
  • Availability of accessible and intuitive video authoring tools such as iMovie and YouTube Video Editor, as well as increased familiarity with and availability of prosumer grade versions of traditional nonlinear video editing tools like Avid and Final Cut Pro.
  • Free or low-cost online distribution channels such as Vimeo, YouTube and Amazon Web Services.

Today, even museums possessing extremely limited technological infrastructure have the means to record and broadcast such events as interviews, lectures, symposia and gallery tours. These recordings, while typically more DIY than professional productions, nevertheless can make a vital contribution to an institution’s archival repository, yielding mountains of invaluable primary resources for use by scholars. 

The Predicament: Barriers to Access
In spite of the increasing abundance of museum-produced video content, however, many barriers exist that prevent scholars from employing these video recordings in their research. Video-specific barriers to access differ radically from those associated with traditional scholarly use of textual resources and thus require information professionals to formulate special approaches to make video content more attractive and accessible to scholars.

Video’s Technical Difficulties
When discussing the challenges attendant to scholarly use of video, it is tempting to emphasize the medium’s technical barriers. These difficulties can be formidable, to be sure, largely due to the following realities:

  • Museums must possess either digitally reformatted versions of, or playback equipment for, born-analog content, for example, Betacam tapes or 16mm film, much of which may be physically deteriorating.
  • Digital video content delivery requires both large-scale file storage – easily climbing into the terabytes for relatively small collections – and sufficient bandwidth for online delivery. 
  • Digital video files are massive and heterogeneous and, therefore, demand considerable, managed digital preservation efforts from collections managers, including ongoing reformatting, migration and inspection of data.
  • Digital video file formats can create interoperability nightmares for all users, including the risk of format obsolescence and problems arising from use of highly compressed, proprietary formats. Heterogeneous file formats are also commonly difficult to play back, to publish, to preserve and to share across different post-production and editing environments. 

Fortunately, significant progress is being made to help even those museums with limited resources overcome video’s technical access barriers. Storage and bandwidth are becoming less expensive; museums are adopting more sophisticated digital or media asset management systems and metadata schemas custom tailored to video’s unique needs, such as the PBCore (Public Broadcasting Metadata Dictionary Project; www.pbcore.org); subscription services are emerging to help small museums with their managed digital preservation efforts and work is being done to encourage the use of more open and cross-compatible file formats.

Scholars’ Attitudes toward Using Video in Research
Notwithstanding those purely technical access barriers, however, video poses numerous content-specific challenges to scholars – arising mostly from the medium’s time-based and visual-aural nature. These challenges must be addressed before video can be effectively deployed as a scholarly resource.

Traditions of scholarly practice have largely inhibited the use of video in research. Questioning whether such a malleable, context-dependent and author-influenced medium could possibly afford authenticity and objectivity, researchers on the whole have been reluctant to employ video as a primary resource. These attitudes may be shifting, however, as a new generation of media-savvy and web-acclimated scholars emerges and as museums overcome the aforementioned technical access barriers. As archivist Rick Prelinger, who has made video content accessible to scholarly audiences via the Internet Archive and other projects, remarked to me, “In our experience, easy online access, downloadability and student use have sparked new scholarly interest in archival moving images… [and] widely accepted and easily editable formats and frequent rederivations also encourage reuse.” [1] However, for scholars actively seeking and using video content in their research, other access barriers, including intellectual property rights and information retrieval issues, exist and must be addressed by museums and other repositories.

Managing Rights for Video
Intellectual property issues threaten to impede scholarly access to video content for a number of reasons: 

  • Video content can be extraordinarily rich and complex, involving numerous authors’ works and potentially many differently owned assets that require separate licensing agreements, including music, images, clips of footage and the publicity and privacy rights of multiple individuals.
  • Because so many different rights are at stake, efforts to make video available entail an increased risk of having to contend with orphan works (that is, works for which no rights holder can be found) or refusal by individual rights holders to allow access and reuse of their content.
  • In the past, museums and other repositories may not have obtained the rights necessary to make video content accessible worldwide and online, thereby limiting opportunities for discovery and reuse.

It is advisable that any repository aiming to publish its video content online –encouraging scholars to reuse its content – first conduct a rights assessment of the content and clear as many necessary rights as possible beforehand, whether with content producers or people whose likenesses or works are represented in video footage. In certain cases, museums may assess their use of rights-protected content within the context of a particular video as making a fair use of that content. Museums may find some guidance in the rights assessment process for certain types of video in the Documentary Filmmaker’s Statement of Best Practices in Fair Use (www.centerforsocialmedia.org/fair-use/best-practices/documentary/documentary-filmmakers-statement-best-practices-fair-use). Repositories can also simplify scholars’ works by publishing their video content in a fashion that encourages reuse, for example, under a “CC BY” Creative Commons license. The rights assessment and clearance process can be arduous; therefore, the most effective approach is to clear necessary rights at the point of creation, that is, when the event is being recorded, as opposed to waiting until a scholar requests access and reuse.

Video-Specific Information Retrieval Issues
Perhaps the greatest challenge yet to be overcome in the provision of access to video content is achieving a level of description and information retrieval sufficient to negotiate the medium’s inordinate demands on time. Due to video’s time-based nature, locating and retrieving a specific idea, event or piece of information within hours of footage can be a needle-in-a-haystack proposition. And unlike text, which can be readily scanned and made searchable using OCR – or even still images, which can be visually analyzed and easily cataloged – video typically can only be fully comprehended and described after having been experienced in real time. This situation promises to become more onerous as the volume of video content increases and video supplements textual forms of museum documentation. Several approaches to providing granular research access to video have been explored in recent years. Still, much progress remains to be made in this area.

Treating Video as a Text
Historically, a common approach to providing access to video has been to create text-based transcriptions of speech and significant events. While this approach does afford information retrieval via full-text search and also yields a document more compatible with traditional scholarly modes of practice, it can be costly and time-consuming for the average museum to produce video transcripts. The process requires a professional transcriptionist or trained volunteer to view video content in some multiple of real time. Depending on the complexity and audibility of the conversation and events, as well as the transcriptionist’s typing speed, one hour of a standard video interview can take between three to six hours to transcribe. Great advances have been made in the area of speech-to-text transcription software and other transcription tools, and they can certainly aid a transcriptionist, but in cases involving multiple speakers, thick accents or events that demand more context than simply the words being uttered, a human transcriptionist or editor is nearly always still required, particularly if transcripts must be verified for accuracy before being delivered to scholars.

Access purely via transcription also threatens to deny video content much of its meaning, insofar as it is impossible to capture the nuances of visual and aural information with punctuation. Information conveyed via physical gesture, facial expression, vocal intonation, pauses, music, ambient sound, camera work and other visual or aural events can be missed entirely, sometimes even occluding meaning or subtext. Transcriptionists will often insert their own descriptions or marks of emphasis to convey these events, but their descriptions can be highly interpretative and may only capture a fraction of the information that would otherwise be conveyed in a direct experience of the video content. This limitation is particularly true when the content engages the viewer in non-verbal, sensory experiences of visual art, graphics or music.

Partially illustrating this problem, the following transcript segment comes from an interview that the Menil Collection conducted with the artist Frank Stella as part of the Artists Documentation Program (www.adp.menil.org). The transcription, while illuminating, loses much of the richness of meaning conveyed in the video clip itself (adp.menil.org/?page_id=31). In this segment, two art conservators, Brad Epley and Elizabeth Lunning, attempt to reconstruct the techniques that Stella used decades ago on a painting. In the process of the interview, the artist reformulates his recollection of those techniques. 

Transcript only:

Brad Epley: Can I talk about this one a little bit? 
[Referring to Takht-i-Sulayman]
Frank Stella: Yeah.
Brad Epley: One of the obvious distinctions, aside from color, is that there’s priming applied underneath all of these layers? With the bands…
Frank Stella: Not that I know of, no. 
Brad Epley: Oh?
Frank Stella: That’s unsized canvas. I mean…
Brad Epley: I noticed in some areas it looks like there’s a white priming kind of bleeding out a little bit from underneath.
Frank Stella: Ah, yeah you’re right. Oh, I know. I think we did for the fluorescent colors. We put the white to make…
Elizabeth Lunning: Because it would make them more fluorescent in a way.
Frank Stella: Right. Right. You’re right. 

Figure 1 Figure 2 Figure 3 Figure 4

These still images from the Stella interview illustrate how much meaning is lost when video content is translated to simple transcription.

Creating Semantically Rich Video Metadata
Another descriptive approach to providing time-based access to video, one that has for years been used by media archives in the film industry, television broadcasting and large-scale moving image archives, is to exploit the time-based nature of the medium and describe footage at the timecode or segment level, creating logs or content indices in order to help end users locate and retrieve a specific idea, event or piece of information within the video. In this context, timecodes point to specific video chapters, clips or frames – a familiar and minimal example of this approach is a DVD menu. Footage can be indexed as broadly or as granularly as required by a particular project. Each chapter, clip or frame is assigned descriptive metadata such as a content summary or abstract and/or controlled vocabulary metatags/keywords. The more deeply footage is indexed (for example, yielding timecode-level records), the more semantically rich the description can become. Once they have been created, timecode-level records can be cross-referenced with each other or linked to external resources, including textual documents, images, other media assets or even other collections and then synchronized with the video content to which they refer. The result can be robust, rich media presentations and easy navigation of video by scholars.

Like transcription, manual cataloging of video at the granular level of timecodes requires a significant investment of time and human application, as well as some degree of interpretation on the part of the cataloger. Moreover, because video can be described in so many dimensions, the lens through which a cataloger views it or through which the end user seeks to access it can vary widely. For instance, a cataloger or scholar might create separate indices to track or find each of the following: 

  • individuals speaking or shown onscreen
  • topical themes
  • references to historical names or subjects
  • editing events (close-up shots, fades, jump cuts, B-roll additions and so forth)
  • facial expressions
  • physical gestures
  • ambient sounds
  • music
  • colors
  • lighting
  • movement
  • narrative arc
  • mood
  • location or setting

Additional indices can be created to record timecode-specific administrative metadata, such as intellectual property rights or information about clips, such as provenance, used in a video. In all of these cases, the more that data is standardized and made interoperable across videos and repositories, the more likely this type of description can become the norm. Identifying and adopting structural and descriptive metadata standards for video is key.

The possibilities for indexing video content are virtually limitless. This effort could be made less time consuming for museum and other information professionals, and also made to employ more subject expertise, by employing a crowdsourced approach, whereby researchers are empowered by video repositories, who would provide them with the necessary tools and instructions in the use of descriptive standards to create their own custom indices, vet catalog records and annotate video stills or clips.

Finding Inspiration in Emerging Tools and Initiatives
Several tools and initiatives have been developed or are in the process of being developed to allow museums and other repositories and scholars to describe and use video in the manners described above – to search, retrieve and annotate video content. In no way is this an exhaustive inventory, but here are a few projects worth mentioning:

  • Glifos Social Media (www.glifos.com/wiki/index.php/Social_Media) – A web-based software application developed by InfoLib S.A. in Guatemala, Glifos offers tools to mark up video content in reusable XML; allows users to bookmark and share segments of marked-up video via email and social media networks; and supports the synching of videos with transcripts in multiple languages and other media assets. Several video archive projects, including the Artists Documentation Program, the Texas Archive of the Moving Image (which invites users to become taggers) (www.texasarchive.org/library), the Archive of American Television (www.emmytvlegends.org/) and the Kigali Genocide Memorial Centre in Rwanda (www.genocidearchiverwanda.org.rw) have used Glifos to synchronize transcripts, indices, images, Google maps and more with their video content.
  • WGBH Digital Library: Participatory Cataloging Project (www.openvault.wgbh.org) – Building upon a multi-phase project to design, build and sustain “an online media archive content delivery system for research and classroom use,” as well as to present open source tools for other archives to use, Boston’s WGBH Media Library and Archives is recruiting students, scholars and filmmakers to access their rich archive of public television programming and assets and contribute user-generated metadata in order to enhance their catalog records. Through their research, WGBH aims to build “a vetted resource suitable for research use through the metadata contributions” of scholar users and to “work with the digital humanities and emerging digital publishing communities to ensure that streaming archival media is supportable and available for these advancing fields.” [2]
  • CollectiveAccess (www.collectiveaccess.org) – a web-based, open source solution providing a unified platform for cataloging (including time-based cataloging of audio and video), collections management and collections publishing.
  • ArtBabble (www.artbabble.org) – Developed by the Indianapolis Museum of Art, ArtBabble is an aggregation site for video content related to art and artists, principally contributed by museums or other arts-related organizations. Although its video synchronization and indexing tools are exclusively for use by ArtBabble partners and have not been applied to all of its collections, the site offers a powerful vision of the potential for museum video content to be made more widely accessible.
  • AVAN (www.archivenetwork.org) – Still under development, the Audiovisual Archive Network (AVAN) seeks to help archives – and other content producers – preserve and manage their digital video content through its digital repository service. The network will also provide aggregated access to video content, with the intent of increasing discovery of content by scholars, students and the public.

Conclusion: How Can Museum Video Archives Support Digital Scholarship?
While the new wave of museum-produced video promises to create vast and rich resources for mining by scholars, it also entails a steep investment curve on the part of museums. Unless an investment is made in implementing an approach (examples of which have been introduced here) or otherwise made to address the issues, the result may be that museums will continue to collect stores of rich video content, while finding themselves unable to describe and manage the content sufficiently for access and use by scholars.

The solution to this problem lies in the continued collaborative development, spearheaded by larger repositories, of low-cost and robust metadata schemata and tools for transcription, cataloging, search and retrieval of video content. While many systems offer the capacity for rich media description, search and retrieval, the tools readily available to museums – particularly with small-scale collections of video content – are limited at this time.

For all repositories, crowdsourced, time-based cataloging by expert scholars affords the greatest potential for yielding unexpected discoveries and connections across rich information resources. But first, descriptive standards must be adopted, and the barriers to implementation mentioned here must be addressed. Only then can museums and scholars benefit from the promise of simple, affordable and usable video content.

Resources Mentioned in the Article
[1] Prelinger, R. (November 5, 2011). Personal communication.

[2] Courtney, M. (July 22, 2010). Mellon Project Report: Media Digital Library for Scholars. Open Vault. Boston: WGBH Media Library and Archives. Retrieved November 25, 2011, from http://openvault.wgbh.org/blog/2010/07/mellon-project-report-media-digital-library-for-scholars/


Heather Nodler is archivist for the Artists Documentation Program, a partnership among the Menil Collection, the Whitney Museum of American Art and the Harvard Art Museums. In this role, she manages a collection of video oral histories with major contemporary artists, collected for purposes of art conservation and scholarship. She can be reached at hnodler<at>gmail.com or hnodler<at>menil.org.