I’d like to propose a session to discuss what the digital humanities can (and cannot) do with binary image and sound files.

Part of my frequent dissatisfaction with recent experiments in data mining, computational distant reading, etc., I think, comes from a tension that’s been in place in DH for a long time. The same set of McGann-Druckerites (a group among which I count myself) who were raised up on a critical methodology that emphasizes the centrality to literary interpretation of material texts, bibliographic codes, etc.—those aspects of texts that are extremely difficult to capture in digital surrogate form—now finds itself in the midst of a digital scene in which plain text, whether marked up or not, is king. As often as not, scanned or photographed images more accurately capture the material situation of a book than marked text does—and text files, though they can operate as “phonotext” with the potential to make sound, as Garrett Stewart points out, cannot embody the sounded performance of poetry in the way audio recordings can. “Multimedia” was once the buzzword that best captured the promise of computers in culture, but those messy image, sound, and video files strewn all about the Internet have proven beyond the purview of most text-heavy DH research.

Some recent attempts to deal more readily with image and sound in DH suggest to me that there might be more we can do on this front:

• Alex Gil‘s work as a Scholars’ Lab fellow on an edition of Aimé Césaire, which aimed to meticulously recreate the original placement and appearance of text in Césaire’s work;
• Lev Manovich‘s work on large sets of images, memorably presented at a Scholars’ Lab talk in fall 2011;
• and Tanya Clement‘s work on using digital tools to illuminate the meaning of sound patterns in the work of Gertrude Stein.

I’m surely unaware of lots of great work being done on this front, and one of the purposes of the session would be to have a bit of a show-and-tell of that existing work. I’d also like to have a conversation about the possibilities for and limitations of multimedia data in relation to the digital humanities. How can we conceive of image and sound files not just as black boxes to be surrounded by metadata, but as data (or capta, as the case may be) in their own right? Do such files offer enhanced versions of the texts we work on, or are they in many respects impoverished? And of course, what knowledge can we actually produce by playing around with the multimedia treasure trove of the Internet?