Data Vis Explorer - Time-based Media (Visualizing Oral History Phase 3)

Short summary

The work involved the iterative design and development of a User Interface to visualise data relevant to the Oral Histories, with various annotations and metadata accessible and variously visualised at different scales of granularity: collection, interview, speech act/exchange, sentence/segment.

The work was grounded in and informed by the prior Phase 2 work and the visualisation conceived and developed by Stef de Sabbata, working with colleagues, including Phase 1/2 Investigation lead, Stefania Zardini, and the insights derived from workshops during Phase 2.

The form developed built on past work by the designers and is intended to provide the basis for a medium/content-agnostic platform for the analysis of time-based media - including in relation to its automated transcription and annotation, and its potential linkage to external sources including collections of relevant contextualising material and media artefacts. It is additionally intended to support user-generated curation and annotation. The design follows principles of ‘subject in transit’ (Butterworth 2022) and, as such, is conceived in relation to a suite of cross-responsive exploratory data visualisation formats that allow fluid movement through complex semantic datascapes.

Research questions

• What forms of visual analysis are required to support a range of expert and non-expert users in the exploration of oral history material in relation to: its transcription, the semantic annotation of its content, its non-discourse elements, its provenance and its material contexts.

• How can contributory participation be integrated to an exploratory User Interface to support editing, validation, annotation in a virtuous cycle model of a digitally mediate Social machine?

People

Andrew Richardson

Conceptualization, Investigation, Methodology, Software, Writing – original draft

Alex Butterworth

Conceptualization, Investigation, Data Curation, Methodology, Formal Analysis, Supervision, Writing – original draft

Stef De Sabbata

Conceptualization, Methodology, Resources, Writing – review & editing

Stefania Zardini Lacedelli

Methodology, Data Curation, Writing – review & editing

Arran Rees

Data Curation

Tim Smith

Investigation

Data sources

Oral history recordings from:

• the Bradford Historical Recording Units (c800)

• Lost Mills and Ghost Mansions project (c 50)

Video interviews:

• ‘Rights of Passage’ Bradford sound systems

• the West Yorkshire Queer Stories

Existing text transcripts from:

• Saltaire Collection of Life Stories (c 10)

• National Coal Mining Museum (c 800)

Investigation methods/ tools/ code/ software

D3.js and React code, developed using an iterative but informal design-focused methodology, Amazon Web Services. Machine Learning methods (Whisper transcription, NLP, SPD,Topic Modelling, entity linkage, etc) inherited from previous phases.

Outputs

• Prototype exploratory data visualisation interface involving novel components integrated to supporting prior codebase. This enhanced interface allows to select, filter and view Topics and Entities created from a combination of topic modelling and manual annotations via OpenRefine. Further additions are the ability to play the original audio sources (indexed to time-coded metadata) and the opportunity to move between different levels of granularity (‘Overview’, ‘Speech event’, ‘Sentences’), getting a sense of the paralinguistic elements of the conversation (rhythm, pace, pauses).

• Chapter of ‘Emergent Histories’ book on data visualisation within the ‘Congruence Engine’ project and contribution to ‘A New Age for Sonic History’ chapter.

Licence

This work is licensed under a Creative Commons Attribution 4.0 License - CC BY 4.0.