MOCO18 Annotation Workshop

Abstract

KIn the last decade, interdisciplinary research projects focused on the documentation and study of choreography and dance including Transmedia Knowledge Base (TKB) and Motion Bank used annotation as a major part of their approach to both analyses of time-based media recordings and presentation of research results. Researchers from both projects continue to develop the methods, approaches and in some cases software platforms supporting annotation of dance recordings. More recently, the Wholodance H2020 project has developed an annotation tool for use in their research aimed at developing and applying technologies to dance. Additionally, several of the MOCO 2017 papers included annotation ‘by experts’ in their research design. With all of these developments, much is still to be discovered and understood about dance annotation. Therefore, the authors of this Practice Works proposal, several who are involved in the projects mentioned above, propose a two-hour workshop to take place during MOCO 2018 in Genoa. This workshop will give the opportunity to try out and compare software platforms and tools, and discuss and debate annotation practice, products and coding schemes. The workshop will be open to another 10-15 interested MOCO attendees making up a total of 20 or 25 people.

Full text

!
1!
MOCO Annotation Workshop
Scott deLahunta
Centre for Dance Research, Coventry University
United Kingdom
scott@motionbank.org
Sarah Fdili Alaoui
LRI-Université Paris-Sud 11
France
sarah.fdilialaoui@gmail.com
Frédéric Bevilacqua
IRCAM – Centre Pompidou
France
Frederic.Bevilacqua@ircam.fr
Rosemary Cisneros
Centre for Dance Research, Coventry University
United Kingdom
ab4928@coventry.ac.uk
Katerina El Raheb
Athena Research Center
Greece
kelraheb@di.uoa.gr
Carla Fernandes
Universidade Nova de Lisboa
Portugal
carla.fernandes@fcsh.unl.pt
Anton Koch
Motion Bank, Hochschule Mainz University of
Applied Sciences
Germany
anton.koch@hs-mainz.de
Thembi Rosa
Universidade Federal de Minas Gerais
Brazil
thembirosa@gmail.com
Sarah Whatley
Centre for Dance Research, Coventry University
United Kingdom
adx943@coventry.ac.uk
Abstract & Keywords: see On Line Submission form
Background. Individual Artist Projects 1999-2014:
The tacit, collaborative and embodied forms of knowledge involved in contemporary dance training, creation and
performance are a challenge to document and communicate to those not taking part in the original performance or
creation work, but who have a vested interested in understanding dance. To address this challenge, several well-known
choreographers including William Forsythe, Anne Teresa de Keersmaeker, Wayne McGregor, Rui Horta, Emio
Greco|PC and Siobhan Davies began to invest in collaborations focused on creating digital prototypes and platforms “to
make, capture, archive, disseminate and study dance”. [1] Annotation of time-based recordings emerged as a
fundamental tool or approach for this work, and it has appeared in various forms between 1999 and 2014. Quite
recognizable, even famous, are the graphic drawings on top of videos integrated into Forsythe’s Improvisation
Technologies multi-media DVD-Rom, developed in the 1990s but first published in 1999. The website Synchronous
Objects (another Forsythe project)1 and Steve Paxton’s Material for the Spine also uses this technique, as did the
Double Skin/ Double Mind Interactive Installation of Emio Greco|PC.2 These dynamic drawings on top of video point
directly toward and/ or augment some aspect of the video content the viewer should be paying attention to (perhaps
following) so as to understand what the artist is trying to communicate. Drawing on top of video has also been
integrated into a real-time creative tool now known as Dance Pro. [2]
Another video annotation platform named Piecemaker (PM) was introduced into the creative process of The Forsythe
Company and in use from 2007 to 2013. Developed by David Kern, PM was intended to support the organisation and
recall of materials created in the rehearsal studio. PM used a free-text based annotation system combined with tags and
other standard elements and was the basis from which the Motion Bank project developed its own versions to support
development and publication of its on-line scores with choreographers Deborah Hay and Jonathan Burrows.3 Also
working intensively with text-based annotation was the Transmedia Knowledge Base (TKB) project based in Lisbon
and developed in close collaboration with the Portuguese choreographer Rui Horta. [3] For the TKB projects, the use of
annotation had a scientific background and the team used ELAN, software developed at the Max Planck Institute for
Psycholinguistics and designed for the analysis of languages, sign languages, and gestures.4 Alongside these projects,
since 2007 the Rekall project in France has been developing a special platform uniquely built for documenting
performing arts and including video annotation.5
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1 Accessed 14 Feb. 2018. https://synchronousobjects.osu.edu/
2 Accessed 14 Feb. 2018. http://insidemovementknowledge.net/
2 Accessed 14 Feb. 2018. http://insidemovementknowledge.net/
3 Accessed 14 Feb. 2018. http://scores.motionbank.org/
4 Accessed 14 Feb. 2018. https://tla.mpi.nl/tools/tla-tools/elan/
5 Accessed 14 Feb. 2018. http://www.rekall.fr/

!
2!
Deepening the Research:
Since 2014 critical and reflexive questions about the impact of dance annotation on analysis, criticism, practice and
reception have emerged. These are being explored by researchers at the Centre for Dance Research (C-DaRE),
Coventry University with the goal of establishing an international hub for digital methods of dance scholarship with an
interdisciplinary focus on digitisation, cultural value, artistic research and the expanding choreographic field. C-DaRE
is also a project partner on the H2020 project WhoLodancE, whose partner Athena Research Center has built an
annotation web-based platform to browse, search and enrich with movement descriptors the data in the repository of
dance libraries, including video and motion capture [4]. C-DaRE works in partnership with Motion Bank, based at the
Hochschule Mainz University of Applied Sciences. Since 2014, Motion Bank focuses on establishing the basis for the
documentation, annotation and mediation of dance by means of standard-compliant (W3C), researchable data, low-
threshold tools and suitable methods. Motion Bank is currently doing development research with a handful of core
partners including the Pina Bausch Foundation and ICKAmsterdam. They work now on the third iteration of
Piecemaker (PM), which uses annotations as connections within a referential network of digital objects and draws on
the concept of Linked Data. [5]
Workshop Questions:
Despite these developments, many questions remain about annotation for dance practice. Questions ranging from basic
definitions and contexts [6] to annotation practices as no ‘natural’ methods of dance annotation exist in the field itself.
Currently, individual dance artists who embrace or are interested in using annotation as a way of enriching their
documentation have to invent their own method. Existing approaches such as Laban Movement Analysis6 can be a point
of reference, but annotating itself raises new questions. For interdisciplinary research projects such as WhoLodancE,
with experienced partners and shared goals, a set of movement principles used for annotation might be agreed to.7
However, how to look and what to look for in complex dance sequences requires expertise [7], and the labour of
annotation that might take place in the context of one scientific project may not be repeatable for others. [8] There is the
question of the product of the annotation, to whom is the annotation communicating? Are annotations being used in the
framework of education or arts research, or if they are part of a scientific project who are they useful for and how? Are
other fields doing annotation, e.g. ethology, anthropology, linguistics or film studies, dance related research might learn
from? What of the future of dance digitisation in relation to annotation and linked data? Are there new learning
resources to be created for future generations from dance data and what part might annotation play? And what about the
link between manual and automatic annotation? These are the kinds of questions the Workshop will address in hands-on
(using the systems that are there) as well as discussion formats.
Workshop Technical Requirements:
For the Workshop we would request a number of tables that can be brought together to form one large table for the
group to sit around or to be used to set up separate stations where different tools and platforms can be explained and
tried out using sample data. We would require a data projector and sound for presentations. See the attached Technical
Requirements spread sheet for details.
References:
[1] Maaike Bleeker. 2016. Introduction. Transmission in Motion: the Technologizing of Dance. Maaike Bleeker (Ed). Taylor &
Francis, London, xx.
[2] J. Gouveia, C. Fernandes and Nuno Correia. 2016. The Creation-Tool: Augmenting the Annotation of Performing Arts
Rehearsals. Multimodality and Performance. C. Fernandes (Ed.) Cambridge Scholars Publishing, Newcastle upon Tyne.
[3] C. Fernandes and S. Jürgens. 2013. Video annotation in the TKB project: linguistics meets choreography meets technology.
International Journal of Performance Arts & Digital Media 9,1 (2013), 113-132.
[4] K. El Raheb, N. Papapetrou, V. Katifori and Y. Ioannidis. 2016. BalOnSe: Ballet Ontology for Annotating and Searching Video
performances. Proceedings of the International Conference on Movement and Computing (MOCO) 5,1 (2016), 8
[5] S. deLahunta and A. Koch. 2017. Dance Becoming Data: Parts One and Two. Computing the Corporeal: Computational Culture,
a Journal of Software Studies. N.S. Sutil and S. deLahunta (Eds). Issue 6 (2017).
[6] S. deLahunta, S. Whatley and K. Vincs. 2015 On An/Notations (editorial). On An/Notations. Performance Research 20, 6 (2015),
1-2.
[7] S. F. Alaoui, C. Kristin, S. Cuyckendal, K. Studd, K. Bradley and T. Schiphorst. 2015. How Do Experts Observe Movement.
Proceedings of the International Conference on Movement and Computing (MOCO) 2015.
[8] V. Evola, J. Skubisz and C. Fernandes. 2016. The Role of Eye Gaze and Body Movements in Turn-Taking during a
Contemporary Dance Improvisation. Proceedings from the 3rd European Symposium on Multimodal Communication. E. Gilmartin,
L. Cerrato and N. Campbell (Eds). Dublin, September 17-18, 2015. LiU Press, Linköping, 24-31.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
6 Accessed 14 Feb. 2018. https://en.wikipedia.org/wiki/Laban_movement_analysis
7 Accessed 14 Feb. 2018. http://www.wholodance.eu/wp-content/uploads/2016/11/D1.5-Data-Acquisition-Plan_Submitted.pdf