Conference PaperPDF Available

WhoLoDancE: Towards a methodology for selecting Motion Capture Data across different Dance Learning Practice

Authors:
Antonio Camurri at University of Genoa
Antonio Camurri
Augusto Sarti
Augusto Sarti
Augusto Sarti
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Stefano Di Pietro
Stefano Di Pietro
Stefano Di Pietro
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Vladimir Viro
Vladimir Viro
Show all 13 authorsHide
Download full-text PDFDownload full-text PDF
Read full-text
Download citation

Abstract

In this paper we present the objectives and preliminary work of WhoLoDancE a Research and Innovation Action funded under the European Union's Horizon 2020 programme, aiming at using new technologies for capturing and analyzing dance movement to facilitate whole-body interaction learning experiences for a variety of dance genres. Dance is a diverse and heterogeneous practice and WhoLoDancE will develop a protocol for the creation and/or selection of dance sequences drawn from different dance styles for different teaching and learning modalities. As dance learning practice lacks standardization beyond dance genres and specific schools and techniques, one of the first project challenges is to bring together a variety of dance genres and teaching practices and work towards a methodology for selecting the appropriate shots for motion capturing, to acquire kinetic material which will provide a satisfying proof of concept for Learning scenarios of particular genres. The four use cases we are investigating are 1) classical ballet, 2) contemporary dance, 3) flamenco and 4) Greek folk dance.
MOCO16_WhoLoDancE_Final_Accept
ed.pdf
Content uploaded by Katerina El Raheb
Author content
All content in this area was uploaded by Katerina El Raheb on Jun 05, 2018
Content may be subject to copyright.
MOCO16_WhoLoDancE_Final_Accept
ed.pdf
Content uploaded by Katerina El Raheb
Author content
All content in this area was uploaded by Katerina El Raheb on Jun 05, 2018
Content may be subject to copyright.
WhoLoDancE: Towards a methodology for selecting
Motion Capture Data across different Dance Learning
Practices
WhoLoDancE partnersi
ABSTRACT
In this paper we present the objectives and preliminary work
of WhoLoDancE a Research and Innovation Action funded
under the European Union’s Horizon 2020 programme,
aiming at using new technologies for capturing and
analyzing dance movement to facilitate whole-body
interaction learning experiences for a variety of dance
genres. Dance is a diverse and heterogeneous practice and
WhoLoDancE will develop a protocol for the creation and/or
selection of dance sequences drawn from different dance
styles for different teaching and learning modalities. As
dance learning practice lacks standardization beyond dance
genres and specific schools and techniques, one of the first
project challenges is to bring together a variety of dance
genres and teaching practices and work towards a
methodology for selecting the appropriate shots for motion
capturing, to acquire kinetic material which will provide a
satisfying proof of concept for Learning scenarios of
particular genres. The four use cases we are investigating are
1) classical ballet, 2) contemporary dance, 3) flamenco and
4) Greek folk dance.
Author Keywords
Dance Learning; Motion Capture; Human Movement;
Whole-Body interaction; Dance practices and genres.
ACM Classification Keywords
H.5.5. Information interfaces and presentation (e.g., HCI):
Sound and Music Computing—Systems. J.5. Arts and
Humanities: Performing arts (e.g. dance, music).
INTRODUCTION
WhoLoDancE is a three years (January 2016-December
2018) Research and Innovation Action, under the framework
of ICT2015 of H2020 aiming at designing and developing
whole body interaction tools to support dance learning.
DANCE LEARNING PRACTICES
Dance education is mainly an example of “learning by
doing” and the development of embodied knowledge,
whereas technological tools can support the analytical and
conceptualization abilities of dance students, and
choreographers. Currently there is not one single repository
of these teaching methods, neither is there a repository of
dance actions that would support the development of the
teaching of dance. Learning based on ICT (Information and
Communication Technologies), and whole-body-interaction
in particular, can largely benefit from the outcomes of
applying it in the field of dance, as dancers are experts in
embodied communication and dance is by its nature
multimodal. WhoLoDancE data-driven tools will be
provided with a measurement of human-system interaction
fostering non-linear approaches to adaptive learning and
cognitive artifacts for effective human learning. Dance is
traditionally a practice that is passed from body to body, and
whilst there are established and codified dance ‘techniques’
that are genre specific these techniques evolve through the
teaching itself. Some techniques are therefore more
prescriptive than others, and those that are more prescriptive
tend to be guarded by a named innovator or the developer of
the technique. But aside from the specifics of these
techniques, there are teaching methods that connect all studio
based teaching of dance and whilst these methods are
moderated according to context (whether within a
professional/vocational teaching context, or a recreational
context, etc.) and according to genre, there are underlying
principles that pertain to all good teaching of dance. The
WhoLoDancE multimodal repository will enable the usage
of data analytics supporting the identification of effective
teaching methods and practices showing commonalities and
differences between them to support the future teaching of
dance within a variety of contexts.
DANCE LEARNING PRINCIPLES
As the aforementioned dance genres provide a huge diversity
in both the kinetic vocabularies and the teaching
methodologies, one of the big challenges is to find learning
objectives that are common across genres. We aim for a
systematic way of selecting shots beyond the differences of
the kinetic material. A focus group, with the participation of
dance teachers of all genres, agreed that the following
Learning Principles summarize the different teaching styles
applied in different practices:
1. Mimesis: imitation/copying: the teacher is teaching the
student a specific movement or sequence of movements
and the student follows the movement. This is a case
where the learning is largely based on observational
abilities of the students as they are asked to see and do;
2. Generative: the teacher gives the student an
exercise/phrase/sequence as a starting point to achieve
technical and creative goals. In this case the student is
allowed to generate new kinetic material, or alter things
as long as he or she is consistent with the technical or
creative goals;
3. Reflexive: the student is given a movement
task/image/to work with, improvising without trying to
achieve a specific phrase/sequence and the teacher
provides feedback. In this case the memorization ability
of the student is challenged, as in contrary to the mimetic
approach the student has to remember the sequence,
rather than see and do, and at the same time is allowed
to alter or generate new material, as in the generative
approach;
4. Traditional also known as “command style teaching”:
where the teacher makes all the decisions and the learner
follows, while the teacher “commands” what the student
must correct or change to achieve the good performance
of the movement. The method requires precision and
accuracy of performance.
MOVEMENT PRINCIPLES
The aim of the WhoLoDancE is to develop learning tools that
will be designed upon the principles of contemporary models
of dance learning and teaching. The research will focus not
only on teaching steps by mimicking, but also on enhancing
the student’s movement literacy, increasing the learners’
ability to analyze her/his own movement, and enhancing
movement skills that will feed into the development of the
student’s creativity, musicality and broader dance
experience.
Following the approach of the interdisciplinary focus group,
we proposed ten essential Movement Principles that can
summarize the embodied skills which are to be improved in
each dance learning process, independent of the dance genre
and style. From our preliminary investigations, we defined a
first list of movement principles
1. Symmetry: The use of the two sides of the body (right
vs. left side, arm, leg) etc., both in position and while
moving. The ability to do the same thing simultaneously
or sequentially using both sides. Each Movement
Principle includes also the opposite. Playing with
asymmetry and isometry is included in this principle.
2. Directionality: The awareness of body orientation in
space. Usually this is derived from the position of hips
and torso, but interesting postures might derive from the
various directions of each body part in relation to a
space, e.g., the audience, the camera, the studio.
i
Antonio Camurri (University of Genova), Katerina El
Raheb (Athena RC), Oshri Even-Zohar (Motek), Yannis
Ioannidis (Athena RC), Amalia Markatzi (Lykeion
Ellinidon), Jean-Marc Matos (K.Danse), Edwin Morley-
3. Balance: The ability to stand and move in balance, but
also out of balance, depending on whether the line of
gravity falls within the line of your supporting limb(s)
or not. The awareness of the different vector forces on
your body.
4. Alignment – Posture Stability: The awareness of the
geometry of the body (e.g., the axes (sagittal, horizontal,
vertical) and planes, and how the relations of different
body parts and joint create “lines” in the body shape.
5. Weight bearing vs. Gesturing: This principle is about the
difference between movement that is concerned with
bearing weight (weight transference, stepping.) and
movement (gesture) that is not bearing weight but which
has intention/expression
6. Gross vs. Fine Motorics/Isolation/Articulation: The
ability to distinguish small movements done by specific
body parts e.g., hand or one hip, or one shoulder, without
moving the rest of the body, vs. moving larger parts of
the body as a whole.
7. Coordination: One of the most important skills practiced
in every kind of dancing, which is about the ability to
synchronize or not different parts of the body that can
move in the same or separate tempos.
8. Motion through Space: Progressing through space or
towards particular directions, paths etc. "Moving
through space vs. dancing on the spot. Also the body as
moving point in space, or as continuously changing
moving volume.
9. Rhythm and phrasing. The ability to move in particular
(predefined or improvised) rhythms. This principle is
also about how the dancer’s movement is related or not
to the music and its rhythmical aspects (tempo, time
signature, rhythmic patterns etc.).
10. Stillness. While movement seems to be the essence of
dance, a dancer needs to improve her/his ability to
remain still, whether this is a part of a choreography or
interpretation of rhythmical pauses, and exercise for
balance and isolation of body parts. Stillness is usually
connected to the notion of being present and has been
investigated in previous interdisciplinary work.
Fletcher (Lynkeus), Pablo Palacio (STOCOS), Muriel
Romero (STOCOS), Augusto Sarti (Politecnico di Milano),
Stefano Di Pietro (Lynkeus), Vladimir Viro (PeachNote),
Sarah Whatley (Coventry University).
... In the literature (Camurri et al., 2016a), there are three representative methods of dance learning and teaching, summarized as follows. ...
... Recent advances in technologies and digital environments can improve the dance learning process. Technologies for capturing and quantifying dance moves, such as motion capture (Camurri et al., 2016a;Chen et al., 2005;Nakamura et al., 2005;Qian et al., 2006) and Kinect (Alexiadis et al., 2011;Kim & Kim, 2018;Kitsikidis et al., 2014;Raptis et al., 2011;Saha et al., 2013;Saha et al., 2016), are used as essential parts in computer-supported dance learning systems by providing partial feedback. These technologies allow abstract dance movements to be recognized and quantified with high accuracy. ...
Dancing on the inside: A qualitative study on online dance learning with teacher-AI cooperation
Article
Full-text available
  • Mar 2023
  • Educ Inform Tech
Recent technologies have extended opportunities for online dance learning by overcoming the limitations of space and time. However, dance teachers report that student–teacher interaction is more likely to be challenging in a distant and asynchronous learning environment than in a conventional dance class, such as a dance studio. To address this issue, we introduce DancingInside, an online dance learning system that encourages a beginner to learn dance by providing timely and sufficient feedback based on Teacher-AI cooperation. The proposed system incorporates an AI-based tutor agent (AI tutor, in short) that uses a 2D pose estimation approach to quantitatively estimate the similarity between a learner's and teacher's performance. We conducted a two-week user study with 11 students and 4 teachers. Our qualitative study results highlight that the AI tutor in DancingInside could support the reflection on a learner's practice and help the performance improvement with multimodal feedback resources. The interview results also reveal that the human teacher's role is essential in complementing the AI feedback. We discuss our design and suggest potential implications for future AI-supported cooperative dance learning systems.
View
... Secondly, dance is a time-varying process that requires not only the accuracy of static poses but also the continuity of poses across frames, further compounding the difficulty of analysis. Additionally, current evaluations of dance performances primarily rely on manual expert assessments, which are time consuming, labor intensive, and challenging in terms of the establishment of objective and quantifiable evaluation criteria [3,4]. In contrast, in motion analysis-particularly in basketball shooting-significant progress has been made in quantifying and analyzing joint kinematics using various technologies, including optical motion capture systems (OMCSs) and magnetic inertial measurement units (MIMUs). ...
Intelligent Dance Motion Evaluation: An Evaluation Method Based on Keyframe Acquisition According to Musical Beat Features
Article
Full-text available
  • Sep 2024
  • SENSORS-BASEL
Motion perception is crucial in competitive sports like dance, basketball, and diving. However, evaluations in these sports heavily rely on professionals, posing two main challenges: subjective assessments are uncertain and can be influenced by experience, making it hard to guarantee timeliness and accuracy, and increasing labor costs with multi-expert voting. While video analysis methods have alleviated some pressure, challenges remain in extracting key points/frames from videos and constructing a suitable, quantifiable evaluation method that aligns with the static–dynamic nature of movements for accurate assessment. Therefore, this study proposes an innovative intelligent evaluation method aimed at enhancing the accuracy and processing speed of complex video analysis tasks. Firstly, by constructing a keyframe extraction method based on musical beat detection, coupled with prior knowledge, the beat detection is optimized through a perceptually weighted window to accurately extract keyframes that are highly correlated with dance movement changes. Secondly, OpenPose is employed to detect human joint points in the keyframes, quantifying human movements into a series of numerically expressed nodes and their relationships (i.e., pose descriptions). Combined with the positions of keyframes in the time sequence, a standard pose description sequence is formed, serving as the foundational data for subsequent quantitative evaluations. Lastly, an Action Sequence Evaluation method (ASCS) is established based on all action features within a single action frame to precisely assess the overall performance of individual actions. Furthermore, drawing inspiration from the Rouge-L evaluation method in natural language processing, a Similarity Measure Approach based on Contextual Relationships (SMACR) is constructed, focusing on evaluating the coherence of actions. By integrating ASCS and SMACR, a comprehensive evaluation of dancers is conducted from both the static and dynamic dimensions. During the method validation phase, the research team judiciously selected 12 representative samples from the popular dance game Just Dance, meticulously classifying them according to the complexity of dance moves and physical exertion levels. The experimental results demonstrate the outstanding performance of the constructed automated evaluation method. Specifically, this method not only achieves the precise assessments of dance movements at the individual keyframe level but also significantly enhances the evaluation of action coherence and completeness through the innovative SMACR. Across all 12 test samples, the method accurately selects 2 to 5 keyframes per second from the videos, reducing the computational load to 4.1–10.3% compared to traditional full-frame matching methods, while the overall evaluation accuracy only slightly decreases by 3%, fully demonstrating the method’s combination of efficiency and precision. Through precise musical beat alignment, efficient keyframe extraction, and the introduction of intelligent dance motion analysis technology, this study significantly improves upon the subjectivity and inefficiency of traditional manual evaluations, enhancing the scientificity and accuracy of assessments. It provides robust tool support for fields such as dance education and competition evaluations, showcasing broad application prospects.
View
... The Whole-body Interaction Learning for Dance Education (WhoLoDancE) pro-ject, funded by the European Union's Horizon 2020 program, conducted groundbreak-ing research on using digital technology to teach dance and choreography [65,66]. They utilize motion capture technology and 3D vision technology, such as Mi-crosoft HoloLens Mixed Reality headsets, to capture dances (including ballet, modern, flamenco, and Greek folk dances) and preserved them for future dancers and choreog-raphers to study. ...
Application of Three-Dimensional Vision Technology in Dance
Article
Full-text available
  • Aug 2024
The development of science and technology constantly injects new vitality into dance performance and creation. Among them, three-dimensional (3D) vision technology provides novel ideas for the innovation and artistry of dance performances, expands the forms of dance performances and the way to present dance works, and brings a brand-new viewing experience to the audience. Nowadays, 3D vision technology in dance has been widely researched and applied. This review presents the background of the 3D vision technology application in the dance field, analyzes the main types of technology and working principles for realizing 3D vision, summarizes the research and application of the 3D vision technology in dance creation, perception, enhancement, and dance teaching, and finally looks forward to the development prospect of the 3D vision technology in the dance.
View
... In the context of the Internet, the use of Internet technology allows teachers to better track students' progress and needs and provide individualized teaching plans for each student, and the Internet enables classical dance schools and teachers to collaborate with students worldwide, share resources and experiences, and promote the internationalization of classical dance education, among other ways to develop and innovate the teaching mode of classical dance in China [9][10][11]. Although the Internet has brought new opportunities for teaching classical Chinese dance, there are also some challenges [12]. ...
Development and Innovation of Chinese Classical Dance Teaching Mode in the Background of Internet
Article
Full-text available
  • Feb 2024
Since the development of Chinese classical dance, its dance technology and dance skills have been maturing for a long time, but the innovation and development of classical dance teaching mode needs to be improved. In this paper, OptiTrack is used to capture the dance movement data of classical dancers, transform the data into three-dimensional coordinates, and extract the features of classical dance data through direction normalization. The Hidden Markov Model obtains the time series information in the features and constructs the correlation model. Using the skeletal model data to get the center of mass of the human body, the convolution-gated recurrent unit network structure is used to improve the accuracy of the classical dance score generation results. Combining the above methods, a new teaching model framework for classical dance is proposed from three aspects and empirically analyzed. The empirical results show that the mean value of each dimension of classical dance teachers’ teaching ability ranges from 3.4562-4.3621, which is an excellent overall performance. In the comparison between students’ classical dance movements and standard movements, there existed higher scores than traditional movements in the two time periods within 20 minutes, which were 30 and 5 points higher than the conventional scores, respectively. The rest of the time was slightly lower than the regular movements, which shows that the innovation effect of classical dance teaching is better, and optimizes students’ classical dance movements to a certain extent.
View
... Departamento de Artes del Movimiento Universidad Nacional de las Artes but centered around the same intention. Current iterations of this impulse to tangibilize dance include: the production of automatic Labanotated scores from hidden Markov models (Li et al., 2017); the customization of users' -virtual selves‖ while dancing (El Raheb, 2018); the facilitation of whole-body interaction learning experiences of dance (Camurri et al., 2016); the training of AI models to produce their own dance movements ; and the usage of motion capture data to quantitatively identify how genre-specific music can alter dance production (Carlson et al., 2020). ...
From Humans That Move Like Machines to Machines That Move Like Humans: A Multidisciplinary Review of the State of the Art in the Digitization of the Dancing Body
Article
Full-text available
  • Oct 2023
The heightened circulation of digital renditions of dance and their reincorporation back into human bodies reveals a kind of permeability of cross-pollinating forces dialoguing fundamentally through gesture. The tensions, digressions, and reformulations of what movement means in the digital era have spawned discourses and innovations that need to be tackled through multiple disciplines, namely the computer sciences, the humanities, and the law, all of which have been considered for this study. First, we offer a review of what are currently the most prominent initiatives in the digital safeguarding of dance and movement globally, seen in tandem with the latest developments in the field of computerized movement recognition in the following section. We then examine the most salient criticisms from the humanities surrounding the digitization of human movement, which we follow with a compilation of relevant jurisprudence of conflicts dealing with the dancing body's merging with the digital. The purpose of this survey is to assist readers and practitioners in positioning their own projects related to the digitization of dance within a thriving field and suggest potential conceptual refigurations that such advances might produce to better grasp the leakages between humans and machines.
View
... Especially, following the development of big data and image information processing technology, a real-time human pose estimation method for dance training has been developed to more effectively correct the human pose in college dance training, and improve the training effect. In view of the above, the study on the relevant HPE for college dance training is of great significance for optimizing the teaching quality of dance training [3]. This paper aims to explore a highly robust, highprecision method for correcting the human poses through the study on the standard pose movement of dances. ...
View
... These could be used to extend the possibilities of movement detection and tracking, as long as it focuses on the users' abilities. A summary of the ten Movement Principles (Partners 2016;Rizzo et al. 2018 Gesture elicitation studies allow participants to map an action, however the result may be an individual list of personalised gesture library. This would allow personalisation, addressing challenges in accessibility and accuracy (Szedel 2020). ...
View
... Our proposal supports individual stopping practice and thus can contribute to improving dance skills. Various researchers have recently proposed analyses of dance movements using motion capture (MoCap) for teaching and training [13][14][15][16][17][18][19][20]. Those analyses have focused on the performer's pose during a dance. ...
Detecting and Visualizing Stops in Dance Training by Neural Network Based on Velocity and Acceleration
Article
Full-text available
  • Jul 2022
  • SENSORS-BASEL
Various genres of dance, such as Yosakoi Soran, have contributed to the health of many people and contributed to their sense of belonging to a community. However, due to the effects of COVID-19, various face-to-face activities have been restricted and group dance practice has become difficult. Hence, there is a need to facilitate remote dance practice. In this paper, we propose a system for detecting and visualizing the very important dance motions known as stops. We measure dance movements by motion capture and calculate the features of each movement based on velocity and acceleration. Using a neural network to learn motion features, the system detects stops and visualizes them using a human-like 3D model. In an experiment using dance data, the proposed method obtained highly accurate stop detection results and demonstrated its effectiveness as an information and communication technology support for remote group dance practice.
View
... While a complete and versatile annotation tool would require hours of training to be mastered, the easier and more intuitive tools might produce less informative annotations. Heaviness and fluidity for example are only two of the movement qualities that are defined in a more complex conceptual framework [4,12,21,53] which served as a model to bridge the gap between high-level computationally analysed features [13,54] and the concepts that where meaningful for the dance experts focusing on dance learning. As we explain further in Section 6, we focused on these two qualities as they were characterised as being amongst the most commonly understood by the that dance experts, representing different genres. ...
Towards a general framework for the annotation of dance motion sequences
Article
Full-text available
In this paper, we present a conceptual framework and toolkit for movement annotation. We explain how the design of the annotation systems, based on the framework, if combined with specific strategies for the process of annotation, can enhance the collection of ground-truth datasets for training algorithms. Computational algorithms, such as machine learning, show promising results for massive and scalable automatic movement annotation. Nevertheless, the need for reliable ground-truth datasets annotated by human experts, to train the machine learning algorithms and for bridging the gap between machine measurable and human perceived expressive aspects remains an open issue. This need constitutes a challenging task, due to the complexity of human movement and diversity of possible descriptors, as well as the high subjectivity that accompanies movement characterisation by both experts and non-expert users. We contribute to addressing this problem, by proposing a conceptual framework for dance movement manual annotation which we evaluate through the development and deployment of the toolkit. Finally, we discuss how the different design choices affect the process and the reliability of collecting data sets regarding qualitative aspects of movement.
View
View
Body as instrument: an exploration of gestural interface design
Thesis
Full-text available
  • Mar 2016
Gestural interfaces broaden musicians’ scope for physical expression and offer possibilities for creating more engaging and dynamic performances with digital technology. Increasing affordability and accessibility of motion-based sensing hardware has prompted a recent rise in the use of gestural interfaces and multimodal interfaces for musical performance. Despite this, few performers adopt these systems as their main instrument. The lack of widespread adoption outside academic and research contexts raises questions about the relevance and viability of existing systems. This research identifies and addresses key challenges that musicians face when navigating technological developments in the field of gestural performance. Through a series of performances utilising a customised gestural system and an expert user case study, I have combined autoethnographic insights as a performer/designer with feedback from professional musicians to gain a deeper understanding of how musicians engage with gestural interfaces. Interviews and video recordings have been analysed within a phenomenological framework, resulting in a set of design criteria and strategies informed by creative practitioner perspectives. This thesis argues that developing the sensorimotor skills of musicians is integral to enhancing the potential of current gestural systems. Refined proprioceptive skills and kinaesthetic awareness are particularly important when controlling non-tactile gestural interfaces, which lack the haptic feedback afforded by traditional acoustic instruments. However, approaches in the field of gestural system design for music tend to favour technical and functional imperatives over the development of the kinaesthetic sense. Building on a growing body of gestural interface design and human–computer interaction (HCI) literature, this research offers practice-based insights that acknowledge the changing face of musicianship in response to interaction with gestural sensing technologies. To encourage enhanced physical aptitude and more nuanced movement control amongst musicians, I have applied embodied interaction design and dance-based perspectives to musical contexts, developing a multimodal environment that provides a range of design strategies for musicians to explore relationships between sound and movement while developing an awareness of their own movement potential.
View
Composing Interactive Dance Pieces for the MotionComposer, a device for Persons with Disabilities
Conference Paper
Full-text available
  • Jan 2015
The authors have developed a new hardware/software device for persons with disabilities (the MotionComposer), and in the process created a number of interactive dance pieces for non-disabled professional dancers. The paper briefly describes the hardware and motion tracking software of the device before going into more detail concerning the mapping strategies and sound design applied to three interactive dance pieces. The paper concludes by discussing a particular philosophy championing transparency and intuitiveness (clear causality) in the interactive relationship, which the authors apply to both the device and to the pieces that came from it.
View
The Composition and Performance of Spatial Music
Article
Full-text available
This thesis will examine the perceptual effectiveness of various works of spatial music in terms of the technical means of spatialization, and also the compositional approach to the use of space as a musical parameter. Particular attention will be paid to the effectiveness of different spatialization techniques in a performance context, and what this implies for compositional strategies which use space as a musical parameter. The results of a large number of listening tests and simulations were analysed to determine the fundamental capabilities of different spatialization techniques under the less than ideal conditions typically encountered during a performance. It was found that stereophonic techniques based on amplitude panning provided the most accurate localization but suffered from a lack of spaciousness and envelopment. Ambisonics provides an improved sense of envelopment but poor localization accuracy, particularly with first order Ambisonics systems. Ambisonics was consistently preferred for dynamically moving sources as this technique eliminated the panning artefacts exhibited by amplitude panning as the source moved from a position at a loudspeaker, to one inbetween a pair of loudspeakers. A single-band, max-rE decoding scheme was found to be the most suitable Ambisonics scheme for a distributed audience, and increasing the order of the system was shown to improve the performance at all listener positions. It is recommended that an octagonal array be adopted as a minimum standard for performances of multichannel spatial music, as this arrangement can be utilized for third order Ambisonics and can also be readily implemented with digital audio hardware. Wavefield synthesis (WFS) was found to be quite distinct from multichannel techniques such as stereophony or Ambisonics. It was found that as the distance between loudspeakers in the array is increased, spatial aliasing results in significant soundfield reconstruction errors. The ability of WFS systems to position virtual sources both behind and in front of the loudspeaker array was found, in practice, to be extremely difficult to achieve. In the latter half of this thesis, a number of landmark works of spatial music were presented and analysed in terms of the perceptual validity of their approach to spatialization. It was shown that many composers have used spatial distribution to improve the intelligibility of different layers of material, and this approach was found to agree with the findings of scientific research in the area of auditory cognition. The use of recognizable spatial motifs was shown to be highly difficult to achieve, and complex, abstract spatial designs were found to only be indirectly related to what is eventually perceived by the audience. A gestural approach to spatial music, and augmented instruments which map the actions of the performer to a spatialization algorithm, would both seem to be highly suitable for performances of mixed-media spatial music. The use of flocking algorithms to control spatialization and sound synthesis also appears to be a novel and effective approach to the creation of spatially dynamic, electronic sounds. Finally, a number of original compositions by the author are presented and analyzed in terms of the perceptual results discussed earlier in the thesis.
View
A system to support the learning of movement qualities in dance: a case study on dynamic symmetry
Conference Paper
  • Sep 2016
In this paper, we present (i) a computational model of Dynamic Symmetry of human movement, and (ii) a system to teach this movement quality (symmetry or asymmetry) by means of an interactive sonification exergame based on IMU sensors and the EyesWeb XMI software platform. The implemented system is available as a demo at the workshop.
View
The Dancer in the Eye: Towards a Multi-Layered Computational Framework of Qualities in Movement
Conference Paper
  • Jul 2016
This paper presents a conceptual framework for the analysis of expressive qualities of movement. Our perspective is to model an observer of a dance performance. The conceptual framework is made of four layers, ranging from the physical signals that sensors capture to the qualities that movement communicate (e.g., in terms of emotions). The framework aims to provide a conceptual background the development of computational systems can build upon, with a particular reference to systems analyzing a vocabulary of expressive movement qualities, and translating them to other sensory channels, such as the auditory modality. Such systems enable their users to "listen to a choreography" or to "feel a ballet", in a new kind of cross-modal mediated experience.
View
A serious games platform for validating sonification of human full-body movement qualities
Conference Paper
  • Jul 2016
In this paper we describe a serious games platfrom for validating sonification of human full-body movement qualities. This platform supports the design and development of serious games aiming at validating (i) our techniques to measure expressive movement qualities, and (ii) the mapping strategies to translate such qualities in the auditory domain, by means of interactive sonification and active music experience. The platform is a part of a more general framework developed in the context of the EU ICT H2020 DANCE "Dancing in the dark" Project n.645553 that aims at enabling the perception of nonverbal artistic whole-body experiences to visual impaired people.
View
Movement Fluidity Analysis Based on Performance and Perception
Conference Paper
  • May 2016
In this work we present a framework and an experimental approach to investigate human body movement qualities (i.e., the expressive components of non-verbal communication) in HCI. We first define a candidate movement quality conceptually, with the involvement of experts in the field (e.g., dancers, choreographers). Next, we collect a dataset of performances and we evaluate the perception of the chosen quality. Finally, we propose a computational model to detect the presence of the quality in a movement segment and we compare the outcomes of the model with the evaluation results. In the proposed on-going work, we apply this approach to a specific quality of movement: Fluidity. The proposed methods and models may have several applications, e.g., in emotion detection from full-body movement, interactive training of motor skills, rehabilitation.
View
Mconduct: Transcending domains and distributed performance
Article
  • Jan 2012
Conductors play an important role in an ensemble. They are essential to an ensemble functioning as a single coherent unit, being responsible for controlling tempo and dynamics, whilst also channeling the emotional intensity of a piece. Traditional conducting requires visual cues; contemporary composers challenge traditional conducting with distributed performance techniques, where the performers' view of the conductor can be obscured. This paper proposes an interface to enhance coordination in distributed performance. This interface is a non-intrusive system for communicating conductor information via haptic feedback. Data is collected using sensors and mapped into haptic feedback and transmitted via a wireless connection. Transcending Domains is a work for a distributed ensemble by the composer Joanne Armitage. The paper discusses the application of the mConduct system in this composition and concludes with the latest findings, future directions and the impact the research may have outside the realm of gesture communication application.
View
View
View