No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Interactive reflexive and embodied exploration of sound qualities with BeSound
Abstract
The embodied and reflexive interaction paradigms separately proved to be effective for learning music in childhood. However, nowadays, there is a scarcity of research addressing the joined adoption of these paradigms, both from a theoretical and a technological point of view. BeSound supports children to explore - by means of their own body - rhythm, melody, and harmony and to creatively combine them together. Firstly, the child is engaged in a game in which she has to mimic the movement of the characters of stories BeSound tells her; then she can ad-lib a music dialogue with the characters. Each character was previously associated with a component of Laban's Effort and it was described through a set of whole-body movement features. These features are automatically detected, analysed, and used to control the music response of BeSound.
... Auditory feedback can also convey intricate information through its qualities (e.g. intensity, pitch, timbre, and directionality) [4,91,99]. It often remains at the periphery of attention, becoming less obtrusive, but potentially also less actionable than visual output [68]. ...
The growing interest in wearables for sports and fitness calls for design knowledge and conceptualizations that can help shape future designs. Towards that end, we present and discuss a design space of wearables for these practices, based on a survey of previous work. Through a thematic analysis of 47 research publications in the domain, we surface core design decisions concerning wearability, technology design, and wearable use in practice. Building on these, we show how the design space takes into account the goals of introducing technology, that design decisions can be either directly designed, or left open for appropriation by end-users; and the social organization of the practice. We characterize prior work based on the design space elements, which yields trends and opportunities for design. Our contributions can help designers think about key design decisions, exploit trends and explore new areas in the domain of wearables for sports and fitness practices.
... The MIROR-Body Gesture application exploits and amplifies a number of elements of gesture analysis proposed by Rudolf Laban (1879-1958: we aimed to take advantage of this theoretical and practical framework coming from the context of dance education, in order to develop children's motor and musical skills through reflexive technologies. The MIROR-Body Gesture implemented in the framework of the MIROR project does not use sensors but the kinect, and focuses mainly on the exploration of rhythm, melody, and harmony (for technical details see Volpe et al., 2012;Addessi et al., 2013a;Varni et al., 2013). An example of the interface is shown in Figure 5. ...
This article presents the pedagogical paradigm of reflexive interaction and its application in the field of technology-enhanced learning and children’s musical creativity. The main feature of reflexive interaction is the repetition-variation mechanism: something is repeated and varied during the interaction, through a continual process of imitation and variation. In the context of the MIROR project (EU-ICT Project), we exploited the educational potential of the reflexive interaction paradigm and implemented the MIROR platform, an educational device consisting of a set of softwares that implement the reflexive paradigm not only in music improvisation (as was the case for the first interactive reflexive system), but also in the field of music composition and dance. The platform was conceived as a tool to stimulate and develop musical and motor creativity in children, although it can also be used to teach a specific musical instrument. The hypothesis of the MIROR project was that reflexive interaction sustains and promotes the learning and creative expression processes of music and movement. This initial hypothesis, which stemmed mainly from the pilot study in which we observed children who were interacting with the first prototype of the interactive reflexive musical system, was subjected to a series of empirical studies conducted within the MIROR project, and was theoretically defined in order to lay down the foundations of the reflexive interaction paradigm and its pedagogical implications. This article summarizes the state-of-the-art of the project and brings together, in a comprehensive overview, the theoretical framework, the pedagogical concepts, the empirical studies, and the description of the MIROR platform, with the aim to reflect on the results achieved so far and point out the contribution of the reflexive perspective to the field of children’s instrumental learning and creativity.
... Pursue a similar concept, but use on a complete computer game and not just a partial aspect. (Varni, Volpe, Sagoleo, Mancini, & Lepri, 2013) In this game, the children are told a story in which they can interact to help a magician fill the world with sounds that have been lost. The findings of Blaine and Perkis are also interesting for the development of DDMM since they also chose drumming interfaces as the simplest option for collaborative improvised music making. ...
Most musical instruments provide children with a rather complex user interface and unfamiliar possibilities of interaction, which do not appear in their daily experience of the world. This complicates the process of creating harmonic melodies and may lead to dissonance and frustration. Thus, the main goal of our reasearch project "Drum-Dance-Music-Machine" is the development of a toolset for early childhood music education, that facilitates a low-threshold access to collaborative music making. For this purpose, a prototypical new instrument has been constructed, offering the simple interface of a drum, but producing notes and melodies. Furthermore, with this system it is possible to create music by dancing and changing its parameters through movement. Amongst others, this aspect can be used to explain and clarify the meaning of abstract musical terms. During the ongoing development and after further enhancements we tested the system in two phases in order to ascertain our predefined goals. Amongst others, the children were asked to give their mind on their rating of this new instrument and tell if it is easy or difficult to create music with. Furthermore, we collected subjective assumptions from parents and educators, test observations and objective measurement data, combined them and analysed the results in terms of various aspects to verify the suitability of the system for use in early education.
... However, the system operates asynchronously, and is therefore not suitable for DDMM. Varni et al. use dialogues in a computer game to introduce children to an interface in which they are able to use their entire body to produce sound [23]. Results from this approach may be partly modified and applied to DDMM. ...
Most instruments traditionally used to teach music in early education, like xylophones or flutes, encumber children with the additional difficulty of an unfamiliar and unnatural interface. The most simple expressive interaction, that even the smallest children use in order to make music, is pounding at surfaces. Through the design of an instrument with a simple interface, like a drum, but which produces a melodic sound, children can be provided with an easy and intuitive means to produce consonance. This should be further complemented with information from analysis and interpretation of childlike gestures and dance moves, reflecting their natural understanding of musical structure and motion. Based on these assumptions we propose a modular and reactive system for dynamic composition with accessible interfaces, divided into distinct plugins usable in a standard digital audio workstation. This paper describes our concept and how it can facilitate access to collaborative music making for small children. A first prototypical implementation has been designed and developed during the ongoing research project Drum-Dance-Music-Machine (DDMM), a cooperation with the local social welfare association AWO Hagen and the chair of musical education at the University of Applied Sciences Bielefeld.
... Nowadays, technologies such as augmented reality (e.g., see Santos et al., 2014) and serious games (e.g., see Connolly et al., 2012) play a relevant role in many educational contexts, both in science and in the humanities. Multisensory technologies enabling embodied interaction were used, for example, to support teaching in computer programming (e.g., Katai and Toth, 2010;Katai, 2011), music (e.g., Varni et al., 2013), and dance (e.g., Rizzo et al., 2018). Baud-Bovy and Balzarotti (2017) reviewed recent research on force-feedback devices in educational settings, with a particular focus on primary school teaching. ...
While technology is increasingly used in the classroom, we observe at the same time that making teachers and students accept it is more difficult than expected. In this work, we focus on multisensory technologies and we argue that the intersection between current challenges in pedagogical practices and recent scientific evidence opens novel opportunities for these technologies to bring a significant benefit to the learning process. In our view, multisensory technologies are ideal for effectively supporting an embodied and enactive pedagogical approach exploiting the best-suited sensory modality to teach a concept at school. This represents a great opportunity for designing technologies, which are both grounded on robust scientific evidence and tailored to the actual needs of teachers and students. Based on our experience in technology-enhanced learning projects, we propose six golden rules we deem important for catching this opportunity and fully exploiting it.
... The auditory sense is also complex, and conveys intricate information [19,38] encoded in multiple sensorial qualities, such as intensity, pitch, timbre, and directionality [3,58,61]. ...
Although the use of technology to support physical training has been gaining traction in HCI, the design space of sensory augmentations to improve and support physical training remains narrow. To help open this design space in the domain of collocated instructed physical training, we first present Sensory Bodystorming, a hybrid ideation method influenced by material and embodied approaches to ideation. Sensory Bodystorming targets the generation of ideas that support, and are underpinned by, a rich and nuanced sensorial experience. Ideas are generated through physically engaging with evocative sensorial stimuli elicited by ideation probes featuring diverse material qualities. Secondly, we present ideas resulting from three Sensory Bodystorming sessions that extend prior sensory augmentation works in the domain. Lastly, we identify interesting research and design directions based on prior works and our own.
... Por tanto, desde la aproximación semiótica de Eco (1975), el signo sonoro es, simultáneamente, icónico o natural, arbitrario o simbólico, e indicial. Se debe buscar constantemente asociaciones sonoras con los personajes, los movimientos o las situaciones (Varni, Volpe, Sagoleo, Mancini y Lepri, 2013). ...
Radio advertising is suffering from a remarkable crisis of creativity as it has yet not found its role in a radio model based on voice locution and information genres. This article suggests the need for implementing a peripheral or heuristic strategy to attract and hold listeners’ attention. Within this framework, the narration and scene representation are proposed as suitable persuasion techniques. The objective is to design a useful conceptual tool for an efficient creative conception of narration at the service of certain commercial strategy. First, the concept of narrative persuasion is grounded according to the possibilities of the sound code. Second, the keys of scene representation and commercial strategy (brand, product, advantage, benefit and target) within the sound message are presented. And third, these keys are articulated in a model. This model is pre-tested by means of analyzing eight different case-radio ads.
... The first one is described previously, the second one exhibits two main differences: 1) it executes quickly, and 2) the configuration of the map can be updated in real-time: that is, it can adapt to the movement features values that characterize a user. We did not yet exploit the latter capability in our prototype, but previous work showed that this approach can be used to create reflexive interfaces [35]. SVM had a performance (F-score) of 0.73 (Precision 0.75, Recall 0.73). ...
In this paper, we investigate the detection of laughter from the user’s nonverbal full-body movement in social and ecological contexts. Eight hundred and one laughter and nonlaughter segments of full-body movement were examined from a corpus of motion capture data of subjects participating in social activities that stimulated laughter. A set of 13 full-body movement features was identified, and corresponding automated extraction algorithms were developed. These features were extracted from the laughter and nonlaughter segments, and the resulting dataset was provided as input to supervised machine learning techniques. Both discriminative (radial basis function-support vector machines, k-nearest neighbor, and random forest) and probabilistic (naive Bayes and logistic regression) classifiers were trained and evaluated. A comparison of automated classification with the ratings of human observers for the same laughter and nonlaughter segments showed that the performance of our approach for automated laughter detection is comparable with that of humans. The highest F-score (0.74) was obtained by the random forest classifier, whereas the F-score obtained by human observers was 0.70. Based on the analysis techniques introduced in the paper, a vision-based system prototype for automated laughter detection was designed and evaluated. Support vector machines (SVMs) and Kohonen’s self-organizing maps were used for training, and the highest F-score was obtained with SVM (0.73).
... We were able to analyze Laban's Effort features heavy/light in children (refer toTable 1), collect videos to be used for subsequent work, test equipment, software, materials and space, as well as share ideas and pedagogical concerns with primary school teachers about the ecological setting for the experiments, the potential uses of MIROR-Body Gesture in schools, and activities for teacher education. Video-recordings were used for automatic analysis and system training by the Genoa team (for more details, see Addessi et al., 2013; Varni et al. 2013). ...
In this article, we present an overview of studies on young children’s music and movement creativity, carried out within the framework of the MIROR project. The project aimed at implementing several software applications based on the “reflexive interaction” paradigm. In particular, here we focus on reflexive interaction and its connections with embodied cognition and creative movement in young children. We present the theoretical background, the methodology for designing the first application devoted to children’s music and movement creativity (i.e. the MIROR-Body Gesture), the exploratory studies carried out with children in a reflexive-embodied environment, and end by offering examples of pedagogical applications.
As art influences science and technology, science and technology can in turn inspire art. Recognizing this mutually beneficial relationship, researchers at the Casa Paganini-InfoMus Research Centre work to combine scientific research in information and communications technology (ICT) with artistic and humanistic research. Here, the authors discuss some of their work, showing how their collaboration with artists informed work on analyzing nonverbal expressive and social behavior and contributed to tools, such as the EyesWeb XMI hardware and software platform, that support both artistic and scientific developments. They also sketch out how art-informed multimedia and multimodal technologies find application beyond the arts, in areas including education, cultural heritage, social inclusion, therapy, rehabilitation, and wellness.
Embodiment and reflexive interaction proved to be effective approaches to music education in childhood. A research challenge consists of merging them. This paper presents BeSound, an application intended to support children in learning the basic elements of composition. Children explore rhythm, melody, and harmony by playing at mimicking objects or characters; the qualities of their whole-body movements are analysed in real-time according to Rudolf Laban's Theory of Effort and used to control sound. The paper focuses on the design of BeSound and describes the analysis performed to distinguish between direct and flexible movements - the Laban's Space component - and between light and heavy movements - the Laban's Weight component.
The relationship between new technology and learning is gaining increasing relevance in the field of music education (Webster, 2002; Folkestad et
al., 1998). However, only a few studies have considered the nature of the interaction between children and musical machines. This article describes an observation study of children aged 3–5 years confronting a particular interactive musical system, the Continuator, which is able to produce music in the same style as a human playing the keyboard (Pachet, 2003). The analysis of two case studies suggests that the Continuator is able to develop interesting child/machine interactions and creative musical processes in young children. It was possible to observe a ‘life cycle’ of interaction, as well as micro-processes similar to those observed in child/adult interactions (Stern, 1985; Imberty, 2002). The ability of the system to attract and hold the attention of children has been interpreted through Csikszentmihalyi's (1990) ‘flow theory’.
The MIROR project is an European Union funded project under the 7th Framework programme (Technology-enhanced learning). The project is in its first year and so the aim of this paper is to explain its background and rationale, to describe its aims and structure. The MIROR Project deals with the development of an innovative adaptive system for music learning and teaching based on the reflexive interaction paradigm. The reflexive interaction paradigm is based on the idea of letting users manipulate virtual copies of themselves, through specifically designed machine-learning software referred to as interactive reflexive musical systems (IRMS). MIROR aims at extending the IRMS paradigm with the analysis and synthesis of multisensory expressive gesture to increase its impact on the musical pedagogy of young children. The MIROR platform will be developed in the context of early childhood music education. It will act as an advanced cognitive tutor, designed to promote specific cognitive abilities in the field of music improvisation, composition and body performance, both in formal learning contexts (kindergartens, primary schools, music schools) and informal ones (at home, kinder centers, etc.).
When we think of digital technologies in schools, we tend to think of computers, keyboards, sometimes laptops, and more recently whiteboards and data projectors. These tools are becoming part of the familiar educational landscape. Outside the walls of the classroom, however, there are significant changes in how we think about digital technologies - or, to be more precise, how we don't think about them, as they disappear into our clothes, our fridges, our cars and our city streets. This disappearing technology, blended seamlessly into the everyday objects of our lives, has become known as 'ubiquitous computing'. Which leads us to ask the question: what would a school look like in which the technology disappeared seamlessly into the everyday objects and artefacts of the classroom? This review is an attempt to explore this question. It maps out the recent technological developments in the field, discusses evidence from educational research and psychology, and provides an overview of a wide range of challenging projects that have attempted to use such 'disappearing computers' (or tangible interfaces) in education - from digitally augmented paper, toys that remember the ways in which a child moves them, to playmats that record and play back children's stories. The review challenges us to think differently about our future visions for educational technology, and begins to map out a framework within which we can ask how best we might use these new approaches to computing for learning.
This work in progress presents the BodyBeats Suite— three prototypes built to explore the interaction between children and computational musical instruments by using sound and music patterns. Our goals in developing the BodyBeats prototypes are (1) to help children engage their whole bodies while interacting with computers, (2) foster collaboration and pattern learning, and (3) provide a playful interaction for creating sound and music. We posit that electronic instruments for children that incorporate whole-body movement can provide active ways for children to play and learn with technology (while challenging a growing rate of childhood obesity). We describe how we implemented our current BodyBeats prototypes and discuss how users interact with them. We then highlight our plans for future work in the fields of whole-body interaction design, education, and music.
In this paper we present the results of a comparative study that explores the potential benefits of using embodied interaction to help children, aged 7 to 10, learn abstract concepts related to musical sounds. Forty children learned to create musical sound sequences using an interactive sound making environment. Half the children used a version of the system that instantiated a body-based metaphor in the mapping layer connecting body movements to output sounds. The remaining children used a version of the same environment that did not instantiate a metaphor in the mapping layer. In general, children were able to more accurately demonstrate sound sequences in the embodied metaphor based system version. However, we observed that children often resorted to spatial rather than body-based metaphors and that the mapping must be easily discoverable as well as metaphorical to provide benefit.
Modern Educational Dance . Macdonald and Evans Ltd ., 1963 . R. Laban. Modern Educational Dance. Macdonald and Evans Ltd
- R Laban
- Laban R.