No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The conductor’s intensity gestures
Abstract
This work is aimed at outlining a repertoire of conductors’ gestures. In this perspective, it presents two studies that investigate a specific subset of the body signals of orchestra and choir conductors, namely, the gestures for musical intensity. First, an observational qualitative study, based on a systematic coding of a corpus of fragments from orchestra concerts and rehearsals, singled out 21 gestures, in which either the gesture as a whole or some aspects of it conveyed indications for forte, piano, crescendo, or diminuendo; some are symbolic gestures, used either with the same meaning as in everyday interaction or with one specific of conductors; others are iconic gestures, both directly or indirectly iconic. Second, in a perception study, a questionnaire submitted to 77 participants tested if 8 gestures of intensity out of the 21 singled out by the coding study are in fact shared and understood, and whether they are better interpreted by music experts than by laypeople. Results showed that the tested gestures are fairly comprehensible, not only by experts but also by non-expert participants, probably due, for some gestures, to their high level of iconicity, and for others to their closeness to everyday gestures.
... Therefore, various studies have attempted to find out the lexicons of the conductors' gestures and gaze: first, qualitative analyses of corpora of conducting hypothesize the correspondences between specific gestures or gaze items and the meanings conveyed by the conductor, and then perception studies test the hypotheses put forward about some gesture or gaze items. After singling out the gestures for intensity-those providing indications like forte, piano, crescendo, diminuendo [56][57][58]-a lexicon of gaze in conducting was proposed [59,60]. ...
... All in all, the fragment analyzed contains 56 gaze items, and the functions of their direct and indirect meanings are distributed as in Table 4. As already undertaken in previous studies on intensity gestures [56][57][58], our method to discover the conductor's lexicon exploits a two steps approach: first, we view the communicative signals investigated, whether gestures, or gaze items, only from the point of view of its production, hence simply attributing the Sender the intention of communicating a given meaning by using that particular signal. This was the aim of Study 2. The second step is to check, by a perception study, if that signal is actually interpreted as bearing that very same meaning by the Receivers; and this is what will be carried out in Study 3. In other words, in Study 2, we only make some hypotheses about what the meaning of each gaze signal in the fragment could be. ...
... During the last few decades, a conspicuous body of research has investigated multimodal communication in music performance, tackling the body movements of performers and conductors between each other and with the audience. Concerning gestures in conducting, they have been investigated including their semantic aspects, the meanings they convey and the semiotic devices they exploit [45,[47][48][49][56][57][58]. In regards to gaze, the majority of studies on the use of eyes in performance, both between co-performers and between performers and conductor, deals with what we would call an "input", not an "output" function: using eyes mainly to acquire information about what another is doing [10], in order to reach synchronization or to catch technical or expressive indications. ...
The paper argues for the importance and richness of gaze communication during orchestra and choir conduction, and presents three studies on this issue. First, an interview with five choir and orchestra conductors reveals that they are not so deeply aware of the potentialities of gaze to convey indications in music performance. A conductor who was utterly conscious of the importance of gaze communication, however, is Leonard Bernstein, who conducted a performance of Haydn’s Symphony No. 88 using his face and gaze only. Therefore, a fragment of this performance is analyzed in an observational study, where a qualitative analysis singles out the items of gaze exploited by Bernstein and their corresponding meanings. Finally, a perception study is presented in which three of these items are submitted to expert, non-expert, and amateur participants. The results show that while the signal for “start” is fairly recognized, the other two, “pay attention” and “crescendo and accelerando” are more difficult to interpret. Furthermore, significant differences in gaze item recognition emerge among participants: experts not only recognize them more, but they also take advantage of viewing the items with audio-visual vs. video-only presentation, while non-experts do not take advantage of audio in their recognition.
... This is based on the creation of psycho-emotional relief and interaction with other people. Conductors have an influence on the formation of different emotions while listening to the same musical composition as a result of a different approach to performance: at a fast or slow tempo, in a changed key, etc. (Poggi et al., 2021). The individual style of playing piano compositions is based on the variation of tempo (base, global, and local). ...
The study aims to investigate the impact of ChatGPT on music education by evaluating creativity and perception indicators resulting from AI integration. The research is based on analyzing the effect of AI (ChatGPT) integrated into music education. To this end, a method for implementing the program in piano teaching sessions was developed, grounded in theoretical principles of creativity development according to flow theory, divergent and convergent thinking. The study involved 566 students from a music conservatory. Creativity was assessed using two forms of the Torrance Tests of Creative Thinking (TTCT). At the beginning of the pre-test, both groups demonstrated uniform levels of creativity within the music education process. Analysis of creativity revealed a significant increase in creative skills within the AI-intervention group (p = 0.000). Considering the aim to more precisely determine the impact of recommendation algorithms on queries based on theoretical concepts of creativity development, a correlation was established between the frequency of application of the proposed query approaches and the dynamics of creativity between the pre- and post-tests (r = 0.001). The results indicate notable improvements in creativity within the experimental group; the difference between the control and experimental groups increased to 23.1849, with the experimental group showing a mean of 67.311. Key concerns among students relate to the precise tuning of question-answer algorithms and the training required for effective use of technology for academic and creative development.
... The in-person social interaction in choir enables singers to adjust their singing technique to achieve better vocal production and avoid vocal tension by watching the conductors' hand and gesture [27] [28]. Another research also showed how conductor's gesture can be suggestive for singers to execute certain dynamics and element of tempo [29]. Gonzo [30] explained that in vocal lesson the idea of modelling where singers need to imitate certain technique as well as body movement is prominent. ...
... Even if, as in our corpus, we cannot exactly monitor or test musicians' reactions to or understanding of these instructions, the observation whether and, if so, how certain passages are repeated or resumed may provide an indication as to whether previous instructions might have been unclear. Additionally, a perception study on instructions about musical dynamics has shown that there is no significant difference in understanding of these movements between participants with and without musical experience (Poggi et al., 2021(Poggi et al., , p. 1493). An expert group was able to give more fine-grained interpretations (e.g., between a crescendo and a forte), but general patterns (louder, softer) were equally recognized by nonexperts. ...
In this contribution, we examine the way in which orchestra conductors
use the space around them to convey aspects of musical dynamics. In
music, dynamics refers to the intensity of volume of notes and sounds
and its interpretation is highly context-bound. We approach dynamics as a
phenomenon of emergent and construed meaning in interaction, induced
by the music score and the interpretation in situ by musicians and the
conductor. Conductors’ movement-based instructions on dynamics result in
highly complex usage events. This study aims at disentangling these instances
by asking how conductors move and use the space around them to instruct on
(un)desired aspects of musical dynamics, zooming in on movement direction
as a central formal feature. We find ourselves at the crossroads of cognitive
and interactional linguistics, aligning with existing studies on the interactional
and contextually embedded nature of music interaction. From a cognitive
linguistic perspective, this endeavor translates as the identification of the
construal mechanisms (metaphor, specificity and viewpoint) that underlie
and therefore motivate movement directions in the specific instances under
examination. The analysis is based on 10h of video data from a corpus
recorded during rehearsals of five conductors instructing their respective
orchestras in Dutch. Our data reveal that conductors use different movement
patterns, some of which appear to involve opposite movement directions for
expressing a similar music dynamical aspect, e.g., depending on the usage
event, a vertical upward movement can mean both a request for playing
louder and softer. By taking into account different construal mechanisms,
we are able to provide an encompassing multimodal analysis, in which these
allegedly deviating oppositional movements appear as consistently motivated
(metaphorical) expressions, which profile a similar target concept involving
different viewpoints.
The study aims to reveal the conducting techniques as a complex unified system. Its components are profoundly interconnected and have inherent functions, content and significance. The research methods chosen were a remote survey, assembly methods, analysis and comparison of the critical elements of the phenomenon called “the means of the conducting system in the 21st century”. The study has highlighted the importance of several modern principles of conducting. The study discovered a set of modern technical means of creating a performance: multimedia resources, software, modernised engineering equipment, and Internet communication tools. The dramaturgical plan of a piece of music is effectively implemented with their help, according to 81% of respondents. The crystallisation of the compositional structure based on modernised conducting tools is fully realised, as 85% of students say. The effect produced by the sound quality and a system of new gestures gazes, and articulation is best achieved, according to 89% of respondents. Prospects for further research in this area lie in the need to systematise modern conducting techniques when performing pieces of classical music and music of other styles, national schools, and eras.
The purpose of this two-experiment study was to examine collegiate musicians’ ability to discern conductor intent. In Experiment 1, participants ( N = 44) viewed two videos of Sir Simon Rattle conducting an ensemble performing an excerpt of the Finale to Gustav Mahler’s Symphony No. 2. One video featured the original performance, whereas the other video was synchronized with a different audio recording from a video of a performance led by Leonard Bernstein. When asked whether any of the two videos were mismatched, results indicated that the proportion of correct responses among our participants differed significantly from chance. In Experiment 2, participants ( N = 127) watched two sets of videos, either matched (unaltered) or mismatched versions of videos featuring Sir Simon Rattle and Leonard Bernstein, and were asked to rate how well the musical intent of the conductor matched the performance of the ensemble they viewed. Although we found a nonsignificant overall result ( p = .053) when comparing the matched and mismatched videos, we found a significant difference in participants’ ratings between the matched and mismatched Rattle videos, but not for those featuring Bernstein. Our findings suggest that a conductor’s visual contribution to the performance can enhance or detract from the observers’ musical experience.
Statement of the problem. Conducting performance in China is still a poorly explored area of the national musical art, which makes it necessary to deeply comprehend its main aspects and determine the originality of this important area of the country’s musical culture. This study is aimed at identifying the influence of the dynamics of historical development in China in the 20th century on the development of a conductor’s profession in the country. The recent researches and publications study the specifics of the conductor’s gesture (M. Mannone, I. Poggi, B. Silvey, L. Reizabal), personal conducting activities of the brightest representatives of this profession in different countries (C. Bongiovanni, J.-C. Branger, J. Orrego Salas, M. Walter and others). However no work mentions the activities of Chinese conductors. This profession is practically not studied among its Chinese representatives, too. Methodology of this research includes the study of scientific works devoted to various aspects of European conducting (R. Komurdzhi, Lee En Zhong, B. Smirnov, O. Tremzina), to the outstanding conductors (S. Bongiovanni, J.-C. Branger, J. Orrego Salas, M. Walter), general problems of connection of conducting with exact sciences (M. Mannone), and, especially, the history of the development of a new type of symphony orchestra in China (Hon-Lun Yang & M. Saffle, Mingyen Lee). Hence, it calls for an integrated approach, which includes historical-typological, comparative, genrestyle and dialectic methods. Results and Conclusions. Active involvement of European music has been taking place in China relatively recently – since the 1920s. In this way, the problems of creating a national conducting school and a symphonic culture of performance in the country became especially acute. The history of the development of the symphony orchestra and conductor’s performance of Chinese musical art of the twentieth century can be imagined as a single culture process that is constantly moving in the same vein encompassing all countries and continents. During the 20th century a new profession of musician-conductor of a new type of orchestra was formed in China, which combined instruments of the European symphony orchestra and national instruments. The formation of national symphonic schools in China is also linked to specific historical events: the modernization of Chinese music, the foreign education of many Chinese musicians, the influence of communist ideology (“exemplary performances”), on the one hand, and Western capitalist (Hong Kong orchestras), on the other. The specificity of the activity of the conductor of a new type of symphony orchestra is directly related to various changes in the quantitative and qualitative composition of the orchestra, special features for use of both individual instruments of the symphony orchestra and whole sections, and to technical capabilities of national and European instruments.
Paperback, 15,3 x 22,7 cm 433 pages, 108 ill., € 56.00 ISBN 978-3-89693-263-1 Body Sign Culture, Volume 19 Editors: Hartwig Kalverkämper, Reinhard Krüger, Roland Posner Communication is multimodal. In everyday interaction we do not communicate only by words, but by our whole body. We talk by gestures, facial expression, gaze, body movements , posture, and these communicative modalities interact with each other in subtle and complex ways. But can we disentangle the different sounds in a symphony, the different pieces in a mosaic? This book claims that the communication scholar can write down the musical score of the communicative symphony by attributing a specific meaning to each single signal-to each gesture, gaze, facial expression-and by finding out lexicons of all communicative modalities. If Linguists have been writing dictionaries of verbal languages for millennia, why not start compiling a new type of dictionaries, and discover the lexicons and the alphabets of gestures, gaze, or touch? Part I of this book (Mind) presents a cognitive model of communication in terms of the notions of goal and belief; Parts II (Hands) and III (Face) analyse gestural and facial communication in detail, by distinguishing universal and cultural aspects in gesture and gaze, showing the differences between gestures that are codified in our mind and gestures that we create on the spot, and teaching how to make a dictionary of touch or how to find the meanings conveyed by the eyebrows. Part IV (Body) presents an annotation scheme to transcribe and analyse signals in all modalities and to capture the meaning of their interaction, that has proved useful for empirical research on multimodality and for its simulation in Embodied Conversational Agents; to illustrate the potentialities of this tool, multimodal discourses are analysed, taken from TV talk shows, political discourse, classroom interaction, speech-therapy sessions, judicial debates, university examinations and comic movies. The subtleties of multimodality are dissected, showing how the whole body can be a tool for indirect and contradictory messages, deception, joke, irony and other sophisticated uses of communication. Prof. Dr. Isabella Poggi teaches General Psychology and Psychology of Communication at Roma Tre University. She works to the construction of a cognitive model of mind, social interaction and communication, through conceptual analysis, observative research and simulation in Embodied Agents. After her first research about the teaching of Italian as a first language, she has published books and papers about emotions (guilt, shame, humiliation, pity, enthusiasm), deception, persuasion, verbal and multimodal communication in humans and machines.
The paper argues for the importance and richness of gaze communication during orchestra and choir conduction, and presents three studies on this issue. First, an interview with five choir and orchestra conductors reveals that they are not so deeply aware of the potentialities of gaze to convey indications in music performance. A conductor who was utterly conscious of the importance of gaze communication, however, is Leonard Bernstein, who conducted a performance of Haydn’s Symphony No. 88 using his face and gaze only. Therefore, a fragment of this performance is analyzed in an observational study, where a qualitative analysis singles out the items of gaze exploited by Bernstein and their corresponding meanings. Finally, a perception study is presented in which three of these items are submitted to expert, non-expert, and amateur participants. The results show that while the signal for “start” is fairly recognized, the other two, “pay attention” and “crescendo and accelerando” are more difficult to interpret. Furthermore, significant differences in gaze item recognition emerge among participants: experts not only recognize them more, but they also take advantage of viewing the items with audio-visual vs. video-only presentation, while non-experts do not take advantage of audio in their recognition.
This chapter focuses on the relationship between gestures and persuasive discourse. It begins with an overview of the history of gesture research and studies about gesture expressivity. It then presents a model of persuasive discourse in terms of goals and beliefs. This model is illustrated by using case studies on the gestural behaviour of famous politicians. Finally, this chapter discusses how such a model can be used to implement persuasive gesturing in an embodied conversational agent.
The research literature concerning gesture in musical performance increasingly reports that musically communicative and meaningful performances contain highly expressive bodily movements. These movements are involved in the generation of the musically expressive performance, but enquiry into the development of expressive bodily movement has been limited. In two studies this paper explores the expressive components of bodily movement in both solo and ensemble musical performance. The first study examined flute and clarinet performers in both solo and duo settings. Whilst each player had a specific way of expressing musical goals through their bodily movement, there were features common to the woodwind instruments investigated. Detailed analyses revealed that, although many movements were possible, performers used only six basic expressive gesture types. The second study described a performance of the internationally celebrated pianist, Lang Lang, focusing on the relationship between musical affect, bodily movement and facial expression. Analysis also revealed extensive and striking use of combined bodily and facial expression, which were involved in articulating structural features of the music and the narrative of the work. Findings suggest the existence of a repertoire of expressive information used for the generation of expressive ideas, and available to the observers of music performers.
Building on the insights of the first volume on Music and Gesture (Gritten and King, Ashgate 2006), the rationale for this sequel volume is twofold: first, to clarify the way in which the subject is continuing to take shape by highlighting both central and developing trends, as well as popular and less frequent areas of investigation; second, to provide alternative and complementary insights into the particular areas of the subject articulated in the first volume. The thirteen chapters are structured in a broad narrative trajectory moving from theory to practice, embracing Western and non-Western practices, real and virtual gestures, live and recorded performances, physical and acoustic gestures, visual and auditory perception, among other themes of topical interest. The main areas of enquiry include psychobiology; perception and cognition; philosophy and semiotics; conducting; ensemble work and solo piano playing. The volume is intended to promote and stimulate further research in Musical Gesture Studies. © Anthony Gritten, Elaine King and the Contributors 2011. All rights reserved.
Musicians sway expressively as they play in ways that seem clearly related to the music, but quantifying the relationship has been difficult. We suggest that a complex systems framework and its accompanying tools for analyzing non-linear dynamical systems can help identify the motor synergies involved. Synergies are temporary assemblies of parts that come together to accomplish specific goals. We assume that the goal of the performer is to convey musical structure and expression to the audience and to other performers. We provide examples of how dynamical systems tools, such as recurrence quantification analysis (RQA), can be used to examine performers' movements and relate them to the musical structure and to the musician's expressive intentions. We show how detrended fluctuation analysis (DFA) can be used to identify synergies and discover how they are affected by the performer's expressive intentions.
Teachers’ communication of musical knowledge through physical gesture represents a valuable pedagogical field in need of investigation. This exploratory case study compares the gestural behaviour of three piano teachers while giving individual lessons to students who differed according to piano proficiency levels. The data was collected by video recordings of one-to-one piano lessons and gestures were categorized using two gesture classifications: the spontaneous co-verbal gesture classification (McNeill, 1992; 2005) and spontaneous co-musical gesture classification (Simones, Schroeder & Rodger, 2013). Poisson regression analysis and qualitative observation suggest a relationship between teachers’ didactic intentions and the types of gesture they produced while teaching, as shown by differences in gestural category frequency between teaching students of higher and lower levels of proficiency. Such reported agreement between teachers’ gestural approach in relation to student proficiency levels indicates a teachers’ gestural scaffolding approach whereby teachers adapted gestural communicative channels to suit students’ specific conceptual skill levels.
The significance of the “physicality” involved in learning to play a musical instrument and the essential role of teachers are areas in need of research. This article explores the role of gesture within teacher–student communicative interaction in one-to-one piano lessons. Three teachers were required to teach a pre-selected repertoire of two contrasting pieces to three students studying piano grade 1. The data was collected by video recordings of piano lessons and analysis based on the type and frequency of gestures employed by teachers in association to teaching behaviours specifying where gestures fit under (or evade) predefined classifications. Spontaneous co-musical gestures were observed in the process of piano tuition emerging with similar general communicative purposes as spontaneous co-verbal gestures and were essential for the process of musical communication between teachers and students. Observed frequencies of categorized gestures varied significantly between different teaching behaviours and between the three teachers. Parallels established between co-verbal and co-musical spontaneous gestures lead to an argument for extension of McNeill’s (2005) ideas of imagery–language–dialectic to imagery–music–dialectic with relevant implications for piano pedagogy and fields of study invested in musical communication.
This paper presents the results of a study of conductor
gesture and examines the effect of that gesture on
musician action within a tertiary music rehearsal and
pedagogical context. This study follows experimental
work on conductor gesture and attempts to complement
the results of experimental studies by using a corpus of
naturally-occurring video recorded data of conducting
classes. The aim is to determine whether the physical
actions (gestures) of conductors can be seen to be
systematically related to musician action (playing of
instruments) in such naturally occurring data. In addition,
we explore how quantitative analytical techniques, more
familiar to experimental approaches, can be used as the
basis for a framework for examining conductor gesture in
data collected using ethnographic research approaches.
This study concludes that conductor gesture has a
significant effect on musician action, and suggests some
salient features of conductor gesture.
Conducting is a complex art that involves, among other things, effective nonverbal communication. While this communication includes hand and arm gestures as well as eye contact and facial expression, it also encompasses nonverbal messages sent by the rest of a conductor's body, such as his/her physical appearance. Assumptions based on what conductors look like and how well they fit the stereotypical role they are portraying often affect opinions of their job performance. Thus, the nonverbal messages sent by a conductor's physical appearance could affect ensemble members' opinions of him or her as a conductor. Therefore, this study attempted to investigate whether perceptions of a conductor's body type were a factor when judging the conductor and ensemble performances. Furthermore, relationships between ensemble performance ratings and the conductor's visual appearance characteristics (eye contact, facial expression, and posture), evaluator confidence in the conductor, and overall conductor effectiveness were explored. Neither the conductor's body type nor the gender or college major of the evaluator affected ratings of the conductor and ensemble performances. However, performance ratings were affected when the conductors were viewed in a certain order. There were moderate to moderately strong relationships between the performance ratings and conductor posture, conductor facial expression, evaluator's confidence in the conductor, and overall conducting effectiveness. Eye contact, however, was not strongly related to performance ratings.
This article reports an empirical investigation into the characteristics of conductors' temporal gestures and people's ability to synchronize with them. It describes a new approach to the investigation of the characteristics of conductors' gestures, the investigation of conductor-musician synchronization, and a first experiment to demonstrate the feasibility of the approach. As part of this approach, a new computer-based environment for manipulating and presenting conductors' gestures, and recording participants' responses, was developed. This involved the creation of several pieces of new software, all of which were developed under Microsoft® Windows® using Visual C++. Used in combination, these programs allowed: (1) the manipulation of the size and viewing angle of high-quality three-dimensional (3D) recordings of conductors' gestures; (2) the presentation of these gestures to participants in a controlled experimental setting; (3) the recording of participants' temporal responses to these gestures; (4) the calculation of various parameters of both the trajectory of the gestures (e.g. instantaneous speed, radius of curvature along the trajectory) and participants' responses (e.g. mean response point and associated standard deviation); and (5) the graphical display of relevant features of both the gestures and participants' responses in a clear visual form. In the experiment, participants tapped in time with simple conducting gestures while several factors that might be expected to affect synchronization accuracy were manipulated. These factors were (1) the radius of curvature with which the beat was defined; (2) the experience level of the conductor; and (3) the experience level of participants. Results indicated that only participants' previous experience affected their synchronization ability; no effects of conductors' previous experience, or radius of curvature with which the beat was defined were found. This first experiment successfully demonstrated the feasibility of the new approach for the investigation of conductors' gestures and conductor- musician synchronization, and this article concludes by suggesting a number of subsequent experiments that could be undertaken in this computer-based environment to investigate these topics further. Copyright
The paper presents a model of the motor, cogni- tive and emotional processes taking place in pi- ano performance, and an annotation scheme for the analysis of the pianist's body movements. By applying the scheme to a concert and a re- hearsal, the paper finds similarities and differ- ences in the pianist's body movements depending on the musical stracture of the piece and on the performance situation, and finally presents a lexicon of the pianist's body movements.
The paper defines as "social signal" a communicative or informative signal that, either directly or indirectly, conveys information about social actions, social interactions, social emotions, social attitudes and social relationships. It proposes a conceptual definition of these social facts and some examples of relevant social signals.
The paper presents a framework for classifying gestures in terms of different parameters, and shows that the parameter of iconicity cuts across that of cognitive construction, which distinguishes codified gestures — those represented in memory as stable signal-meaning pairs — from creative ones — those invented on the spot on the basis of a few generative rules. While creative gestures are necessarily iconic, because they can be understood only thanks to their iconicity, codified gestures can be iconic too. A model for the generation of iconic gestures is presented, according to which, to create a new gesture, people select the features of the meaning to imitate in terms of their distinctiveness, ease of representation and ontological type. Finally, some principles for measuring the iconicity of gestures of Hearers and Signs of the Deaf are illustrated, thus outlining the continuity of iconic devices between creative and codified gestures.
This study investigated the perception of conductors' body movements under different viewing conditions with a multi-modal within-subjects design. Five conductors with different levels of experience each conducted four excerpts from a Beethoven symphony that varied in musical expressiveness. Video recordings were manipulated according to three occlusion conditions: a) only the face was visible, b) only the arms were visible and c) the whole body was visible in simulated peripheral vision. 127 musically trained or untrained participants first watched, without sound, randomly presented video sequences according to these conditions. For each sequence, they were asked to rate affective and communicative items. Complete video sequences with sound were then presented as a reference and rated similarly. Video sequences that presented the conductors' faces resembled the reference significantly better than the arms-only or the peripheral conditions in terms of expressiveness ratings. Sequences showing the arms were judged higher in amount of information. For the movements of all conductors, clear interpretation differences between the four musical excerpts appeared even for conditions without sound. Differences between conductors were related to their general affective behaviour and to evaluations of four conducting experts. The findings generally highlight the importance of facial affective behaviour for expressive conducting.
The paper discusses how distributed logic programming can be used to define and control the hand gestures of embodied agents in virtual worlds, by using the STEP language as an interface between the constructs of logic pro-gramming and the humanoid model defined in VRML. By using this frame-work, different gesture dictionaries can be defined and variants of a hand ges-ture, according to dynamically changing factors, can be generated on the fly. The approach is tested on the demanding demonstrator of conducting, providing experience, also on time performance of the animation.
In this paper we describe a system which allows users to use their full-body for controlling in real-time the generation of an expressive audio-visual feedback. The system extracts expressive motion features from the user's full-body movements and gestures. The values of these motion features are mapped both onto acoustic parameters for the real-time expressive rendering of a piece of music, and onto real-time generated visual feedback projected on a screen in front of the user.
In previous computer systems for "conducting" a score, the control is usually limited to tempo and overall dynamics. We suggest a home conducting system allowing an indirect control of the expressive musical details on the note level. In this system, the expressive content of human gestures is mapped into semantic expressive descriptions. These descriptions are then mapped to performance rule parameters using a real time version of the KTH rule system for music performance. The resulting system is intuitive and easy to use also for people lacking formal musical education, making it a tool for the listener rather than the professional performer.
This paper presents a virtual embodied agent that can conduct musicians in a live performance. The virtual conductor conducts music specified by a MIDI file and uses input from a microphone to react to the tempo of the musicians. The current implementation of the virtual conductor can interact with musicians, leading and following them while they are playing music. Different time signatures and dynamic markings in music are supported.
Ensemble musicians coordinate their actions with remarkable precision. The ensemble cohesion that results is predicated upon group members sharing a common goal; a unified concept of the ideal sound. The current chapter reviews research addressing three cognitive processes that enable individuals to realize such shared goals while engaged in musical joint action. The first process is auditory imagery; specifically, anticipating one's own sounds and the sounds produced by other performers. The second process, prioritized integrative attention, involves dividing attention between one's own actions (high priority) and those of others (lower priority) while monitoring the overall, integrated ensemble sound. The third process relates to adaptive timing, i.e., adjusting the timing of one's movements in order to maintain synchrony in the face of tempo changes and other, often unpredictable, events. The way in which these processes interact to determine ensemble coordination is discussed.
The KTH rule system models performance principles used by musicians when performing a musical score, within the realm of Western classical, jazz and popular music. An overview is given of the major rules involving phrasing, micro-level timing, metrical patterns and grooves, articulation, tonal tension, intonation, ensemble timing, and performance noise. By using selections of rules and rule quantities, semantic descriptions such as emotional expressions can be modeled. A recent real-time implementation provides the means for controlling the expressive character of the music. The communicative purpose and meaning of the resulting performance variations are discussed as well as limitations and future improvements.
The work presents two studies investigating the existence of a lexicon of gaze in conducting, and its possible different mastery in musicians and laypeople. An observational qualitative study singled out 17 items of gaze used by Conductors in music rehearsal and concert, conveying interactional, affective and musical meanings to musicians in the ensemble, and exploiting four semiotic devices: the Conductor may use the same gaze types as laypeople and with the same meaning (generic codified), or with meaning more specific of musical performance (specific codified), and directly or indirectly iconic gaze items. In a subsequent perceptual study, 8 of the gaze items singled out were submitted to 177 between musicians and naïf subjects asking them to interpret their meanings through open and closed questions. Results show that some gaze items, especially those conveying intensity (piano, forte) and other technical indications (high note, attack) are fairly recognized; yet, no significant differences result between expert and naïf subjects. Gaze constitutes a lexicon also in music performance, and exploits the same semiotic devices as gaze in everyday life.
Based on a model of communication according to which not only words but also body signals constitute lexicons (Poggi, 2007), the study presented aimes at building a lexicon of conductors’ multimodal behaviours requesting intensification and attenuation of sound intensity. In a corpus of concerts and rehearsals, the conductors’ body signals requesting to play or sing forte, piano, crescendo, diminuendo were analysed through an annotation scheme describing the body signals, their meanings, and their semiotic devices: generic codified (the same as in everyday language); specific codified (shared with laypeople but with specific meanings in conduction); direct iconic, (resemblance between visual and acoustic modality); indirect iconic, (evoking the technical movement by connected movements or emotion expressions). The work outlines a lexicon of the conductors’ signals that in gesture, head, face, gaze, posture, body convey attenuation and intensification in music.
Gestures are ubiquitous and natural in our everyday life. They convey information about culture, discourse, thought, intentionality, emotion, intersubjectivity, cognition, and first and second language acquisition. Additionally, they are used by non-human primates to communicate with their peers and with humans. Consequently, the modern field of gesture studies has attracted researchers from a number of different disciplines such as anthropology, cognitive science, communication, neuroscience, psycholinguistics, primatology, psychology, robotics, sociology and semiotics. This volume presents an overview of the depth and breadth of current research in gesture. Its focus is on the interdisciplinary nature of gesture. The twenty-six chapters included in the volume are divided into six sections or themes: the nature and functions of gesture, first language development and gesture, second language effects on gesture, gesture in the classroom and in problem solving, gesture aspects of discourse and interaction, and gestural analysis of music and dance.
As of March 2017, this e-book is freely available, thanks to the support of libraries working with Knowledge Unlatched. It is licensed under a Creative Commons CC BY-NC-ND license.
Indian vocalists trace intricate shapes with their hands while improvising melody. Although every vocalist has an idiosyncratic gestural style, students inherit ways of shaping melodic space from their teachers, and the motion of the hand and voice are always intimately connected. Though observers of Indian classical music have long commented on these gestures, Musicking Bodies is the first extended study of what singers actually do with their hands and voices. Matthew Rahaim draws on years of vocal training, ethnography, and close analysis to demonstrate the ways in which hand gesture is used alongside vocalization to manifest melody as dynamic, three-dimensional shapes. The gestures that are improvised alongside vocal improvisation embody a special kind of melodic knowledge passed down tacitly through lineages of teachers and students who not only sound similar, but who also engage with music kinesthetically according to similar aesthetic and ethical ideals. Musicking Bodies builds on the insights of phenomenology, Indian and Western music theory, and cultural studies to illuminate not only the performance of gesture, but its implications for the transmission of culture, the conception of melody, and the very nature of the musicking body.
Body movement plays a role in the construction, execution, and perception of musical performances. This chapter explores the interface between technical matters of physical control and the expressive components of physical gestures, and discusses the bodily origins of musical meaning, expressive performance, and musical skill acquisition. For example, bodily gesture and rhythm in protomusical mother-child exchanges influence the development of thought and knowledge, and expressive slowing in music (ritardando) corresponds to the deceleration of runners coming to a halt. Specific movement gestures in music performance function as illustrative and emblematic cues and clearly indicate the focus of the performer's attention, whether on the narrative content of a song or on showing off to the audience. Thus, through body movement thoughts and concerns are communicated to the audience. Performers, educators, and students can use this knowledge to enhance their performing, teaching, and learning capacities.
The present study investigates expressive non-verbal interaction in the musical context starting from behavioral features extracted at individual and group levels. Four groups of features are defined, which are related to head movement and direction, and may help gaining insight on the expressivity and cohesion of the performance, discriminating between different performance conditions. Then, the features are evaluated both at a global scale and at a local scale. The findings obtained from the analysis of a string quartet recorded in an ecological setting show that using these features alone or in their combination may help in distinguishing between two types of performance: (a) a concert-like condition, where all musicians aim at performing at best, (b) a perturbed one, where the 1 violinist devises alternative interpretations of the music score without discussing them with the other musicians. In the global data analysis, the discriminative power of the features is investigated through statistical tests. Then, in the local data analysis, a larger amount of data is used to exploit more sophisticated machine learning techniques to select suitable subsets of the features, which are then used to train an SVM classifier to perform binary classification. Interestingly, the features whose discriminative power is evaluated as large (respectively, small) in the global analysis are also evaluated in a similar way in the local analysis. When used together, the 22 features that have been defined in the paper demonstrate to be efficient for classification, leading to a percentage of about 90 % successfully classified examples among the ones not used in the training phase. Similar results are obtained considering only a subset of 15 features.
This chapter reviews theoretical concepts and empirical findings on musical emotions. First, it examines the notion of music as a means of communicating emotion and presents some relevant evidence concerning the issue. Then it provides a working definition of emotions and some conceptual distinctions for the study of musical emotion. It reviews mechanisms through which music may express and induce emotions. Finally, it considers various objections to music-as-communication and provides an agenda for future research. The discussion is limited to Western music, especially classical and popular music from the 18th century to present day.
This paper examines the development and implementation of general social and specific non-verbal communication between two expert pianists who prepared and gave a recital of piano duo and duet music. All ensemble rehearsals and the final performance were video-taped. Following the performance, the musicians were interviewed in order to document their thoughts on the learning and performance processes. From the video-taped rehearsals and performance, data concerning musical coordination, social interaction, non-verbal gestures and looking behaviour were coded and counted. The results show that these excellent sight-readers used rehearsals to consolidate the timing, phrasing and sense of musical style. Moreover, an emergent set of coordinated, non-verbal gestures and eye-contact developed, with these actions increasing significantly over the rehearsal process at locations in the music identified by the pianists as “important for coordinating performance and communicating musical ideas”. Thus, the two performers acquired a deepening expressive and communicative assurance along with a familiarisation with the musical material. The findings are discussed in relation to their implications for musical performance by highlighting the elements of co-performer interaction that were negotiated and coordinated throughout the rehearsal process.
Body movements during music performance have been found to be indicative of the performer’s musical intentionality, and contribute to an observer’s perception of expressive playing. This study investigates the effect of structural elements of the score, and the playing of different levels of expression on body movements during a piano performance. Pianists were required to play the same piece in four different performance conditions. Their movements were tracked by an optical motion capture system, and the comparisons that were made between specific parts of the body used, performance condition, and musical score locations were subsequently statistically examined. We found that the head and shoulders exhibited more movement per measure, as well as larger differences between each condition, than the fingers, wrists and lower back. Differences between performance conditions were observed primarily at structurally significant portions of the score, and biomechanical factors also played a role. Moreover, our data supports the view that performers equate playing without expression to playing without nonessential movements.
The purpose of this study was to begin to explore the link of gesture and voice when teaching children and to examine how the children used these gestures in singing. Semi-structured interviewing techniques and observation were used extensively to collect information from the respondents. Five children were asked to participate in this preliminary work individually. The results demonstrated that there was a link between children's singing voice and their use of gesture by observing the Dalcrozian mirror and follow games. The functions of gestures could be defined as being for focus, guidance, support, good intonation and sensation. In addition, some specific teaching techniques were required for achieving a more effective gesture and a more successful sound. Gesture techniques helped the children to improve their vocal techniques and correct their vocal faults. This study opened the field for the full study of gesture in aiding children's singing.
This article is a contribution to the psychology of performance, investigating the role of body movements both globally and locally in expressive musical performance. A case study of a single pianist is used to explore qualitative similarities and differences within and across different performances of a Beethoven bagatelle. Using observational analyses of the performance movements, the results reveal that the performer uses particular movement shapes that are specific and identifiable `expressive locations' within the context of a whole performance. These movement shapes cannot simply be categorized as being either intention specific or musical structure specific, for there are movement shapes common to locations across different performances and different locations within a performance. These expressive movements have some consistency over time, with the locations of the expression being common, but the movements being used flexibly within and across manner and time. The results are theorized in terms of how these movements may be produced.
The study of music perception has focused almost exclusively on sound, ignoring the role of seeing the performer's body movements. Whilst anecdotes frequently refer to the importance of the performer's movements, there is scant psychological evidence to support this finding. The closest equivalent work in visual event perception research has shown that covert mental dispositions (for instance, an intention to deceive an observer) are specified in body movements, and therefore provide important information for the observer.. With these findings in mind, this article investigates the information conveyed by the movements of a musical performer when s/he is asked to play-the same piece in three different expressive manners. These performance manners are presented to observers in three modes: vision alone, sound alone and sound and vision together to investigate the relative contributions of the different perceptual modes. The results reveal that not only is vision a useful source of information about manner, but that it actually more clearly specifies manner than the other modes. These findings emphasise the need to consider visual as well as sound information in psychological enquiries into music perception.
The work described in this paper interprets the body movements of singers in an attempt to understand the relationships between physical control and the musical material being performed, and the performer's implicit and explicit expressive intentions. The work builds upon a previous literature which has suggested that the relationship between physical execution and the expression of mental states is a subtle and complex one. For instance, performers appear to develop a vocabulary of expressive gestures, yet these gestures – though perceptually discreet – co-exist and are even integrated to become part of the functional movement of playing. Additionally, there is the matter of how both musical and extra-musical concerns are coordinated between performer, co-performers and audience using body movements. A case study shows how, in the interaction between body style, musical expression and communication movements of both an individual and culturally-determined style are used. Many of these performance movements have clear functions and meanings: to communicate expressive intention (for instance, a sudden surge forwards to facilitate the execution of a loud musical passage, or a high curving hand gesture to link sections of the music during a pause); to communicate to the audience or co-performers a need for co-ordination or participation (for example, nodding the head to indicate “now” for the audience to join in a chorus of a song; or exchanging glances for the co-performer to take over a solo); to signal extra-musical concerns (for example, gesturing to the audience to remain quiet); and to present information about the performer's personality, with their individualized characteristics providing important cues (muted contained gestures, or large extravagant gestures, for example); to show off to the audience. From these results a theory is developed to explain how gestural elements help to make a performance meaningful.
This article presents results from an ongoing exploratory analysis of ancillary gestures of several advanced clarinetists playing standard concert solo repertoire. We build upon our previous work involving the analysis of clarinetists' gestures by focusing on the timing of various performance manners, the relation that ancillary gestures have to the musical score, the different styles of expressive movements among performers, and the perception of these movements by the audience.
We present the first multimedia system to conduct a realistic electronic orchestra. Users can control tempo, dynamics, and instrument emphasis of the orchestra through natural conducting gestures with an infrared baton. Using gesture recognition and tempo adjustment algorithms, the system plays back an audio and video recording of an actual orchestra that follows the users conducting in real time. A major achievement of this system is its ability to vary playback speed in real time while avoiding audio artifacts such as pitch changes. The system has been deployed as an exhibit and has become a major attraction of a large Vienna-based music exhibition center.
Thesis (Ph. D.)--University of Wisconsin-Madison, 1985. Vita. Includes bibliographical references (leaves 159-169). Microfiche. s
Thesis (Ph. D.)--Ohio State University, 1992. Includes bibliographical references, (leaves 95-98). Advisor: A. Peter Costanza, School of Music.
It is approaching a half century since Bill Stokoe published his revolutionary monograph, Sign Language Structure: An Outline of the Visual Communication Systems of the American Deaf It is rare for a work of innovative scholarship to spark a social as well as an intellectual revolution, but that is just what Stokoe's 1960 paper did. And it is indicative both of Stokoe's genius and of his commitment that he did not simply publish his groundbreaking work and then sit back to watch the revolutions unfold. He actively promoted important changes in at least three areas of social and intellectual life. First, and perhaps most important, his work, that was ultimately generally accepted as showing the signing of deaf people to be linguistic, supported significant changes in the way deaf children are educated around the globe. Second, his work led to a general rethinking of what is fundamental about human language; and, third, it helped to reenergize the moribund field of language origin studies. This truly revolutionary paper has been reprinted at least twice, in revised and original versions, since its initial release in 1960, and now, five years after Bill's death, it is good to see it once again brought before the general public. - David F. Armstrong, Gallaudet University.