Conference PaperPDF Available

Man in |e|space.mov / Motion Analysis in 3D Space.

Authors:
  • res publica, e-art sup

Abstract and Figures

The article documents the theoretical and aesthetical basis of the interactive dance performance "man in |e|space.mov". The text discusses the abstraction of the human body in this performance by an interactive costume of light whose motion is analyzed by a 3D motion-rendering programme, which assembles and recombines the captured frame in real time in electronic 3D space. Thus appears a juxtaposition of 3 visions on the body making up the representation: the eye of the spectator, the camera and the 3D camera view: 3 visions constituting the contemporary body. Furthermore, the text questions the dislocation of the sensual body in physical space to the reading of body as a data in the matrix of virtual space in performing arts.In order to investigate the meaning of the aesthetics of abstraction, and dislocation of human body in performance art and motion analysis, the article puts 'man in |e|space.mov' in perspective to historical references of the 20th century, in particular to the work of the physiologist and pioneer of cinema J. E. Marey and the Bauhaus artist O. Schlemmer.
Content may be subject to copyright.
Man in |e|space.mov / Motion Analysis in 3D Space
Wolf Ka
company res publica
13, rue Germain Pilon
75018 Paris
+33 1 42 62 29 03
compagnie_respublica@yahoo.fr
ABSTRACT
The article documents the theoretical and aesthetical basis of the
interactive dance performance “man in |e|space.mov”. The text
discusses the abstraction of the human body in this performance
by an interactive costume of light whose motion is analyzed by a
3D motion-rendering programme, which assembles and
recombines the captured frame in real time in electronic 3D space.
Thus appears a juxtaposition of 3 visions on the body making up
the representation: the eye of the spectator, the camera and the 3D
camera view: 3 visions constituting the contemporary body.
Furthermore, the text questions the dislocation of the sensual
body in physical space to the reading of body as a data in the
matrix of virtual space in performing arts.
In order to investigate the meaning of the aesthetics of
abstraction, and dislocation of human body in performance art
and motion analysis, the article puts ‘man in |e|space.mov’ in
perspective to historical references of the 20th century, in
particular to the work of the physiologist and pioneer of cinema J.
E. Marey and the Bauhaus artist O. Schlemmer.
Categories and Subject Descriptors
J.5 [Arts and humanities] --- Performing arts- dance, D.4.7
[Interactive systems] --- real-time systems and embedded
systems, I.2.10 [Vision and Scene Understanding] --- Motion,
Video analysis
General Terms
Performance, documentation.
Keywords
Interactive dance performance, real time 3D rendering
application, digital motion analysis, Marey, Schlemmer, res
publica, Lab[au].
1. INTRODUCTION
The performance “man in |e|space.mov” was elaborated in
collaboration between the French digital performance art
company “res publica” and the Belgian laboratory for architecture
and urbanism LAb[au]. It is the second interdisciplinary work
after “Enjeux” [2] between dance, architecture and digital media
of the two groups. Both groups question the relation of man and
space in the age of digital technologies. Res publica investigates
its viewpoint from theatre, LAb[au] from architecture.
In “man in |e|space.mov”, we refer to and adapt the experimental
work of artists such as Oskar Schlemmer and his research on the
relation “man in space” as well as the cinematic work of Etienne
Jules Marey (.mov). Man in |e|space.mov combines both
approaches while applying them to digital media (|e|), which
cover various art forms such as choreography, cinematography,
scenography, dance, music and architecture.
The article provides a brief overview on relevant aspects of
Schlemmer’s and Marey’s works. We then describe the elements
of “man in |e|space.mov” and conclude with a short evaluation
and some remarks of future work.
2. HISTORICAL REFERENCES
2.1 Marey - From written records to iconic
language
The works by the physiologist J. E. Marey [1] established the
foundation for a quantitative analysis of human motion resulting
in the scientific representation of classified parameters. He
developed a number of devices and sensors to record mainly
human motion but also other living beings. In the 1870s he
recorded movements first by the graphic method that translates
movement into lines, curves or graphs inside the Cartesian
coordinate system. In the 1880s, he integrated photography in his
motion analysis, which culminates in its famous partial
chronophotography (see figure 1).
With these new methods, he represented the shift from the written
record of scientific observation to the treatment of data coming
from devices between the observer and the object. By the
extraction, decomposition and discrimination of data, the method
permits the schematic visualization of complex movements. His
method substituted the terms, the verbal description and the
written argumentation by the synthetic comprehension of the
iconic description. The body with its cultural connotation
disappears in these devices and an abstract vision of pure (human)
motion reappears in the Cartesian matrix.
The shift from the verbal description of the “sensual body” to the
production of data from specific parameters of the disconnected
and reconnected body, influenced performing arts, especially
dance. At the same period the modern or contemporary dance was
born, that refuses the narrative, literature-based aspects of
traditional ballet. Modern dance develops strategies of
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
MM’05, November 6–11, 2005, Singapore.
Copyright 2005 ACM 1-59593-044-2/05/0011…$5.00.
representation of motion where the body interacts with the
abstract space of the stage. This period represent also the shift
from a written-based society towards a society of visual and
communication technologies. [4]
2.2 Oskar Schlemmer: man and space, a
codified relationship
Oskar Schlemmer [5] was one of the first theatre artists who
worked in a systematic and radical way on the human body as a
code, and the relation of man and space. He proposed a
perspective of body on stage as an abstract and symbolic
representation, as opposed to the psychological and expressive
approaches of German Dance in the 1920’s. In his stage work, he
intended to represent the exemplary, the universal instead of
showing the individual or the typical. His theatre works did not
intend to be a copy of reality, but an artificial (art) product in
order to bring up the essential, namely the (pure) idea.
He examined the gesture/ movement as a relation of man and
space that results in the formalization of human motion to
geometrical forms. He opposes the laws of the cubic space of the
stage to the laws of natural man. If the space is adapted to man,
the stage becomes naturalistic or illusionist. If man is adapted to
the cubic space the stage becomes abstract. From his viewpoint,
the laws of the abstract stage are the invisible lines of planimetric
and stereometric relationships [see Figure 3, left top corner –
Figur und Raumlineatur, (figure and spatial delineations)].
Schlemmer was aware of Marey’s work when he designed his
project “Stäbetanz” (Stick dance). In here, he extended the limbs
with white wooden sticks of a dancer dressed in black to
emphasize the relation between man and space and the spatial
delineations. He proceeded in the same way as Marey did, when
he produced his partial chronophotography: the elaboration of a
device, which erased the body (black costume in front of a black
background) to reveal a particular data of human motion (white
lines/ white sticks on the limbs). In both examples, there is a
dislocation of the represented body as a psychological- physical
unit towards a representation of selected principles, parameters/
data inside a matrix (the film/ the cubic stage). This represents the
shift from the approaches of the body as a cultural unity towards a
mathematical relation between man and space. [6]
3. Man in |e|space.mov DISCRIPTION
The performance ‘man in |e|space.mov ’, created in 2004,
examines the notion of space and movement, facilitating new
perception and understanding of the relationship between body
and space, through the integration of communication and
computation technologies into a dance performance.
It is based on techniques that marked the beginning of the cinema
but introduces a new reading of the body movement. Thus, the
project renews the process of reduction and abstraction of the
human body and its decomposition in 24 images/sec. through the
contribution of electronics and digital technologies. In that way it
questions the representation of movement by merging two forms
of “motion writings”: cine[mato]graphy and choreo-graphy.
3.1 Man in |e|space.mov: abstraction of the
human body
The first element of the project, a costume of light, questions the
movement of the body through structuring it by a luminous
device, which focuses the perception on the geometric abstraction
of the body and stripped of any cultural and psychological
connotations.
The costume consists of electroluminescent wires aligned to the
limbs of the dancer, which make the movement of the body
visible via vertical lines. An electronic interactive component
pilots wireless the luminous-electronic infrastructure to offer the
possibility to fragment the perception of the body. The body thus
becomes a medium of information that is the result of the
electronic infrastructure and the choreography. The costume
reveals a reading of the movement through its perceptive
transformation, structured by a geometrical and abstract approach
of the body. The relation between time, space and body/
movement is focalized through the signalization of the body by a
few lines, revealing it in a pure way.
Figure 1. Jules Etienne Marey, partial
chrono
p
hoto
g
ra
p
h
y
and motion ca
p
ture costume
Figure 3. O. Schlemmer, “Stäbetanz”, 1928 and “Figur und
Raumlineatur”, 1926
Its conception refers to the work of Schlemmer, by reinforcing the
dislocation of the represented body towards a representation of
selected principles (the movement of the limbs), parameters/ data
(the representation of the movement) inside a matrix (the stage).
3.2 Man in |e|space.mov: motion analysis in
3D space
The second element is the additional visualization of the dancer’s
movements. For that the dancer is filmed on a dark stage and
projected on two video-screens. One screen is at the back of the
stage and the second one, disposed in 45° at the front of the stage.
The movement of the dancer signalized by the electroluminescent
device is analyzed and represented in a virtual 3D environment.
The main analysis mechanism is based on isolating the bodylines
from the rest of the captured image. As the body is illuminated by
itself, no other light is needed, and the black part of the captured
images can be put in transparency by using their alpha-channels.
The captured image in real time is assembled and recombined in
the virtual 3D space (Figure 4), and projected on both screens, the
frontal view of the virtual 3D camera on the rear projection and a
45° sloped vision on the front projection: the scenography
recreates the virtual space as a physical setting.
In fact, there are two adjacent and complementary
representations, the three-dimensional one of the dancer-inter-
actor, represented in a diagrammatic and abstracted way, the other
representing the movement, through the effects of persistence or
geometrical construction in electronic space. The performance
integrates the process as well as the result of Marey’s device.
In the following section we describe the 3 key parts of the
performance in order to explain the play with the relation of
temporal and spatial construction of cinematographic movement
and the transcription or representation of movement in 3 D space.
4. Man in |e|space.mov EXAMPLES
The performance of about 40 min is composed by 7 sequences,
each assembling and combining differently the captured frames.
This dramaturgy is established while a VJ, present on stage,
manipulates in real-time a number of parameters of the
construction of the virtual space, such as the navigation of the
virtual camera in the 3D space or the transparency level of the
frames. Of the seven sequences, we describe the three most
essential: the first exploring in particular movement, the second is
devoted to time, and the last covers space.
4.1 First sequence: man walking at ordinary speed
A camera films the slow motion walking dancer on stage, while
capturing only 2 frames per second. These captured images are
placed inside a 3D space loop of 2 seconds, and is projected in
real time. Each loop adds new images to the sequence, thus
slowly building up a strip-man like animation out of the captured
images. Once 24 fps have been reached, the formed strip-man
juxtaposed to the performer on stage and thus constructs a second
‘virtual’ dancer moving in normal time. Through this
juxtaposition a dance or a movement, is constructed between the
dancer and the extracted images (see Figure 5). On the one hand,
the cinematic construction is directly derived from the dancer’s
movements but the temporal delay during the extraction capturing
distinct movements constitutes an unpredictable parameter that
facilitates autonomy to the self-residual images.
The choreography proceeds in real-time whereas the temporal
parameters of the different media employed are used to construct
the movement of man in |e|space.mov. This parametric approach
creates a mathematical, fragmented, analytic and linguistic
perspective of the dancer’s body to the digital space. That doesn’t
mean the subordination of the body under the laws of a machine,
but a way to prolong the investigations of Marey and to
deconstruct the non-linear and non-continuous construction of
cinematographic time and space [3].
As the sequence refers to the iconography of the well-established
work by Marey on motion-analysis by photography, it provides
the grounds for the ongoing development of movement
investigation in the performance. This sequence is, therefore, used
as the general introduction to acquaint the audience with the
technology as well as the thematic means of the performance.
4.2 Second sequence: body-particles
The sequence “body-particles” highlights the relationship
between the frequency of the captured images and the movement,
or between the captured image and the covered space in a given
time while referring to the most obvious analogy: the hand of a
watch. The dancer describes a circle with her forearm during one
minute while the camera captures and records one image/second,
which creates the equation between movement and time and its
denoted and connoted representation: a clock (see Figure 6).
Once the clock is recorded, the dancer disappears from the stage,
and particles in form of simple 3D shapes invade the screen
progressively, constructing the landmarks of the virtual 3D space.
The watch keeps on being present as a trace of the physical body
in the virtual space. It is mostly like the transformation of a
parametric presence of human motion that continues moving in
the 3D space. Through navigation, the trace of the body breaks up
its 2D reference marks X+Y and shows the construction of the
sequence in time, represented on axis Z. The decomposition of the
analogical image of time by the figure of the watch in the 3D
space finally reveals its construction in time.
Figure 5. Sequence “man walking at ordinary speed”
with scheme of the construction of frames in 3D s
p
ace
Figure 4. Scheme of the motion analysis device and the
frame construction in 3D s
p
ace.
4.2.1.1 Third sequence: 3D shadow
The last sequence explores the potential of the established
vocabularies to blend the real space, namely the stage, and the
auditorium, with the virtual representations of body movement in
time and space.
The sequence, as visualised in Figure 7, shows the individual
captured frames of the movement on the time axis Z, one behind
the other. The transparency of the superposed pictures does not
produce the illusion of a movement through retinal persistence,
but deconstructs the technique of this illusion in a 3D space. The
dancer, reduced to few lines and thus deprived far-going its 3
dimensional body (body space), finally produces a 3 dimensional
“space-body”, as time becomes the 3rd dimension. The reference
to Marey is here extended to the simultaneous presence of the
dancers movement and its process of analyse by the real-time
device while the 3D space reveals its temporal construction. Over
the navigable virtual space the represented movement becomes
the result of the choreography and the navigation in 3D space.
The three viewpoints, the eye of the public, the camera, filming
the dancer’s body and the virtual 3D camera, modifying the view
of the frames in 3D space, constituting together the two
synchronous representations of the dancing body: the dancer and
the dancing 3D “space-body”. In fact, it is the dancer and the VJ
who interact in real time, or in other words, they dance together
through the computer.
5. CONCLUSION
Since his creation, the performance has been shown several times
in varied contexts with different cultural backgrounds, such as
digital music festivals, exhibition openings, electronic arts or
dance festivals. The spectators with an art background, for who
Schlemmer and Marey are well known, appreciated the link
between science, modern art and digital culture. (Schlemmer is
more known as painter than choreographer and Marey is known
through abstract paintings’ reception, namely by M. Duchamps)
For the ‘dance public’ the relation between virtual and physical
representation of human motion was less evident as it focalized its
attention on the gesture quality of the dancer. For the music
public, involved in audio-visual culture by music clips and VJ
performances, it was simply a good show.
One of the main issues in digital performance art and also in this
performance is the transparency and the experience of the real-
time processing while the performance, which depends often on
the visibility of the minimal gestures operating a mouse or a
keyboard. This is one of the reasons that the following project
will leave the spatial separation of stage and auditorium to create
a symbiotic space. The public’s movement will be integrated into
the device by motion tracking in order to transpose the experience
of 3D space navigation to the public and to build up an interactive
play between the motion of the dancer and the audience.
6. ACKNOWLEDGMENTS
The theoretical and practical approaches of the creation
“man in |e|space.mov” was developed together with the laboratory
for architecture end urbanism LAb[au]. I would like to thank
Frank Nack, Frederic Bevilacqua and Laure Pelayo for the
reading and correction of this article.
7. REFERENCES
[1] Frizot, Michel: Etienne-Jules Marey, Chronophotographie,
Nathan, 2001.
[2] Ka, Wolf, From text to interface, Theatre and digital media,
COSIGN-2003, p 42-47 University of Teesside (UK),
http://www.cosignconference.org/cosign2003/papers/Ka.pdf
[3] LAb[au], e.motion space_the cinematic construct of
electronic space, ARCA n° 187, December 2003.
[4] Leeker, Martina, Der Körper des Schauspielers/ Performer
als Medium: Von der Ambivalenz des Theatralen 1998-
http://userpage.fu-berlin.de/~sybkram/medium/leeker.html
[5] Schlemmer, Raman C, Oskar Schlemmer, Musées de Marseille
reunion des Musée nationaux Musée Cantini, 1999.
[6] Schlemmer, Oskar, Tänzerische Mathematik , Vivos Voco,
Zeitschrift für neues Deutschtum 8/9, Sonderheft Bauhaus,
Leipzig, 1926 p. 279-293, Urban.
Figure 6. Sequence “body particles” with scheme of the
construction of frames in 3D s
p
ace
Figure 7. Sequence “3D shadow” with scheme of the
construction of frames in 3D s
p
ace
... In category 1.5, we find five papers (12%) that focus on specific 'artistic work and processes' at the intersection of contemporary dance and various kinds of technologies, e.g. zero-gravity environments [9], interactive system design for the stage [40], or automatized and markerless motion capture for the creation of online scores [41]. This category is distinguishable from 1.2 as there is a stronger emphasis on the artistic side, and authors often present specific pieces. ...
... [2], [13], [16], [27], [37], [47], [55] C1.4 Support tools for dance 11 (26.2%) [10], [11], [12], [14], [17], [19], [22], [31], [60], [61], [64] C1.5 Artistic work and process 5 (11.9%) [9], [40], [41], [49], [ To deepen our analysis and subsequent discussion of the results, we also included references to concrete artistic work (2.4), and references to processes and practices (2.5). In this coding scheme, multiple mentions of papers in different categories were possible to analyze more precisely how the authors understood the term 'contemporary dance'. ...
... Without a clear scope, research on specific aspects of contemporary dance will be mistaken to represent research of the entire field of contemporary dance. For example, some researchers assert that "movement qualities are central in contemporary dance" [31] or contend that "contemporary dance explores the qualities [4], [9], [10], [11], [12], [19] [22], [23], [26], [36], [37], [40] [41], [47], [49], [54], [53], [65] C2.2 Contemporary dance as a dance genre 24 (36.4%) [1], [2], [9], [13], [14], [15], [16], [17], [23], [27], [28], [29], [31], [38], [48], [55], [56], [57], [60], [61], [62], [63], [64], [67] C2.3 Contemporary dance as a dance technique 7 (10.6%) ...
... Shallhorn [44] created a digital stage machine -a stage augmented with interactive real-time technology. Ka [24] aimed to achieve human body abstraction via an electroluminescent interactive costume. Both demonstrated novel approaches for digital technology use in performance. ...
Conference Paper
In the 1920s, Oskar Schlemmer, artist in the Bauhaus movement, created the Triadic Ballet costumes. These re-strict movement of dancers, creating new expressions. In-spired by this, we designed an interactive wire costume. It restricts lower body movements, and emphasizes arm movements spurring LED-light 'sparks' and 'waves' wired in a tutu-like costume. The Wire Costume was introduced to a dancer who found that an unusual bond emerged be-tween her and the costume. We discuss how sensory altera-tion (sight, kinesthetic awareness and proprioception) and bodily training to adjust to the new soma, can result in nov-el, evocative forms of expression. The interactive costume can foster a certain mood, introduce feelings, and even embody a whole character -- only revealed once worn and danced. We describe a design exploration combining cul-tural and historical research, interviews with experts and material explorations that culminated in a novel prototype.
... In other areas of the performing arts, such as dance, interactive technologies are used more often than in theatre or opera. Here, technology is typically attached to the moving human body and generates sound [1,7,15] or influences interactive projection [5,6,13]. While these examples demonstrate how to apply interactive systems, only some investigate the audience's perception, and none study adaption by different stakeholders. ...
Conference Paper
Our case study explores how interactive systems may influence and enhance theatre performances. We pro-duced a short play featuring four interactive systems. Using a mixed-method evaluation approach we strive to answer two main research questions: How do users (actors and directors) adapt to the technology, and how do spectators perceive interactive scenes? We found the success of using interactive systems in theater to be based mainly on how interaction is conceptualized in reference to the script. Adaptions are done on both sides -- system and user need to "understand" each other, and the audience's perception is not necessarily characterized by recognizing interactivity per se but ra-ther by experiencing it in a theatre play as a whole.
... Our study case has been developed and evaluated during our collaboration with the choreographer Wolf Ka, creator of an interactive piece designed for dancer and spectator [4]. ...
Conference Paper
The growing overture towards interacting virtual words and the variety of uses, have brought great changes in the performing arts, that worth a profound analysis in order to understand the emerging issues. We examine the performance conception for its embodiment capacity with a methodology based on interaction analysis. Finally, we propose a new situation of multimodal choreography that respects the aforementioned analysis, and we evaluate the results on a simulation exercise.
Chapter
This chapter deals with smart costumes that are made for the usage on professional stages, e.g., theatre, ballet or performances of pop stars. Such costumes can be active or interactive depending on whether they are reactive to their wearers or the environment, or not. The requirements of (inter)active costumes are different in comparison with conventional costume design because the individual computational features have to be developed and integrated properly. Creating, staging and maintaining them challenge in particular the established structures of traditional production houses in theatre. This chapter analyses the crafting process of (inter)active costumes for the performances of singers or musicians and for professional theatre or ballet. Furthermore, it seeks to outline the requirements that are needed to succeed.
Conference Paper
In this paper, we discuss the requirements and critical challenges for creating and staging interactive costumes in the theatre. Different to other types of performance, theatre costumes are secondary to acting. Our investigations are based on two practice-based case studies: a self-directed design research within a student project, and a collaboration with a local theatre house, where interactive costume elements were developed in a real-life setting. These reveal requirements and challenges for the design process as well as the effective staging of interactive costumes, the biggest challenge being how to integrate these into existing structures of traditional theatre houses, and requirements for the costumes themselves. Because interactive costumes integrate technological features and traditional analogue crafts, they require interdisciplinary collaboration and transcend established boundaries between departments in theatre houses, challenging established work processes and structures.
Performer als Medium: Von der Ambivalenz des Theatralen
  • Martina Leeker
  • Der Körper Des Schauspielers
Leeker, Martina, Der Körper des Schauspielers/ Performer als Medium: Von der Ambivalenz des Theatralen 1998- http://userpage.fu-berlin.de/~sybkram/medium/leeker.html
  • Oskar Schlemmer
  • Tänzerische Mathematik
  • Vivos Voco
Schlemmer, Oskar, Tänzerische Mathematik, Vivos Voco, Zeitschrift für neues Deutschtum 8/9, Sonderheft Bauhaus, Leipzig, 1926 p. 279-293, Urban.
From text to interface, Theatre and digital media
  • Wolf Ka
Ka, Wolf, From text to interface, Theatre and digital media, COSIGN-2003, p 42-47 University of Teesside (UK), http://www.cosignconference.org/cosign2003/papers/Ka.pdf
e.motion space_the cinematic construct of electronic space
LAb[au], e.motion space_the cinematic construct of electronic space, ARCA n° 187, December 2003.
Musées de Marseille reunion des Musée nationaux Musée Cantini
  • Schlemmer
  • C Raman
  • Oskar Schlemmer
Schlemmer, Raman C, Oskar Schlemmer, Musées de Marseille reunion des Musée nationaux Musée Cantini, 1999.
Schlemmer Musées de Marseille reunion des Musée nationaux Musée Cantini 1999. Schlemmer Raman C Oskar Schlemmer Musées de Marseille reunion des Musée nationaux Musée Cantini
  • C Schlemmer Raman
  • Oskar
Zeitschrift für neues Deutschtum 8/9
  • Oskar Schlemmer
  • Tänzerische Mathematik
  • Vivos Voco
  • Schlemmer Oskar