Content uploaded by Florent Levillain
Author content
All content in this area was uploaded by Florent Levillain on Apr 28, 2015
Content may be subject to copyright.
Moving on its Own: How do Audience
Interacts with an Autonomous Moving
Artwork
Abstract
In contemporary art, a new type of artworks use
motion as a material from which to create the illusion of
life. These autonomous robotic artworks have a
behavioral specificity; they tend to be perceived as
living, and by some account intentional entities. To
account for this behavioral specifity and how it affects
the audience experience, we propose a data-driven
approach to reveal specific visit patterns. Through a
cluster analysis performed on visitors’ path inside an
installation involving autonomous objects, we highlight
four different attitudes characterized by patterns of
approach or withdrawal, passive observation and
exploration.
Author Keywords
audience experience; interactive arts; behavioral
artworks; engagement; cluster analysis
ACM Classification Keywords
I.2.9 [Artificial Intelligence]: Robotics---Autonomous
vehicles; J.4 [Social and Behavioral Sciences]:
Psychology; J.5 [Arts and Humanities]: Performing arts
Permission to make digital or hard copies of all or part of this work for personal
or classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this notice
and the full citation on the first page. Copyrights for components of this work
owned by others than the ACM must be honored. Abstracting with credit is
permitted. To copy otherwise, or republish, to post on servers or to redistribute
to lists, requires prior specific permission and/or a fee. Request permissions
from permissions@acm.org.
CHI'15 Extended Abstracts, April 18–23, 2015, Seoul, Republic of Korea.
Copyright is held by the owner/author(s). Publication rights licensed to ACM.
ACM 978-1-4503-3146-3/15/04...$15.00.
http://dx.doi.org/10.1145/2702613.2702973
Florent Levillain
Laboratoire CHART-LUTIN
EA 4004
Université Paris 8
2 rue de la Liberté, 93526
Saint Denis – Cedex 02
FRANCE
flevillain@mac.com
Sébastien Lefort
Sorbonne Universités
UPMC Univ Paris 06
CNRS, UMR 7606, LIP6, F-
75005 Paris, FRANCE
sebastien.lefort@lip6.fr
Elisabetta Zibetti
Laboratoire CHART-LUTIN
EA 4004 Université Paris 8
2 rue de la liberté, 93526
Saint Denis - Cedex 02
FRANCE
ezibetti@univ-paris8.fr
Case Study: Art & Life
CHI 2015, Crossings, Seoul, Korea
695
Introduction
Take a mental snapshot of a museum or an art gallery,
what do you see inside this picture? Objects most
probably, pictures, sculptures or installations, objects
captive to your glance, objects here for you to admire,
to walk around, to contemplate, quietly waiting for you
to be revealed as works of art. But imagine that one of
these objects now suddenly starts moving on its own,
wanders in the room, takes a glimpse, maybe even at
you. Who is the observer now? This situation is not
unusual in contemporary art. Take for instance Jeppe
Hein’s 360° Presence (2002), a large ball of steel that
moves randomly inside a room, progressively ravaging
the walls as it keeps banging into them. While
extremely simple, this artwork takes a life of its own, it
seems to move on purpose and can be described using
psychological attributes (aggressive, stubborn, erratic,
etc.). There is a tradition of autonomous objects in art.
Robert Breer's majestic Floats (1970), the animal robot
by Edward Ihnatowicz (The Senster, 1970), Jean
Tinguely's immense self-destructive mechanism,
Homage to New York (1960) are examples of artworks
that have in common to use motion as a material from
which to create the illusion of life [1].
Autonomous robotic objects in art are part of a larger
trend involving the design of interactive computational
systems that evolve in real-time, capturing certain
aspects of the audience behavior and adjusting their
output accordingly. These systems have been
categorized by [2] as either dynamic-passive (objects
that change spontaneously or are modified by an
environmental factor), dynamic-interactive (with the
possibility to react to the audience), and dynamic-
interactive (varying) (with the conditions of interaction
changing as well). Autonomous robotic objects may be
considered either dynamic-passive or dynamic-
interactive depending on whether they adjust or not to
the audience. However, their specificity lies in the fact
that they tend to suggest a genuine behavior to a
human observer. The ball of steel previously described
is dynamic as it moves spontaneously, but what is
striking about it is its sense of aliveness, and the
impression that it acts purposefully. This 'behavioral'
specificity is something more than the mere autonomy
as it depends both on the motion cues produced by the
object and on the interpretative skills of the observer.
For this reason we will now refer to the previously
described autonomous objects in art as “behavioral
artworks”. In essence, behavioral artworks are objects
that, because of the way they move, elicit a perception
of animacy, animacy perception being the perception
and categorization of an entity as a living, and by some
account intentional being [3].
Audience experience with behavioral artworks
A large amount of literature has been devoted to the
interaction with computer-controlled interactive
artworks [4, 5, 6, 7]. As experience is a key issue in
interactive art [8], HCI researchers try to define how
people engage into the specific interaction modalities
proposed by an artwork and how the interaction is
sustained over time [9].
The way an audience interacts with a behavioral
artwork is likely to depend on the psychological
properties attributed to this object. The fact to perceive
a robotic object as having goals or intentions may give
rise to feelings of sympathy, empathy, or uneasiness
and fear if its behavior is interpreted as aggressive
[10]. In turn, these psychological attributions may yield
Case Study: Art & Life
CHI 2015, Crossings, Seoul, Korea
696
a tendency to approach or to keep a distance from the
artwork.
Specifying the interaction modalities with a behavioral
artwork would require to better understand how visitors
interpret its behavioral properties, and to evaluate
systematically the interaction patterns that emerge at
the contact of such an artwork. In this study we
undertake the latter goal, by examining the relation
with a behavioral artwork through the recordings of
visitors’ tour inside an installation involving several
autonomous objects. With a data-driven approach
grounded in the observation of visitors’ displacements,
we look for visit patterns that could reveal different
ways of engaging and interacting with a behavioral
artwork.
The present study
In this article, we examine the recent installation Off
Road (2013), created by Céleste Boursier-Mougenot at
Les Abattoirs, Toulouse, France during the 2014
summertime. Céleste Boursier-Mougenot, a french
artist initially trained as a musician, creates
installations that merge sound and movement,
exploiting the unique characteristics of a location to let
emerge works of art that evolve through their
surrounding environment. Off Road is a mechanical
ballet involving three motorized pianos (Figure 1) that
move slowly and in a random fashion inside the
exhibition room1.
In evaluating audience experience with interactive
artworks, observational and qualitative research
methods are generally used [11, 12]. Those methods
1 https://www.youtube.com/watch?v=0y01Ghx2Lxc
are meant to evaluate whether the audience experience
relates to the artist intentions or if the audience
appreciated the artwork. In this study, the evaluation
of audience's experience with a behavioral artwork is
not targeted at the aesthetic level (pleasure, goodness,
beauty) [13, 14] but rather at the type of audience
engagement with the artwork, that is the degree to
which a visitor is willing to approach and interact with
the artwork.
Examining the physical displacements and behaviors of
visitors inside an artistic installation provide a unique
opportunity to evaluate how they position themselves
spatially with respect to an artwork, and the type of
interaction prompted by it. Techniques and tools form
ethnomethodology [15] and from HCI [4, 16] have
been deployed to understand the audience experience.
If such an experience may be deemed profoundly
subjective in essence, these approaches suggest that
visitors can be categorized according to stereotyped
behaviors. [7] describe for instance 8 categories of
audience behavior. Among them the "Stop-and-
observe": visitors that stand still for a short while
before leaving the installation; "Skimmer": visitors that
evolve slowly round the room; "Serious, quiet and
contemplative engagement": visitors that spend a long
time trying to immerse themselves inside the
installation. These three categories indicate stereotyped
visit patterns that can be observed consistently across
visitors, possibly revealing some personality traits and
possibly related to motivational components [5].
In the same spirit, we propose a data-driven approach
to reveal visit patterns when an audience is confronted
to a behavioral artwork. Through a cluster analysis
performed on visitors’ path inside an installation room,
Figure 1. Three pianos moving
on their own.
Case Study: Art & Life
CHI 2015, Crossings, Seoul, Korea
697
we highlight some patterns of approach or withdrawal,
passive observation or exploration. We also examine
how these patterns relate to variations in the artwork’s
movements.
Method
Setting
The pianos, as well as the visitors, were captured by a
webcam located on the ceiling of the room. The pianos
and the visitors’ movements were tracked through an
algorithm based on contour recognition. The recording
of the audience behavior inside the installation’s room
took place during three entire days.
Algorithm for the pianos movement
The pianos are mounted on wheels. The wheels are
coupled up to electric motors through a bike chain. The
motors receive commands from a computer located
next to the installation room, this computer gets it
input from a webcam located on the ceiling of the room
and performs contour recognition to locate the pianos
and the visitors’ positions, which in turn determines the
pianos evolution inside the room. The pianos obey to a
sort of virtual landscape in constant evolution that
simulates upslopes and downslopes such that pianos
are ‘sliding’ on these slopes. Walls create a virtual
downslope, such that a piano approaching from the wall
would turn back. Pianos tend to be repelled by each
other, although they can sometimes (gently) collide.
Through the use of an anemometer, the speed of the
wind outside the speed of the wind outside the museum
is measured and has a subtle influence on the way the
pianos interact. For instance, when the wind is regular,
the pianos tend to be attracted toward each other. In
principle, the visitors cannot influence the pianos’
movements. However a mechanism has been
implemented that allows one single visitor at a time to
do such a thing. While the room is divided into several
zones, the direction of the wind is measured
determining which zone is selected. If a visitor is inside
this zone she will be able to repel the pianos.
Visitors’ selection and video analysis procedure
We used three days of recordings from the ceiling
camera of the exhibition room. These recordings depict
the installation in its full extent with visitors roaming
around the pianos (Figure 2). However, we kept only a
portion of the total recording, picking up the sequences
in which no more than 10 people were simultaneously
present inside the room and, among these sequences,
those in which the visitor had no interaction with other
people. In total, 120 sequences were selected each
corresponding to a visitor's path inside the installation.
The visitors and the pianos’ trajectories were extracted
with Mocha from Adobe After Effect; their position (in
pixels) was measured for each frame, on a 25 frames
per second basis. We developed a software in order to
evaluate some variables regarding the visitors' position
and some temporal aspects of their visit. To do so, we
delineated two different zones inside the video frame:
While the dimensions of the room are 580 x 711 pixels,
we drawn a 520 x 680 rectangle inside this space (see
Figure 2) to delimit an area ('visit zone') where visitors
are considered to be evolving among the pianos instead
of staying withdrawn and observe passively from the
edge of the installation ('edge zone'). To determine the
time spent interacting directly with the pianos, we
considered that a visitor was in interaction with a piano
each time she entered a 100 pixels diameter circle
around a piano center ('piano zone'). Furthermore, we
Figure 2. Three different zones
inside the installation room.
Case Study: Art & Life
CHI 2015, Crossings, Seoul, Korea
698
considered that a visitor is in a passive watching state
each time she spends more than 0.8s standstill.
We measured the 8 following variables (for each visitor,
in pixels and in seconds - the durations are normalized
with respect to the total visit duration):
Overall visit duration
Overall distance travelled
Average walking speed
Average distance to the three pianos
Average distance to the closest piano (whose
identity vary from frame to frame)
Time spent inside the ‘piano zone’
Time spent inside the ‘edge zone’
Time spent in a watching state.
Cluster analysis procedure
In order to let emerge different patterns of behaviors
among visitors from the data, we performed a C-
Medoids clustering analysis [17]. To determine the
relevant number of clusters, we used the “elbow
method” [18] with a threshold of explained variance set
at 80%. To find out the variables combinations that let
emerge a reasonable number of clusters, we tested all
these combinations with two constraints: at least 2
variables are considered, and a maximum of 4 clusters
have to emerge.
Results
Among all the 256 possible combinations, only 4 let
emerge a maximum of 4 clusters: average distance to
the pianos and average distance to the closest piano;
average distance to the pianos and time spent in the
piano zone; average distance to the closest piano and
time spent in the piano zone; and time spent in the
piano zone and time spent in the edge zone. Inasmuch
as the first three pairs concern variables related to each
other, we only considered the last combination.
Considering the time spent in the piano zone and the
time spent in the edge zone, 4 clusters emerge: the
first (C1) and the second (C2) correspond to visitors
that tend to spend a short time both inside the visit
zone and in the proximity of the pianos. The third
cluster (C3) corresponds to visitors that tend to spend
a long time at the edge of the installation and tend to
avoid close encounters with the pianos. Visitors from
the fourth cluster (C4) have the opposite behavior:
they tend to spend a long time close to the pianos and
a very short time at the edge of the visit zone.
In the following analyses, we do not consider the
average distance to the pianos since this variable is
related to the distance to the nearest piano. To ensure
that the visitors clusters we obtained are statistically
different from each other, we performed an ANOVA
with clusters as independent factor on the different
parameters (Tab. 1). The clusters differ significantly
regarding the time spent in the piano zone, the time
spent in the edge zone, and the time spent in a
watching state. However they do not differ with respect
to variables related to the overall time spent inside the
exhibition room or to the scene dynamic: the visit
duration, the travelled distance and the average
velocity, average distance between pianos, and mean
velocity of pianos. The results indicate therefore that
the groups we found differentiate according to
parameters related to the spatial interaction with the
artwork.
Table 1. Mean values (in pixels
and seconds) for the different
visit variables as a function of the
visitors clusters.
Case Study: Art & Life
CHI 2015, Crossings, Seoul, Korea
699
We performed post-hoc tests (Tukey’s HSD) on average
distance to the nearest piano, time inside the piano
zone, time inside the edge zone, and time in a watching
state. Regarding the average distance to the nearest
piano, all the clusters are differing from each other.
Visitors in C3 are on average the farthest from the
pianos, whereas those in C4 are the closest. C4 is the
only cluster to differentiate significantly from the other
groups with respect to the time in the piano zone.
Regarding the time spent in the edge zone, C4 and C1
differ significantly from the other groups but do not
differ from each other. Regarding the time spent in a
watching state, C1 and C2 differ significantly from each
other, as well as C2 and C3.
Four different profiles emerge that represent four
different ways to engage with the installation. Certain
visitors have a pattern of withdrawal (C3), staying at
the edge of the installation, contemplating quietly the
pianos as they move around. On the opposite side,
certain visitors (C4) engage directly with the artwork,
spending time in the immediate proximity of the
pianos, trying sometimes to seek an interaction with
them. Somewhere in between, visitors adopt a mix of
these two fundamental attitudes. Some (C1) stay
relatively close to the pianos (while not in contact),
remaining relatively standstill. Some (C2) adopt a more
cautious (or perhaps indifferent) attitude; farther from
the pianos they also change their position more often.
We could surmise that the artwork, due to its
behavioral properties, is prompting these patterns of
approach and withdrawal. This would require to be
tested comparing the variations of these patterns
depending on the mobility of the artwork. Since we
could not control directly the pianos’ mobility, we
evaluated if the pianos’ dynamic, measured by the
average distance between the pianos (i.e. the total
length of the triangle formed by the pianos) and their
average speed (i.e. the sum of the average speed of
each piano), had an impact on visitors’ trajectory.
We evaluated the correlation between the variables
related to the pianos’ movements and the variables
related to audience behavior (Tab.2). We found a small
yet significant correlation between the pianos average
speed and the visit duration (r2 = .189; p<.05). The
average walking speed seems also to be correlated with
the distance between the pianos (r2 = .213; p<.05).
Visitors tend to stay longer inside the installation room
when the pianos are moving faster, and they tend to
walk more slowly when the pianos are more scattered.
The impact of the pianos’ dynamic seems to depend on
the previously identified clusters. Walking speed and
average pianos separation are negatively correlated for
C2 (r2 = .549; p<.005), while we found a correlation
between pianos’ speed and distance to the pianos for
C1 (r2 = .425; p<.05), indicating that these visitors
tend to distance themselves from the pianos as they
move faster.
In sum, the way the pianos move inside the installation
seems to have an influence on audience behavior,
although only for a portion of the audience. Some
visitors remain at the edge on the installation no matter
what (C3), while other visitors interact directly with the
pianos no matter what (C4). The remaining visitors (C1
& C2) are those adjusting their behavior according to
the pianos evolution inside the room: C1s, more
engaged into the visit zone, adjust their distance as a
function of pianos’ movements; C2s, less engaged,
move more slowly as the pianos are more scattered.
Case Study: Art & Life
CHI 2015, Crossings, Seoul, Korea
700
Summary and discussion
Through a cluster analysis performed on an audience
reacting to autonomous objects, we see different
profiles emerge, corresponding to different ways to
engage with a behavioral artwork. Distinct patterns of
exploration and observation characterize four groups of
the approximatively same size of visitors. Some visitors
may be considered “explorers” in that they engage
systematically with the artwork; some others are
“stalkers”, contemplating the work from a distance.
Aside these two extreme groups, some visitors have a
more flexible behavior: they approach then retreat,
they observe, and they adjust their distance and pace
as a function of the pianos’ speed and the way they are
scattered inside the room.
The choice to approach the artwork, to interact directly
with it, or, on the contrary, to stay at the edge,
observing passively, represent different ways to
appreciate the artwork. The visit patterns echo the
aesthetic pleasures the artwork offers to the visitor. A
behavioral object is not something only to contemplate,
but something you can engage with, something that
you can test through micro-interactions (in the case of
Off Road people try to come as close as possible from
the pianos to observe if they are reacting to their
presence) to gauge the range of its reactions. The
behavioral properties of the artwork prompt curiosity in
the visitors, but may also invite caution.
Thus, this is through a very specific combination of
properties that a behavioral artwork shapes the
audience experience. Some interaction patterns arise
from immediate feedback given by basic motion cue,
while others come from higher-order attributions
regarding the object’s psychological properties. An
investigation of the audience attributions and
expectations regarding the artwork would be necessary
to qualify the audience experience, but from the visit
patterns we observe we can suggest a range of
different pleasures and associated motivations that may
explain the variation in audience behavior. Following
[5], a combination of the pleasure one gets from
exploring an unfamiliar setting and the pleasure one
gets from figuring out how something ticks may
account for the “explorer” profile. The “stalker” profile
is probably more related to a perceptive pleasure drawn
from the act of contemplating something whose
evolution cannot be easily predicted.
Taking into account the modalities of engagement with
a behavioral artwork is an opportunity for artists and
curators to devise their work according to visitors
profiles. In the future, drawing a systematic
relationship between the psychological inferences
drawn from motion cues and the emerging patterns of
interaction would allow to understand the nature of the
aesthetic experience derived from the contemplation of
this original kind of entities that are now populating our
museums.
Acknowledgements
We would like to thank Céleste Boursier-Mougenot and
the Abattoirs team for kindly letting us investigate the
installation. This study was made possible with the help
of Samuel Bianchini, Emanuele Quinz, and Naoko Abe.
This publication is part of a research program (The
Behaviors of Things) that is supported by the Labex
Arts-H2H, by Investissements d'Avenir (ANR-10-LABX-
80-01) of the French National Research Agency (ANR).
Case Study: Art & Life
CHI 2015, Crossings, Seoul, Korea
701
References
[1] Bianchini S., Bourganel R., Quinz E., Levillain
F., Zibetti E., (Mis)Behavioral objects. Empowerment of
users vs. empowerment of objects, in Empowering
Users through Design, Bihanic D. Ed., Springer, 2015,
pp. 129-152.
[2] Cornock, S., & Edmonds, E. (1973). The creative
process where the artist is amplified or superseded by
the computer. Leonardo, 11-16.
[3] Rutherford, M.D., & Kuhlmeier, V.A. (2013).
Section introduction: The perception of animacy and
intentional behavior. In M.D. Rutherford & V.A.
Kuhlmeier (Eds) Social Perception. MIT Press.
[4] Höök, K., Sengers, P., & Andersson, G. (2003,
April). Sense and sensibility: evaluation and interactive
art. In Proceedings of the SIGCHI conference on Human
factors in computing systems (pp. 241-248). ACM.
[5] Costello, B., & Edmonds, E. (2007) A study in play,
pleasure and interaction design. In Proceedings of the
2007 conference on Designing pleasurable products and
interfaces (pp. 76-91). ACM.
[6] Bilda, Z., Muller, L., & Edmonds, E. (2009). Artist,
evaluator and curator: three viewpoints on interactive
art, evaluation and audience experience. Digital
Creativity, 20(3), 141-151.
[7] Loke, L., & Robertson, T. (2009). Design
representations of moving bodies for interactive,
motion-sensing spaces. Int.J.Human-Computer Studies,
67, 394-410.
[8] Boehner, K., Thom-Santelli, J., Zoss, A., Gay, G.,
Hall, J. S., & Barrett, T. (2005, April). Imprints of
place: creative expressions of the museum experience.
In CHI'05 extended abstracts on Human factors in
computing systems (pp. 1220-1223). ACM.
[9] Edmonds, E., Muller, L., & Connell, M. (2006). On
creative engagement. Visual Communication, 5(3),
307-322.
[10] Saerbeck, M., & Bartneck, C. (2010). Attribution of
affect to robot motion. Proceedings of the 5th ACM/IEEE
International Conference on Human-Robot Interaction,
Osaka pp. 53-60.
[11] Bilda, Z (2006) Evaluating audience experience
Engage: interaction, art and audience experience.
Creativity and Cognition Studios Press, Australia 248-
260
[12] Candy, L, Amitani, S and Bilda, Z (2006) Practice-
led strategies for interactive art research, CoDesign:
International Journal of Co Creation in Design and the
Arts Vol 3 pp 209e223
[13] Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What
is beautiful is usable. Interacting with computers,
13(2), 127-145.
[14] Hassenzahl, M. (2004). The interplay of beauty,
goodness, and usability in interactive products. Human-
Computer Interaction, 19(4), 319-349.
[15] Vom Lehn, D., Heath, C., & Hindmarsh, J. (2001).
Exhibiting interaction: Conduct and collaboration in
museums and galleries. Symbolic Interaction, 24(2),
189-216.
[16] Costello, B., Muller, L., Amitani, S., Edmonds, E.
(2000). Understanding the experience of interactive
art: iamascope in beta_space. In Pisan, Y. (Ed.),
Australasian Conference on Interactive Entertainment.
Creativity and Cognition Studio Press, pp. 49-56.
[17] Krishnapuram, R., Joshi, A., & Yi, L. (1999,
August). A fuzzy relative of the k-medoids algorithm
with application to web document and snippet
clustering. In Fuzzy Systems Conference Proceedings,
1999. FUZZ-IEEE'99. 1999 IEEE International (Vol. 3,
pp. 1281-1286). IEEE.
[18] Thorndike, R. L. (1953). Who belongs in the
family? Psychometrika, 18(4), 267-276.
Case Study: Art & Life
CHI 2015, Crossings, Seoul, Korea
702