Content uploaded by Petra Gemeinboeck
Author content
All content in this area was uploaded by Petra Gemeinboeck on Mar 14, 2021
Content may be subject to copyright.
1
Moving beyond the mirror:
relational and performative meaning-making in human-robot communication
Petra Gemeinboeck
Rob Saunders
Department of Media Theory
University of Applied Arts Vienna, AT
Leiden Institute of Advanced Computer Science
Leiden University, NL
Centre for Transformative Media Technologies
School of Arts, Social Sciences and Humanities
Swinburne University of Technology, AU
1 Introduction
You enter the foyer in a gallery and see a large white box sitting next to a shelf and chair. You wouldn’t
even have taken note of the box but suddenly it rotates then slides along the side of the shelf. Probably
some sort of vacuum cleaner, you think. As you turn away, you catch something unexpected: one of the
box’s top corners delicately raises upwards and the whole box seems to gently tilt towards you, before it
begins to skitter in your direction. Really? You look around for other witnesses to this curious event, or
better, someone controlling the box, half expecting a child with a joystick grinning at you. As you turn
back, you find the artefact precariously teetering on one of its edges before forcefully tipping onto a
corner, where it slowly sways, as if pondering what to do next. You decide to move closer and it seems to
suddenly halt its swivel in mid-air, but you notice an ever so slight tremble. When you bend down to
investigate, it rambunctiously bumps onto the ground. Bomp bada bomp.
The scene may be reminiscent of a Disney animation, where shapes and objects ‘come alive’. Abstract
animated screen characters, however, are often deliberately anthropomorphized by imbuing the object
with human features and a humanlike disposition. Yet while the ‘box’ in the above scene moves in ways
that display some form of communicative intention, it does not offer googly eyes or framing strategies
that guide our gaze (see Lasseter 2001) to easily pinpoint its expressions. The artefact in question is a
robotic prototype, coming out of our Machine Movement Lab (MML) project, which investigates the
relational potential of movement and, in particular, movement qualities in human-robot interaction. What
the robotic artefact offers, in contrast to screen-based characters, is embodied dynamics and material
relations that we can bodily share and kinaesthetically grasp. Even though ‘giving life’ to an artefact is
not what we aim for, the effects of a simple artefact moving in dynamic and delicate ways open up an
ambiguous zone between subject and object. Such “behavioural objects”, according to Levillain and
Zibetti (2017:5), “carry spatial transformations that can be interpreted as actions executed toward a goal,
possibly motivated, and possibly intelligent”, without resembling humans or animals.
Human-Robot Interaction (HRI) research frequently sides with traditional animation strategies with
regards to portraying artefacts as humanlike subjects, whether seen as technological artefacts that humans
interact with as if they were a person or made to look and behave humanlike, thus masking the issue
altogether. Considerable ongoing efforts to overcome the ontological gap between people and robots as
communicators (see Guzman 2020) rely on blurring the profound differences and “deep asymmetries”
(Suchman 2007:11) between them. Yet at its core, human-robot interaction is about interacting and
communicating with social entities that dramatically differ from us—ontologically as well us culturally
(see Guzman 2020). While dominant HRI research approaches assume that machine ‘otherness’ disrupts
successful communication with people (see Sandry 2016), from an artistic research viewpoint, this
otherness opens up a productive challenge and ample potential for communication that is about situated
meaning-making, rather than information exchange. Human-machine communication (HMC), according
to Guzman (2018), is not only about transmitting information, but also about how meaning is created
between human and machine participants. This article attempts to develop a counter position to human-
centric approaches in HRI by examining how meaning is bodily and dynamically enacted in human-
nonhuman encounters.
1.1 Communicating with an ‘other’
Hegel et al state that “with a functional designed robot it is impossible to express human facial
expressions and consequently emotional displays” (2009:173). This limited notion of affective
communication starkly contrasts artistic practices exploring the capacity of machinelike artefacts to evoke
Preprint of peer-reviewed, accepted article; camera-ready version; to be published in:
AI & Society: Knowledge, Culture and Communication
Special issue on: Critical Robotics Research (forthcoming in 2021)
2
affective responses or communicate in non-verbal ways. Artists developing kinetic sculptures, robotic
artworks or machine performances have studied the potential of movement, sound and other relation-
making modes, such as staging, to render artefacts relational or transform their machine identity. The
Table: Childhood (1984–2001) by Max Dean, Raffaello D'Andrea and Matt Donovan, for example,
produces surprising relational dynamics between audience members and the familiar object of a table (see
Drouin-Brisebois 2008). This interactive artwork attempted to reverse the roles of viewer and object by
staging a white yet otherwise non-descript table following audience members, apparently selected by the
table itself, around the gallery space. The seemingly intentional movement of the table not only renders
the object strange but also enables it to open up unexpected, evocative moments of encounter. Simon
Penny’s Petit Mal (1989–2005) resembles a strange dicycle, which takes on the role of “an actor in social
space” (Penny 2000:400). The work’s unique, non-verbal behavioural performance arises from an
eccentric, yet simple mechanism based around a double pendulum, which brings an unpredictable and
charming quality to its movements; seemingly struggling to balance, it sways through the gallery space,
inviting audiences to bodily interact with it (Penny 2016:57).
From a communication perspective, interactions with such ‘machine performers’ are not governed by
familiar turn-taking protocols (see Sandry 2016). Rather, they evoke, or sometimes provoke, flowing
interactional coordination, based on spatial or empathic interpretations of movement. Penny argues that
artistic practice favours “subtle and evocative modes of communication” (2000:400) over goal-oriented
functionality. Communication here is not predicated on the transmission of information but rather focuses
on modes of engagement, embodied experience and emergent meaning-making. This often involves a
multimodal, embedded approach, as well as an understanding of complex sociocultural contexts. By
embracing and promoting multiple interpretations and the emergence of meanings in the interaction, arts-
led approaches resonate with conceptions of communication as “an emergent property of systems”
(Sandry 2016:188), arising between communicators, rather than being directly produced and transmitted
by them.
Our ongoing MML project brings together creative robotics, choreography, performance techniques and
machine learning, grounded in an enactive, performative framework (see section 3.2) to promote a
performative understanding of communication by exploring the relational potential of human-machine
asymmetries. Designing a social artefact, in MML, is about scaffolding a machine’s ability to participate
in embodied meaning-making, rather than imbuing the machine with a predesigned sociality, e.g., through
humanlike features. To create this intra-corporeal scaffold, our design methodology opens up an intimate
link to embodied, performance-based inquiries into the generative potential of movement and its dynamic
qualities to enact meaning with abstract robotic artefacts. Starting in 2015, we first set out to search for
possible abstract shapes that, when in motion, take on relational, affective qualities without relying on
humanlike features. To explore the communicative potential of shapes-in-motion, we asked dancers to
bodily extend into them and their material affordances by inhabiting and moving with them. A small
selection of simple geometric shapes were then rebuilt as wearable costumes of various sizes to serve as
embodied mapping interfaces. Standing in for a becoming-robot’s shape, a costume harnesses a dancer’s
tactile-kinaesthetic expertise for movement creation to socioculturally situate (see Lindblom 2020;
Lindblom and Ziemke 2003) a robot’s learning to move in relational ways (see Figure 1). We discuss this
mapping methodology, called Performative Body Mapping (PBM), in more detail in section 3.
Figure 1: PBM cube costume inhabited by Audrey Rochette (on the left) with Cube Performer #1
3
Based on our PBM dancer-costume entanglements, we have so far realised two robotic prototypes, Cube
Performer #1 (see Figures 1 and 2) and Cube Performer #2. Observing and talking to exhibition
audiences and interaction study participants, the blank canvas offered by a plain, regular cube shape when
juxtaposed with the rich dynamics of energetic movement qualities seems to open up a large potential for
spatial transformations that we can read as meaningful actions (see Levillain and Zibetti 2017).
Such a relational-performative approach towards HMC is not suitable for all human-robot interaction
scenarios. In particular, the bodily, kinaesthetic immersion that is at the heart of our methodology may
not be feasible for a particular robot design. Furthermore, our emphasis on the enactment of situated,
emergent meanings in our encounters with robotic social entities poses new challenges for interaction
scenarios in narrowly defined tasks. However, shifting the focus from representationalism to
performativity could advance our understanding of how a robot’s sociality emerges and reveal new
pathways for robot design. Machinelike robots whose communicative skills exploit the relational,
aesthetic potential of dynamic movement qualities could produce novel, more diverse human-machine
relationships that pose fewer ethical risks to potentially vulnerable users than those formed with
humanlike robots (see Lee at al 2016; Turkle 2011).
In the following, we begin by outlining a critical perspective onto human-robot interaction relying on
human likeness (section 2) before taking a closer look at our design methodology. The latter includes a
discussion of our methodological approach (section 3.1), participatory studies (3.3) and our conceptual
framework for performative, relational human-machine interaction (3.2), bringing concepts from
posthuman performativity and embodied, affective sense-making into a unique conjunction through
performance-based, creative practice.
Figure 2: Machine Movement Lab: Cube Performer #1 at the Games & Performing Arts Festival, UK, 2018
2 Making machines in our own image
The promotional video for Pepper
1
promotes the humanoid robot as an emotional friend—a robot that can
be thought of as “high tech you can high five”. Pepper is one example of a range of commercial
humanoid robots marketed as companions, assistants, and carers, heralding a not-too-distant future in
which we will live with robots that exhibit traits reminiscent of ourselves—smart, polite and gendered.
Robots that are human enough that we can easily empathise with their familiar performances or ‘high
five’ them. This marketing sentiment is often sustained by the young fields of Social Robotics and HRI,
where great effort is invested in studying how anthropomorphic attributes positively affect people’s
acceptance of and interaction with robots (De Graaf and Allouch 2013; Fink 2012; Fong et al 2003).
According to popular definitions, a social robot should exhibit design features that permit ‘natural
interaction’ (Dautenhahn 2013; Hegel et al 2009), including a distinctive personality and the capacity to
1
SoftBank Robotics, promotional video: www.youtube.com/watch?v=oDeQCIkrLvc
4
express and perceive emotions (Becker 2006; Fong et al 2003). The aim is to render communication
between people and machine artefacts ‘more natural’ (Becker 2006; Guzman 2018; Hegel et al 2009),
founded on the belief that humanlike features and familiar behaviours can be orchestrated to give a robot
a social façade (see Jones 2017; Broadbent 2017; Alač 2015). Despite the dominance of humanoid robots
in social robotics, we can also find many studies that question underlying assumptions such as ‘natural
interaction’ (Dautenhahn 2013) and the appeal of human likeness (Lee et al 2016; Vlachos et al 2016) and
investigate potential social impacts (Šabanović 2010; Turkle 2007).
Successful human-machine communication, seen from this mimetic perspective, is based on what
communicators either already have or could have in common (see Sandry 2016). Communication here is
framed as a process that has a correct outcome or predefined protocol, where potentially ambiguous
meanings or multiple interpretations would be, in Sandry’s words, “an undesirable risk that should be
eliminated” (Sandry 2016:179). This narrow view promotes a transmission model of communication,
where messages are sent and received or information is passed from one communicator to another (Craig
1999). A constitutive model of communication, in contrast, is based on the production of shared
meanings (Craig 1999). The former, focusing on a neatly bounded channelling of pre-scripted meanings,
lends itself well to technological functionalist thought and notions of technological control (see Craig
1999). In addition to omitting meanings arising from interaction dynamics, a transmission-focused
approach to HMC is also prone to be blind to the sociocultural context that human communicators
inevitably bring with them (see Guzman 2018; Craig 1999).
Researchers in Science and Technology Studies (STS) have been calling for more diversified robot
designs to allow for human-robot relations that do not rely on beguiling users through imitation (Jones
2018; Castañeda and Suchman 2014; Turkle 2011). Talk of peoples’ or robots’ agency “presupposes a
field of discrete, self-standing entities” (Suchman 2007:263), fuelling the assumption that a machine, as a
discrete entity, can be rendered social by mimicking human capacities (Alač 2016). Agency and sociality
here are seen as intrinsic qualities that can be reverse engineered and programmed into a robot (Jones
2018), irrespective of the wider sociomaterial context. But what if social capacity cannot simply be
‘given’ to a robot and human-robot relationships cannot simply be modelled after human-human
relationships (see Jones 2018)? What if, instead, we need to look at human-robot interaction as a
relationally enacted, situated meaning-making process, in which a machine’s social agency is brought
forth and sustained by the dynamics unfolding in the encounter?
Our arts-led design approach, discussed below, aims to mobilise some of the critical views of dominant
assumptions about a machine’s social agency and how they shape the ways we understand successful
human-robot communication. MML’s methodological contribution promotes an embodiment-centred,
relational model of communication that places movement and its dynamic qualities at the centre of
meaning-making.
3 Beyond the mirror and straight through the looking glass: Machine Movement Lab (MML)
Our argument that robots should be conceived of as machinelike, rather than humanlike, communicators
has also been raised by researchers in communication studies (Guzman and Lewis 2020; Hoorn 2018;
Sandry 2016, 2019). However, the relational dynamics that both robot design and resulting robots are
embedded in and how they contribute to a social actor’s capacity to participate in the encounter are often
overlooked (see Guzman and Lewis 2020). In fact, we believe that the relational dynamics that both
constitute and unfold in the design process play a key role in the relations we can have with a machine.
That is to say that the practice of enacting human-machine communication starts at the very beginning of
the design process, not once a robot design is apparently complete. The remainder of this article will take
a closer look at our MML project and how it seeks to embrace, and aesthetically exploit, the asymmetries
between human and machine embodiments to facilitate meaning-making.
3.1 Performative Body Mapping (PBM)
Robot designs that do not rely on mimicking familiar, organic bodies allow for rich encounters, where
meaning-making is not predetermined or constrained by the expectations, conceptions, or projections we
form in advance of the experience of encounter (Dautenhahn 2002). Guzman observes that people’s
perception of the differences between humans and machines co-shape their overall interpretations of the
machine communicator, which, in turn, informs their decisions and actions over the process of
communicating with the machine (Guzman 2020). To embrace the difference of machines, the challenge
is to find a starting point from which to explore the social potential of machinelike agents.
5
MML takes movement and its “spatio-temporal-energetic” (Sheets-Johnstone 2011:432) qualities as its
starting point to investigate the relational potential of abstract, non-humanlike machines and situate these
strange artefacts in our social environment (see section 3.2). Movement and, with it, bodily perception
and affect are core to embodied meaning-making (Johnson 2007, 2018). Contemporary dance, which
purposely and systematically develops movement “for its own sake” (Stevens and McKechnie 2005:243),
is a natural ally for investigating embodied, relational meaning-making and bringing to the fore the
potential of qualitative movement dynamics (see Sheets-Johnstone 2012). Leach and deLahunta describe
the relationality of bodies, affectively reaching through movement as “an extension of feeling, knowing,
and sensing into the world with, and of, other bodies” (Leach and deLahunta 2017:464). According to
Manning and Massumi, movement “bodies forth” (2014:39), rather than “something the body does”
(2014:40). While collaborations between robotics and dance or performance have provided a testbed for
evaluating robots’ expressive capacity (Jochum et al 2017), many of these interdisciplinary projects still
involve humanlike robots or integrate existing robots within a performance event.
Figure 3: PBM cube costume, inhabited by Audrey Rochette
MML investigates how the generative capacity of movement can render human-machine differences
relational by harnessing choreographic knowledge and dancers’ kinaesthetic expertise to inform the robot
design and situate its machine learning. As briefly laid out in the introduction, our experimental,
investigatory design process revolves around an embodied mapping interface that combines ideas that
underly theatrical costumes
2
(Suschke 2003) and demonstration learning in HRI (Billard et al 2008). The
Performative Body Mapping (PBM) costume serves to bridge between human-machine asymmetries by
enabling dancers to: (1) corporeally experience the ‘other’, machinelike morphology and learn to
kinaesthetically extend into and move with it, and (2) bypass the correspondence problem, commonly
posed by the challenge of mapping between two very different embodiments (Dautenhahn et al 2003).
The PBM costume not only serves as a mapping interface but also an instrument for recording the kinetic
traces of the dancer’s bodily activation of the costume, manifesting in the resulting movement qualities of
the dancer-costume entanglement (see Figure 3). PBM thus allows (1) delegating much of the difficult
morphological mapping to the movement expert without relying on simulating human movement
mechanisms, and (2) for the robot prototype to learn from the motion capture data as if it was trained by
another robot performer with the same physical shape
3
.
2
Costumes in theatre and performance not only serve to situate a character in the performance but also to deliberately
‘transform’ performers’ bodies. For example, for his 1993 production of Tristan and Isolde, Heiner Mueller asked
Yohji Yamamoto to design costumes for the singers “that would impede on the movement they are used to” (Suschke
2003:205). MML, in contrast, looks for a productive intermeshing of bodies.
3
More details on our costume-based demonstration learning can be found in (Gemeinboeck and Saunders 2017,
2018).
6
Figure 4: Early PBM workshop with tube-like costumes, inhabited by performers
Importantly, we did not begin with a predefined shape for the becoming-robot and, consequently, the
costume. Early PBM workshops with choreographer Tess de Quincey and dancers from De Quincey Co
4
explored a wide range of forms and materials in motion, with the goal to challenge our assumptions and
preconceptions with regards to possible machine forms and their movement capacities (see Figure 4).
Later workshops focused on a small selection of simple, geometric forms to investigate the
communicative potential of movements resulting from the dancer-costume entanglement, including
cuboids, cubes and a tetrahedron (see Figure 5)
5
. Since we are not interested in examining the robot’s
social capacities in terms of how well it performs existing human social tasks, our approach explores how
far we can push the relationship between abstract robotic forms and their potential to elicit empathic and
affective responses through relational movement qualities.
Figure 5: PBM workshop, showing a dialogue between two PBM costumes inhabited by dancers
(Tess de Quincey, on the right)
Ultimately, we selected perhaps the most obvious, abstract form, yet not the most apparent in terms of its
evocative capacity—a cube—to shape the first robotic prototype
6
. Starting with experiments, in which
dancers inhabited a simple cardboard box, this familiar object demonstrated great potential for being
transformed into something more than an ‘object’ when moving in unexpected ways. The cube’s regular,
4
See: https://dequinceyco.net
5
A detailed account of this earlier form-finding stages, selection criteria and movement studies can be found in
(Gemeinboeck and Saunders 2017).
6
We also experimented with a four-jointed tetrahedron structure, which serendipitously turned into a versatile, five-
jointed artefact (see Figure 5, on the right; more details can be found in (Gemeinboeck and Saunders 2017). The
cubic shape, however, proofed more readily transformable into a skilful, mobile robotic artefact.
7
omnidirectional geometry presents a counterpose to organic structures with limbs, two-sided symmetries
and the hierarchy of front and back. Furthermore, an intricately moving cube, capable of lifting off the
ground, twisting upwards or gently swaying through space, quickly loses its simple, predictable nature.
We believe that it may be the apparent schism between the unassuming shape of a cube and dynamic or
delicate movement qualities that opens up an aesthetically rich space for transformation. So far, we have
iteratively realised two 75x75x75cm robotic artefacts: Cube Performer #1 and Cube Performer #2. The
movement requirements for their mechanical design were derived from an analysis of over ten hours of
motion capture recordings to determine the required velocity, acceleration and ranges of vertical,
horizontal and rotational movement (Gemeinboeck and Saunders 2018; see Figure 6).
Figure 6: The mechanical frame of Cube Performer #1
Our relational, performative approach proposes that the “spatio-temporal-energetic” (Sheets-Johnstone
2011:432) dimensions of movement can serve to bootstrap the robot’s learning to situate the machine in
the “social and cultural scaffolds” (Lindblom 2020:4) that are fundamental to embodied, social meaning-
making (Lindblom 2020; see also section 3.2). The PBM costume enables dancers to ‘step into’ the other,
machinic shape to explore and put to work its enabling constraints rather than anthropomorphizing or
imprinting their human intents onto the becoming-robot. Hence, our goal is to socioculturally situate (see
Lindblom 2020; Lindblom and Ziemke 2003) the abstract artefact by entangling human dancers with the
artefact and its transformational potential. Motion capture data of this dancer-cube entanglement then
serves to bootstrap the learning of the robotic artefact. It comprises granular, discrete movement patterns,
derived from short choreographic abstractions (Aviv 2017) that, in the machine learning process, take on
the role of aesthetically and socioculturally coded biases and constraints. The latter allow the robotic
artefact to learn to compose new movements that are both, grounded in its own unique material
embodiment (see Gemeinboeck and Saunders 2018; Saunders and Gemeinboeck 2018) and embedded in
our social and cultural scaffolds (see Lindblom 2020). According to Rotman, “[m]otion-capture
technology allows the communicational, instrumental, and affective traffic of the body in all its
movements, openings, tensings, foldings, and rhythms into the orbit of ‘writing’” (2008:47). Intermeshing
human and nonhuman affordances, the resulting kinetic alphabet of the PBM entanglement captures a
wide range of kinetic dynamics that serve to render the Cube Performer a highly skilled participant in the
affective exchanges of the encounter (see Damiano and Dumouchel 2020).
3.2 Performativity and embodied meaning-making in human-machine communication
Underlying our PBM methodology is a conceptual framework, which meshes theoretical work from
performative new materialism (Gamble et al 2019; see also Barad 2003, 2007), embodied meaning-
making (Johnson 2007; Fuchs 2016; Sheets-Johnstone 2010) and kinaesthetic empathy (Koch 2014;
Reynolds and Reason 2012; Behrends et al 2012) and brings them into a unique conjunction through
performative, embodied practice. In a nutshell, we argue for and aim to put in practice a relational,
performative view of human-machine communication.
8
3.2.1 Agency cannot be given to a robotic artefact
Both agency and meaning in HRI design are often taken for granted and amenable to technical
reappropriation, determined by individually held representations (e.g., external features or pre-
programmed gestures). Agency understood performatively, however, is not a property that someone or
something can have or be imbued with, rather “agency is a matter of intra-acting … an enactment” (Barad
2007:178). Humans and machines here are no longer regarded as distinct subjects and objects with
discreet signals and messages sent in-between. Instead, we are looking at interaction as “a multiplicity of
more and less closely aligned, dynamically configured moments of encounter within sociomaterial
configurations, objectified as persons and machines” (Suchman 2007:268). More so, we believe that the
boundaries between subjects and objects become elastic and renegotiable in this dynamic encounter (see
Gemeinboeck 2019; 2021)—a view, which our observations and conversations with audiences and
participants seems to support (see section 3.3). Hence, rather than asking how a social agent should look
like or what behaviours display ‘its’ agency, we should instead look at human-machine couplings and
how they enact agency through the interactional exchange, evolving over time. The robot’s design still
plays a constitutive role in this interactional exchange, as it affects how it can act and how its actions are
interpreted, both of which are core to meaning making. Yet a distributed, coupled view brings to the fore
an artefact’s performative, participatory capacities instead of predefined representational attributes.
Shifting the design focus from representation to performativity can unlock an understanding of how
robots’ sociality emerges in the interactional exchange itself, beginning with, for instance, PBM’s dancer-
costume entanglement. Our relational, performative stance places meaning making firmly in the ‘here and
now’ of the interactional encounter, interrelating human and nonhuman co-agents in a particular situation.
Meaning making in HMC then happens as part of a process of embodied, situated material engagement
(see Malafouris 2013), rather than based on meanings (pre)ascribed to a certain appearance or behaviour.
Both human and machine interactors here are rendered active participants in the meaning-making process,
thus empowering them in terms of what they can bring to this negotiation and how they can evolve. A
relational understanding of the “ongoing reconfigurings of the world” (Barad 2007:141; see also Haraway
2007; Latour 2005; Law 2004), which we are never outside of, profoundly changes the potential of
human-robot communication and opens up new pathways for robot design and studying possible human-
robot relationships. It opens up a horizontal ethics of relationality (see section 4) and, from a pragmatic
design viewpoint, lays open a potentially rich field of opportunities that may lead to greater freedom and
novel, yet unknown ways of communicating with machines (see Sandry 2019). While ‘freeing’ the human
communicator to actively negotiate and continuously renegotiate meaning may place a bigger burden on
them, it also frees the machine communicator to become its own ‘thing’; a more or less social, unique
artefact, depending on both its machinelike abilities to participate in this negotiation and the relational
affordances of the unfolding situation.
3.2.2 Meaning making happens in the embodied interactional exchange
Our conceptual framework aligns Barad’s posthuman notion of agency as an intra-active enactment with
a notion of meaning-making that is fundamentally “relational, experiential and enactive” (Johnson
2018:244), situated in a particular social, material, cultural and historical context. In contrast to traditional
cognitive science, embodied cognition
7
places embodiment and interaction dynamics at the centre of the
meaning-making process (Lindblom 2020; Di Paolo et al 2010; Johnson 2007; Gallagher 2005; Varela et
al 1991). Instead of accessing our world through representations, we bodily participate in the generation
of meaning, “often engaging in transformational and not merely informational interactions; [we] enact a
world” (Di Paolo et al 2010:39). Such a radical embodied view that emphasizes sociocultural situatedness
and environmental embeddedness and favours a mutual link between action and perception over internal
representations (Lindblom 2020; see also Fuchs 2018; Gallagher 2005) significantly affects how we can
think about and consequently design HMC. Maturana and Varela (1987) characterise communicative
behaviours as occurring in processes of social coupling; communication then is the observable effect of
behavioural coordination. Hence, from a biological perspective, “there is no ‘transmitted information’ in
communication” and, equally, social interaction “cannot be reduced to so-called ‘social information
transfer’” (Lindblom 2020:10). Rather, social interaction is always relational (Fuchs 2018; Di Paolo et al
2010; see also Maturana and Varela 1987; Varela et al 1991), where meaning is negotiated in the
7
Embodied cognition rejects traditional cognitive-scientific notions of internal representation and computation in
favour of studying the fundamental role of bodily mechanisms and the environment, including interactions with other
agents, artefacts, etc. (Ziemke 2002; see also Lindblom 2020; Gallagher 2005).
9
encounter itself, dynamically arising from intermeshing processes that are corporeal, affective, creative,
social, cultural and interactive (see Di Paolo et al 2010; Lindblom 2020; Colombetti 2014).
Current MML research expands its investigations of a robot’s spatial-relational affordances by studying
and experimenting with processes of social coupling and the spatial-relational affordances that they
produce. Yet social couplings of humans and robots happen across a divide of perceptual worlds; like
embodiment, a robot’s perception differs greatly from ours. Hence, while humans and robots can
physically share a social space, from a biosemiotic viewpoint, they are each embodied in their own,
distinctly different umwelt (Uexküll 1957; see also Ziemke and Sharkey 2001). An entity’s umwelt is the
perceptual world in which it exists and acts or, in Colombetti’s words, its “lived or phenomenal
environment” (Colombetti 2010:5). Meaning making between humans and robots is thus an intra-bodily
enactment across differentiated ecological niches. Designing for embodied HMC then is about developing
pathways to negotiate humans’ and machines’ distinct ecological affordances (Fiebich 2014; see also
Gibson 1979) through embodied interactional coordination.
To afford dancers an embodied insight into the Cube Performer’s unique machine umwelt, we are
extending the costume interface to allow for mapping between human and nonhuman perceptual worlds,
the Relational Body Mapping (RBM) costume. Equipped with the same set of sensors that the robot uses,
the goal is to enable dancers inhabiting the RBM costume to experience the Cube Performer’s sensorium,
made ‘tangible’ to the dancer in the form of a dynamic soundscape. Current workshops with
choreographer Marie-Claude Poulin
8
and dancer Audrey Rochette experiment with the meaning-making
potential of dynamically emerging relational spaces in various social couplings and the intra-bodily
resonances they produce (see 3.2.3). Relational spaces between agents here are understood as situated
enactments that are aesthetically evoked, sculpted and rendered elastic through movement and its
dynamic qualities. Being able to tap into the robot’s different ecological affordances permits the dancer-
in-costume to creatively work with the asymmetries between the two perceptual worlds and how they
affect social coordination. The goal is then to bootstrap the Cube Performer’s learning with the
constraints and biases of movement dynamics recorded in relation to specific social couplings; while
grounded in its own machine umwelt, the learning is expanded by the dancer’s relational negotiation of
two umwelts.
3.2.3 Intra-bodily resonance between bodies-in-motion
At its core, MML seeks to generate possibilities for social coupling (see Maturana and Varela 1987)
between humans and machinelike artefacts. In this human-nonhuman coupling, meaning is created where
rhythmic coordination (Di Paolo et al 2010) meets embodied, affective sense-making (see Colombetti
2014; Froese and Fuchs 2012). According to Colombetti’s phenomenological approach, embedded in a
radical embodied understanding of cognition, our empathic experience of others does not rely on
ascribing mental states to them but rather happens ‘directly’, as we perceive bodies and their expressions
“as a locus of bodily subjectivity and sensations” (2014:175). Our thoughts, feelings, and behaviours are
grounded in our bodily interaction with other bodies and the environment (Meier et al 2012; see also
Lindblom 2015; Colombetti 2014). Vice-versa, these thoughts, feelings and behaviours manifest in
embodied ways in what Froese and Fuchs have termed “intra-bodily resonance” (Froese and Fuchs
2012:212). As they manifest, they express themselves to others, who interpret them based on their own
intra-bodily resonance. Much of our embodied, social meaning-making process thus involves movement
and, in particular, movement qualities, allowing us to rhythmically coordinate with others through
interaction (Di Paolo et al 2010). Fuchs and Koch thus understand motion and emotion as inherently
interconnected: “one is moved by movement (perception; impression; affection) and moved to move
(action; expression; e-motion)” (2014:1).
Intra-bodily resonance between bodies-in-motion is referred to by researchers in dance and dance studies
as kinaesthetic empathy (Behrends et al 2012). Kinaesthetic empathy is a key interdisciplinary concept
concerned with a moving body’s capacity to resonate with us and to literally move and bodily affect us;
thus facilitating our understanding of social interaction and embodied communication (Reynolds and
Reason 2012; Foster 2008). From a performance perspective, intra-bodily resonance is a bodily
processing of forces and tensions expressed in dynamic variations of movement qualities. Importantly,
resulting intra-bodily relations serve as scaffolding for non-verbal meaning-making (see Meekums 2012),
allowing for social interaction to be initiated and sustained without relying on stereotypical and limited
modelling of artificial emotions (see Damiano and Dumouchel 2020). Hence, rather than programming
social abilities into robots’ cognitive interior and relying on their anthropomorphic exterior to express
8
Codirector of kondition pluriel, see konditionpluriel.org
10
them (see Damiano and Dumouchel 2020), we look to movement qualities and how they afford robots
active participation in socially meaningful encounters. Our relational, performative approach thus
mobilises the intra-affective capacity of movement to actualise Damiano and Dumouchel’s “affective
loop” (2020:190), which locates sociality in the interactional dynamics, coenacted by the robot’s ability to
engage human interactors in affective encounters.
3.3 First encounters
The relational, embodied communication, which MML promotes, is already familiar to us from the
relationships we form with our animal companions. Embodied communication is, in Haraway’s words,
“more like a dance than a word: the flow of entangled, meaningful bodies in time—whether jerky and
nervous or flaming and flowing, whether both partners move in harmony or are painfully out of synch or
something else altogether—is communication about relationship, the relationship itself, and the means of
reshaping relationship and so its enacters” (Haraway 2008:26). Watching participants, whether audience
members or study participants, encounter our Cube Performer for the first time, we are often reminded of
Haraway’s sometimes jerky, sometimes flowing dance that is embodied communication. Dynamics
unfold in unpredictable configurations and participants find themselves, alternating, in moments of
harmony or ‘painfully out of synch’ with the cube. In one of our studies, five out of ten participants
compared their responses to the Cube Performer with the kinds of responses they have towards animals.
“I was surprised how intimate it was. I responded to it like another species and increasingly so”, said one
participant. Another commented, “[i]t comes across as playful with an honest curiosity, like a wild
animal”. We believe that this view of relational inter-species communication presents promising
pathways for human-robot communication, but without the need to render machines animal-like. The
difference between the Cube Performer and, for instance, a robot vacuum cleaner is that the machine
performer, situated by dancers, is better equipped to spatially and temporally coordinate with other
bodies. This, in turn, mobilizes us to correlate our bodies in response to produce “moments of moving
complicity” (Suchman 2007:265) by forming new constellations with the machine that are likely to
produce alternate openings for entanglement and so on.
We have so far studied social encounters with the Cube Performer (at various prototyping stages) as part
of four public exhibitions/events
9
and two dedicated participatory studies
10
. The main aim of sharing even
early prototypes in exhibitions and studies was to gain some insights into their capacity to generate intra-
bodily resonance in unscripted, first-encounter scenarios. The robot’s mechanical structure has been
conceived to allow changing its outer ‘shell’ to allow the Cube Performer to integrate itself in various
(performance) contexts. For the first public exhibition, we decided to stage Cube Performer #1 as a
gallery plinth, disguised amongst a group of other, immobile plinths (see Figure 7). In the open-studio
scenario, without an exhibition context, the robot took on the utilitarian identity of a simple wooden box
(see Figure 8). These ‘humble’ stagings suited our prototype stages and the contexts of encounter; at the
opening of Repair (2017), for instance, two audience members jumped when the apparent plinth, which
they had placed their glasses on, began to twist toward them.
9
The Cube Performer was shown in two exhibitions, RePair at the The Big Anxiety Festival, Sydney, 2017, and
Games and Performing Arts Festival, UK, 2018, and two further public events (Printemps Numérique, VUB,
Brussels, BE, 2019, and the Performing Robots Conference partnering with SPRING Festival, Het Huis Utrecht, NL,
2019).
10
We conducted two participatory studies: a three-day study involving 48 participants within the frame of the RePair
exhibition, Sydney, 2017 (https://www.thebiganxiety.org/events/repair), and a more detailed follow-up study, in
2019, involving ten participants as part of a two-day open lab, both situated in the same performance space at the
UNSW Art & Design campus. A description of the study designs can be found in (Gemeinboeck and Saunders 2018;
2019). Extracts of video documentation of first encounters can be viewed at: http://machinemovementlab.net/first-
encounters/
11
Figure 7: Cube Performer #1, shown at RePair, The Big Anxiety Festival Sydney, 2017
At an early prototyping stage, we were interested in getting feedback on the robot’s expressive and
affective qualities and whether they would, in the participants’ eyes, render the robot more humanlike. On
average, participants reported that they perceived the robot as evocative and affective, although not
humanlike. Participants also reported perceiving the robot as spontaneous and responsive, which was
surprising given its limited adaptive capabilities at the time of the study. Audiences often used affective
terms, e.g., ‘curious’, ‘shy’, ‘cheeky’ or ‘playful’ to describe the ways in which they perceived the Cube
Performer. Granted, it seems fair to say that we tend to attribute more agency and intent to machines that
behave with a certain degree of complexity than is technically warranted (see also Levillain and Zibetti
2017), particularly in first-encounter scenarios. However, having observed people’s surprise, affect and
curiosity in the course of the exhibitions and participatory studies, we believe that the rich movement
qualities performed by the Cube Performer and the dynamic, affective relations they produce are a
contributing factor to rendering encounters with this very plain artefact evocative and meaningful. In
general, audiences either preferred to observe the robot from a distance, often circulating around it, or
engaged with it directly for more than five minutes and sometimes significantly longer. These latter
encounters can be characterised as engagements, where people (1) are occupied with bodily probing how
the robot ‘works’ and/or how they are being sensed, (2) meet the Cube Performer ‘on its own terms’,
often based on an interplay of following the robot’s movements, tilting with it, crouching or trying to
keep up with it on their hands and knees and moving in sudden, unexpected ways to elicit responses from
it, or (3) behave in a combination of the two, at first being inquisitive regarding its workings and
increasingly developing a ‘dance’ with the robot.
Figure 8: Study participants engaging with Cube Performer #2, 2019
12
In contrast to our first study, which involved members of the public, the follow-up study aimed at
evaluating our PBM methodology with ten professional experts in interaction design and dance
improvisation
11
. To do this, we developed a three-minute-long sequence with the PBM costume to study
how the cube’s plain, omnidirectional geometry could be transformed in different spatial-affective ways
when moving in different dynamics of tension, amplitude and projection (see Sheets-Johnstone 2012).
We then trained the robot to perform similar movement trajectories and grouped the different spatio-
temporal-energetic dynamics in three categories, described by the choreographer and dancer as ‘light–
airy’, ‘boisterous–chunky’ and ‘playful–unpredictable’. Engaging our participants with this specific set of
movement qualities in multiple configurations allowed us to compare their perception of these qualities
and the different relational-affective affordances they produce to the dance experts’ interpretation (see
Table 1). The study involved a later iteration of the robot prototype (Cube Performer #2, see Figure 8)
which could perform more movement qualities with a higher sensorimotor fidelity, and participants
engaged between ten and fifteen minutes with the robotic artefact. Although the robot prototype lacked
improvisational skills at the time, participants tended to relate its movements to their own and reported
affective responses akin to intra-bodily resonances. Table 1 shows that all participants described the
artefact’s qualitative dynamics in terms that closely align with the choreographer’s and dancer’s
descriptions. One participant commented, “it’s obvious that it does what it does because I’m here”; others
expressed surprise about connecting with a ‘wooden box’: “I felt quite tender towards it” or “I like its
non-humanness … there is a companionability to it. Wow”. The results thus indicate that the dynamics of
the PBM entanglements when reinterpreted by the robot can, in the participants’ “kinetically-sensitive
eyes” (Sheets-Johnston 2010:124), produce kinaesthetic empathic responses. We previously referred to
this form of HMC as human-robot kinesthetics (Gemeinboeck and Saunders 2018).
quality
choreographer’s and
dancer’s description
participants’ descriptors
1
light–airy
sensitive, tender, tentative, gentle, delicate, timid, less
dynamic than other two stages
2
boisterous–chunky
aggressive, more violent, agitated, sharp, competitive,
purposeful, show-off, decisive
3
playful–unpredictable
playful, dynamic, attention seeking, intense, animal-like,
broader repertoire, moving with attitude
Table 1. Participants’ descriptors of different movement qualities (Gemeinboeck and Saunders 2019).
We also deployed PBM in design workshops with primary school children, aged 8–10
12
, to engage them
in creative, embodied explorations of possible relationships with machinelike robots. Children were
invited to get together in pairs to design a simple robot using cardboard boxes and to develop a playful
human-robot scenario with it, where one of them wears the cardboard robot costume (see Figure 9).
While some children opted for cutting holes in the box to be able to use their arms, many of the children
were inspired by our demonstrations of the Cube Performer and developed intricate movement patterns
with the box, sometimes coupled with moving tentacles or flaps, to give their robot a distinct affective
identity.
11
None of our participants were familiar with the project or its aims and only two of them had some prior experience
with robotic design. This narrow selection of participants offered us expert feedback in design areas most relevant to
our methodology. Future participatory studies are planned to include a much wider range of participants.
12
Four participatory workshops with 60 school children in total, as part of the public outreach program of the Games
and Performing Arts Festival at The Exchange Gallery, Penzance, and AMATA, Falmouth University, Penryn, UK,
2018.
13
Figure 9: PBM design workshop with school children, demonstrating Cube Performer #1 at AMATA, Falmouth
University (on the left), and building cardboard robot costumes at The Exchange Gallery, UK, 2018
4 Embodied meaning-making in HMC—some concluding reflections
In this article, we introduced a movement-centric design approach to HMC that counters HRI approaches
whose communication potential relies on rendering the machine and its behaviours as humanlike as
possible. Building on a conceptual framework that meshes concepts from performativity and embodied,
affective meaning-making, HMC in this arts-led approach is fundamentally relational, embodied and
performative. Its central premise is that social capacity is not a property of the machine but rather is
enacted in the encounter or evolving relationship, thus shifting the design focus from the representation of
social agency to how it is performatively enacted.
A core insight from our performative, relational approach is that agential enactment in meaning-making
between human and nonhuman agents not only plays a key role in the interactional process but is equally
core to the design process. Hence, human-machine encounters begin with the design process, not only
once the robot is apparently “ready for relationships” (Turkle 2005:288). Talking about effective
encounters at the human machine interface, Suchman points to “those moments of moving complicity
between persons and things achieved through particular, dynamic materialities and extended socialities”
(Suchman 2007:245). Moments of complicity brought about by dynamic configurations of specific
materialities and distributed socialities not only shape the interactional encounter but also constitute
significant junctures in the design process that shape the social potential of our human-machine
relationships. Conceived to socioculturally situate the learning of an abstract robotic artefact, the PBM
costume also serves to explore and experiment with the first instances of human-machine encounter. It
permits dancers to corporeally extend and kinaesthetically probe into a particular set of material,
performative possibilities to reimagine and bodily coenact the artefact’s spatial-affective affordances. It is
along the embodied interface and the dancer’s kinaesthetic-material probings that the framing of possible
human-machine encounters and the potential for relational meaning-making is beginning to take shape,
quite literally.
Arguing for an extended notion of HMC that considers machinelike robots as social communicators,
Sandry (2019) distinguishes between evocative or relational artifacts, depending on their ‘personality’.
An evocative artefact, according to Sandry (2019), reveals itself in the process of interaction based on its
behaviour, without suggesting a humanlike capacity for reciprocal communication, while a relational
artefact directly engages the human interactor. This focus on direct engagement, she argues, is more
likely not only to be considered humanlike, but also to be compared against a human assistant or
companion. But opposing the evocative and relational seems unnecessarily limiting, particularly as it
assumes the relational to be confined to and compared against human qualities. Both from a performance
or an embodied cognition perspective, embodied meaning-making is always relational and contextual,
without being confined to the human domain. Indeed, it could be argued that only relational forms of
HMC situate machines (whether machinelike or humanlike) in a wider sociomaterial ecology. Beyond a
human-centric perspective, relational artefacts, such as the Cube Performer, open up HMC to the
enactive, relational nature of meaning-making that we can observe in embodied animal communication
(see Smuts 2008; Haraway 2008), as well the performative, agential role that artefacts play in our
embodied, material engagements (see Malafouris 2013).
A relational, performative notion of HMC could open up a horizontal ethics of human-machine relations,
one that releases the robot from Čapek’s original vision of humanlike non-humans built to serve mankind
(see Madigan 2009), which arguably still weighs on current social robot design. Jones (2017) detected the
beginnings of a relational turn in HRI, arguing that researchers begin to consider people’s experience and
social judgement as part of the making of a robot’s sociality. Hegel et al (2009), for instance, speak of
14
robots with a social interface, where the latter is a metaphor for a machine’s properties that, in the
observer’s eyes, render it social. However, shifting the attribution of sociality from an internal property to
peoples’ perception still understands social agency as something given to the artefact by humans, albeit
through psychological attribution (see Levillain and Zibetti 2017). Aligned with our argument here,
Damiano and Dumouchel (2020) argue that a genuine relational turn warrants a further step, which
requires a radically different understanding of sociality and how it emerges. The relational only opens up
once we position ourselves on a horizontal plane and in the middle of the encounter.
Observing audiences and participants encountering the Cube Performer, it is the in-between where
transformations are not only triggered but literally take on shape: without an interactor, the Cube
Performer may twist, tilt and sway but it is just an object-in-motion. But as soon as a person (or another
artefact for that matter) approaches the cube and, for instance, crouches facing one of its gently twisting
corners, the same movement turns into a gesture toward them. Hence, the same spatial-temporal
dynamics take on a meaning when in relation to someone/thing. This transformation, we claim, does not
only happen in the interactor’s or the observer’s eyes as it requires two bodies to intra-act to co-construct
this relational space. Movement becomes a relational gesture in the dance that is embodied
communication (see Haraway 2008), a form of empathic ‘being toward’ that renders the previously
unseen or obscure meaningful (see Dimitrova 2017). A horizontal, relational ethics thus unhinges and
breaks open traditional visions of what a robot is, requiring and at the same time opening up “a more
differentiated set of starting points for the robot” (Castañeda and Suchman 2014:340).
Our Cube Performer, in many ways, can be considered a material, situated research proposition, and we
recognise, as earlier noted, that our relational, performative approach to HRI design is not suitable for all
robot designs. Yet to advance our understanding of possible human-robot relationships, it is critical that
we question our assumptions and widen our perspective to include more differentiated notions of human-
nonhuman communication and its social potential. Robots, such as the Cube Performer, that are able to
skilfully utilise motion dynamics as building blocks for affective, social coordination could not only open
up novel, participatory forms of human-nonhuman meaning-making but also bring to the fore alternative
ethical dimensions. In contrast to human-robot relationships that remodel human relationships, such an
alternative, horizontal human-nonhuman ethics is more reflective of our embeddedness in the “ongoing
reconfigurings of the world” (Barad 2007:141). Intra-acting with a plain cube, meaning- and relation-
making cannot follow familiar social protocols but can only unfold and evolve in the intra-actional
process. In fact, we selected this simple yet commonly expressionless shape precisely because it cannot
rely on already known social attributes. Instead, we found that a cube equipped to participate in the
encounter through the relational dynamics of movement can engender a surprisingly broad potential for
social meaning to be enacted in-between, far beyond the mirror image.
Acknowledgments
This research was funded in part by the Australian Government through the Australian Research Council
(DP160104706); an EU Framework Programme (FP7) European Research Area Chairs Scheme project
(621403); and the Austrian Government through the Austrian Science Fund (FWF): AR545.
The authors would like to thank former co-investigator Maaike Bleeker (Utrecht University, NL);
choreographer and current co-investigator Marie-Claude Poulin (kondition pluriel: konditionpluriel.org)
and dance performer Audrey Rochette; De Quincey Co (dequinceyco.net), in particular director and
choreographer Tess de Quincey and dance performers Linda Luke and Kirsten Packham; as well as dance
performers Katrina Brown and Sarah Levinsky (Falmouth University, UK).
References
Alač M (2015) Social robots: Things or agents? AI & Society 31(1):519-535.
https://doi.org/10.1007/s00146-015-0631-6
Alač M (2016) Zeigt auf den Roboter und schüttelt dessen Hand. Intimität als situativ gebundene
interaktionale Unterstützung von Humanoidtechnologien (In English: Points to and Shakes the Robot’s
Hand. Intimacy as Situated Interactional Maintenance of Humanoid Technology). Zeitschrift für
Medienwissenschaft 8(15):41-71. http://dx.doi.org/10.25969/mediarep/1902
Aviv V (2017) Abstracting dance: detaching ourselves from the habitual perception of the moving body.
Frontiers in Psychology 8: 776. https://doi.org/10.3389/fpsyg.2017.00776
15
Barad, K (2003) Posthumanist performativity: toward an understanding of how matter comes to matter.
Signs: Journal of Women in Culture and Society 28(3):801-831. https://doi.org/10.1086/345321
Barad K (2007) Meeting the universe halfway: quantum physics and the entanglement of matter and
meaning. Duke University Press, Durham
Becker B (2006) Social robots – emotional agents: some remarks on naturalizing man-machine
interaction. International Review of Information Ethics 6(12/2006):37–45
Behrends A, Müller S, Dziobek I (2012) Moving in and out of synchrony: a concept for a new
intervention fostering empathy through interactional movement and dance. The Arts in Psychotherapy
39(2):107–116. https://doi.org/10.1016/j.aip.2012.02.003
Billard A, Calinon S, Dillmann R, Schaal S (2008) Robot programming by demonstration. In: Siciliano
B, Khatib O (eds) Springer handbook of robotics. Springer, Berlin, Heidelberg, pp 1371–1394.
https://doi.org/10.1007/978-3-540-30301-5_60
Broadbent E (2017) Interactions with robots: the truths we reveal about ourselves. Annual Review of
Psychology 68:627–652. https://doi.org/10.1146/annurev-psych-010416-043958
Castañeda C, Suchman L (2014) Robot visions. Social Studies of Science 44(3):315–341.
https://doi.org/10.1177/0306312713511868
Craig RT (1999) Communication theory as a field. Communication Theory 9:119–161.
https://doi.org/10.1111/j.1468-2885.1999.tb00355.x
Colombetti G (2010) Enaction, sense-making, and emotion. In: Stewart J, Gapenne O, Di Paolo E
(eds) Enaction: towards a new paradigm for cognitive science. MIT Press, Cambridge, pp 145–164
Colombetti, G (2014) The feeling body: affective science meets the enactive mind. MIT Press,
Cambridge
Damiano L, Dumouchel P (2020) Emotions in relation. Epistemological and ethical scaffolding for mixed
human-robot social ecologies. HUMANA.MENTE Journal of Philosophical Studies, 13(37)181–206.
http://www.humanamente.eu/index.php/HM/article/view/321. Accessed: 30 October 2020
Dautenhahn K (2013) Human–robot interaction. In: Soegaard M, Dam RF (eds) Encyclopedia of human-
computer interaction, 2nd edn. Interaction Design Foundation, pp 2283–366
Dautenhahn K (2002) Design spaces and niche spaces of believable social robots. In: Proc. 11th IEEE Int.
Workshop on Robot and Human Interactive Communication (RO-MAN 2002), pp 192–197
Dautenhahn, K, Nehaniv, C, Alissandrakis, A (2003) Learning by Experience From Others—Social
Learning and Imitation in Animals and Robots. Kühn, R., Menzel, R., Menzel, W., Ratsch, U., Richter,
M. and Stamatescu, I. (eds) Adaptivity and Learning: An Interdisciplinary Debate (Berlin, Heidelberg:
Springer), 217–421.
De Graaf MM, Allouch SB (2013) Exploring influencing variables for the acceptance of social robots.
Robotics and Autonomous Systems, 61(12):1476–1486. https://doi.org/10.1016/j.robot.2013.07.007
Dimitrova Z (2017) Robotic performance: an ecology of response. Performance Philosophy. 3(1):162–
177. https://doi.org/10.21476/PP.2017.3135
Di Paolo E, Rohde M, De Jaegher H (2010) Horizons for the enactive mind: values, social interaction,
and play. In: Stewart J, Gapenne O, Di Paolo E (eds) Enaction: towards a new paradigm for cognitive
science. MIT Press, Cambridge, pp 33–87
Drouin-Brisebois J, Hill GA, Horne S, Ninacs A-M (2008) Caught in the act: the viewer as performer.
National Gallery of Canada, Ottawa
Fiebich A (2014) Perceiving affordances and social cognition. In: Gallotti M, Michael J (eds)
Perspectives on social ontology and social cognition. Studies in the Philosophy of Sociality 4. Springer,
Dordrecht, pp 149–166. https://doi.org/10.1007/978-94-017-9147-2_11
Fink J (2012) Anthropomorphism and human likeness in the design of robots and human-robot
interaction. In: Ge SS, Khatib O, Cabibihan JJ, Simmons R, Williams MA (eds) Social robotics, ICSR
2012. Springer, Berlin, Heidelberg, LNCS 7621:199–208. https://doi.org/10.1007/978-3-642-34103-8_20
Fong T, Nourbakhsh I, Dautenhahn K (2003) A survey of socially interactive robots. Robotics and
Autonomous Systems 42(3-4):143–166. https://doi.org/10.1016/S0921-8890(02)00372-X
16
Foster SL (2008) Movement’s contagion: the kinesthetic impact of performance. In: Davis TC (ed) The
Cambridge Companion to Performance Studies. Cambridge University Press, Cambridge, pp 46–59
Froese T, Fuchs T (2012) The extended body: A case study in the neuro-phenomenology of social
interaction. Phenomenology and the Cognitive Sciences 11:205–236
Fuchs T (2018) Ecology of the brain. Oxford University Press, Oxford
Fuchs, T (2016) Intercorporeality and Interaffectivity. Meyer, C., Streeck, J. and Jordan, S. (eds)
Intercorporeality: Emerging Socialities in Interaction, (Oxford: Oxford University Press). https://
doi.org/10.1093/acprof:oso/9780190210465.003.0001
Fuchs T, Koch S (2014) Embodied affectivity: on moving and being moved. In: Frontiers in Psychology.
5(3):508. https://doi.org/10.3389/fpsyg.2014.00508
Gallagher S (2005) How the body shapes the mind. Oxford University Press, Oxford
Gamble CN, Hanan JS, Nail T (2019) What is new materialism? Angelaki. 24(6):111–134.
https://doi.org/10.1080/0969725X.2019.1684704
Gemeinboeck P (2021) The aesthetics of encounter: a relational-performative design approach to human-
robot interaction. Front. Robot. AI - Human-Robot Interaction. https://doi.org/10.3389/frobt.2020.577900
Gemeinboeck, P (2019) Dancing with the nonhuman. J. Bennett, M. Zournazi (eds) Thinking in the
World, (London /New York: Bloomsbury Academic), 214-239.
Gemeinboeck P, Saunders R (2019) Exploring social co-presence through movement in human robot
encounters. In: Proc. AISB 2019 Symposium on Movement that Shapes Behaviour
Gemeinboeck P, Saunders R (2018) Human-robot kinesthetics: mediating kinesthetic experience for
designing affective non-humanlike social robots. Proc. 27th IEEE Int. Conference on Robot and Human
Interactive Communication (RO-MAN 2018), pp 571–576.
https://doi.org/10.1109/ROMAN.2018.8525596
Gemeinboeck P, Saunders R (2017) Movement matters: how a robot becomes body. In: Gillies M,
Niehaus, K (eds) MOCO ’17: Proc. 4th Int. Conference on Movement Computing. Association for
Computing Machinery, New York, pp 1–8. https://doi.org/10.1145/3077981.3078035
Gibson, JJ (1979) The ecological approach to visual perception. Houghton Mifflin, Boston
Guzman AL (2020) Ontological boundaries between humans and computers and the implications for
human-machine communication. Human-Machine Communication, 1:37–54.
https://doi.org/10.30658/hmc.1.3
Guzman AL, Lewis SC (2020) Artificial intelligence and communication: a human-machine
communication research agenda. New Media & Society, 22(1):70-86.
https://doi.org/10.1177/1461444819858691
Guzman AL (2018) What is human-machine communication, anyway? In: Guzman AL (ed) Human-
machine communication: rethinking communication, technology, and ourselves. Peter Lang, New York,
Bern, Berlin, Bruxelles, Oxford, Wien, pp 1–43
Haraway DJ (2008) When species meet. University of Minnesota Press, Minneapolis
Hegel F, Muhl C, Wrede B, Hielscher-Fastabend M, Sagerer G (2009) Understanding social robots. In:
Proc. Second Int. Conferences on Advances in Computer-Human Interactions (ACHI '09). IEEE, New
York, 169–174
Hoorn JF (2018) Theory of robot communication: I. The medium is the communication partner.
arXiv:1812.04408 [cs.HC]. https://arxiv.org/abs/1812.04408. Accessed 14 Nov 2020
Jochum EA, Millar P, Nuñez D (2017) Sequence and chance: design and control methods for
entertainment robots. Robotics and Autonomous Systems 87:372–380.
https://doi.org/10.1016/j.robot.2016.08.019
Johnson M (2018) The aesthetics of meaning and thought: the bodily roots of philosophy. University of
Chicago Press, Chicago
Johnson M (2007) The meaning of the body. In: Overton WF, Mueller U, Newman JL (eds)
Developmental perspectives on embodiment and consciousness. Psychology Press, Hove, pp 35–60
17
Jones RA (2018) Human-robot relationships. In: Bess M, Pasulka DW (eds) Posthumanism: the future of
homo sapiens. Macmillan Reference USA, New York, pp 365–375
Jones RA (2017) What makes a robot ‘social’? Social Studies of Science 47(4):556-579
Lasseter J (2001) Tricks to animating characters with a computer. ACM SIGGRAPH Computer Graphics
35(2):45–47. https://doi.org/10.1145/563693.563706
Latour B (2005) Reassembling the social: an introduction to actor-network-theory. Oxford University
Press, New York
Law J (2004) After method: mess in social science research. Routledge, London
Leach J, deLahunta S (2017) Dance becoming knowledge: designing a digital “body”. Leonardo
50(5):461–467. https://doi.org/10.1162/LEON_a_01074
Lee HR, Šabanović S, Stolterman E (2016) How humanlike should a social robot be: a user-centered
exploration. In: Taylor ME, Sukthankar G (eds) In: Proc. AAAI Spring Symposium on Enabling
Computing Research in Socially Intelligent Human-Robot Interaction. AAAI, Palo Alto, pp 135–141
Levillain F, Zibetti E (2017) Behavioural objects: the rise of the evocative machines. Journal of Human-
Robot Interaction 6(1):4–24. https://doi.org/10.5898/JHRI.6.1.Levillain
Lindblom, J (2020) A radical reassessment of the body in social cognition. Frontiers in Psychology
11:987 https://doi.org/10.3389/fpsyg.2020.00987
Lindblom J (2015) Embodied social cognition. Springer, New York
Lindblom J, Ziemke T (2003) Social situatedness of natural and artificial intelligence: Vygotsky and
beyond. Adaptive Behavior 11(2):79-96. https://doi.org/10.1177%2F10597123030112002
Madigan T (2009) Food for Thought: RUR or RU ain’t a person? Philosophy Now 72:48.
https://philosophynow.org/issues/72/RUR_or_RU_Aint_A_Person. Accessed 2 Nov 2020
Malafouris L (2013) How things shape the mind: a theory of material engagement. MIT Press, Cambridge
Manning E, Massumi B (2014) Thought in the act: passages in the ecology of experience. University of
Minnesota Press, Minneapolis
Maturana H, Varela F (1987) The tree of knowledge. Shambalaya, Boston
Meekums B (2012) Kinesthetic empathy and movement metaphor in dance movement psychotherapy. In
Reynolds D, Reason M (eds) Kinesthetic empathy in creative and cultural practices. Intellect Books,
Bristol, pp 51–65
Meier BP, Schnall S, Schwarz N, Bargh JA (2012) Embodiment in social psychology. Topics in
Cognitive Science 4(4):705–716. https://doi.org/10.1111/j.1756-8765.2012.01212.x
Penny S (2000) Agents as artworks and agent design as artistic practice. In: Dautenhahn K (ed) Human
cognition and social agent technology. John Benjamins, Amsterdam, pp 395–413
Penny S (2016) Robotics and art, computationalism and embodiment. In: Herath D, Kroos C, Stelarc
(eds) Robots and art: exploring an unlikely symbiosis. Springer, Berlin, Singapore, pp 47-65
Reynolds D, Reason M (2012) Introduction. In: Reynolds D, Reason M (eds) Kinesthetic empathy in
creative and cultural practices. Intellect Books, pp 17–25
Rotman B (2008) Becoming beside ourselves: the alphabet, ghosts, and distributed human being. Duke
University Press, Durham
Sandry E (2016) The potential of otherness in robotic art. In: Herath D, Kroos C, Stelarc (eds) Robots and
art: exploring an unlikely symbiosis. Springer, Berlin, Singapore, pp 177–189
Sandry E (2019) Taking social machines beyond the ideal humanlike other. In: Papacharissi Z (ed) A
networked self: human augmentics, artificial intelligence, sentience. Routledge, New York, pp 69–82
Šabanović S (2010) Robots in society, society in robots: mutual shaping of society and technology as a
framework for social robot design. Int. Journal of Social Robotics 2(4):439–450.
https://doi.org/10.1007/s12369-010-0066-7
18
Saunders R, Gemeinboeck P (2018) Performative body mapping: a creative robotics method for learning
expressive movement. In: Machine Learning for Creativity and Design, NeurIPS 2018 Workshop.
https://nips2018creativity.github.io/doc/performative_body_mapping.pdf. Accessed 19 Mar 2020
Sheets-Johnstone M (2010) Kinesthetic experience: understanding movement inside and out. Body,
Movement and Dance in Psychotherapy 5(2):111–127. https://doi.org/10.1080/17432979.2010.496221
Sheets-Johnstone M (2011) The primacy of movement, expanded 2nd edn. Series on Advances in
Consciousness Research 14. John Benjamins, Amsterdam
Sheets-Johnstone M (2012) From movement to dance. Phenomenology and the Cognitive Sciences
11:39–57. https://doi.org/10.1007/s11097-011-9200-8
Smuts B (2008) Embodied communication in nonhuman animals. In: Fogel A, King B, Shanker S (eds)
Human development in the 21st century. Cambridge University Press, Cambridge, pp 136–146
Suchman L (2007) Human-machine reconfigurations: plans and situated actions. Cambridge University
Press, Cambridge
Suschke, S (2003) Müller macht Theater: Zehn Inszenierungen und ein Epilog. Berlin: Theater der Zeit
Stevens C and McKechnie S (2005) Thinking in action: thought made visible in contemporary dance.
Cognitive Processing 6:243–252. https://doi.org/10.1007/s10339-005-0014-x
Turkle S (2011) Alone together: why we expect more from technology and less from each other. Basic
Books, New York
Turkle S (2007) Authenticity in the age of digital companions. Interaction Studies 8(3):501–517
https://doi.org/10.1075/is.8.3.11tur
Turkle S (2005) The Second Self: Computers and the Human Spirit, Twentieth Anniversary Edition. The
MIT Press, Cambridge
Uexküll, JV (1957) A Stroll Through the Worlds of Animals and Men: A Picture Book of Invisible
Worlds. In: Schiller, CH (ed, trans) Instinctive behavior: the development of a modern concept.
International Universities Press, New York, pp 5–80
Varela FJ, Thompson E, Rosch E (1991) The Embodied Mind: Cognitive Science And Human
Experience. MIT Press, Cambridge
Vlachos E, Jochum E, Demers LP (2016) The effects of exposure to different social robots on attitudes
toward preferences. Interaction Studies 17(3):390–404. https://doi.org/10.1075/is.17.3.04vla
Ziemke T, Sharkey N (2001) A stroll through the worlds of robots and animals. Semiotica 134:1–4.
https://doi.org/10.1515/semi.2001.050
