ChapterPDF Available

Distributed Embodiment: Real Presence in Virtual Bodies

Authors:

Abstract

This chapter discusses the notion of mediated presence, the feeling of being experientially present in a virtual or mixed reality, and describes how this form of virtuality is developing into “distributed embodiment.” When we experience strong mediated presence, our experience is that the technology has become part of the self. Distributed embodiment describes how our sense of being present in the world is becoming separated from our sense of ownership of a particular body, through the development of new approaches to deploying the technologies of virtualization that give rise to what is known as “mediated presence,” or “telepresence.” The possibility for distributed embodiment comes from the physical-virtual nature of familiar, first-person embodiment. We move from a sense of presence in the physical world, though a mediated sense of presence in virtuality, to the mediated sense of being in the physical-virtual world in another body than our own.
Part VIII
Technology and Applications
Chapter 36
Distributed Embodiment: Real Presence in Virtual Bodies
John A. Waterworth and Eva L. Waterworth
Abstract
This chapter discusses the notion of mediated presence, the feeling of being experientially
present in a virtual or mixed reality, and describes how this form of virtuality is developing
into “distributed embodiment.” When we experience strong mediated presence, our
experience is that the technology has become part of the self. Distributed embodiment
describes how our sense of being present in the world is becoming separated from our sense
of ownership of a particular body, through the development of new approaches to deploying
the technologies of virtualization that give rise to what is known as “mediated presence,” or
“telepresence.” The possibility for distributed embodiment comes from the physical-virtual
nature of familiar, first-person embodiment. We move from a sense of presence in the
physical world, though a mediated sense of presence in virtuality, to the mediated sense of
being in the physical-virtual world in another body than our own.
Keywords
mediated presence, telepresence, virtual reality, mixed reality, distributed embodiment, self
In this chapter, we discuss the notion of mediated presence, the familiar feeling of being
experientially present to a greater or lesser extent in a virtual or mixed reality, and describe
how this form of virtuality is developing into something new that we call “distributed
embodiment.” Distributed embodiment describes how our sense of being present in the world
is becoming separated from our sense of ownership of a particular body, through the
development of new approaches to deploying the technologies of virtualization that give rise
to what is known as “mediated presence,” or “telepresence” (Bracken and Skalski 2010). The
possibility for distributed embodiment comes from the physical-virtual nature of familiar,
first-person embodiment. We move from a sense of presence in the physical world, through a
mediated sense of presence in virtuality, to the mediated sense of being in the physical-virtual
world in another body than our own.
Presence is the feeling of being in an external world (Waterworth et al. 2010). As
discussed in chapter 12 in this volume, by Riva and Waterworth, human consciousness of
being present in an external environment has its roots in the animal feeling of something
happening from outside the self rather than from within. In other words, the sense of presence
distinguishes the self from the nonself. This suggests two key principles. First, the feeling of
presence is an embodied phenomenon, a source of information (analogous to emotional
engagement) through which we monitor and adjust our reactions and level of attention to our
surroundings. Second, mediated presence is fundamentally the same phenomenon as natural
presence; it concerns the extent to which we feel ourselves to be in our present surroundings,
at the present time. Mediated presence is the feeling of being present through virtuality
experienced as a convincing perceptual “illusion of non-mediation” (Lombard and Ditton
1997). As in natural presence, this varies moment by moment according to what is presently
happening in our surroundings (see Waterworth et al. 2010 for a fuller account). We may feel
hardly present at all in the physical world (a state we call absence; Waterworth and
Waterworth 2001) if nothing is happening there that is of interest or that impacts on our well-
being, and so it is with mediated presence. Presence arises from active awareness of our
embodiment in a present world around us. Presence is not consciousness, and we may be
highly conscious while feeling absent, at those times when we are relatively unaware of our
own embodiment.
Our earlier publications suggest that presence is the means by which an organism
knows when something is happening in the present world at the present time, and is the
manifestation of an encoded ability to know when consciousness is occupied with situations
in the immediate, outside world. For organisms in a natural environment, it is obviously vital
for survival to pay conscious attention and respond rapidly to present threats and
opportunities. This need is a key driver for development, both within the developing
organism and when viewed as evolutionary change. Through evolution, this fundamental
ability of all conscious organisms has developed in humans into the ability to distinguish
external, physical events and situations from events and situations realized mentally, as
internal reflections in thought and imagination. This is a necessary distinction that cannot be
made on the basis of emotional appraisal or reality judgments, because imagined situations
trigger the same emotional responses as physical situations (Russell 2003)—and may also be
judged real or unreal (as may physical events). To do this, people need to be able to feel
directly when they are attending to the current external world; this is the feeling of presence.
It is closely bound up with the intention to act, of mental and bodily readiness for action in
the physical or in a virtual world (see Riva et al. 2011).
We see the development of increasing virtuality as part and parcel of the evolution of
the human sense of presence, but also suggest that not all forms of virtualization play a
similar role in this. We argue that only when the boundary of the self is experientially altered
by technology can we say that presence has evolved into new forms, that the sense of
presence and the virtual forms are coevolving. This perspective provides the grounding for
understanding how our sense of presence could shift from our own bodies to other, virtual,
bodies. Recent experiments with the sense of presence, as well as new gaming and
communication innovations that are starting to appear on the market, already point to a new
way of being conscious, a way of feeling real presence in a virtual body, that we suggest is
the next step in the evolution of presence. This implies that the virtual other body is
incorporated (literally) into our sense of self.
Can You Tickle Yourself?
René Descartes gets a bad press these days, along with any other suggestions that smack of
dualism. Almost no one takes seriously the view that we form a representation of the world in
our brains, which we then somehow “view” mentally to know what the world is like. The
very idea is widely ridiculed as suggesting some sort of private internal cinema where a
projection of the external world is watched by a homunculus (in whose head sits another
mini-observer, and so on, in infinite regress)—the so-called Cartesian theater (Dennett 1991).
And we also all know that how we experience the external world does depend on what goes
on in our heads, specifically our brains. We know that drugs, sickness, aging, and trauma, to
name but a few, change the way our brains process information and so alter our perception of
what is “out there” in the world. We need to have a body, too, with its capacities for sensing
and for action, to be able to have any impression of the world around us. Some theorists talk
of “enacting a world” out there through action (Varela et al. 1999). By this view, it is not that
there is a world out there, at least not an objectively definable one, it is rather that we bring
one into being through our actions.
And yet we claim that, in a sense, the world we experience is a Cartesian theater, but
it is one that we experience as existing outside our heads. This is the world in which we feel
ourselves to be, bodily, at the present time and place. It is a Cartesian theater because it exists
for us by virtue of what is happening, electrically and chemically, in our brains; and we are
experiencing it, though not as merely a passive member of the audience but as an actor in the
show.
Some thoroughgoing antidualists (for example Dennett 1991, in the process of
explaining consciousness away) conveniently ignore the fact that there is a world that we do
experience as being inside our heads, because it exists nowhere else. When we read a book,
or fantasize about the future, or fret about the past, the world in which we are absorbed is not
experienced as being out there, in the present. We divide our mental and especially our very
limited attentional resources between these two worlds: one we experience as internal, which
is not tied to the present time and place, and the other that we experience as external, which
is. Media such as novels or fairy stories, represented on the page with the abstract symbols of
written language, create a vivid and believable world (fictional or factual, or usually a bit of
both) inside our heads. But we do not experience presence in this way, since there is no
perceptual illusion of nonmediation.
Fact or fiction can be represented in both abstract and concrete forms of media, and
the increasing development of virtuality brings with it a shift toward concrete
representations—forms that look, sound, and even feel like the things they are experienced as
being. In cinemas and homes throughout the developed world, movie and especially game
devices seek to evoke ever-stronger impressions of reality in virtuality (see Bracken and
Skalski 2010). This involves closely matching the Cartesian coordinates of reality with
appropriate, body-position-sensitive sights, sounds, and tactile stimulation. Our perceptual-
motor systems then project their “findings” about what is happening onto whatever really
does exist “out there.” Information technology thus fools the mind-body by matching its
expectations of how a three-dimensional world of physical objects looks, sounds, feels, and
behaves. This is not necessarily always what is thought of as increased “realism” or exactly
related to the level of technological immersion. High realism may magnify the impact of
whatever mismatches are perceived, as in the Uncanny Valley effect (Misselhorn 2009). And
while experienced presence tends to increase with immersion, the use of high-end immersive
technology does not guarantee a convincing experience of presence.
So, can you tickle yourself? The point of this question is that it brings forth a quick-
and-dirty appraisal of where the boundary between self and other lies. Normally, the self is
roughly collocated with the physical body. In other words, self-image and body image are
mentally collocated. Even though we can conceive of the body as an object, it does not have
the same status as other objects in the world. We (again, normally) only see this particular
object from the inside, with what we call a first-person perspective.
We know that some technology can change the boundary of the body, by becoming
part of the self—the blind man’s stick is the classic example from phenomenology (Merleau-
Ponty 1962), the constantly worn spectacles, even the car we drive daily. Most people cannot
tickle themselves (at least, not with amusing effect), but a tickling machine can do the job.
For most people, only something other, the nonself, can successfully tickle them. By our
view, the feeling of presence tells us what is other and what is ourselves. When we mistake
our own thoughts or actions for those of another, it is an indication that the presence
mechanism has broken down.
Presence and Distributed Cognition
The relation between self and other is reflected in prevalent arguments about the impact of
information and communication technology on consciousness, and on the relationship
between the mind and the body. One is the common observation that since the world of
digital communications is increasingly part of our bodies—not only embedded or attached
devices such as sensors or electrodes on the brain, but also carried devices such as mobile
phones or laptops—it is becoming clear that we are basically and even naturally cyborgs,
blends of animal and machine with “extended minds” (e.g., Clark 2003). Another view is that
tangible interaction characterizes the future of virtualization—this is “where the action is”
(Dourish 2001). Yet another is that the mind exists as a result of the whole range of
interactions between people and artifacts in the pursuit of activities (e.g., Kaptelinin and
Nardi 2006. Related themes have also been made in the literature of distributed and extended
cognition (e.g., Perry 2003). All of these views have merit, but we can put their claims into a
new perspective when we consider the sense of mediated presence, stressing the significance
of presence in distinguishing self from other.
When we are deprived of the electronic tools we have become used to and depend on
in everyday activities, we naturally feel somewhat lost and confused. This is one of the
stronger arguments for viewing new information technology as part of the extended self. The
loss may feel as if a part of memory has been erased, as when the address book on one’s
mobile phone suddenly disappears through an electrical fault. But it may also feel as if some
aspect of one’s world no longer exists, for example when the Internet connection is down and
one’s normal interactions there are not possible. These are quite different psychological
effects that reflect the sense of presence in operation. We do feel strong presence in some
kinds of mediated environments, such as videoconferences and VR, and relatively strong
presence in cinemas and even some websites. But we do not feel present within an electronic
address book; nor would we want to.
We use the general term of “distributed cognition” to cover these views that neglect
the role of presence. The common theme of such views is a perspective that suggests that the
mind is not located in and is not only a function of the brain, but it is also distributed among
the information artifacts we encounter in the world, and including those we carry with us
(Clark 2003; Perry 2003). This may seem surprising, but is it really? The mind evolved to
deal with objects and events in the world through our own movements and actions. If I
perceive a tree, for example, physically located in my immediate surroundings (among other
trees, and many other things that I also perceive), it is not experienced by me as residing in
my head—that would not be helpful for survival. It is out there, while I, myself am in here, in
my head. If I am able to think about the tree, it is because my cognition is in a sense
distributed to include the tree. The mind does not build a detailed internal representation of
the tree because it does not need to; the tree is there in all its glory to be inspected at any
moment. It is not such a conceptual leap to suggest that the same applies to information
artifacts in the surroundings, which in themselves also function as processors of information
(see also Hutchins 1996). But recent developments in virtual technology take us beyond the
extended mind and toward the initially more surprising idea of what we call distributed
embodiment.
The strength of the feeling of presence experienced is a potentially powerful factor in
understanding the extent to which technology has become experientially internalized as part
of the self. Information can be seen as “realized,” that is, given concrete form, either
internally in the mind, or externally, in the physical world. When information must be
realized internally to be given form and understood, such as that expressed in the abstract
symbols of language, any information technology involved in its expression is experienced as
part of the other. A character in a novel, for example, is in this sense realized in the mind of
the reader and not on the page; the technology involved—the book—remains external. In
contrast, when information is realized externally, as concrete forms in or as a surrounding
environment in which one can act directly, the technology may become part of the self. An
action-based computer game is one common example. The expert player acts directly in a
virtual world that has been realized externally, while the technology creating the world
effectively disappears from view—in what we have termed perceptually seductive technology
(Waterworth 2001). To be part of the self, information technology must create or modify an
external other of which it is not perceived to be a part; this will be another in which, or with
which, we can feel consciously present in virtuality.
We suggest that rather than focusing on notions such as “distributed cognition,” by
which artifacts perceived as situated in the external environment contribute to the problem-
solving work of the singular and body-centered mind, it is time for a new focus on distributed
embodiment.
Developing First-Person Presence and Social Personas
As with simpler organisms, the sense of self of the newborn infant is underdeveloped, but we
believe the feeling of presence is already there when the newborn is conscious. The newborn
infant is either present or unconscious, since the capacity for mental reflection has not yet
emerged. Mental reflection is conscious mental activity that does not elicit a sense of
presence; it also underlies the development of the self. The self develops largely through
social interaction, and as this increases through development, so does the capacity for varying
degrees of presence. As the child becomes increasingly mobile and also capable of reflective
thought, a calibrated sense of presence supports action on and in the external world (Riva et
al. 2011), in parallel with increased capacity for internal thought, imagination, and planning.
The developed adult draws continuously on a sophisticated sense of presence to
support successful actions in the world, actions that embody both long- and short-term
intentional strategies for carrying out activities. In old age, capacities for action decline as the
sense of presence diminishes. Attention is increasingly directed toward the self and away
from the external world. The common dementias of old age are accompanied by a partial
breakdown in the sense of presence. This is reflected in the familiar problems experienced by
the demented elderly in distinguishing internal worlds of reflection from the external world
around them and in completing planned activities. Without a conscious sense of presence, we
can no longer act successfully on our intentions.
The phenomenon of mediated presence, the compelling experience of being
perceptually located in a computer-generated three-dimensional world surrounding the self, is
already well recognized and documented, although no single theoretical perspective to
explain presence has yet been agreed upon (Lombard and Jones 2006). We are also all
familiar with the notion of social presence in virtuality, and millions have experienced the
sharing of a virtual space with other people, and of the potential to be represented in that
space in a wide variety of ways, such as a cartoon character in a social virtual world or a
persona illustrated on a social network page. The first is closely analogous to the feeling of
“natural” presence in the physical world around us, but the world in which we are immersed
is not the physical world in which our body is located. Our perspective is a first-person one;
the world surrounds the self who is experiencing it. The second demonstrates that our social
personas can be numerous and do not necessarily reflect our embodied physical reality. Our
avatars may represent beings quite unlike our physical bodies or our everyday social
characters. We experience them from a third-person perspective.
A first-person view on the world has been the norm throughout human development,
both in the physical world and in classic VR (virtual reality). Both are experiences in which
we view the virtual or physical world from within our own body, looking out and acting on
the world around it. In such VR, we move our physical head and the virtual view changes
accordingly; we move our physical arms and hands and we see a representation of these body
parts depicted as if they were collocated with our internal image of our physical body. Such
body-eye coordination is essential, but we are very adept at dealing with mappings of bodily
actions onto the behavior of tools, as long as the behavior of the tool is closely coordinated
with movements of the body. This is how we can use a computer mouse, drive a car, or fly a
remote control model airplane.
Increasingly often, we see ourselves represented in the third person in social virtual
spaces—look, that’s me over there. But this is generally not in a realistic way, and with
minimal coordination between body and virtual image—as when mouse actions or arrow
buttons control gross movements and preprogrammed gestures of our avatar. (There are
examples of first-person perspective social spaces [distributed VRs] too, but these tend to be
restricted to a limited, projected own-body [e.g., just an arm or a gun].) In these social spaces
we can usually choose the appearance of our virtual persona from a selection of avatars or
avatar parts. And these social spaces do give us a degree of copresence, even though we are
looking at ourselves from the outside, as a third-person self among the third-person selves of
one or more other people (Turkle 2005).
Changing our experiences of our own body is a key aspect of the future development
of presence, not least the potential to experience events from a wide variety of observational
perspectives. Standard perceptual effects such as “the rubber hand illusion” (Botvinick and
Cohen 1998) have been successfully reproduced in virtual reality and, with reduced
vividness, mixed reality situations (IJsselsteijn et al. 2006: Holmes N., & C. Spence. 2007).
The body image can be remarkably flexible, and may be “stretched” well beyond the confines
of the biological body. It has been known for some time that it is possible for virtual reality to
achieve a kind of “sensory rearrangement” resulting in modified experiences of one’s own
body (Biocca and Rolland 1998; Castiello et al. 2004; Normand et al. 2011; Riva 1998; Riva
et al. 2011.
Distributed Embodiment: This Mind in That Body
Technologically induced “out-of-body experiences,” using relatively simple technology, have
been reported for several years (e.g., Ehrsson 2007; Lenggenhager et al. 2007). In these
cases, it is as if we observe media representations of ourselves from the outside while
simultaneously experiencing individual presence as the observed person. This is a truly novel
mode of consciousness for people in normal mental states, and opens up a wealth of new
possibilities for entertainment experiences, in areas such as game playing, sports broadcasts,
and many other types of TV show.
Manipulation of the visual perspective, in combination with the receipt of
correlated multisensory information from the body was sufficient to trigger the
illusion that another person’s body or an artificial body was one’s own. This
effect was so strong that people could experience being in another person’s
body when facing their own body and shaking hands with it. Our results are of
fundamental importance because they identify the perceptual processes that
produce the feeling of ownership of one’s body. (Petkova and Ehrsson 2008)
We can already, at least to some extent, produce the feeling of being in a virtual body
that is also experienced as remotely located, separated from our own body. This is the feeling
that “that is me over there, and I am present in that body.” That body might look like this
body, or not. If not, then it is as if I have different selves. If the other self looks like someone
else, I might think that I have the experience of having that person’s body—and I might have
that experience, at least to some extent, as revealed in physiologic responses indicating
appropriate emotional change (see New Scientist 2010; Slater et al. 2010).
New, cheap gaming technologies (at the time of writing, most obviously, Microsoft’s
Kinect system) open these possibilities and more for everyday use. The most significant
aspect of these new systems, apart from their affordability and “hackability,” is that they can
readily and quite accurately locate and track several individual bodies, voices, and faces in
three-dimensional physical space. This means that, for example, an avatar or a robot can
relatively easily be programmed to mirror the movements and facial expressions of a person,
either locally or at a distance. This leads to developments such as the following (among many
others):
Teleconferencing applications, where the participants are represented as
avatars that move their bodies and faces in exact accord with those of the
distributed participants
Wall-sized responsive displays, controlled by the body and in almost any
location
Interactive “fitting rooms” for trying on new clothes before buying
Remote control of (as) robots
Interactive “workbenches” and other surfaces for close, highly dexterous
interaction (architecture, inspecting medical imagery)
Physiotherapeutic application and sports training
The other is not confined to human bodies. With the right visual and tactile
stimulation, one could in principle feel present in an animal body, or even an inanimate
object. If I see myself as that creature or thing, and feel myself to be present in that body,
might I come to know what it feels like to be, for example, a bat (Nagel 1974) or a box? This
is more than virtualization, more than the representation of things and their behavior, and my
ability to identify with them. It is the feeling that I am present as them—not metaphorically,
but literally.
Presence transference to another body—distributed embodiment—need not involve
much simulation. All of this can be accomplished through virtual reality, where every pixel
and all behavioral and physical responses must be specified precisely to cover all possible
events. But it can also, and more easily and flexibly, be done in the augmented reality we
increasingly inhabit. This includes sensors in objects, people, and even animals, and
distributed large and small displays that respond to sensed events nearby or at a distance.
This reality mixes the real and the virtual in a blended world of almost unimaginable
possibilities.
Direct Brain-Computer Interaction: Can We Sidestep Perception?
At its core, presence—in physical or virtual environments—is perceptual. The perception of
an environment is not the same thing as imagining the environment, and the point of a sense
of presence is to allow us to distinguish between what we imagine and where we really are
right now (Waterworth et al. 2010). It is sometimes said that VR works because the world is
virtual. In some sense, this is true, in that we do not perceive everything about what is out
there in the world, and often we misperceive. As Gregory suggested, our perception presents
us with hypotheses of what exists (Gregory 1997). But it does not follow from this that the
imagined is the same as the virtual. Presence in virtuality is significant not because reality is
virtual, but because—for the organism if not the intellect—VR is real, in a way that mental
imagery is not.
But there is a way to go other than simulating the physical world and presenting the
results for perception via the senses. Why not use technology to stimulate the brain directly?
Placing a few electrodes in the correct places allows us to track the changing electrical
activity corresponding to perception. And by applying electrical fields in the right places, we
should be able to produce electrical activity corresponding to whatever perceptions we would
like to produce in a head. The expanding field of brain-computer interaction (BCI) focuses on
the potential of the first approach—detecting electrical brain activity and using it to interact
with a device or virtual environment, whether this is a physical wheelchair or a computer-
based game or other virtual world. The approach has been successful (e.g., Leeb et al. 2007),
but the experience is not that of presence, but of absence, with a significant effort of action in
the world. Through biofeedback, we can learn when we are producing certain electrical
patterns and how to control them, to some extent, especially if highly motivated by disability
(Hochberg et al. 2006). But this is a far cry from experiencing a convincing illusion of acting
directly in a virtual world.
On the other side, direct brain stimulation—transcranial magnetic stimulation
(TMS)—has been applied in psychotherapy, for example in the treatment of neurological and
psychiatric disorders, such as depression and auditory hallucinations, although there is little
reliable evidence of its effectiveness (Slotema et al. 2010). It is perhaps best known for
producing extraordinary experiences, some producing feelings of being in the presence of the
divine. The so-called God Helmet (see Persinger et al. 2010), invented by Stanley Koren,
applies weak, fluctuating magnetic fields to the right hemisphere parietal and temporal lobes.
Typically, this is said to produce a sense of the presence of another in the room, and quite
often this has been experienced as a divine other. With some individuals, however, there is no
effect, and other kinds of presence are common, sometimes these are experienced as malign
or threatening. But other groups (e.g., Larsson et al. 2005) have failed to replicate these
findings, and the evidence for specific effects remains sparse. The unanswered question, then,
is can specific information be realized by TMS? This would be a direct computer-mind
interface and would be experienced as other, although the perceptual channels are not
involved.
Conclusions: The Future of Distributed Embodiment
When we experience strong mediated presence, our experience is that the technology has
become part of the self, and the mediated reality to which we are attending has become an
integrated part of the other. When this happens, there is no conscious effort of access to
presented information (the simulated reality), nor effort of action to carry out overt responses.
We can perceive and act directly, as if unmediated. The extent to which we experience
presence through a medium thus provides a measure of the extent to which that technology
has become part of the self. This is not simply a matter of “sensory replacement” as
addressed in classical virtual reality research—the personal significance of the mediated
situation and other factors relating to content are known to cut across technological
sophistication in inducing presence. It is a matter of how and where we experience the
boundary between self and other. We predict that presence will increasingly be experienced
and studied in blended realities of the physical and virtual. Our changing experiences of
presence reflect the changing virtual/physical world in which we live.
The virtual world can replace the sense of presence in the physical world, or enhance
it. The former case comprises the definition of virtual reality (VR); the latter is the aim of
augmented or mixed reality (AR). Distributed embodiment, real presence in another virtual
body, is something else. It depends on and confirms the surprising ease with which our body
image can be manipulated and transferred in relation to our sense of self. We know that it is
possible to transfer ourselves to other bodies (or body parts, or objects), to have a real sense
of presence in them, or as them. This can be done with rubber arms, with manikins or other
people, and also in VR. That it works in VR is not surprising, since in VR we always have
the illusion of being embodied in a virtual body—though conventionally this is collocated
with our own physical body.
Future research on distributed embodiment should systematically implement and
experiment with different viewpoints of media experience, involving singular and multiple
first-, second-, and third-person virtual representations of self and others. To date, the
evidence is both limited and weak. Factors such as the degree of coordination between body
and virtual image, sensory-motor coupling, and visual similarity (among others) will need to
be varied and the impact on the sense of presence assessed (by means of triangulations of
introspective, behavioral, and neuropsychological data). A search for quantum shifts in the
nature and quality of presence in response to specific manipulations of perceived
embodiment should be the focus. This work has already started (e.g., Ehrsson 2007;
Lenggenhager et al. 2007; Slater et al. 2008; 2010), but many questions remain open at this
time. We still don’t know with any precision: To what extent does collocation of body and
virtual image affect our sense of presence? What are the limits of out-of-the-body
experiences? Can direct brain stimulation match or even exceed the potential of virtual body
image manipulation for vivid re-embodied experiences?
The results of this ongoing research will open up new possibilities. New knowledge
will lead to techniques that will result in products that manipulate our sense of self—of when,
where, who, and even what we feel ourselves to be. The future applications of real presence
in virtual bodies will be numerous, including psychotherapy to restore body image distortions
(such as anorexia), entertainment (imagine watching a movie and feeling really present in the
body of a main character), mediation (experiencing situations as if one were the other party),
and mental and physical rehabilitation, among many others. An understanding of how and to
what extent we can experience real presence in other virtual bodies will be enormously
important in these and other fields.
But we should remember that many people already demonstrate that they want to be
in both the physical world and in virtuality at the same time. Distributed embodiment in the
real world may be what will have mass appeal. The major challenge for the future will be
effectively and cheaply to shift the sense of presence from one’s own body to another,
without replacing or excluding the physical world in which we all exist.
References
Biocca, F. A., & Rolland, J. P. 1998. Virtual eyes can rearrange your body: Adaptation to
visual displacement in see-through, head-mounted displays. Presence, 7, pp. 262–
277.
Botvinick, M., & Cohen, J. 1998. Rubber hands 'feel' touch that the eye sees. Nature, 391,
756.
Bracken, C. C., and P. Skalski. 2010. Immersed in Media: Telepresence in Everyday Life.
New York: Routledge.
Castiello, U. , Lusher, D., Burton, C., Glover, S. & Disler, P. 2004. Improving left
hemispatial neglect using virtual reality. Neurology, 62, pp. 1958-1962.
Clark, A. 2003. Natural Born Cyborgs: Minds, Technologies, and the Future of Human
Intelligence. Oxford: Oxford University Press.
Dennett, D. 1991. Consciousness Explained. London: Allen Lane.
Dourish, P. 2001. Where the Action Is: The Foundations of Embodied Interaction.
Cambridge, MA: MIT Press.
Ehrsson, H. H. 2007. The Experimental Induction of Out-of-Body Experiences. Science 317
(5841): 1048.
Gregory, R. L. 1997. Eye and Brain. 5th edition. Oxford: Oxford University Press.
Hochberg, L. R., M. D. Serruya, G. M. Friehs, J. A. Mukand, M. Saleh, A. H. Caplan, A.
Branner, D. Chen, D. Penn, and J. P. Donoghue. 2006. Neuronal Ensemble Control of
Prosthetic Devices by a Human with Tetraplegia. Nature 442 (7099): 164–171.
Holmes N., and C. Spence. 2007. Dissociating Body Image and Body Schema with Rubber
Hands. Behavioral and Brain Sciences 30: 211–212.
Hutchins, E. 1996. Cognition in the Wild. Cambridge, MA: Bradford Books, MIT Press.
IJsselsteijn, W.A., de Kort, Y.A.W., & Haans, A. 2006. Is this my hand I see before me? The
Rubber Hand Illusion in Reality, Virtual Reality, and Mixed Reality. Presence:
Teleoperators and Virtual Environments 15, 455-464.
Kaptelinin, V., and B. Nardi. 2006. Acting with Technology: Activity Theory and Interaction
Design. Cambridge, MA: MIT Press.
Larsson, M., D. Larhammarb, M. Fredrikson, and P. Granqvist. 2005. Reply to M. A.
Persinger and S. A. Koren’s Response to Granqvist et al. Sensed Presence and
Mystical Experiences Are Predicted by Suggestibility, Not by the Application Of
Transcranial Weak Magnetic Fields. Neuroscience Letters 380 (3): 348–350.
Leeb, R., D. Friedman, G. R.ller-Putz, R. Scherer, M. Slater, and G. Pfurtscheller. 2007.
Self-Paced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: A
Case Study with a Tetraplegic. Computational Intelligence and Neuroscience 20
(April 2007): Article 7.
Lenggenhager, B., T. Tadi, T. Metzinger, and O. Blanke. 2007. Video Ergo Sum:
Manipulating Bodily Self-Consciousness. Science 317 (5841): 1096–1099.
Lombard, M., and T. Ditton. 1997. At the Heart of It All: The Concept of Presence. Journal
of Computer Mediated-Communication 3 (2).
http://jcmc.indiana.edu/vol3/issue2/lombard.html. Accessed July 29, 2013..
Lombard, M., and M. T. Jones. 2006. Defining Presence. Paper presented to Presence 2006:
The 9th International Workshop on Presence, Cleveland, OH.
Merleau-Ponty, M. 1962. Phenomenology of Perception. London: Routledge & Kegan Paul.
Misselhorn, C. 2009. Empathy with Inanimate Objects and the Uncanny Valley. Minds and
Machines 19 (3): 345–359.
Nagel, T. 1974. What Is It Like to Be a Bat? Philosophical Review 83 (4): 435–450.
New Scientist. 2010. The Real Avatar: Body Transfer Turns Men into Girls. May 13.
http://www.newscientist.com/article/dn18896-the-real-avatar-body-transfer-turns-
men-into-girls.html. Accessed June 27, 2013.
Normand, J.-M., E. Giannopoulos, B. Spanlang, and M. Slater. 2011. Multisensory
Stimulation Can Induce an Illusion of Larger Belly Size in Immersive Virtual Reality.
PLoS ONE 6 (1): e16128.
Perry, M. 2003. Distributed Cognition. In HCI Models, Theories, and Frameworks: Toward
an Interdisciplinary Science, edited by J. M. Carroll, 193–223. San Francisco, CA:
Morgan Kaufmann.
Persinger, M. A., K. Saroka, S. A. Koren, and L. S. St-Pierre. 2010. The Electromagnetic
Induction of Mystical and Altered States within the Laboratory. Journal of
Consciousness Exploration & Research 1 (7): 808–830.
Petkova VI, Ehrsson HH 2008. If I Were You: Perceptual Illusion of Body Swapping. PLoS
ONE 3(12): e3832. doi:10.1371/journal.pone.0003832
Riva G. 1998. Modifications of body-image induced by virtual reality. Perceptual & Motor
Skills, 86(1), pp. 163-70.
Riva, G., J. A. Waterworth, E. L. Waterworth, and F. Mantovani. 2011. From Intention to
Action: The Role of Presence. New Ideas in Psychology 29 (1): 24–37.
Russell, J. A. 2003. Core Affect and the Psychological Construction of Emotion.
Psychological Review 110 (1): 145–172.
Slater M., D. Perez-Marcos, H. H. Ehrsson, and M. Sanchez-Vives. 2008. Towards a Digital
Body: The Virtual Arm Illusion. Frontiers in Human Neuroscience. 2, Article 6, Page
1-8, 20 August 2008 | doi: 10.3389/neuro.09.006.2008.
Slater, M., B. Spanlang, M. V. Sanchez-Vives, and O. Blanke. 2010. First Person Experience
of Body Transfer in Virtual Reality. PLoS ONE 5 (5): e10564.
Slotema, C. W., J. D. Blom, H. W. Hoek, and I. E. C. Sommer. 2010. Should We Expand the
Toolbox of Psychiatric Treatment Methods to Include Repetitive Transcranial
Magnetic Stimulation (rTMS)? Journal of Clinical Psychiatry 71 (7): 873–884.
Turkle, S. 2005. The Second Self: Computers and the Human Spirit. Twentieth Anniversary
Edition. Cambridge, MA: MIT Press.
Varela, F. J., T. Thompson, and E. Rosch. 1999. The Embodied Mind: Cognitive Science and
Human Experience. Cambridge, MA: MIT Press.
Waterworth, E. L. 2001. Perceptually-Seductive Technology: Designing Computer Support
for Everyday Creativity. Umeå: Umeå University, Department of Informatics.
http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-65867. Accessed June 27, 2013.
Waterworth, E. L., and J. A. Waterworth. 2001. Focus, Locus and Sensus: The Three
Dimensions of Virtual Experience. Cyberpsychology and Behavior 4 (2): 203–214.
Waterworth, J. A., E. L. Waterworth, F. Mantovani, and G. Riva. 2010. On Feeling (the)
Present: An Evolutionary Account of the Sense of Presence in Physical and
Electronically-Mediated Environments. Journal of Consciousness Studies 17 (1–2):
167–189.
... La définition de référence de l'incarnation dont l'objet porte sur les mondes virtuels immersifs et que nous retiendrons est celle de Kilteni et al. (2012) L'embodiment, d'un point de vue technologique, est possible grâce aux technologies de virtualisation et de réalité mixte dont la RV fait partie. Selon Waterworth et Waterworth (2014), la RV crée un effet de présence identique à la présence naturelle. Pour ces auteurs, c'est l'effet de présence qui permet à l'individu de considérer la technologie comme partie intégrante de sa personne en « passant d'un sentiment de présence dans le monde physique, à travers un sentiment médiatisé de présence dans la virtualité, au sentiment médiatisé d'être dans le monde physique-virtuel dans un autre corps que le sien » (ibid., p. 589). ...
Thesis
Full-text available
Résumé : La réalité virtuelle (RV) réunit les plus grandes marques et les consommateurs dans le métavers pour vivre des expériences de consommations ludiques sur le « web 4.0 », révolutionnant l’expérimentation et la découverte d’offres. En interagissant physiquement, le consommateur devient l’acteur principal, au plus près de l’expérience réelle. L’objectif de la thèse est de comprendre dans quelle mesure les composantes expérientielles influencent ce sentiment d’incarnation et quels sont les effets de ce dernier sur l’attitude et le comportement du consommateur. La démarche de recherche hypothético-déductive nous amène à construire un cadre théorique en marketing expérientiel puis d’explorer le phénomène par des études qualitatives à visées exploratoires afin d’élaborer un modèle conceptuel. Le modèle intègre la position du corps (assis vs. debout) et les défis (absence vs. modérés vs. avancés) tels les déterminants de l’incarnation ainsi que l’immersion et le plaisir tels les facteurs explicatifs de l’incarnation et des intentions comportementales. L’étude quantitative (n=328) est analysée par un modèle d’équations structurelles PLS. Six unités expérimentales composent le contexte expérientiel relatif la visite d’une destination touristique virtualisée. Les résultats confirment le rôle des effets prédictifs ainsi que ceux joués par des médiateurs et modérateurs entre certaines relations du modèle conceptuel. La recherche présente les apports méthodologiques et théoriques et formule des préconisations à destination des professionnels et de chercheurs dans le cadre de futures recherches.
... Our body is a single instance in our physical space, but it is possible to extend it with virtual reality or robotics technology (Waterworth and Waterworth, 2014). To have multiple bodies with a single user can be considered as an implementation of human augmentation. ...
Article
Full-text available
Human behavior and perception are optimized for a single body. Yet, the human brain has plasticity, which allows us to extend our body schema. By utilizing technology like robotics or virtual reality (VR), we can modify our body parts or even add a new body to our own while retaining control over these parts. However, the update of body cognition when controlling multiple bodies has not been well examined. In this study, we explore the task performance and body cognition of humans when they have multiple full bodies as an extended embodiment. Our experimental system allows a participant to control up to four bodies at the same time and perceive sensory information from them. The participant experiences synchronizing behavior and vision perception in a virtual environment. We set up three tasks for multiple bodies and evaluated the cognition of these bodies with their gazing information, task performances, and subjective ratings. We found that humans can have the sense of body ownership and agency for each body when controlling multiple bodies simultaneously. Furthermore, it was observed that people manipulate multiple bodies by actively switching their attention in a static environment and passively switching their attention in a dynamic environment. Distributed embodiment has the potential to extend human behavior in cooperative work, parallel work, group behavior, and so on.
Chapter
The music and sounds of video games have become an inescapable part of our world. Not only do these sonic elements profoundly shape the experiences of billions of players every day, but also the soundscapes of games have stretched out from our living rooms to encompass spaces as diverse as pinball arcades, concert halls, museums, and classrooms across the globe. Research on game music and sound is equally diverse—a vibrant, innovative, and multifaceted field that incorporates approaches from media studies, musicology, sound studies, music theory, psychology, and more. Drawing on the expertise of leading scholars and practitioners from around the globe, The Oxford Handbook of Video Game Music and Sound features nearly fifty chapters on topics ranging from the earliest pinball machines to the latest in virtual reality technology. The resulting volume provides both a comprehensive introduction to the study of game audio and an indispensable resource for experts.
Article
The persisting usage of virtual means for the completion of activities usually or traditionally held in person stimulates the reflection about the possible effect that doing philosophy online could have on the philosophical integrity of the process. The body question seems to be pivotal in this context not only as far as concerning virtuality issues but also philosophy’s care to integrate the body into its routines – when it is practiced physically – especially in the frame of an education still hesitating to embrace embodied experience. The body deficiency, its vanishing, is first of all a question for philosophy and its ways to be practiced today – on that basis, passing through virtuality’s discourses can enlighten the relevant reflection.
Article
Full-text available
In the present text, I will reflect upon the composition of works that use virtual 3D environments in which the performers are represented as interacting avatars. In particular, I will explain the musical possibilities offered by this work with virtual environments, detailing the effects that the design of virtual topologies has on the interactive sound sources and sound events contained in these topologies. I will focus particularly upon how performers can musically influence said effects by navigating with the avatars acting within the topology.
Chapter
The article presents a brief introduction to the concept of sonic virtuality, a view of sound as a multi-modal, emergent perception that provides a framework that has since been used to provide an explanation of the formation of environments. Additionally, the article uses such concepts to explain the phenomenon of presence, not only in virtual worlds but also in actual worlds. The view put forward is that environment is an emergent perception, formed from the hypothetical modelling of salient worlds of sensory things, and it is in the environment that we feel present. The article ends with some thoughts on the use of biofeedback in computer games as part of the immersive technology designed to facilitate presence in such worlds.
Preprint
Ce texte présente une exploration différentiée du rapport entre le terme anglais embodiment et le terme français incarnation qui est généralement tenu pour sa traduction. Cependant quand on y regarde de plus près, la relation entre ces deux termes est diversifiée et complexe. Parfois croisement, parfois translation cette ambiguïté n'est pas sans conséquences sémantiques, voire symboliques.
Conference Paper
We present a theoretical model of how presence is achieved in virtual worlds, from which future frameworks prescribing the sensory design of such worlds can be constructed. Drawing on the study of cognitive metaphors and the concept that sound is an emergent perception we offer an account of the environment as a salient and dynamic construct that functions as a metonym for the nonself. Separating environment from world, we discuss the role of sound in the forming of the environment and argue that it is this environment that establishes the means for presence because it is the process behind the construction of the environment that individuates self from nonself and such differentiation is a defining feature of the concept of presence.
Article
Full-text available
The human brain is the locus of all human experiences. The substantial microstructural and neuroelectrical differences between the two cerebral hemispheres predicts two major classes of mystical experiences which involve the sensed presence and the out-of-body experience. Their occurrence and their attributions to cosmic origins have been reported for centuries and have been the bases for social belief systems. Direct cerebral electrical stimulation during the 20th century evoked these experiences. In the 21st century the non-invasive, external transcerebral application of complex, physiologically-patterned weak magnetic fields has been shown to produce similar experiences that can be correlatively mapped by quantitative electroencephalographic inferences of interhemispheric coherence. The experimental production and control of these powerful experiences by more sophisticated technologies might be employed to understand the intricate nature and function of mystical/altered states within large populations of human beings.
Article
Full-text available
The sense of presence is the ability to experience the fact that one is physically present in the place in which the body is currently located. In other words, it refers to the part of the contents of consciousness that relate to the current time and place in which the body is located. One may have a relatively strong or weak feeling of presence at any given moment, since one may be more or less aware of one’s location within the current surroundings - this experiential aspect of one’s physical presence may be more or less psychologically salient. In what follows, we are not concerned with physical presence per se but only the extent to which we have the experience of being present in our surroundings. We present an account of the sense of presence understood as the result of an evolved neuropsychological process, created through the evolution of the central nervous system, and which solves a key problem for an organism’s survival: how to differentiate between the internal and the external. We suggest that in its simplest form this originated as the feeling of something happening to an organism from outside rather than within. In this sense, presence distinguishes self from other. In its highly evolved form in humans, the degree of presence we feel tells us the extent to which we are focusing conscious attention on the outside world as opposed to attending to the internal world of thoughts and imaginings. Our suggestion is that we need this calibration of the current focus of attention in order to act effectively and indeed to know that we are acting. We cannot do this through emotional appraisal of situations, nor through reality judgments, because imagined situations trigger the same emotional responses as physical situations (Russell, 2003), and may also be judged realistic or unrealistic (as may physical events). In short, the sense of presence - distinguishing the physically present from the imagined - is necessary to survive, and this is why is has evolved. Our view is that calibrating the focus of attention in this way is necessary, and that when we do so we experience presence to varying degrees . Since more and more of our experiences are now mediated by digital information and communication technology - from mobile phones to home entertainment centres to video conferences to game environments, it is reasonable to see the future of the human sense of presence as reflecting the rapid development of ever more pervasive digital technologies - technologies which will increasingly mediate our experiences of the world around us. We therefore also consider how the feeling of presence can be invoked in computer-mediated environments, to produce what is currently termed mediated presence, and how this relates to the sense of presence experienced in and of the physical world. The sense of presence has not been much discussed in the psychological literature relating to consciousness, although research on mediated presence has received considerable attention in relation to technological developments and applications in virtual reality (VR), and digital media generally.
Book
A new edition of a classic work that originated the “embodied cognition” movement and was one of the first to link science and Buddhist practices. This classic book, first published in 1991, was one of the first to propose the “embodied cognition” approach in cognitive science. It pioneered the connections between phenomenology and science and between Buddhist practices and science—claims that have since become highly influential. Through this cross-fertilization of disparate fields of study, The Embodied Mind introduced a new form of cognitive science called “enaction,” in which both the environment and first person experience are aspects of embodiment. However, enactive embodiment is not the grasping of an independent, outside world by a brain, a mind, or a self; rather it is the bringing forth of an interdependent world in and through embodied action. Although enacted cognition lacks an absolute foundation, the book shows how that does not lead to either experiential or philosophical nihilism. Above all, the book's arguments were powered by the conviction that the sciences of mind must encompass lived human experience and the possibilities for transformation inherent in human experience. This revised edition includes substantive introductions by Evan Thompson and Eleanor Rosch that clarify central arguments of the work and discuss and evaluate subsequent research that has expanded on the themes of the book, including the renewed theoretical and practical interest in Buddhism and mindfulness. A preface by Jon Kabat-Zinn, the originator of the mindfulness-based stress reduction program, contextualizes the book and describes its influence on his life and work.
Book
In The Second Self , Sherry Turkle looks at the computer not as a "tool," but as part of our social and psychological lives; she looks beyond how we use computer games and spreadsheets to explore how the computer affects our awareness of ourselves, of one another, and of our relationship with the world. "Technology," she writes, "catalyzes changes not only in what we do but in how we think." First published in 1984, The Second Self is still essential reading as a primer in the psychology of computation. This twentieth anniversary edition allows us to reconsider two decades of computer culture--to (re)experience what was and is most novel in our new media culture and to view our own contemporary relationship with technology with fresh eyes. Turkle frames this classic work with a new introduction, a new epilogue, and extensive notes added to the original text. Turkle talks to children, college students, engineers, AI scientists, hackers, and personal computer owners--people confronting machines that seem to think and at the same time suggest a new way for us to think--about human thought, emotion, memory, and understanding. Her interviews reveal that we experience computers as being on the border between inanimate and animate, as both an extension of the self and part of the external world. Their special place betwixt and between traditional categories is part of what makes them compelling and evocative. (In the introduction to this edition, Turkle quotes a PDA user as saying, "When my Palm crashed, it was like a death. I thought I had lost my mind.") Why we think of the workings of a machine in psychological terms--how this happens, and what it means for all of us--is the ever more timely subject of The Second Self .
Article
First published in 1945, Maurice Merleau-Ponty’s monumental Phénoménologie de la perception signalled the arrival of a major new philosophical and intellectual voice in post-war Europe. Breaking with the prevailing picture of existentialism and phenomenology at the time, it has become one of the landmark works of twentieth-century thought. This new translation, the first for over fifty years, makes this classic work of philosophy available to a new generation of readers.