ChapterPDF Available

Enter the Avatar: The Phenomenology of Prosthetic Telepresence in Computer Games

Authors:

Abstract

In this paper I will give a phenomenological account of embodied presence through computer game avatars, based on Maurice Merleau-Ponty’s analysis of bodily intentionality, bodily space and bodily extensions (2002 [1962]). The core idea is that when we play, directly controllable avatars like Mario or Lara Croft, as well as racing cars or other kinds of controllable vehicles, function as prosthetic extensions of our own body, which extend into screen space the dual nature of the body as both subject and object. Because they act as proxies or stand-ins for our own body within the gameworld, prosthetic avatars are crucially different from more familiar kinds of bodily extensions, like tools or musical instruments. In navigable 3D environments, the main “body” of the avatar, in the phenomenological sense, is not the controllable marionette itself (for example Mario or Lara), but the navigable virtual camera, which becomes an extension of the player’s locomotive vision during play. In this way, the navigable camera, extending from the player’s eyes and fingers, re-locates the player’s bodily self-awareness – the immediate sense of “here” as opposed to “there” – into screen space. This displacement of our visual perceptual apparatus through prosthetic avatars creates a distinctive kind of prosthetic telepresence, a phenomenon that nineteenth-century philosophy could not imagine or foresee. Prosthetic telepresence operates at the ground level of the phenomenology of the body, and does not rely on imagination or fictionality. Prosthetic telepresence offers – and indeed demands – full perceptual immersion, yet is not dependent on technologies of audiovisual immersion.
1
Enter the Avatar. The phenomenology of prosthetic telepresence
in computer games.
Rune Klevjer
Department of Information Science and Media Studies
University of Bergen
This is a self-archived postprint version of "Enter the Avatar. The phenomenology
of prosthetic telepresence in computer games." In The Philosophy of Computer
Games, edited by Hallvard Fossheim, Tarjei Mandt Larsen and John Richard
Sageng, 17-38. London & New York: Springer.
The final publication is available at SpringerLink:
http://link.springer.com/book/10.1007/978-94-007-4249-9/page/1
Abstract
In this paper I will give a phenomenological account of embodied presence
through computer game avatars, based on Maurice Merleau-Ponty’s analysis of
bodily intentionality, bodily space and bodily extensions (2002 [1962]. The core
idea is that when we play, directly controllable avatars like Mario or Lara Croft, as
well as racing cars or other kinds of controllable vehicles, function as prosthetic
extensions of our own body, which extend into screen space the dual nature of the
body as both subject and object. Because they act as proxies or stand-ins for our
own body within the gameworld, prosthetic avatars are crucially different from
more familiar kinds of bodily extensions, like tools or musical instruments.
In navigable 3D environments, the main “body” of the avatar, in the
phenomenological sense, is not the controllable marionette itself (for example
Mario or Lara), but the navigable virtual camera, which becomes an extension of
the player’s locomotive vision during play. In this way, the navigable camera,
extending from the player’s eyes and fingers, re-locates the player’s bodily
2
self-awareness – the immediate sense of “here” as opposed to “there” – into screen
space. This displacement of our visual perceptual apparatus through prosthetic
avatars creates a distinctive kind of prosthetic telepresence, a phenomenon that
ninenteenth-century philosophy could not imagine or foresee. Prosthetic
telepresence operates at the ground level of the phenomenology of the body, and
does not rely on imagination or fictionality. Prosthetic telepresence offers and
indeed demands – full perceptual immersion, yet is not dependent on technologies
of audiovisual immersion.
**
To what does it refer when we talk about “being” in a game, or when we say that
we are “in the shoes of” Lara Croft, Mario or Master Chief? How is it possible
that we can, in certain types of games, act and react in an intuitive fashion, as if
actually being inside the gameworld, when we are in fact in front of the screen,
moving buttons and sticks on a game controller?
In internet- and computer game discourses, the notion of “avatar” has two
common uses. It is usually taken to simply mean playable character, in all its
variants, from Pac-Man to Guybrush Threepwood. In online environments like
World of Warcraft (2004), the term tends to highlight, more specifically, the
player’s virtual persona in the game world. This latter meaning of the term has
migrated into online virtual spaces of all kinds, where users' accounts and profiles
are typically linked to personas or “avatars”.
A central premise for this paper, however, is that we must make a distinction
between ”avatar” understood as a playable character (or persona), and ”avatar”
understood as a vehicle through which the player is given some kind of embodied
agency and presence within the gameworld. In the action-adventure genre of
computer games, from Super Mario Bros. (1985) to Call of Duty: Modern
Warfare 2 (2009), these two different aspects or functions are combined into one
avatar. Nevertheless, character-play must clearly be seen as independent from
embodied presence, and vice versa. Playable characters can be interacted with via
email, for example, or in numerous other ways that would not imply any kind of
embodied presence within a computer-simulated environment. Conversely, the
vehicle of agency and presence in a gameworld does not at all need to be also a
character; the paradigmatic category here would be racing games or flight
simulators, but there are also games like Marble Madness (1984), in which our
avatar is a rolling marble. Indeed we could say that, when playing those kinds of
games, we get to be a racing car, a rolling marble, or a spaceship, just like we get
to be (some equivalent of) a human body in an action-adventure game.
3
Agency: the cursor analogy
Many have noted that the character dimension of avatars like Mario or Lara Croft
appears to be relatively insignificant in comparison to their function as a mediators
of players' agency in the gameworld. In Nintendo and New World Travel Writing:
A Dialogue (1995), Mary Fuller and Henry Jenkins point out that action-adventure
avatars should not be mistaken for characters or protagonists in a narrated story:
In Nintendo®'s narratives, characters play a minimal role, displaying traits that are largely
capacities for action: fighting skills, modes of transportation, preestablished goals (…).
The character is little more than a cursor that mediates the player's relationship to the
story world (Fuller & Jenkins 1995).
“Little more than a cursor” seems to imply that the avatar is no more than a tool,
a capacity for action, an instrument. The cursor analogy has also been used also by
Marie-Laure Ryan, who in her influential Narrative as Virtual Reality (2001)
suggests that the cursor is the “the minimal form” of third-person avatars like
Mario (2001:309).
In “The Myth of the Ergodic Videogame. Some thoughts on player-character
relationships in videogames” (2002), James Newman broadly shares Fuller &
Jenkins’ approach, and argues that the primary function of avatars is to mediate
the player’s agency rather than to be characters with whom the player is supposed
to identify in a similar fashion as in novels or films. When playable characters
are ”On-Line”, Newman argues – that is, when being played, as opposed to merely
appearing in cutscenes – they are not ”characters” in the traditional sense at all:
Thus, On-Line "character" in the sense we understand it in non-ergodic media, dissolves.
Characters On-Line are embodied as sets of available capabilities and capacities. They are
equipment to be utilised in the gameworld by the player. They are vehicles. This is easier
to come to terms with when we think of a racing game like Gran Turismo where we drive
a literal vehicle, but I am suggesting that, despite their representational traits, we can think
of all videogame characters in this manner. On-Line, Lara Croft is defined less by
appearance than by the fact that "she" allows the player to jump distance x, while the
ravine in front of us is larger than that, so we better start thinking of a new way
round…(Newman 2002:9).
According to this analysis, we could say that playable characters are being
driven or piloted by the player. We should note here, however, that Newman’s
account goes beyond the minimal cursor analogy of Fuller & Jenkins, at least by
implication. If we recognise that Lara Croft is indeed an ”embodiment” of the
player, this would imply not only that she mediates the player’s ability to jump or
walk, but also that she embodies the player’s risk of falling down the ravine. This
latter aspect is arguably central to what the game is about. A mouse cursor does
not make the player belong to or be in the game environment in the same way.
Still, Newman's choice of emphasis, when he says that, for example, avatars are
«equipment to be utilised», indicates that the notion of embodied agency is
thought of in relatively narrow terms: avatars are mainly seen as tools or
instruments, resources for the player. An influential strand of computer game
4
research (Salen & Zimmerman 2004, Linderoth 2005, Carr 2002, Dovey &
Kennedy 2006) has adopted a similar approach, however with an important
addition or modification, drawing on insights from cultural and text-oriented
theory: even if we do recognise the argument goes that avatars are primarily
tools or equipment, this does not mean that character is unimportant. These
theorists argue that the avatar is important as both tool and character, and point
out that the relative balance or configuration between the two aspects will vary
greatly between games, players and playing situations. Sometimes we care about
character, sometimes we are interested only in the tool.
Prosthetic agency and the camera-body
This dual-function approach to the analysis of player-avatar relationships,
because it sticks to an instrumental notion of agency, tends not to focus so much
on the question of being in a gameworld, via the avatar. Before turning to
Merleau-Ponty, let me pick up two important cues from recent computer game
theory that address this question, and which have informed my own approach.
First, Ulf Wilhelmsson’s notion of the Game Ego:
As a player you incorporate an agent, a Game Ego function, within the
game environment. This exertion of control is an extension of the player’s
own sensory motor system via a tactile motor/kinaesthetic link, why it is not only the
controlled and perceived motion on a screen but also the
experience of locomotion within an environment that is the result of this control.
(...)The Game Ego is that function; the agency within the game that manifest the player’s
presences allowing him or her to perform actions (Wilhelmsson 2006:67).
What we may call prosthetic agency, which functions as an extension or a
prosthesis of the player’s body, a ”tactile motor/kinaesthetic link”, is a defining
characteristic of action- sport- and action-adventure avatars since Spacewar!
(2006 [1962]). Through the magic of real-time control, it is as if the player is
reaching directly into the gameworld through a prosthesis, an extended limb. In
Wilhelmsson’s account, among the various manifestations of the Game Ego would
be, for example, the controllable blocks in Tetris: through practice, the control
exerted on the blocks by the player may become second nature, similar to the way
in which we are able to control our own hand, directly, without planning or
thinking.
Whether a block in Tetris should also qualifiy as an avatar is a different
question, however, as I will explain below. Still, the notion of the Game Ego
overlaps with the concept of the prosthetic avatar: both mediate agency from
within the game environment, and both are hooked up to our hands and eyes in
such a way as to become extensions of our body. Finally, the Game Ego,
according to Wilhelmsson, is manifested not only through visible elements like
blocks, vehicles or characters, but also through the player’s experience of
5
locomotion, of putting oneself into motion via the prosthetic link.
A similar observation is made by Bob Rehak, who in his psychoanalytically
informed analysis of the player-avatar relationship also emphasises the avatarial
role and status of the «camera-body» as much as the visible avatar:
Avatarial operations flow from two elements that interdepend in various ways. First is the
foregrounding of an onscreen body, visible in whole or in part. Second is the conceit of an
offscreen but assumed body constituted through the gaze of a mobile, player-controlled
camera. Different articulations between camera-body and avatar-body lead to different,
though related, modes of play and subject effects. In every case, the intent – to produce a
sense of diegetic embodiment – announces itself from the dawn of video game history
(Rehak 2003:109).
Even if the notion of’ ”diegetic” embodiment may be misleading in a certain
sense, as I will return to below, Rehak’s central observation is important: the
action- and action-adventure strand of computer game history has been pushing
towards an ever more immersive and visceral sense of being in the gameworld.
From Spacewar! and onwards, prosthetic avatars do indeed offer, as Rehak says,
experiential simulations of being a body in a world. Through prosthetic avatars,
we get to play with, and play through, extensions of our own being.
The paradox of the prosthetic avatar
It is important to emphasise, however, at this point, that avatarial extensions are
not like other bodily extensions. A prosthetic avatar is more than a mere extension
of the player’s ego function, more than the extension of the player as acting and
perceiving subject. At the heart of the player-avatar relationship lies a tension and
a paradox, reflected in our intuitive understanding of what it means to be
immersed in a navigable 3D environment through an avatar. How can we say that
the player is extending or reaching into the gameworld, while at the same time
also saying that the player is «being within» and «acting from within» the
gameworld? How can avatarial embodiment be both a kind of extension and a
kind of re-location at the same time? The idea of the bodily prosthesis seems to
contradict the idea of embodied being or presence, especially as it relates to the
navigable “camera-body” that is the primary vehicle of perceptual immersion in
contemporary games.
Hopefully, a phenomenological analysis of this paradox can contribute to our
understanding also from the point of view of game design and analysis. What is
the core difference between a first-person 3D computer game and a Virtual Reality
installation – actual or imagined one – in terms of their immersive characteristics?
And, looking in the opposite direction: what is the difference between navigable
3D environments and 2-dimensional game spaces? Is the prosthetic nature of 2D-
and 3D-avatars basically of the same kind, or is there a radical leap between the
two?
6
Finally, there is the question of fictionality, which is part of my motivation for
investigating the tension between the «here» and «there» of avatar-based play.
Previously, as part of a broader genre study of space and interaction in
avatar-based games (Klevjer 2007), I have suggested a concept of ”vicarious”
embodiment that combines a phenomenological notion of the bodily prosthesis
with theories of fiction and simulation.
The key idea here is, in very simple terms, that the avatar is different from a
cursor because it belongs to the simulated world of the game. According to this
approach, the avatar’s status as a simulated and fictional body becomes essential
to its definition. However, on closer scrutiny, could it really be said that avatarial
embodiment is, at its heart, simulated embodiment? It is an attractive proposition,
because it would seem to solve the conflict between extension and re-location. It
would allow us to say that, whereas the concept of prosthesis addresses the nature
of our actual embodiment here, the notion of simulated or fictional embodiment
would adequately capture our re-located presence therethe latter given to us via
the «conceit», in Rehak's terms, of the avatarial apparatus.
However, while simulated bodies and simulated worlds are certainly crucial in
the concrete articulations of the player-avatar relationship, as I will return to
below, I would argue that, contrary to the claims I made earlier, theories of
simulation and fiction are not necessary to explain the defining mechanisms of
avatarial embodiment. Indeed, the notion of the avatar as a simulated body,
however correct in any particular instance, can neverthless be a misleading one,
obscuring from view important phenomenological parameters of embodied
engagement.
So let us instead take a closer look at the notion of the bodily extension, as laid
out in Phenomenology of Perception. The central idea I will suggest from this
work is that avatarial extensions mediate particular kinds of relationships between
the body as subject and the body as object, and between “bodily space” and
“external space”. This duality in the nature of the body is rooted in the general
phenomenological idea of intentionality, as developed by Edmund Husserl and
Martin Heidegger.
The “I can”
"Intentionality" means that perception always, by its very definition, is directed
or intended towards a meaningful world. Perception implies the perception of
something, and this something will always be, in some sense, anticipated, already
given as significant for us, something that is purposefully reached for by our
senses and our actions. In Phenomenology of Perception, Merleau-Ponty is
concerned with the embodied nature of this intentionality. Heidegger’s Dasein,
according to Merleau-Ponty, must be understood as an intentional body, an
embodied being-in-the-world. The subject is not a mind that has a body, but a
7
mind that is a body; I am constituted as subject by virtue of being a
body-in-the-world. The subject is not, as Descartes argued, a cogito or “I think”,
but rather an “I can”, an intentional body-subject. The way in which we perceive
the world and our position in it is grounded in this I can.
Avatar-based computer games are unique because they play directly to the
constitution of our experienced body. The defining appeal of games like Super
Mario 64 (1996) or Grand Theft Auto III (2001) is that we get to be a different I
can, stepping into the shoes (or wheels) of another body, in another world. Let us
first look more closely at Merleau-Ponty’s account of body intentionality and the
bodily extension, before turning to the implications for avatarial embodiment.
Body intentionality and body image
The body, Merleau-Ponty says, ”is our general medium for having a world”
(2002 [1962]:169). This is a radical formulation to emphasise the status of the
body as subject, that is: as that for which there is a world. At the same time, of
course, our bodies are also objects in the world, along with other objects; we can
look at and measure our hand, just like we can look at and measure other objects
in the world. Merleau-Ponty describes this duality in a passage that is best quoted
at length:
My visual body is certainly an object as far as its parts far removed from head are
concerned, but as we come nearer to the eyes, it becomes divorced from objects, and
reserves among them a quasi-space to which they have no access, and when I try to fill
this void by recourse to the image in the mirror, it refers me back to an original of the
body which is not out there among things, but in my own province, on this side of all
things seen. It is no different, in spite of what may appear to be the case, with my tactile
body, for if I can, with my left hand, feel my right hand as it touches an object, the right
hand as an object is not the right hand as it touches: the first is a system of bones, muscles
and flesh brought down at a point of space, the second shoots through space like a rocket
to reveal the external object in its place. In so far as it sees or touches the world, my body
can therefore be neither seen nor touched. What prevents it ever being an object, ever
being 'completely constituted', is that it is that by which there are objects. It is neither
tangible nor visible in so far as it is that which sees and touches. The body therefore is not
one more among external objects (2002 [1962]:105).
The invisibility of the body, as “that which sees and touches”, also includes
movement and the body's ability to move other objects. The intentionality of the
body, Merleau-Ponty explains, is a “motor intentionality”, a “motor project” (2002
[1962]:127). Like the movement of the eyeballs, the movement of the hand is
equally “on this side of all things seen”:
I move external objects with the aid of my body, which takes hold of them in one place
and shifts them to another. But my body itself I move directly, I do not find it at one point
of objective space and transfer it to another, I have no need to look for it, it is already with
me-I do not need to lead it towards the movement's completion, it is in contact with it
from the start and propels itself towards that end. The relationships between my
8
decision and my body are, in movement, magic ones. (2002 [1962]:108).
Merleau-Ponty then turns to the intentionality of spatial perception. Our
awareness of our own body in space, he argues, our body image, is as an
intentional stance or posture towards the world. My body-image, he explains, is a
“total awareness of my posture in the intersensory world» (2002 [1962]:114). The
body-image is a form, or gestalt, through which external space appears as
meaningful, in a figure-ground structure. So when I am engaged in an activity, for
example playing a video game, the different parts of my body – eyes, feet, thumbs
will all be, in Merleau-Ponty's words, part of this «total awareness (...) only in
proportion to their value to the organism's project».
Psychologists often say that the body image is dynamic. Brought down to a precise sense,
this term means that my body appears to me as an attitude directed towards a certain
existing or possible task. And indeed its spatiality is not, like that of external objects or
like that of ‘spatial sensations’, a spatiality of position, but a spatiality of situation. (...)
The word ‘here’ applied to my body does not refer to a determinate position in relation to
other positions or to external coordinates, but the laying down of the first co-ordinates, the
anchoring of the active body in an object, the situation of the body in front of its task.
Bodily space can be distinguished from external space and envelop its parts instead of
spreading them out, because it is the darkness needed in the theatre to show up the
performance, the background of somnolence or reserve of vague power against which the
gesture and its aim stand out, the zone of not being in front of which precise things,
figures and points can come to light. In the last analysis, if my body can be a ’form’ and if
there can be, in front of it, important figures against indifferent backgrounds, this occurs
in virtue of its being polarized by its tasks, of its existence towards them, of its collecting
together of itself in its pursuit of its aims; the body image is finally a way of stating that
my body is in-the-world (2002 [1962]:114).
So the here of my body, its own bodily space”, in Merleau-Ponty’s terms,
whether in videogame play or any other activity, is intentional in nature, directed
towards a situation – ”the situation of the body in front of its task”. The body
exists towards its tasks. So when we attempt to describe our own body in action,
caught in the act, as it were, we must try and look at it in the reverse direction,
from the point of view of its tasks and aims, by which the body is being ”collected
together”, filtered or "polarised” in its spatial awareness, polarised as body-image.
The bodily extension
A bodily extension, according to Merleau-Ponty, is that which becomes
incorporated into our own body as ”motor project”, integrated as part of that
which is ”...neither tangible nor visible in so far as it is that which sees and
touches”. Getting intuitiely familiar with a hat, a car, or a stick is to be
transplanted into them, he explains, ”or conversely, to incorporate them into the
bulk of our own body.” (2002 [1962]:166). When we get used to a typewriter, so
that we are mastering its operation fluently and intuitively, the typewriter has
9
become like a seamless prosthesis, incorporated into bodily space, along with our
hands and eyeballs. Our body-image is being extended and re-wired through
technology, now "existing towards” and polarised by a new horizon of tasks.
Merleau-Ponty emphasises the way in which objects (stick, typewriter, hat),
when incorporated into our body, become invisible, unexpressed, cease to exist as
external objects. They instead become part of the body as gestalt, part of ”the
darkness needed in the theatre to show up the performance”. Extensions enter into
our bodily awareness as articulated by the situation or tasks towards which they
exist.
The blind man’s stick has ceased to be an object for him, and is no longer perceived for
itself; its point has become an area of sensitivity, extending the scope and active radius of
touch, and providing parallel to sight. In the exploration of things, the length of the stick
does not enter expressly as a middle term: the blind man is rather aware of it through the
position of objects than of the position of objects through it (Merleau-Ponty 2002
[1962]:165-166).
The learning and effort that goes into the transformation of an external object
into a bodily extension – relocating it, as it were, from the visible to the invisible –
is referred to by Merleau-Ponty as a kind of tacit knowledge, a habit, or
"knowledge in the hands”:
If habit is neither a form of knowledge nor an involuntary action, what then is it? It is
knowledge in the hands, which is forthcoming only when bodily effort is made, and
cannot be formulated in detachment from that effort. The subject knows where the letters
are on the typewriter as we know where one of our limbs is, through a knowledge bred of
a familiarity which does not give us a position in objective space (Merleau-Ponty
2002[1962]:166).
Summing up so far: our bodily experience of "here" and "there" is defined by
the actual and potential possibilities and demands of the situation, by what we can.
Our body is intuitively directed and postured towards a set of aims and tasks. A
keyboard, a musical instrument, a gamepad, as a result of our hard effort and
habituation, will alter the I can and thereby alter our bodily awareness, as it
becomes part of the invisible, part of that by which we perceive and act. We could
say that our experience of how our senses and organs relate to each other and to
the world, and our sense of how we are placed in front of a situation, is being
re-situated through the incorporation of a bodily extension.
This is not an extension of pure subjects or egos, but of bodies, in their dual
nature of being both subject and object. The blind man, extended by his stick, is
both touching and being touched.
The extending touch
So what does this mean for computer games and computer game avatars? It is
clear that games that are controlled in real-time, whether via sticks, mouse,
10
steering wheel, motion capture, or any other kind of direct interface, do rely on
our ability to learn, in some way or another, bodily intuitive control. In general
terms, therefore, we do somehow ”transplant” into these kinds of computer games
in a similar fashion as we do when we learn to play an instrument or drive a car.
However, unlike cars and walking sticks and pianos, video games extend our
bodies across a material divide, into screen space. This material gap is a major
complication, which obviously Merleau-Ponty does not address.
Things get particularly complicated when games require us to cross that
material divide via an avatar. What kind of object is it, exactly, that can be said to
plug into our body as a prosthesis? The controller? The screen? The avatar? When
I am playing, say Mario 64 (1996) or Halo (2001), what would be the ”here” of
my bodily space, and what would count as ”external objects”? What would I, in
Merleau-Ponty’s words above, be ”moving directly”, as opposed to the stuff that I
am moving «with the aid of my body”?
The first answer is fairly straightforward, at least in its general formulation: the
core prosthetic element, which plugs into our body in order to disappear under the
radar, is the controller interface. Without our ability to learn to act and react
intuitively into screen space via the sticks and buttons of the controller, we would
never approach any mastery of action-oriented games.
The ideal type for this form of play would be arcade action games like
Pac-Man (1980) or Breakout (1978). And the classic text would be jazz pianist,
sociologist and philosopher David Sudnow’s phenomenological self-study in
Pilgrim in the Microworld: Eye, Mind and the Essence of Video Skill (1983).
Sudnow draws heavily on Merleau-Ponty’s work, including its characteristic style
of formulation. He painstakingly records how, after hundreds of hours of training,
he learned to become a master of the home console version of Breakout, and
describes in poetic detail his own appropriation of the game as a prosthetic
extension.
First, Sudnow draws attention to the “electro-umbilical hookup” (1983:23 ) that
connects our hand to the responsive image of a paddle.
There’s that space over there, this one over here, and we traverse the wired gap with
motions that make us nonetheless feel in a balanced extending touch with things.
(Sudnow 1983:37).
Sudnow here goes straight to the heart of real-time controlled games, as
pioneered by Spacewar! and Pong (1972). The player feels like he or she is in an
extending touch with things on the screen. As players, we reach across the
material barrier, and “traverse the wired gap” between our body’s space here and
screen space there. This “mysterious transformation of our movements” (1983:23),
I would argue, which simulates a tangibility of the screen space, is the primary
conceit, the primary “as if” of the player-machine interaction. The simulation of
tangibility is not dependent on there being anything figuratively recognisable
going on the screen (spaceships, gardens); all that is required is an experience of
continuous physicality, of being in extended touch with on-screen images.
11
It is important to note that it is not the bodily extension itself, but the
experience of being in touch with physical objects – just like we would with a
pinball machine or a mechanical coin-up game – that is a conceit, a pretence, a
(real-time) simulation run by a computer. In other words: simulation comes into
play at the level of materiality. Borrowing Umberto Eco’s terms, we can say that
screen space is given an “analogous function” (1976:209) in relation to the
physical reality of natural embodiment.
The extending touch, Sudnow goes on to observe, comes to function like a
prosthetic extension of his own body, an implement, via the interface of the
gamepad.
When a paddle or a bat is incorporated by the body, becoming a continuation of ourselves
into and through which we realize and aim in a certain direction, such implements lose all
existence as things in the world with the sorts of dimensions you measure on rulers. They
become incorporated within a system of bodily spaces that can never be spoken of in the
objective terms with which we speak of objects outside ourselves (Sudnow 1983:122).
It is tempting to say that the paddle of Breakout becomes a prosthetic avatar,
and in a certain limited sense this would be true; the paddle, or the “bat” in Pong,
directly hooked up to the player's fingers as if connected by a mechanical link, is
indeed a privileged mediator of agency within screen-space. The on-screen paddle
becomes the logical counterpart to the physical extension of the controller device.
However, as Sudnow is lead to realise, after hours and days of practice, there is
a higher ambition to strive for: the game as a whole, gamepad and screen, can be
transformed into bodily prosthesis, incorporated as second nature in a way that is
similar to the mastery of a musical instrument. Sudnow finds that, with this kind
of “game” (which is, he argues, no longer a game), one can, in a
phenomenological sense, literally lose oneself, disappear in the game:
It’s as if instead of truly incorporating the events on the screen within the framework of
the body’s natural way of moving and caring, the action on the screen must incorporate
me, reducing or elevating me to some ideal plane of synaptic being through which the
programmed co-incidences will take place (Sudnow 1983: 138-139).
At the end of his learning experience, Sudnow finds a state of play that
approaches the hypnotic, the hallucinatory:
It’s becoming an instrument. Instantly punctuated picture music. Supercerebral crystal
clear Silicon Valley eye jazz (1983:191).
What is this, in light of the above? It seems to me that this is not an extension
of the body in its dual nature, but rather something approaching the extension of
pure subjectivity, a Game Ego prosthesis in Wilhelmsson's terms, a kind of bodily
self-awareness without external space, bodily habit as trance. So we could call this
kind of play, which is typically associated with classic arcade action, instrument
play, because of its similarity to mastering a musical instrument, its patterns and
rhythms. Instrument games do have a prosthetic avatar, in its minimal form, but
the relationship between avatar and its on-screen environment, its external
counterpart, its screen ecology, is indistinct and blurred, washed out along the path
12
to fluent mastery. In the end, there is no speaking of the avatar versus the
environment, only the controller and the screen as one organ, a hypnotic machine.
The prosthetic marionette
In avatar-based games, in contrast, there is always going to be an external
screen space, an environment, a world into which we extend our bodies via the
avatar. In terms of genre history, the key moment would be the transition from the
static screens of Breakout and Pong to the scrolling environments, the travelling
frame, of Super Mario Bros. Through the playable figure of Mario, the player
could go journeying and exploring into an unknown landscape that stretched
beyond the frame of the screen. The action-adventure was born.
Like the spaceship in Spacewar!, or the paddle in Breakout, Mario's
movements are controlled in a way that simulates a tangible relationship.
Controlling Mario is like controlling a marionette, hooked up to the player's
fingers by invisible strings. In order to play well, the player must work to
incorporate Mario as the on-screen extension of his or her own body, via the
physical extension of the gamepad.
In the action-adventure genre, however, in contrast to the arcade-action genre,
the avatar’s relationship to its environment is put into focus. The rationale behind
this type of prosthetic habituation is not to reach a delirium of the Game Ego, but
rather to be able to perceive and act intuitively within an environment (navigating,
exploring, combating), through an avatar. The world of the game does not offer
itself to be absorbed by the marionette (or vice versa), as in arcade action, but is
instead something external, autonomous and unknown, to be discovered and
conquered through the avatar as part of a journey.
There are of course, it should be noted at this point, avatars that allow the
player to act indirectly within screen-projected environments without becoming
prosthetic in nature, or which at least are not designed to function in this way.
Such avatars are not controlled through tangible interaction (simulating an
extending touch), but rather through symbolic interaction, which means that the
player gives instructions to the avatar, via the controller interface. Symbolic avatar
interfaces simulate not physical tangibility but an anthropomorphic, perceiving
agent who is able to respond to communication. In strategy and roleplaying games,
the player typically enters commands from a menu, and uses mouse clicks or
similar to indicate the positions to where the characters or unit should move.
There are interesting cases that are somewhat ambiguous in this respect. In the
action-roleplaying hit game Diablo (1996), combat control operates through
symbolic interaction, as in other roleplaying games, and the player navigates the
avatar through mouse-clicking at designated positions. However, because the
clicking happens so fast, the experience nevertheless approaches a sense of
"pulling" the avatar through a tangible interface.
13
Proxy embodiment
The prosthetic avatar plays with the phenomenology of the body, by extending
its dual nature into screen space. The on-screen marionette becomes part of that
through which a world comes into existence, part of the player's “I can”. The
player is being re-wired and re-directed towards “important figures against
indifferent backgrounds” through the integrated prosthetic apparatus of controller
and on-screen avatar
When we play, because the avatar extends the body rather than pure agency or
subjectivity, screen space becomes a world that we are subjected to, a place we
inhabit and where we struggle for survival. We learn to intuitively judge, like we
do in the real world, the opportunities and dangers of the environment. James
Gibson’s formulation of the ecological dimension of visual perception is a fitting
description of the dual nature of avatarial embodiment:
Any substance, any surface, any layout has some affordance for benefit or injury to
someone. Physics may be value-free, but ecology is not (Gibson 1986[1979]:140).
In general terms, the prosthetic avatar re-configures our body’s "ecology", in
Gibson's terms. The avatar alters our bodily space so that it (magically) extends
into screen space, across the material divide, a new field of affordances, a new
perceptual ecology.
However, this explanation does not yet address our intuitive experience, in
navigable 3D environments, of also being transported into screen space through
the prosthetic avatar. As noted above, the relationship between the avatar as a
bodily extension and the avatar as embodied presence is a paradox.
Merleau-Ponty’s theory of the body’s dual nature as both subject and object can
help us clarify the nature of this paradox.
It seems to me that the avatar does indeed re-locate our body into screen-space,
not as fiction but through a re-configuration on the level of the phenomenology of
the body. The avatar is no mere extension, I will suggest, but a prosthetic proxy,
which extends the phenomenal body while also – unlike a walking stick or a
musical instrument filtering or channeling our body into shape and place, into
screen space, and thereby also in an important sense “hiding” and protecting it,
making it irrelevant in its original (non-extended) configuration. However, the
extended marionette performs this operation only in a very limited way, as
compared to the more radical channeling of the navigable camera-body.
The marionette’s key function is this: while it extends the body-subject and the
corresponding bodily space into screen space, as argued above, it functions as a
stand-in or replacement of our objective body, a proxy on our behalf. The
prosthetic avatar allows us to engage in a playful and temporary separation of
subjective and objective body, across the material divide. In the moment of being
captured by and channelled through the avatar, the body that is here, safely seated
on the couch, will be rendered irrelevant in its objective dimension, as an object
among other objects, in Merleau-Ponty’s terminology – as that which is being
14
touched. Because the extended body-subject is instead directed towards what is
happening on the screen, the marionette comes to function as a replacement of the
objective body, becoming the new, temporary manifestation of the player’s body
in external space. In other words: as a body-subject I may be throwing myself into
the playground, no barrels held, but as body-object I am participating through a
stand-in, a proxy, an incarnation of myself, an avatar.
This means that our experience of being taken into the game world by our
avatars can be explained without recourse to fictionality. Undoubtedly,
make-believe plays an important role, insofar as computer game marionettes
would also be conceived as humanoid agents or characters who somehow acts on
our behalf. Nevertheless, proxy embodiment is a trick at the level of the
phenomenology of the body, not a trick of fiction. The sense of bodily immersion
that is involved in avatar-based play is rooted in the way in which the body is able
to intuitively re-direct into screen space a perception of itself as object, which is
the perception of itself as part of external space. A mouse cursor cannot function
as a proxy in this way, not because it lacks fictional elaboration, but because it has
no objective presence within screen space.
The principle of prosthetic proxy embodiment has been a dominant paradigm in
computer games since Spacewar!. It responds to a desire to enter into the
gameworld not as yourself, in your actual physical body, but as incarnated in
another body, a body made to fit all kinds of strange and alien worlds, and into
which you can seamlessly transplant, via minimal movements of eyes, hands,
fingers.
Let me note, here, that proxy embodiment, as a general interface paradigm in
computer gaming, is incompatible with two other general principles of tangible
interaction in real-time environments. First, in what we may call direct interaction,
the user or player is allowed to point at or touch objects in the on-screen
environment directly, in a way that simulates Sudnow’s “extending touch”, but
which leaves no place for a proxy body as mediator. This can be done via mouse
and cursor, via a pointing device like for example a light-gun, via a touch-screen
interface, or to a certain extent via motion control interfaces. In all such games of
direct tangible interaction, the only “avatar” is your familiar physical self, in front
of the screen. Direct tangible interaction is typically found in casually oriented
games like, for example, Sneezies on the Iphone or the online Flash game Shoot
Bin Laden.
Secondly, the principle of 1:1 motion control, as currently promised by the Wii
MotionPlus, Kinect, and Playstation Move peripherals, discards the idea of proxy
embodiment in favour of projecting instead an on-screen body that mirrors the
shape and movements of the player’s physical body, in as much detail as possible.
Again, it is your familiar physical body that is made the embodied subject during
play, but this time in a dialogue with its own mirrored self across the divide, its
projected mirror image. It is through this mirror image, rather than through a
prosthetic avatar, that the player gets to be in a simulated direct physical contact
with the elements in on-screen game space.
15
Full-body mirroring control, as an alternative to standard avatar control, opens
up a range of playful possibilities, while at the same time closing down or
marginalising experiences that are specifically linked to the principle of proxy
embodiment. In particular, the player’s self-movement or locomotion within
on-screen game space, which is at the heart of avatarial embodiment in computer
games, becomes a major challenge in motion control interfaces, as it requires
some kind of multi-directional treadmill interface of the kinds that have been tried
out in Virtual Reality installations. 1:1 motion control also implies that your
embodied self in the game cannot, for example, do triple jumps Mario style or fall
down a deep ravine, unless some kind of proxy representation, some kind of
avatar, is being temporarily added to the mix, detached from the straitjacket of the
1:1 mirroring.
Telepresence and the camera-body
Extended 2D avatars or marionettes do, in a sense, “transport” our body into
screen space, in so far as we can relate to them as manifestations of our own body
as object in external space. It would not make sense, however, to say that
prosthetic marionettes re-locate our bodily space – our spatial self-awareness as
bodily subjects – even if they do extend and re-shape it. They are remotely
controlled proxies. Whereas the movements of our fingers are being entirely
swallowed by screen space, integrated into our bodily self-awareness only in
proportion to their on-screen value, our visual perception (as well as, although
relatively less significant, our hearing) will still operate from a bodily “here” that
is outside screen space. With 2D avatars, bodily space remains anchored in
physical space, even if extended and projected into screen-space via the prosthetic
marionette. In this respect, marionettes are comparable to its non-prosthetic
siblings like figures on a flannelgraph or pieces on a chessboard.
In terms of their spatiality, traditional 2-dimensional game spaces are framed
surfaces, contained within physical space, and in this respect comparable to other
framed sub-spaces like a whiteboard or a computer desktop. The framed screen of
Donkey Kong (1981), is a minature world, like a fish tank or a Lego village. It
becomes a miniature when we relate to it from the outside, perceiving it as part of
this space here, the space of my natural body. The miniature world, or
“microworld”, in Sudnow’s terms, has a very strong and distinctive appeal. In
Miniature Gardens & Magic Crayons: Games, Spaces & Worlds (2003), Chaim
Gingold suggests the metaphor of the miniature garden:
A miniature garden, like a snow globe, model train set, or fish tank, is complete; nothing
is missing, and nothing can be taken away. Clear boundaries (spatial and non-spatial),
overviews, and a consistent level of abstraction work hand in hand to make the miniature
world believable, complete, and tractable for both the author and player. Miniatureness
makes a garden intelligible in the mind of the player, and emotionally safe in his heart.
16
Miniature scale, clear boundaries, and inner life help players to wrap their heads, hands,
and hearts around a world. (Gingold 2003: 7-8).
However, avatarial embodiment through a camera-body, for which the First
Person Shooter genre would be the ideal type, is a very different kind of game.
Navigating real-time 3D environmens, we do not perceive the environment from
the outside, as if looking into a fish tank, but we are very acutely present in that
other space. So the distinctive appeal of miniature worlds is lost. Channeled
through the first-person avatar, in the heat of the action, the “here” of my bodily
space is no longer my physical body’s natural space, in front of the miniature
sub-space of the screen. Instead, paradoxically, my new “here” has been re-located
into screen space there; I am tele-present in that space. When captured by the
avatar, I am phenomenally present elsewhere.
How does this work? Unlike 2D avatars, the camera-body of the first person
avatar offers the screen itself as the principal prosthetic hookup, working as an
extension of our body’s “motor project” of moving-and-looking. The physical
mechanism that allows this radical re-wiring of bodily space to take place is
referred to as vection: the experience of bodily locomotion caused by visual
perception alone. Our locomotive vision that is: the way we move not just our
eyeballs but our whole body as an organ of visual perception – has been detached
and re-attached to the minimal movements of our hands and fingers. Our spatial
self-awareness has become relocated, so that we are moving and perceiving
intentionally only in relation to the screen-space of our our temporary
camera-body, our avatar. Our body has become “polarised”, in Merleau-Ponty’s
terms, through the first person avatar.
Prosthetic locomotive vision through an avatarial camera-body has been a
dominant paradigm of action-adventure computer gaming since the mid-nineties.
It has manifested itself strongly in the design of the controller hardware. The
characteristic and still dominant dual-axis paradigm was established in the
mid-nineties through the so-called “mouselook” on the PC platform, first made
default in Quake (1996), and through the twin-stick setup pioneered by
Playstation’s Dual-Shock controller.
We should note that marionettes do not necessarily need to be incorporated as
prosthetic proxies in order to be playable. Because they are remotely controlled
and observed from the outside, it is possible to relate to them as regular external
objects during play, lining them up with other on-screen objects in order to create
the desired effects. In Merleau-Ponty’s terms, we could say that marionettes may
be moved “with the aid of the body” rather than being “moved directly”. In
contrast, the camera-body cannot be “moved with the aid of the body” as an
external object, because we cannot look at our own eyes in the same way as we
can look at our own hand. You could say that our eyes are more radically “on this
side of things” than our hand. As consequence, in a First Person Shooter, you have
to learn to internalise camera control, or you will not be able to play the game at
all.
So we could say that the navigable camera has a radical subject-status, which
17
means that it is de facto prosthetic in nature; there is no other option. After you
have learned to incorporate the camera as a bodily habit, so that you are in
intuitive control of your own new body, if the computer then takes camera control
away from you, if only for a brief second, this will not break the strong prosthetic
link, but instead produce a sensation of being moved, of being taken for a ride.
The built-in prosthetic nature of the navigable camera means that when you
start playing an FPS for the first time in your life, a choice between all or nothing
quickly presents itself: until you learn to incorporate this strange perceptual
apparatus, responsive to the slightest movement of your fingers, as second nature,
as a prosthetic organ, you will be permanently disoriented, like a drunk person,
unable to cope with anything in the on-screen environment, and possibly also
feeling a bit sick. When habituated, however (if you ever get that far), your new
camera-body becomes like a part of your own body, part of the invisible in
Merlau-Ponty’s terminology, part of that for which there are visible objects.
I observe external objects with my body, I handle them, examine them, walk round them,
but my body itself is a thing which I do not observe: in order to be able to do so, I should
need the use of a second body which itself would be unobservable (Merleau-Ponty 2002
[1962]:104).
After the 3D revolution, it is through this kind of proxy body that we now
belong to and inhabit the world, looking (and listening) for opportunities and
dangers, investigating objects, peeking around corners, scanning the horizon.
Channeled through and “transplanted into” the 3D avatarial apparatus, we are like
Merleau-Ponty’s blind man with a stick:
Once the stick has become a familiar instrument, the world of feelable things recedes and
now begins, not at the outer skin of the hand, but at the end of the stick. 175-176.
Similarly, we could say that when playing competently a First Person Shooter,
the world of visual appearances begin behind the surface of the screen. The screen,
when appropriated as camera-body, has become gestalt, the tacit “third term”
against which a structure of background and figure appears (Merleau-Ponty 2002
[1962]:115).
In the phenomenological sense, therefore, the notion of remote control is no
longer accurate when we move from 2D worlds to navigable 3D. The
incorporation of the screen as a new perceptual organ sets up a new, “double
horizon of external and bodily space” that is not directed towards screen-space, as
when playing through a marionette into 2D space, but which is spatially re-located
and anchored within it. The first person avatar, therefore, is a distinctive modality
of perceptual immersion. Being re-located and telepresent through the
camera-body means that we have become perceptually encapsulated without being
sensorially encapsulated.
It is important to emphasise that the prosthetic 3D avatar, like its 2D
counterpart, is a proxy, not just a mechanism of prosthetic locomotive vision. The
camera-body that extends from our fingers is not an extension of a pure vision, not
a vehicle of visual “perspective”. It is an extension of our moving-and-perceiving
18
body, in its dual nature as both subject and object in the world. Crucially, this
implies objective embodiment within the screen space. By definition, playing
through an “avatar” means belonging to and being affected by the
screen-projected environment – otherwise there would be no ecology, no threat or
obstacle, no struggle, no being-in-the world, no game. In comparison, a navigable
workspace camera, as typically found in graphical modelling and animation
software, is distinctively body-less. While it does offer prosthetic vision, it is not
submitted to any ecology, obeys no physical laws of the environment, is not
recognised by any other agents, is not being shot at, in short: it lacks existence as
an external object in the environment.
The immersive experience of first person proxy embodiment is neither fictional
telepresence nor an illusion of telepresence. Analysing it through the theoretical
prism of Phenomenology of Perception, we must conclude that our embodied self
is actually being re-located, transported into screen space. Our familiar
body-image, our intuitive awareness of where we are as perceiving and acting
subjects, is being dramatically altered once we step into a prosthetic relationship
with the avatarial camera-body. For the body-subject, when directed towards its
tasks and aims as they come to light through locomotive visual perception, the
screen as an external object has been made irrelevant, or in Merleau-Ponty’s terms,
invisible, non-existing. In the moment of being captured by the first person avatar,
there is no longer a bodily space here, in front of the screen, from which actions
extend. The only way to escape this situation would be to step back from the
avatar, to detach from it, acting and perceiving independently of the camera-body.
Third person
A possibly unexpected implication of this approach is that also the wider group
of so-called “third-person” 3D action-adventures, from Tomb Raider (1996) to
Grand Theft Auto III (2001), must be included in the category of “first person”, in
spite of the central role given to marionette control. The key parameter in deciding
their status will be the function of the virtual camera, which in these kinds of
games is only indirectly controlled, in various kinds of ways, via the prosthetic
marionette, as if being pulled by the marionette via an invisible string.
In some games, as in the early classic Tomb Raider, the exact behaviours of the
tag-along camera, and how it ends up framing the action in any particular situation,
is entirely controlled by automatic procedures. In other games, notably the
influential Prince of Persia: The Sands of Time (2003), while the link to the
marionette is still intact, the virtual camera can be moved 360 degrees around the
marionette via the right analogue stick or mouse. In both cases, it is indeed as if
the avatarial “follow-cam” takes the role of a “second body which would itself be
unobservable”, whereas the marionette carries the full burden of objective
embodiment by facing acrobatic challenges, being attacked and shot at, falling
19
down ravines and so on.
Most of the time, at least in its ideal form, the relative independence of the
follow-cam in relation to the marionette will be geared towards framing the action
in adequate and predictable ways, always respecting the unbreakable bond to the
marionette and the player. However, it is worth noting that the camera can also
become more autonomous, unpredictable and purposefully inadequate, most
typically in games of the “survival horror” sub-genre of the action-adventure, like
the Silent Hill series (1999). This unpredictability and detachment of the camera
from the player’s prosthetic apparatus undermines a sense of coherent avatarial
embodiment (producing quite unsettling effects), approaching instead the kind of
projected “embodiment” that is characteristic of the language of cinema.
In spite of these modifications and exceptions, the main point I want to make is
that an indirectly controlled camera, to the extent that it functions as a navigable
camera, and to the extent that it is part of an apparatus of objective embodied
presence, is nevertheless avatarial in its function, undermining a sense of
miniatureness in favour of encapsulating telepresence.
To the extent that the player can navigate the third-person camera however
indirectly – in a predictable and intuitive fashion, it embodies the function of
locomotive prosthetic vision. It will be integrated as part of a re-located bodily
space, and linked to the objective presence of the marionette. It seems to me that
this is true even if a follow-cam does not give the player the same level of exact
control over locomotive vision as in a First Person Shooter. We could say that in
third-person 3D, the re-located player operates his or her marionette like an
extended hand, much like in a traditional 2D game, only this time from a position
inside the screen-rendered world, travelling along with the marionette like a
Siamese twin.
Corporeality
Finally, back to the question of simulation. Surely our bodies inside
screen-rendered synthetic space must be, at the end of the day, simulated bodies?
True enough, in a racing game, when I am driving through the wintery landscapes
of Sweden, in my blue Subaru Impreza, while still seated on the couch in my
living room, controller or steering wheel peripheral in hand, simulation and fiction
is very much a part of it, on many levels. However, my argument is nevertheless
that I am, in the moment of play, actually being re-embodied within a different
space. Proxy embodiment is no “conceit”, as Rehak seems to imply, or a projected
or “diegetic” embodiment constructed from moving images, like a mental
simulation, as in cinema. Avatarial space is real external space, navigable,
inhabitable, negotiable. When I am playing, I am actually there, as a composite of
flesh and technology, objectively existing within synthetic space.
Phenomenological analysis, as I have attempted to show in the above, helps us
20
describe this kind of extended embodiment and understand what is going on.
The aspect of simulation comes into play on two primary levels. First, as
argued above, the experience of tangible physicality, of being inside synthetic
space as if being in a world that shares the materiality of the world of natural
embodiment, is indeed a conceit, a simulation.
Secondly, simulation comes into play, I think, in terms of the necessary
familiarity of the particular experiences of avatarial embodiment. Computer games
are perceptual simulations insofar as they evoke familiar corporealities, activating
and utilising already established bodily schemas (or “images”) and bodily spaces.
Without this familiarity, which is a resonance on the level of embodied
self-awareness, first person avatars would be far too hard to incorporate as bodily
habit.
To illustrate this final point, without going into much detail, I would like to
point out how first person avatars, as we are currently familiar with them in their
actual design, owe much of their distinctive corporeality to 3 established bodily
schemas.
The first, which is implied by the term navigable (or virtual) camera, is the
monocular vision of the cinematographic camera lens. First Person Shooters are
especially characteristic in this respect: the camera-body is also a camera-gun, as
if the weapon has been attached on top of it, merging looking and aiming into one
movement. The player is locked into a tunnel vision along the barrel of a gun (–
expanding the virtual focal length would produce a fish-eye effect), optimised for
fast and precise aiming, providing a strong sense of speed and disorientation, and
encouraging the persistent awareness of threat. This avatarial body is highly
focused, highly restrictive, and, one could argue, inherently paranoid in nature. In
popular genre leaders like Half-Life (1998) or Halo, the player not so much walks
or drives as floats around the environment, embodying a type of machine-like
corporeality for which the closest real-world comparison would be a Steadicam.
Secondly, an important capability of our natural embodiment that has been
reproduced in the 3D action-adventure genre is what we may call, for the sake of
simplicity, dual-axis movement: our body is able to operate locomotion and
turning independently of each other, on separate axes, so that the direction in
which we are looking does not have to be the direction in which we are moving.
Without the simulation of this particular embodied capability, there would hardly
be any 3D action-adventure, and there would be no need for the characteristic
twin-stick controller interface that dominates the market. This flexibility of
looking and turning is also, I would guess, an important factor in turning many
people off so-called “hardcore” avatar-driven games, as the dual-axis setup can be
tricky to learn for inexperienced players.
Finally, as mentioned above, and as pointed out by James Newman, controlling
a prosthetic first person avatar evokes the bodily disposition of driving or piloting
a vehicle. In a very concrete sense, this is evident from comparing computer game
controllers with the control devices for remote-controlled vehicles of various kinds.
If we look at scientific and military technology, there is a clear analogy between
21
telepresence through avatars and telepresence through so-called drones, or
Remotely Operated Vehicles (ROV).
This vehicular nature of prosthetic avatars is further emphasised by the
typically vehicular and machine-like corporeality designed into most first person
bodies, as they are being conventionally implemented in commercial games. This
familiar corporeality is obviously evident in racing games and other games that
explicitly simulate vehicular embodiment: flight simulators, space combat games,
and so on. However vehicular embodiment is also typical for the First Person
Shooter, which only very rarely attempts to simulate, on the perceptual level,
something that would approach a human, walking body.
The evocation of driving or piloting as a familiar bodily schema is a natural
response, it seems to me, to the basic nature of prosthetic avatars. The notion of
driving is not a metaphor in this case; the reason why computer game avatars feel
like piloted vehicles or machines is that they are actually driven by the player.
This is the main reason why game designers and storytellers in most cases choose
to elaborate on a vehicular experience rather than attempt to break away from it.
Proxy VR
Summing up, the phenomenology of proxy embodiment in computer games is a
unique and playful paradox. The prosthetic avatar functions both as the player’s
bodily extension into screen space and as a proxy or replacement for the player’s
body as an object in external space. This is a highly unique if we compare with the
examples discussed by Merleau-Ponty in Phenomenology of Perception. It is also
unique if we compare with other forms of bodily play, like sport and various
forms of embodied make-believe, although there are clear analogies to puppetry
and remote-controlled vehicles.
Avatarial embodiment in 3-dimensional environments, via a navigable camera,
is different from embodiment through a marionette only, even if the basic
principle of prosthetic proxy embodiment remains the same. The virtual camera’s
prosthetic locomotive vision re-locates the player in terms of his or her intuitive
bodily awareness, and sacrifices the safe appeal of the miniature world for the
sake of perceptually encapsulating telepresence. Whereas proxy embodiment via
marionettes only, as we find in traditional 2-dimensional game spaces games,
would be comparable to driving a radio-controlled miniature truck or aeroplane,
first person embodiment is more accurately compared to driving a real car. An
important difference, obviously, is that the 3D avatar takes us for a ride into
synthetic space only, which implies no (or at least, in comparison, very modest)
danger or risk to our physical body.
Prosthetic telepresence is not a conceit or mental projection, but actual
embodied presence. In the cinema, in contrast, there is no actual space to be
inhabited, only images, and there is nothing off-screen except our own mental
22
projections.
The central aspect of simulation involved in our engagement with prosthetic
avatars in computer games is the simulation of tangibility. Experiencing
tangibility means that it is as if we were not just crossing the material gap but in
fact closing it; it is the conceit of continuous materiality.
At the next level, avatars also simulate familiar corporealities, evoking familiar
bodily schemas in order to be accessible. Adding to this come layers of fictional
and narrative significance, starting with a projection of the avatar as a humanoid
agent, a character. These fictionalising layers add further substance and meaning
to the basic phenomenological mechanism of prosthetic proxy embodiment. In this
respect, we may compare avatar-based 3-dimensional computer game worlds with
theme parks: there are layers of fiction all around us, but our own being there
perceiving, acting, and being acted upon – is neither fiction nor illusion.
Finally, is prosthetic telepresence Virtual Reality? Not if we mean the kind that
is being experimented with in various types of VR installations, and which has
been fantasised about in Star Trek’s Holodeck, The Matrix, or James Cameron’s
Avatar. Proxy embodiment does not offer a duplication of our own body in virtual
space. Instead, it extends from our body as a hard-earned habit, allowing us to
inhabit synthetic space through a prosthetic vehicle, a different kind of body.
During play we are piloting, via minimal movements of our eyes and fingers, a
different body in a different world. Still, telepresence by proxy is no less
immersive than its more ambitious VR sibling. In navigable 3D gameworlds,
perceptual re-location does not depend on sensory encapsulation, but follows from
the intentional nature of bodily space.
23
Bibliography
Carr, Diane. 2002. “Playing With Lara” in ScreenPlay, Cinema/Videogames/Interfaces, edited
by Tanya Krzywinska and Geoff King. London: Wallflower.
Dovey, Jon, and Helen Kennedy. 2006. Game Cultures: Computer Games as New Media. Milton
Keynes: Open University Press.
Eco, Umberto. 1976. A Theory of Semiotics. Bloomington: Indiana University Press.
Fuller, Mary, and Henry Jenkins. 1995. Nintendo and New World Travel Writing: A Dialogue.
In Cybersociety: Computer-Mediated Communication and Community, edited by S. Jones.
Thousand Oaks: Sage. Also available at
http://www.stanford.edu/class/history34q/readings/Cyberspace/FullerJenkins_Nintendo.html.
Gibson, James J. 1986. The Ecological Approach to Visual Perception. Edited by R. E. Shaw, W.
M. Mace and M. E. Turvey, Resources for Ecological Psychology. London: Lawrence
Erlbaums Associates. Original edition, 1979.
Gingold, Chaim. 2003. Miniature Gardens & Magic Crayons: Games, Spaces, & Worlds.
Master's Thesis, School of Literature, Communication and Culture, Georgia Institute of
Technology, Atlanta. Also available at
http://www.slackworks.com/~cog/writing/thesis/toc.php.
Klevjer, Rune. 2007. What is the Avatar? Fiction and Embodiment in Avatar-Based Singleplayer
Computer Games. Doctoral dissertation, Department of Information Science and Media
Studies, Bergen: University of Bergen.
Linderoth, Jonas. 2005. Animated game pieces. Avatars as roles, tools and props [online].
Aesthetics of Play Conference Proceedings [cited 12 May 2006]. Available from
http://www.aestheticsofplay.org/king.php.
Merleau-Ponty, Maurice. 2002. Phenomenology of Perception. London: Routledge Classics.
Original edition, 1962.
Newman, James. 2002. The Myth of the Ergodic Videogame. Some thoughts on player-character
relationships in videogames [online]. Game Studies 2 (1) [cited 25 July 2005]. Available
from http://www.gamestudies.org/0102/newman/.
Rehak, Bob. 2003. Playing at Being: Psychoanalysis and the Avatar. In The Video Game Theory
Reader, edited by M. J. P. Wolf and B. Perron. London: Routledge.
Ryan, Marie-Laure. 2001. Narrative as Virtual Reality. Immersion and Interactivity in Literature
and Electronic Media. Baltimore: The Johns Hopkins University Press.
Salen, Katie, and Eric Zimmerman. 2004. Rules of Play: Game Design Fundamentals.
Cambridge, Massachusetts: MIT Press.
Sudnow, David. 1983. Pilgrim in the Microworld. Eye, Mind and the Essence of Video Skill.
New York: Warner Books.
Wilhelmsson, Ulf. 2006. Computer games as playground and stage. In Proceedings of the 2006
international conference on Game research and development. Perth, Australia: Murdoch
University.
Games
Breakout. 1978 (coin-op). Developed by Atari. Published by Atari.
Call of Duty: Modern Warfare 2. 2009 (Xbox 360). Developed by Infinity Ward. Published by
Activision.
Diablo. 1996 (PC Windows). Developed by Blizzard. Published by Blizzard.
24
Donkey Kong. 1981 (coin-op). Developed by Nintendo. Published by Nintendo.
Doom. 1993 (DOS). Developed by id Software. Published by id Software.
Grand Theft Auto III. 2001 (Playstation 2). Developed by DMA Design. Published by Take Two
Interactive Software.
Half-Life. 1998 (PC Windows). Developed by Valve. Published by Sierra On-Line.
Halo: Combat Evolved. 2001 (Xbox). Developed by Bungie. Published by Microsoft Game
Studios.
Marble Madness. 1984 (coin-op). Developed by Atari. Published by Atari.
Pac-Man. 1980 (coin-op). Developed by Midway. Published by Midway.
Pong. 1972 (coin-op). Developed by Atari. Published by Atari.
Prince of Persia: Sands of Time. 2003 (PS2). Developed by Ubisoft Montreal Studios. Published
by Ubisoft.
Quake. 1996 (PC Dos). Developed by id Software. Published by id Software.
Silent Hill. 1999 (Playstation). Developed by Team Silent. Published by Konami.
Spacewar! 2006 (Java version). Developed by Russel/Graetz/Wiitanen. Published by
Silverman/Silverman/Gerasimov. Available from
http://lcs.www.media.mit.edu/groups/el/projects/spacewar/. Original Version, 1962.
Super Mario 64. 1996 (Nintendo 64). Developed by Nintendo. Published by Nintendo.
Super Mario Bros. 1985 (Nintendo Entertainment System). Developed by Nintendo. Published
by Nintendo.
Tomb Raider. 1996 (Playstation). Developed by Core Design. Published by Eidos Interactive.
World of Warcraft. 2004 (PC Windows). Developed by Blizzard. Published by Blizzard.
... Presence is the successful feeling of existing in a synthetic environment. Telepresence narrows the term further, describing the feeling of presence achieved through the use of a communication medium (Klevjer, 2012). This definition, while originally reserved for telecommunication, applies to the feeling of being present in a game. ...
... This definition, while originally reserved for telecommunication, applies to the feeling of being present in a game. Klevjer (2012) describes this as being "tele-present in that space. When captured by the avatar, I am phenomenally present elsewhere." ...
... The controls used for this kind of game are usually either a dual-axis controller, or in case of keyboard and mouse, the mouse, rather than the keyboard, is used to move the avatar by issuing commands to move to a certain place. This type of movement control might seem unintuitive, however Klevjer (2012) proposes that "because the clicking happens so fast, the experience nevertheless approaches a sense of "pulling" the avatar through a tangible interface." and as such, the control interface can still create a sense of high correspondence between the player's actions and movements of the avatar, reaching a real-time synchrony as mastery of the control interface is reached. ...
... In some cases, the camera can be rotated or moved around slightly to peek around a box while crouching for example, but almost always the camera and the avatar are in tight synchrony. Klevjer (2012) considers all of these camera modes to fall under the same umbrella of camera indirectly controlled by the movements of the avatar, as if the camera is pulled by the avatar around with an invisible string. ...
Preprint
Full-text available
... player agency) išplėtimas ir subalansavimas. Klevjer (2012) pateikia tokį žaidėjo atstovybės (angl. agency) apibūdinimą: "Realistinė atstovybė pasiekiama, kai tau nereikia žaisti žaidimo sekant instrukcijomis ir kai veikėjų, objektų ir procesų elgesys žaidime gali būti priskiriami prie savo savybių ir gebėjimų, o ne taisyklių, kurios priklauso jų išorei." ...
Book
Vaizdo žaidimų industrija vis stipriau įsitvirtina kaip inovatyvus ir pelną duodantis ekonomikos sektorius. Sparčiai tobulėjanti techninė bei programinė įranga, interneto plėtra, žaidimų elektroninių parduotuvių atsiradimas, mokslininkų bei meno kritikų susidomėjimas, netgi kompiuterinio žaidimų sporto reiškinio susiformavimas negrįžtamai pakeitė visuomenės požiūrį į šią interaktyvią mediją. Pastaruoju metu apie kompiuterinius žaidimus kalbama ne tik kaip apie laisvalaikio praleidimo priemonę, bet ir kaip apie patrauklų mokymo ar mokslinių tyrinėjimų įrankį. Žaidimuose taikomi principai vis dažniau perkeliami ir į kitus sektorius, taip siekiant palengvinti naujų dalykų mokymąsi, padidinti klientų lojalumą verslui ar paslaugai, paskatinti visuomenę dalyvauti socialiniuose projektuose. Šalia viso to kompiuteriniai žaidimai vis dažniau prilyginami ir tarpdalykiniams meno kūriniams, atskleidžiantiems specifinę idėją per vaizdą, garsą, personažus, žaidimo mechanikas ir interaktyvų siužetą. Lietuvoje kuriasi vis daugiau tarptautiniu mastu pripažintų kompiuterinių žaidimų kūrimo kompanijų, kurių plėtra yra glaudžiai susijusi ir su aukštosiose mokyklose parengiamų žaidimų industrijai reikalingų specialistų kompetencija. Tačiau tam, kad tokių specialistų parengimas ir įsiliejimas į darbo rinką būtų išties sklandus, iki šiol labai trūko lietuvių kalba išleisto leidinio, padedančio gilintis į kompiuterinių žaidimų kūrimo ypatumus. Šis vadovėlis ir buvo parašytas siekiant užpildyti susidariusią spragą. Versdami šios knygos puslapius susipažinsite su potraukio žaisti prigimti, žaidimų mechanikų ir dinamikų svarba, naratyvą turinčių žaidimų kūrimo ypatumais, linijinio ir interaktyvaus naratyvo vystymo priemonėmis. Taip pat sužinosite, kodėl kompiuterinių žaidimų dizaineriams svarbu suprasti žaidėjų psichologiją, kaip sustiprinti žaidėjo atstovybę ir jo įsitraukimą į žaidimą, kokios naudotojo sąsajos tam yra tinkamiausios. Rašydami šį vadovėlį pirmiausiai orientavomės į kompiuterinių žaidimų dizaino paslapčių besimokančius aukštųjų mokyklų studentus, tačiau jis turėtų būti naudingas ir pradedantiesiems žaidimų dizaineriams bei kitiems asmenims, norintiems perprasti kompiuterinio žaidimo kūrimo kompleksiškumą ir įvairiapusiškumą. Linkime gero skaitymo, kūrybinio įkvėpimo ir įdomių bei įtraukiančių jūsų pačių sukurtų žaidimų. Knygos autoriai
... Mennecke et al 2011, Biocca 1997, Riva and Triberti 2016, phenomenological description (e.g. Dreyfus 2009, Gleason 2016, Klevjer 2012, Hougaard 2021, Zahorik and Jenison 1998, Grabarczyk and Pokropski 2016, and social interaction analysis (e.g. Schulze and Brooks 2019). ...
Article
An axiom of Per Aage Brandt’s approach to conceptual blending, known colloquially as the “Aarhus model”, is that semiosis only makes sense when grounded in communicative interaction. Here we adopt that approach in relation to the reality of current, daily communication which is increasingly mediated by digital audio-visual technology platforms. We pursue this goal via a small set of case studies that explore how this technology changes and challenges social interaction and how participants exploit and adapt cognitive, embodied, technological, and semiotic resources in creating meaningful, collective, virtual spaces of joint social activity. In so doing, we expand the horizon of inquiry and contribute insights that have relevance for the new media ecology. This application of cognitive semiotic analyses of video meetings confronts the nature of “mediation” and its accomplishment, the status of “virtual spaces”, and “social presence.”
... Rune Klevjer, using Maurice Merleau-Ponty's bodily intentions, describes the embodiment of the game player as using the character on screen as an extension of their own. The avatar thus becomes a proxy for the self, within the screen world (Klevjer 2012). Nathaniel Stern adds to this discussion by integrating the body into interactive art pieces, further demonstrating the embedding the physical within the virtual, literally incorporating the body presence within a virtual narrative (Stern 2013) However, there is significant research to demonstrate that the relationship the user has to their avatar is still strong, regardless of the distance between realism and fantasy, as an example of the Proteus effect. ...
Thesis
Full-text available
This thesis aims to semiotically model hyperreality in contemporary digitized society and beyond into the burgeoning Web 3.0 era of ubiquitous virtuality. The upcoming Web 3.0, as an evolution of the current digital space, leads us to an evolution of hyperreality (where the virtual is more real than reality). We call this next step “hypervirtuality” as a portmanteau of “virtual” and “hyperreality”, where the physical becomes a meaningless canvas, overlaid by digital information, which is itself appended by virtual spheres of digital signs that attempt to differentiate themselves from the mundane digital world. We investigate virtual reality, augmented reality, and mixed reality models, stating that virtual reality and augmented reality lack the interaction with the physical space to create hypervirtuality, but mixed reality could overlay the physical with a digital layer of information that increasingly becomes more significant and symbolic than mundane physicality. These spaces can replicate the physical or exaggerate the virtual, pointedly marking themselves as more virtual-than-virtual. The transmediality of identity between the user and their avatar is one of the last relationships where the physical, offline, signs form a primary foundation to the semiotic communication model. Our thesis concluded that an enhanced digital literacy education within the classroom would enable the contextualisation of self within the physical space, thus avoiding the loss of the physical sign as a meaningful part of one’s identity schema. The virtual space should add meaning and not replace offline signs completely.
Chapter
Full-text available
Digitale teknologier og medieplattformer er grunnleggende for kunst- og kulturfeltet i dag, og de griper inn i stadig flere sider av vårt hverdagsliv. Gjennom seksten artikler fra en rekke norske og internasjonale forskere, undersøker Estetiske praksiser i den digitale produksjonens tidsalder hvordan digitalisering forandrer og preger produksjon, formidling og bruk av kunst og kultur i samtiden, og hvordan de estetiske praksisene selv utforsker, tematiserer og problematiserer endringsprosessene. Estetiske praksiser i den digitale produksjonens tidsalder tilbyr en bred vifte av perspektiver, begreper og modeller som belyser kunsten og kulturen i dag. Artiklene drøfter strømmekonserter, instagramdikt, massedigitaliseringsprosjekter, kulturpolitikk, hacking som estetisk praksis og scenekunstens forhold til det digitale. De drøfter bruken av sosiale medier på motefeltet, hvordan kulturinstitusjoner endres, digital formidling på det visuelle kunstfeltet og på minnesteder, og – endelig – behovet for å trekke seg tilbake fra det digitale. Estetiske praksiser i den digitale produksjonens tidsalder er resultatet av et forskningsprogram igangsatt av Kulturrådet. Programmet har også mottatt støtte fra Kultur- og likestillingsdepartementet.
Chapter
Full-text available
Denne artikkelen handler om utviklingen av en digital kulturpolitikk for barn og unge, og den retter særlig blikket mot dataspillpolitikken. For å illustrere og understreke kompleksiteten i den digitale kulturpolitikken vil vi også se på hvordan politikken forholder seg til ulike sosiale medier, og spesielt til den bruken av ulike kulturuttrykk som foregår på YouTube. Barn og unges digitale kulturbruk favner bredt. Den inkluderer for eksempel strømming av musikk, strømming av filmer og serier, kulturelle uttrykk i sosiale medier, digitale kulturuttrykk i Den kulturelle skolesekken, e-bøker, digital formidling i museene. I sin helhet vil en digital kulturpolitikk for barn og unge derfor måtte forholde seg til flere kulturuttrykk og formidlingskanaler enn dataspill og YouTube. Dataspill og YouTube skiller seg riktignok ut i tidsbruk, og fordi de langt på vei representerer en type heldigitale kulturuttrykk. De overordnede spørsmålene i artikkelen er disse: Hvilken utvikling kan vi se i de holdninger til barn og unges digitale kulturbruk som kommer til uttrykk i kulturpolitikken generelt og dataspillpolitikken spesielt? Hvordan kan den digitale kulturpolitikken utvikles fremover for å nå de overordnede kulturpolitiske målene på dette feltet?
Chapter
Full-text available
The essay discusses the role and function of cinematic cut-scenes in single-player games, in particular with respect to ongoing debates over in-game vs. cinematic storytelling. The first part gives an account of key historical developments and main current trends. The second part gives an introduction to the notion of narrative framing, emphasizing in particular the relevance of a rhetorical perspective. The third part argues that cut-scenes set up a separate cinematic space in parallel to ordinary game space, which redefines rather than necessarily excludes player agency. The final note suggests that successful storytelling through cut-scenes is rooted in a mirroring relationship between the player's experience and the characters' story. *
Article
https://www.idunn.no/doi/epdf/10.18261/blft.13.1.1 Digital formation: Videogames as an arena for self-representation and identity play Å spille dataspill er en av de mest utbredte fritidssyslene blant norske barn og unge. Likevel finnes det foreløpig fåundersøkelser av denne praksisen i en norsk kontekst. Basert påsvar fra en spørreundersøkelse jeg foretok blant eleverpå 9. trinn i desember 2019, analyserer og diskuterer jeg i denne artikkelen et utvalg norske ungdommers forhold tilsine digitale identiteter, eller spillavatarer. Resultatene fra undersøkelsen viste at flere av ungdommene ofte utformetspillavatarer som liknet dem selv. På bakgrunn av disse funnene, og med støtte i spillteori, empirisk spillforskning ogteorier om sansning og kognisjon, argumenterer jeg for at dataspill kan være viktige redskaper i ungdommers iden-titetseksperimentering, og at spillopplevelser kanbidra til nye former for posthuman selvforståelse
Conference Paper
Since Turkle’s seminal work Life on the screen (1995), avatars have been described as a form of alter egos, being means for exploring and playing with identity. Following this line of reasoning, computer gaming is seen as an activity where we become immersed in a fictitious world, pretending to be the character we play. Drawing upon empirical observations of children’s game-play I argue that the relation between the avatar and the player is a more multifaceted affair. The meanings of avatars depend upon how they are framed by the player, thus they can have at least three different functions. Avatars can become roles for socio-dramatic interaction. As extensions of the player’s agency, avatars can become tools for handling the game state. Finally when choosing and using avatars in the presence of others, avatars can become a part of our identity, not as alter egos but as props for our presentation of self on the social arena surrounding the game.
Article
In this paper the environment in computer games is discussed from a narratological and ludological perspective focused on the idea of environmental affordances and their inflictions on game play. The basic terminology used stems from Roger Caillois, Michail Bakhtin, James J. Gibson and Ulf Wilhelmsson respectively. The conclusion drawn in the paper is that the environment in games, free play and computer games serve as a playground and stage of purposeful objects that propose affordances and constraints upon the Game Ego manifestation within the game.
Article
From the Publisher:Is there a significant difference in attitude between immersion in a game and immersion in a movie or novel? What are the new possibilities for representation offered by the emerging technology of virtual reality? As Marie-Laure Ryan demonstrates in Narrative as Virtual Reality, the questions raised by new, interactive technologies have their precursors and echoes in pre-electronic literary and artistic traditions. Formerly a culture of immersive ideals—getting lost in a good book, for example—we are becoming, Ryan claims, a culture more concerned with interactivity. Approaching the idea of virtual reality as a metaphor for total art, Narrative as Virtual Reality applies the concepts of immersion and interactivity to develop a phenomenology of reading. Ryan's analysis encompasses both traditional literary narratives and the new textual genres made possible by the electronic revolution of the past few years, such as hypertext, electronic poetry, interactive movies and drama, digital installation art, and computer role-playing games. Interspersed among the book's chapters are several "interludes" that focus exclusively on either key literary texts that foreshadow what we now call "virtual reality," including those of Baudelaire, Huysmans, Ignatius de Loyola, Calvino, and science-fiction author Neal Stephenson, or recent efforts to produce interactive art forms, like the hypertext "novel" Twelve Blue, by Michael Joyce, and I'm Your Man, an interactive movie. As Ryan considers the fate of traditional narrative patterns in digital culture, she revisits one of the central issues in modern literary theory—the opposition between a presumably passive reading that is taken over by the world a text represents and an active, deconstructive reading that imaginatively participates in the text's creation. About the Author: Marie-Laure Ryan is an independent scholar and former software consultant. She is the author of Possible Worlds, Artificial Intelligence, and Narrative Theory and the editor of Cyberspace Textuality: Computer Technology and Literary Theory.