ArticlePDF Available

The intangible material of interactive art: agency, behavior and emergence


Abstract and Figures

This paper presents a conceptual analysis of some of the basic notions for the practice of interactive art and the relations among them. A sound understanding of these notions is essential for the creation of the aesthetics of artistically behaving systems. Interactivity, agency, behavior and emergence are presented as the building blocks of this practice, understanding that they are at least as important as the materials that physically instantiate the pieces and installations that constitute the body of interactive art. Interactivity is defined and confronted to the metaphor of the conversation and to the idea of designing interactive systems with artistic purposes. The notions of agency, behavior and performativity are reviewed through the reading of Andrew Pickering’s account for the ontology of Cybernetics and in relation to interactive art practices. Finally, the concept of emergence and Peter Cariani’s emergence-relative-to-a-model are presented as a theoretical framework with which the analysis and creation of unexpected and non pre-designed behaviors in interactive systems can be based.
Content may be subject to copyright.
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
Universitat Oberta de Catalunya
This paper presents a conceptual analysis of some of the basic notions for the practice of
interactive art and the relations among them. A sound understanding of these notions is
essential for the creation of the aesthetics of artistically behaving systems. Interactivity, agency,
behavior and emergence are presented as the building blocks of this practice, understanding
that they are at least as important as the materials that physically instantiate the pieces and
installations that constitute the body of interactive art. Interactivity is defined and confronted to
the metaphor of the conversation and to the idea of designing interactive systems with artistic
purposes. The notions of agency, behavior and performativity are reviewed through the reading
of Andrew Pickering’s account for the ontology of Cybernetics and in relation to interactive art
practices. Finally, the concept of emergence and Peter Cariani’s emergence-relative-to-a-model
are presented as a theoretical framework with which the analysis and creation of unexpected
and non pre-designed behaviors in interactive systems can be based.
interactivity, behavior, agency, aesthetics, emergence, design, interactive art
Submission date: August, 2015
Accepted date: October, 2015
Published in: November, 2015
The intangible material of
interactive art: agency, behavior
and emergence*
Joan Soler-Adillon
Universitat Pompeu Fabra
Joan Soler-Adillon
2015 by FUOC
* This work is part of the Project “Active Audiences and Journalism. Interactivity, Web Integration and Findability of Journalistic Information”. CSO2012-
39518-C04-02. National Plan for R+D+i, Spanish Ministry of Economy and Competitiveness.
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
E-JOURNAL ON ART, SCIENCE AND TECHNOLOGY The intangible material of interactive art…
Universitat Oberta de Catalunya
CC Joan Soler-Adillon
2015 by FUOC
El material intangible del arte interactivo: agencia,
comportamiento y emergencia
Este artículo presenta un análisis conceptual de algunas de las nociones básicas para la práctica
del arte interactivo y las relaciones entre estas. Un buen conocimiento de estas nociones es
fundamental para la creación de la estética de los sistemas con un comportamiento artístico.
La interactividad, la agencia, el comportamiento y la emergencia se presentan como los pilares
fundamentales de esta práctica, entendiendo que son como mínimo tan importantes como
los materiales que físicamente instancian los componentes e instalaciones que constituyen
el cuerpo del arte interactivo. La interactividad se define y confronta con la metáfora de la
conversación y la idea del diseño de sistemas interactivos con fines artísticos. Las nociones de
agencia, comportamiento y performatividad se revisan a través de la lectura de la interpretación
de Andrew Pickering para la ontología de la cibernética y en relación con las prácticas artísticas
interactivas. Por último, el concepto de emergencia y la emergencia relativa a un modelo de
Peter Cariani se presentan como un marco teorético en el que se puede basar el análisis y la
creación de comportamientos inesperados y no diseñados previamente en sistemas interactivos.
Palabras clave
interactividad, comportamiento, entidad, estética, emergencia, diseño, arte interactivo
1. Introduction
This paper presents a conceptual analysis of some of the basic
notions for the practice of interactive art and of its relation. A sound
understanding of these notions is essential for the creation of the
aesthetics of artistically behaving systems. Interactivity, agency,
behavior and emergence are presented as the building blocks of
this practice, understanding that they are at least as important as
the materials that physically instantiate the pieces and installations
that constitute its body of work.
Interactive art materializes in computer screens, installation
spaces, robotic devices and the like. The interactor (the one who
dialogues with the piece) relates with it through sensor interfaces:
keyboard, mouse, proximity sensors, customized buttons, etc. and
is addressed through actuators: loudspeakers, screens, motors,
etc. All of these materially instantiated elements are certainly a
part of the interactive experience, and it is through them that the
artistic experience is brought about. In addition, more often than
not, interactive art pieces are characterized through these elements:
categories such as, installation art or robotic art are examples
of this.
However, the choosing of components and the designed interfaces
is only a part of the creative endeavor of interactive art. Arguably, what
is essential to this practice is precisely that which is not physically
instantiated, even in robotic pieces. Agency and behavior, which
form the basis of performativity, are instead the defining and very
central aspects of interactive art. They are the intangible material of
interactive art. In this context, the concept of emergence appears
as an essential piece in order to understand the possibility of non
pre-defined behaviors to appear in such pieces.
2. Interactivity
There is a body of literature that has addressed the definition of
interactivity in relation to both general and computer media, mostly
since the mid-late 1990s. The general consensus is that there are
three main approaches to defining the concept from the media studies
point of view, depending on whether the defining efforts situate the
focus of the definition on the structure of media, on the user, or on
the process of communication that develops between them (Downes
and McMillan, 2000; Kiousis, 2002; Quiring, 2009; Mechant, 2012;
Weber, Behr and DeMartino, 2014).
A first group of authors is that which focuses on interactivity as
a characteristic of media (Durlak, 1987; Baecker and Buxton, 1988;
Wilson, 1994; Jensen, 1998; 1999; 2008; Sundar, 2004; Lee, Park
and Jin, 2006). The center of interest within these approaches is on
how media is structured in order to afford interactivity: what it offers
to the potential user and how, so that interaction can take place.
Interactivity is often presented in these cases as a continuum, and
different media are placed in different points along that continuum
(from less interactive to more interactive media). The types of activities
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
E-JOURNAL ON ART, SCIENCE AND TECHNOLOGY The intangible material of interactive art…
Universitat Oberta de Catalunya
CC Joan Soler-Adillon
2015 by FUOC
that each of these media offer to users is also read within these
parameters. On the other end, some authors place the focus of study
on the user, mainly around the concept of perceived interaction (Wu
1999; Liu and Shrum, 2002; Leiner and Quiring, 2008; Quiring 2009).
What is most important here is what interactivity means for the person
that uses the interactive system and how he or she relates to media
on its terms. Interactivity here is an information-based process that is
relevant in terms of individual perception (Newhagen, 2004). Finally,
there is a third group of authors that center their approach on the
communicational process that interactivity represents (Kiousis, 2002;
Rafaeli, 1988; Rafaeli and Sundweeks, 1997; Crawford, 2003; Noble
2009; Penny 2000; 2011; Green, 2010). The focus in these cases is
on the process that develops as the interactive system and the user
act and react to one another.
I have defined interactivity as “a series of related actions between
two or more agents where (1) at least one of them is an artificial
system that (2) processes its responses according to a behavior
specified by design and (3) takes into account some of the previous
actions executed by them” (Soler-Adillon, 2015). This definition aims
not at trying to solve the everlasting discussion on how to define the
term but at clarifying the concept in the context of interactive art.
The definition addresses the three main aspects that should be taken
into account when trying to frame such a complex concept within a
particular context:1 first, specify the system under analysis (ie, are
we analyzing the computer as a whole, for instance, or a particular
piece of software?); second, analyze what the system is capable of
doing (ie, how it responds to its environment); third, following one
of the classic definitions of the term (Rafaeli, 1988), discern if these
responses are merely reactive or are a case of interactivity in terms
of acknowledgement of already performed actions.
This definition corresponds to the third category mentioned above.
It aims at clarifying the communicational process of interactivity, and
it understands it in terms of design and functionality, not of subjective
experience by the user or interactor. The point of view for the analysis
is on how the systems are designed in order to respond to the users’
actions in a way that is read as interactive; on the process of dialogue
that is created between the behaving system and the human interactor
and not on the subjective experience of the latter.
Within this discussion, the metaphor of the conversation is
useful: the dialogue between the human and the machine that
inspired Cybernetics and that is the foundation of Human-Computer
Interaction. After all, the idea is that computers exchange information
with users, as the back and forth of messages advances. User does,
computer reacts, then user does again in response to this reaction,
and so on and so forth. This conversation can be understood in a
very simplistic manner, in terms of the control of systems where
the artificial system responds to the commands of a human user, or
with a more ambitious interpretation of the metaphor, which aims at
generating, through the interactive system, a series of meaningful
Gordon Pask’s Conversation Theory is an important effort to
theorize this (Pask, 1975; 1976; 1996), although it is a general
theory not specifically centered on man/machine conversation. Pask’s
writings on the subject find its context in his interest in learning
devices. Conversation is in his work presented – and interpreted by
latter authors – as an advanced form of interaction by systems that
have the ability to learn. More recently, Paul Pangaro has elaborated
on Conversation Theory in relation to social media and design
(Pangaro, 2009; Dubberly and Pangaro, 2009) and to relevant models
of interactivity (Dubberly, Pangaro and Haque, 2009). Following Pask,
Pangaro and colleagues understand that conversing systems are
a sophisticated assemblage of (second-order cybernetic) learning
systems that feed on and react to each other. This conversation is,
according to them, the most elaborate form of interaction.
In an effort to differentiate systems that are merely reactive
from those that afford interactivity, Dubberly, Panagaro and Haque
distinguish among static, dynamic-reactive, and dynamic-interactive
systems. The first are systems that “cannot act and thus have little
or no meaningful effect on their environment (a chair, for example)”.
Dynamic systems do have the ability to act and relate to their
environment. Within those, they distinguish between systems that react
(they have a linear relation between system activation and response)
and systems that interact. The categorization of interactive systems
is further elaborated and, as said above, they present conversation
as the most sophisticated form of interaction (Dubberly, Pangaro
and Haque, 2009). A similar categorization of passive, reactive and
interactive systems was presented in (Soler-Adillon, 2012).
3. Designing artistically behaving artifacts
When designing interactive artistic artifacts, the approach to
interactivity differs from that of designing functional and efficient
digital systems. In the latter, system’s interfaces should be easy
to use, robust, consistent with existing standards, etc. and users
should know what to expect from the system and how to achieve
their goals. The interactivity of interactive art, on the other hand,
needs not to be constrained by these principles, as it fundamentally
differs from functional interaction in its intentions. This dichotomy
has been characterized, from the point of view of the creator, as
the ‘design for efficiency’ and ‘design for motivation’ approaches
to creating interactive media (Ribas, 2001). Along the same lines,
1. Jens F. Jensen labeled interactivity as a multi-discursive concept (Jensen, 1998), following Tim O’Sullivan and colleagues’ definition, which states that this kind
of concepts are, precisely, those the meaning of which changes depending on the context in which they are used (O’Sullivan et al., 1994).
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
E-JOURNAL ON ART, SCIENCE AND TECHNOLOGY The intangible material of interactive art…
Universitat Oberta de Catalunya
CC Joan Soler-Adillon
2015 by FUOC
Simon Penny has elaborated the notion of poetic interaction, which is
at the heart of his theorization of the aesthetics of behavior (Penny,
2000; 2012; 2015). The basic idea is that when designing interactive
artistic artifacts, the type of interactivity that one is seeking to create
is very different from the one in the case of functional interaction.
Poetic interaction is the kind of interactivity that develops within
a system formed by an interactor and a behaving system that has
been designed with artistic purposes. The aesthetics of behavior
refer to the analysis of such type of devices and the experiences
they afford.
In respect to functional systems, aesthetically behaving systems
move towards unpredictability. Systems are neither necessarily easy
to control in terms of how the interface is presented, nor are they
necessarily predictable in their behavior. The idea is that they do so
without falling into what is perceived by the interactor as complete
random behaviors. Otherwise they become uninteresting in terms
of interactivity, since when that is the case all sense of dialog or
control is lost. Thus, artistic interaction is situated in the search for
a point of equilibrium between what is predictable and what is not.
According to Simon Penny it “should not be predictably instrumental,
but should generate behavior which exists in the liminal territory
between perceived predictability and perceived randomness, a zone
of surprise, of poetry” (Penny, 2011). And it is in this liminal territory
that it seeks to create conversation. Moving away from the linear
responsive model of interaction, as described by Dubberly, Pangaro
and Haque in (2009), conversation is here a more elaborated mode of
relation to the artificial system: “artistic interaction [can be conceived]
as an ongoing conversation between system and user rather than the
conventional (Pavlovian) stimulus and response model” (Penny, 2000).
Poetic interaction lives in the realm of art and experimentation. In
it, interactivity and behavior become aesthetic concerns and this has a
series of implications in this type of art creation and on the discourses
around it. In terms of understanding how interactivity is designed
and understood, this approach is to be interpreted as differentiated
from functional interaction. The objective of poetic interaction is to
experiment with the possibilities of interactivity. As noted by Krueger
when describing his early video tracking experiments, interactivity
becomes here a central aesthetic concern (Krueger, 1989) and, as
such, becomes a fundamental part of the artistic interest in these
pieces, although not necessarily in an exclusive manner.
The methods of designing artifacts for poetic interaction may
coincide with those of functional interaction, and some of the design
principles that apply to the latter apply to the former. However, since
the objectives are fundamentally different when designing such
systems, their interpretation has to be made from a different point
of view. These experiences and the processes that develop as the
interactor and the system act and react to each other shall be read
as any other aesthetic experience: a dance between expectations
and surprise where the interactive artifact seeks to engage the user
into exploring and experimenting with it; a dance of agency, to use
Andrew Pickering’s terms (1995). The artistic interactive system is
not conceived as a tool. Instead of being a system to help a user
perform a task efficiently, in this paradigm it exists on its own right,
as an agent – a behaving entity – that will drive the experience of
the visitor to the piece.
Thus, creating artistic interactive systems means designing
performative artifacts that will relate in a meaningful way to an
interactor. This means that there are mainly two focuses of interest
in terms of design: interface and behavior. Within the system formed by
interactor and performative artifact, the first is the point of connection
between them. In accordance, its design is crucial in establishing
this relation. The artist creating the artifact knows that the interactor
relates to his or her environment by doing things with it. According to
this approach, the idea is that he or she will acquire the necessary
knowledge to relate to the interactive object through this embodied
doing, not through an objective abstract examination of it. This means
that, as opposed to the design of a website from the point of view of
usability, for instance, the design approach must take into account
the actions of the interactor at all levels. The analysis of the artifact’s
behavior comes through the doing things with it in relating to its
interfaces (using the buttons, moving in front of it, etc.).
Behavior is the essential element in defining the artifact, and the
interfaces are the means of establishing the relation to it. All other
aspects of the piece are accessory to behavior. This doesn’t mean
they are not important, but external appearance, for instance, is here
one step below in aesthetic relevance. Indeed, interactive art pieces
can be analyzed aesthetically from the point of view of appearance
or in relation to the technology of the time. But according to the
approach proposed here, it is in terms of performativity, of behavior,
that are most relevant. Since they aim at relating to their visitors in
a meaningful way, and are designed to exploit such relations and to
provoke exploration of behavior on the part of the human interactor,
they are aesthetically relevant mostly in terms of interactive behavior.
4. Agency, behavior and performativity
An agent is a behaving entity. Andrew Pickering has described agency
as a basic building block of the influence that an entity can have on
its environment. Put simply, agency is about “doing things in the
world” (Pickering, 2002), a notion that he elaborates in detail in his
work (Pickering, 1995; 2002; 2010). Other accounts take agency
further into the ontological and metaphysical, and place it at the very
center of the configuration of matter and reality. According to Barad,
agency cannot be an attribute of things as it ontologically precedes
the things themselves (Barrad, 2007, p. 178). In any case, arguably in
terms of designing interactivity, it is sufficient to state that an agent
is a non-passive entity. That is, one that either responds reactively or
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
E-JOURNAL ON ART, SCIENCE AND TECHNOLOGY The intangible material of interactive art…
Universitat Oberta de Catalunya
CC Joan Soler-Adillon
2015 by FUOC
interactively to the actions that it is able to perceive from whomever
is relating to it.
The creation of agents with artistic purposes (agents as artworks)
has been studied by Simon Penny (2000). An agentic artwork,
according to Penny, is not just one that does things in responding
to its surroundings. It can be understood as a cultural actor in its
own right. These agents are designed and built within a cultural
tradition, and in a particular social context. And it is when they are
understood and read as such, and not abstracted from their social
and technological context, that they can be better understood.
Penny’s own Petit Mal is a paradigmatic example of this approach.
This robotic piece intentionally eschews all anthropomorphic or
zoomorphic appearance, but its behavior is infallibly attributed to
some sort of animal-like entity, as it hesitantly moves towards and
away from the visitors in a constant loop of recognition, retreat
and forgetfulness. The materialization of the piece is crucial, of
course. Penny acknowledges that it did have the right size so as
not to be scary, but at the same time not to be regarded as a sort
of pet robot (Penny, 2015). However, what defines Petit Mal as
much as its material instantiation is its behavior. Penny’s naming
it after the disease that informs how it responds to its environment
is no coincidence here. The defining trait of the piece is its agency;
one that performatively relates to the visitors in a neat example of
embodied interaction.
With these type of pieces, then, the aesthetic focus on interactivity
is a step away from the object and a step closer towards its relation
to the interactor. The work is neither just the behaving system nor its
behavior, but it too encompasses the situated actions of the visitor,
the embodied ‘contextualized doing’ of whomever interacts with the
system, much in accordance to the performative ontology that Andrew
Pickering describes as the basis of Cybernetics. As Simon Penny puts
it, in connection to ideas of embodied interaction and enaction, “the
lesson of performativity is that the doing of the action by the subject
in the context of the work is what constitutes the experience of the
work. It is less the destination, or chain of destinations, and more the
temporal process which constitutes the experience” (Penny, 2011).
As said, an important issue here is with what Pickering has
identified as the performative ontology of Cybernetics, which
is arguably extensible to interactive art practices in general. The
basic idea is that, in opposition to the representational ontology of
modern science, Cybernetics is based on a performative ontology: “a
decentered perspective that is concerned with agency – doing things
in the world – and with the emergent interplay of human and material
agency” (Pickering, 2002). Within this context, Cybernetics appears as
a paradigmatic discipline of the ‘performative idiom’. The discipline
as a whole “is all about this shift from epistemology to ontology, from
representation to performativity, agency and emergence, not in the
analysis of science but within the body of science itself” (Pickering,
Thus, the performative idiom, according to Pickering, is an ontology
in its own right, and a nonmodern one in the sense that it differentiates
itself from the reductionist approach of the scientific method (the
modern ontology). In contrast to Bruno Latour’s characterization of
modernity as being determined by the dualism of people and things
(Latour, 1993), the cybernetic approach, in which this frontier would
be blurred, “stages for us a nonmodern ontology, in which people and
things are not so different at all” (Pickering 2010, p. 18). As he has
explained (Pickering, 2008; 2010), this ontology allows Cybernetics
to propose an image of the world that is performative rather than
representational. He calls it the ‘ontological theater’, a nonmodern
understanding of the world and our relationship with it that implies
“a vision of knowledge as ‘part of’ performance rather than as an
external controller of it” (Pickering 2010, p. 25).
Here the idea of the black box is useful to understand how
this is brought into practice. According to Pickering, Cybernetics
proposes a theory of knowledge that is largely built up through a
performative relationship with what we can understand as black
boxes. Rather than being about control in a classical sense, “the
entire task of Cybernetics was to figure out how to get along in
a world that was not enframable, that could not be subjugated to
human designs – how to build machines and construct systems
that could adapt performatively to whatever happened to come
their way” (Pickering 2010, pp. 30-31).
In this context, he argues how Cybernetics assumes in fact an
ontology that, contrary to the reductionist approach, involves a certain
degree of unknowability, as it “tries to address the problematic of
Figure 1. Petit Mal. © Simon Penny.
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
E-JOURNAL ON ART, SCIENCE AND TECHNOLOGY The intangible material of interactive art…
Universitat Oberta de Catalunya
CC Joan Soler-Adillon
2015 by FUOC
getting along performatively with systems that can always surprise
us” (Pickering 2010, p. 23). These are what Stafford Beer labeled
as exceedingly complex systems. That is, systems which are,
unlike simple and merely complex systems, neither predictable nor
susceptible to be treated by the methods of modern science and
engineering. Exceedingly complex systems, like the interior of the
black boxes, are unknowable. They are too complex to be grasped
representationally, and they change over time, so that future behavior
cannot be anticipated trough current knowledge.
So if Cybernetics was about that, as Pickering argues, it was
not so much about control, as some literature has portrayed it, as it
was about the study of the conditions under which the interaction
among different parts of technology could result in these complex,
unanticipated patterns of behavior. Some of the devices built by the
cybernetic practitioners were clearly built in order to experiment in this
direction, and were in this respect anticipative of interactive art. Ross
Ashby’s Homeostat was a machine whose sole purpose was to exist
performatively. Once set in motion a homeostat unit would connect
to others and seek its own equilibrium, with no other goal than that.
Norbert Wiener defined it as the “brilliant idea of the unpurposeful
random mechanism which seeks for its own purpose through a
process of learning” and qualified it as “one of the great philosophical
contributions of the present day” (Wiener, 1989, p. 38). Grey Walters
tortoises, which behaved much similarly to the contemporary Roomba
robots – certainly not by chance –, also represent an example of
such practices. These avant-la-lettre bottom-up robotic devices were
capable of surprising its own designer, Walter, who described them
as having a ‘remarkably unpredictable’ behavior (Walter, 1950). As
Owen Holland has noted, they were a clear anticipation of Artificial Life
(Holland, 1997; 2003). These two pieces, along with Gordon Pask’s
Musicolour and Colloquy of Mobiles, suffice to exemplify the early
Cybernetic’s remarkable anticipation of interactive art.
5. Emergence
The element of surprise, of impossibility of a full anticipation that
Pickering attributes to Cybernetics, and that can be expanded to
interactive art in general, clearly resonates with the idea of emergence,
a concept that is often explained with the idea of a whole being ‘more’
than just the sum of its parts; of being irreducible to the mere addition
of its constituting elements. These explanations are usually articulated
in terms of different levels of complexity where parts or agents are the
constituents of a larger systemic level (the whole) in which emergent
phenomena appear. What is emergent are properties and behaviors
that can be observed at the system level, and they have a degree of
complexity that, as it is argued, cannot be accounted for by a simple
addition of the properties and behaviors of the parts.
In the academic literature the concept of emergence appears
as related to two different and not necessarily related phenomena:
self-organization and the appearance of novelty (Soler-Adillon and
Penny, 2014). Many authors present it as the result of multiple local
interactions among agents within a system that produce observable
patterns at the system level. These patterns are emergent phenomena
in the sense that they could not be understood, nor anticipated, through
the analysis of the elements and their behaviors in isolation (Langton,
1988; Holland, 1998; Bedau, 2008). For other authors, emergent
phenomena are related to fundamental novelty and, thus, to creativity.
For them, emergence is synonymous to the appearance of new
functions or behaviors in a known system (Steels, 1995; Cariani, 2012).
The typical examples of emergence are systems that exhibit
complex behaviors from a relatively small set of simple rules and
behaviors. Ant or termite colonies and their social complexity, the human
mind understood as a product of the interconnectivity of neurons in the
brain, chemical clocks, traffic jams or cellular automata are some of
the most cited examples. When related to novelty, emergence is often
used when referring to learning systems or adaptive devices.
Prominent among the authors that are concerned with emergence
as a generator of novelty and, thus, as creativity, Peter Cariani has
elaborated the theory known as emergence-relative-to-a-model (eg
Cariani 1992; 2009; 2011; 2012). Cariani articulates a discourse
that aims at identifying emergence as novelty in a given system
in a way that can be scientifically communicated. His approach is
concerned with how new functions can appear in systems or devices
that perceive and act on their environment. The basic idea is that
this newness can only be accounted for scientifically if, first, the
observer of the system defines its states and state-transitions by
creating a model of it. Once this is done, these observations are
used to make predictions on the futures states of the system. In
this context, emergence occurs whenever unanticipated behaviors,
states or functions appear: “emergence is the appearance of novel
entities that in one sense or another could not have been predicted
from what came before” (Cariani, 2009).
Figure 2. Walter’s tortoises. © Bristol Robotics Laboratory.
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
E-JOURNAL ON ART, SCIENCE AND TECHNOLOGY The intangible material of interactive art…
Universitat Oberta de Catalunya
CC Joan Soler-Adillon
2015 by FUOC
Cariani bases his modeling on three aspects: how the system
reads and acts on its environment (semantics); how it decides how
to act according to these readings (syntactics); and how it evaluates
the actions performed according to its goals (pragmatics). All these
actions are performed according to the basic building blocks of what
the system can operate with: the primitives, in Cariani’s terminology.
Within this framework, he identifies two ways in which emergence
can occur. The first is Combinatoric Emergence, which consists in the
appearance of new system functions through new combinations of the
primitives with which the system operates (eg, genetic algorithms).
The second is Creative Emergence, which is the appearance of new
functions trough the introduction of new primitives in the computations.
This second type of emergence, equivalent to the introduction of a new
sensory organ in an animal species through the course of evolution,
is extremely rare in artificial systems, and in fact Cariani identifies
in his literature only one case: Gordon Pask’s electrochemical ‘ear’
(Cariani, 1993; 2012). Despite these difficulties in the case of artificial
systems, however, Cariani opens a door to mixed computer-human
systems (ie, interactive systems) to be generators of this latter kind
of emergence (Cariani, 2009; 2012).
The relationship between emergence and interactive art can be
exemplified by the idea of the unpredictable black box (Soler-Adillon,
2011). In the prototypical engineering black box, the researcher can
figure out the mappings between inputs and outputs; i.e. he or she
must be able to develop a protocol to map what goes in and what
comes out of the box (Ashby, 1957). In contrast, unpredictable black
boxes would be devices that appear as black boxes even to their own
creator; systems in which the relation of inputs and outputs is not
fully foreseeable, and the inside of which is not only unknown but
unknowable. Thus, the only possible relation with these unpredictable
black boxes is the continuous performative act of interacting with
them. Not even the eventual opening of the box (or zooming in into
the system) would solve the problem of the unpredictability that they
would have up to some degree.
Ideally, the result of the unpredictable black box is not mere
randomness but something that, although coherent with the general
behaviors of the system, was not explicitly built in it by its designer –
something that lives in the liminal territory of poetic interaction and
can be qualified as emergent. As often happens in interactive art (eg
in generative art and especially in Artificial Live Art), it is sought by
the artist to create systems or processes that exceed his or her own
expectations. The idea is to do so not through some blind trial and
error, but through the creation of the conditions where emergent
phenomena can occur. This creative effort can be conceptualized with
the idea of emergence as generation of novelty (Soler-Adillon, 2015).
In terms of designing interactive artifacts potentially capable
of exhibiting emergent behavior, Cariani’s emergence-relative-to-
a-model is arguably the most solid framework to approach the
challenging task. Designing such devices means facing the apparent
paradox of designing something that is, in principle, impossible to
be designed. However, designing the possibilities for emergence to
appear is indeed possible, and it has been the goal of a series of
practitioners, mostly in relation to Artificial Life Art.
A successful example of such pieces is Ruairi Glynn’s Performative
Ecologies (Glynn, 2008), a series of robots that learn how to best
attract the gaze of the visitors to the installation space by ‘dancing’
in front of them. As they perform their moves, they calculate (through
facial recognition) how much of the attention of a visitor these
moves are capable of attracting. Then, using genetic algorithms, the
individual robots will create new dance moves based on those that
were successful within the group. With this, eventually the robots
should be capable of attracting more and more of the users’ attention
as they learn over time to do so. The new behaviors that the robots
learn and perform, and that were not pre-designed by the artist, can
be regarded as a case of emergence – particularly, of combinatoric
emergence according to Cariani’s categorization (Soler-Adillon, 2015).
Figure 3. Performative ecologies. © Ruairi Glynn.
6. Conclusions
Interactive art is about behaving entities, agents, performatively
relating to their environment, which includes the interactor (visitor,
user, etc.). The theorization and practices of early Cybernetics,
through what Pickering has labeled the performative idiom, can
be a good theoretical framework to understand how these ideas
relate to each other. The embodied doing of the interactor with the
interactive systems, and the connection to ideas of unpredictability
and emergence offer an interesting arena for the analysis and
creation of interactive art pieces. According to this, the guidelines
of design for emergent behavior are possible to produce. Cariani’s
emergence-relative-to-a-model and the theorization of emergence as
self-organization and as generation of novelty constitute a theoretical
framework within which this task can be undertaken.
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
E-JOURNAL ON ART, SCIENCE AND TECHNOLOGY The intangible material of interactive art…
Universitat Oberta de Catalunya
CC Joan Soler-Adillon
2015 by FUOC
However, any aesthetic discussion on this area requires a sound
conceptual analysis; a clarification of terms that helps avoid the
ambiguities of discussing interactivity and emergence without a clear
statement of what is exactly meant by such multi-discursive concepts.
In this respect, the proposed definition of interactivity, along with the
theorization of emergence as self-organization and as generation of
novelty (Soler-Adillon, 2015) represents an effort to contribute to the
elaboration of the aesthetics of artistically behaving systems.
ASHBY, W. R. (1957). An Introduction to Cybernetics. New York: Wiley.
BAECKER, R. M.; BUXTON, W. A. S. (1988). Readings in Human-
Computer Interaction: A Multidisciplinary Approach. San Mateo,
CA: Kaufmann.
BEDAU, M. A. (2008). “Downward Causation and Autonomy in Weak
Emergence”. In: M. A. BEDAU, P. HUMPHREYS (eds.). Emergence.
Contemporary Readings in Philosophy and Science. Cambridge,
Massachussetts: MIT Press, pp. 155–188. <http://dx.doi.
CARIANI, P. (1992). “Emergence and artificial life”. In: C. G. LANGTON,
C. TAYLOR, J. D. FARMER, S. RASMUSSEN (eds.). Artificial Life II
Reading, Massachussetts: Addison-Wesley Publishing Company,
pp. 775–797.
CARIANI, P. (1993). “To evolve an ear: epistemological implications
of Gordon Pask’s electrochemical devices”. Systems Research,
vol. 10, no. 3, pp. 19–33. <
CARIANI, P. (2009). “Strategies for Creating New Informational
Primitives in Minds and Machines”. Proceedings of the Dagstuhl
Seminar on Computational Creativity. Wadern, pp. 1–13.
CARIANI, P. (2011). “The Semiotics of Percept-Action Systems”.
International Journal of Signs and Semiotic Systems, vol. 1, no.
1, pp. 1–17. <>.
CARIANI, P. (2012). “Creating New Informational Primitives in Minds
and Machines”. In: J. MCCORMACK and M. D’INVERNO (eds.).
Computers and Creativity. New York: Springer, pp. 395–430.
CRAWFORD, C. (2003). Chris Crawford on Game Design. San Francisco:
New Riders.
DOWNES, E. J.; McMILLAN, S. J. (2000). “Defining Interactivity:
A Qualitative Identification of Key Dimensions”. New
Media & Society, vol. 2, no. 2, pp. 157–179. <http://doi.
DUBBERLY, H., & PANGARO, P. (2009). “What is conversation, and how
can we design for it?”. Interactions Magazine (ACM), vol. XVI, no. 4.
DUBBERLY, H.; PANGARO, P.; HA QUE, U. (2009). “What is interaction? Are
there different types?”. Interactions Magazine (ACM), vol. XVI, no. 1.
DURLAK, J. T. (1987). “A Typology for Interactive Media”. In: M. L.
MCLAUGHLIN (ed.). Communication Yearboook. Newbury Park,
CA: Sage, vol. 10, pp. 743–757.
GLYNN, R. (2008). “Conversational Environments Revisted”. 19th
European Meeting of Cybernetics & Systems Research. Viena.
GREEN, J. (2010). “Interactivity and Agency in Real Time Systems”.
Soft Borders Conference & Festival Proceedings. São Paulo, Brazil:
Centro Universitário Belas Artes de São Paulo. Pp. 1–5.
HOLLAND, J. (1998). Emergence. From Chaos to Order. New York:
Basic Books.
HOLLAND, O. (1997). “Grey Walter: The Pioneer of Real Artificial Life”.
In: C. G. LANGTON and K. SHIMOHARA (eds.). Artificial Life V:
Proceedings of the Fifth International Workshop on the Synthesis
and Simulation of Living Systems. Cambridge, Massachussetts:
MIT Press. Pp. 34–41.
HOLLAND, O. (2003). “The first biologically inspired robots”. Robotica, no. 21,
pp. 351–363. <>.
JENSEN, J. F. (1998). “Interactivity” Tracking a New Concept in Media
and Communication Studies”. Media and Communication Studies,
Nordicon R, no. 1, pp 185–204.
JENSEN, J. F. (1999). “The Concept of “Interactivity” in Interactive
Television”. In: J. F. JENSEN and C. TOSCAN (eds.). Interactive
Television: TV of the Future or the Future of TV?. Aalborg: Aalborg
Universitetsforlag, pp. 25–66.
KIOUSIS, S. (2002). “Interactivity: A concept explication”. New Media
& Society, vol. 4, no. 3, pp. 355–383. <
KRUEGER, M. (1991). Artificial Reality II. New York: Addison-Wesley
LANGTON, C. G. (1988). “Artificial life”. In: C. G. LANGTON (ed.).
Artificial Life: Proceedings of an Interdisciplinary Workshop on
the Synthesis and Simulation of Living Systems. Boston: Addison-
Wesley Publishing Company. Pp. 1–47. <
LATOUR, B. (1993). We Have Never Been Modern. Cambridge,
Massachussetts: Harvard University Press.
LEE, K. M.; PARK, N.; JIN, S.-A. (2006). “Narrative and interactivity in
computer games”. In: P. VORDERER and J. BRYANT (eds.). Playing
video games: Motives, responses, and consequences. Nahwah,
NJ: Erlbaum, pp. 259–274.
LEINER, D. J.; QUIRING, O. (2008). “What Interactivity Means to the User
Essential Insights into and a Scale for Perceived Interactivity”. Journal
of Computer-Mediated Communication, vol. 14, no. 1, pp. 127–
155. <>
LIU, Y.; SHRUM, L. J. (2002). “What Is Interactivity and Is It Always Such
a Good Thing? Implications of Definition, Person, and Situation
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
E-JOURNAL ON ART, SCIENCE AND TECHNOLOGY The intangible material of interactive art…
Universitat Oberta de Catalunya
CC Joan Soler-Adillon
2015 by FUOC
for the Influence of Interactivity on Advertising Effectiveness”.
Journal of Advertising, vol. 31, no. 4, pp. 53–64. <http://dx.doi.
MECHANT, P. (2012). “An illustrated framework for the analysis of
Web2.0 interactivity”. Contemporary Social Science, vol. 7, no. 3,
pp. 263–281. <>
NEWHAGEN, J. E. (2004). “Interactivity, Dynamic Symbol Processing,
and the Emergence of Content in Human Communication”. The
Information Society, vol. 20, no. 5, pp. 395–400. <http://doi.
NOBLE, J. (2009). Programming Interactivity. A Designer’s Guide
to Processing, Arduino, and OpenFramework. Sebastopol, CA:
O’Reilly Media.
PANGARO, P. (2009). “How Can I Put That? Applying Cybernetics to
“Conversational Media”. In: Proceedings of the Annual Meeting
of the American Society for Cybernetics, pp. 1–16.
PASK, G. (1975). Conversation, Cognition and Learning. A Cybernetic
Theory and Methodology. New York: Elsevier.
PASK, G. (1976). Conversation Theory: Applications in Education and
Epistemology. New York: Elsevier.
PASK, G. (1996). “Heinz von Foerster’s Self-Organisation, the
Progenitor of Conversation and Interaction Theories”. Systems
Research, vol. 13, no. 3, pp. 349–362. <http://dx.doi.
PENNY, S. (2000). “Agents as Artworks: and Agent Design as Artistic
Practice”. In: K. DAUTHENHAHN (ed.). Human Cognition and Social
Agent Technology. Amsterdam, Netherlands: John Benjamins
Publishing Company, pp. 395–414. <
agentdesign.html> <>
PENNY, S. (2011). “Towards a Performative Aesthetics of
Interactivity”. Fiberculture, no. 19, pp. 72–108. <http://nineteen.
PENNY, S. (forthcoming). “Between rocks and hard places – Robotics
and Art, Computationalism and Embodiment”. In: D. HERATH, C.
KROOS, and S. STELARC (eds.). Robots and Art: Exploring an
Unlikely Symbiosis. Springer.
PICKERING, A. (1995). The Mangle of Practice: Time, Agency, and
Science. Chicago: The University of Chicago Press. <http://dx.doi.
PICKERING, A. (2002). “Cybernetics and the Mangle: Ashby, Beer and
Pask”. Social Studies of Science, vol. 32, no. 3, pp. 413–437.
PICKERING, A. (2008). “Ontological Theatre. Gordon Pask, Cybernetics,
and the Arts”. Cybernetis And Human Knowing, vol. 14, no. 4, pp.
43–57. <>
PICKERING, A. (2010). The Cybernetic Brain. Sketches of Another
Future. Chicago: The University of Chicago Press.
QUIRING, O. (2009). “What do users associate with “interactivity”?: A
qualitative study on user schemata”. New Media & Society, vol. 11,
no. 6, pp. 899–920. <>
RAFAELI, S. (1988). “Interactivity: From new media to communication”.
Sage Annual Review of Communication Research: Advancing
Communication Science, no. 16, pp. 110–134.
RAFAELI, S., SUNDWEEKS, F. (1997). “Networked Interactivity”. Journal
of Computer-Mediated Communication, vol. 2, no. 4, pp. 0–0.
RIBAS, J. I. (2001). “Difusió cultural i comunicació audiovisual
interactiva”. Temes de Disseny, no. 18, pp. 170–204. <http://>
SOLER-ADILLON, J. (2011). “Creating Black Boxes: Emergence
and Interactive Art”. In: Proceedings of the 17th International
Conference on Electronic Art. <
SOLER-ADILLON, J. (2012). Principios de diseño de interacción
para sistemas interactivos. <
SOLER-ADILLON, J. (2015). Emergence as Self-Organization and as
Generation of Novelty. A Framework for Understanding Emergence
in the Context of Interactive Art. Universitat Pompeu Fabra.
SOLER-ADILLON, J.; PENNY, S. (2014). Self-organization and novelty:
pre-configurations of emergence in early British Cybernetics. In:
Norbert Wiener in the 21st Century (21CW), 2014 IEEE Conference
on. Boston, MA: IEEE, pp. 1-8. <
STEELS, L. (1995). “The Artificial Life Roots of Artificial Intelligence”.
In: C. G. LANGTON (ed.). Artificial life. An Overview1. Cambridge,
Massachussetts: MIT Press, pp. 75-110. <http://dx.doi.
SUNDAR, S. S. (2004). “Theorizing Interactivity’s Effects”.
The Information Society, vol. 20, no. 5, pp. 385–389.
WALTER, W. G. (1950). “An Imitation of Life”. Scientific American,
vol. 182, no. 5, pp. 42–45. <
WEBER, R., BEHR, K.-M., DEMARTINO, C. (2014). “Measuring
Interactivity in Video Games”. Communication Methods and
Measures, vol. 8, no. 2, pp. 79–115. <
WIENER, N. (1989). The Human Use of Human Beings. Cybernetics
and Society. London: Free Association Books.
WILSON, S. (1994). “The aestheteics and practice of designing
interactive computer events”. SIGGRAPH ’94 Visual Proceedings.
ACM Press.
WU, G. (1999). “Perceived interactivity and attitude toward web sites”.
Proceedings of the 1999 Conference of the American Academy
of Advertising.
A UOC scientific e-journalArtnodes, no. 16 (2015) I ISSN 1695-5951
2015 by FUOC
2015 by FUOC
Sergio Martínez Luna
FUOC, 2015
Universitat Oberta de Catalunya
The intangible material of interactive art…
Joan Soler-Adillon
2015 by FUOC
Joan Soler-Adillon
Universitat Pompeu Fabra
Universitat Pompeu Fabra
Campus de la Comunicació-Poblenou
Roc Boronat, 138 Desp. 52.803
08018 Barcelona
Graduate in Philosophy from the Universitat Autònoma of
Barcelona, Master’s degree from New York University’s Interactive
Telecommunications Program (Tisch School of the Arts) and Official
Master’s Degree in Cognitive Systems and Interactive Media from
Universitat Pompeu Fabra. He holds a PhD in Social Communication from
Universitat Pompeu Fabra, awarded for his thesis entitled “Emergence
as Self-Organization and as Generation of Novelty. A Framework for
Understanding Emergence in the Context of Interactive Art”. At UPF he is
a lecturer and researcher in the Interactive Communication Area of the
Communication Department since 2006 and member of the DIGIDOC
research group since 2009. He has directed the Digital Video Graduate
Program, and lectured at the Digital Art’s Master and Online Digital
Documentation Master. He has also lectured in the Computer Science
undergraduate degree and in Universidad San Francisco de Quito, in
Ecuador. He has participated in interactive installation, performance and
video projects, which have been shown in New York and Barcelona, as
well as several workshops and conferences. His research is centered
on the concept of emergence and its relation to interactive media and
art, and also on the analysis and creation of interactive documentaries
and experimental interactive work.
Recommended citation
SOLER-ADILLON, JOAN (2015). “The intangible material of interactive art: agency, behavior and
emergence”. In: Pau ALSINA and Ana RODRÍGUEZ GRANELL (coord.). “Art Matters II”. Artnodes.
no. 16, pp. 43-52. UOC [Accessed: dd/mm/yy].
This article is – unless indicated otherwise – covered by the Creative Commons Spain Attribution 3.0
licence. You may copy, distribute, transmit and adapt the work, provided you attribute it (authorship,
journal name, publisher) in the manner specified by the author(s) or licensor(s). The full text of the
licence can be consulted here:
... Additionally, emergence refers to how these organizational processes can appear as surprising to an observer (i.e. they are new in respect to some frame of reference). It has been argued that, in fact, emergence refers to two separate (but combinable) ideas: selforganization and novelty (Nagel, 1961;Soler-Adillon and Penny, 2014;Soler-Adillon, 2015a, Soler-Adillon, 2015b. ...
... In the academic literature, the concept of emergence appears as related to two different and not necessarily related phenomena: selforganization and the appearance of novelty (Soler-Adillon and Penny, 2014;Soler-Adillon, 2015a). The first refers to the observable systemic patterns that result from multiple local interactions among agents within a system. ...
Article ------- Emergence has been suggested as one of two basic game structures that distinguish between open and closed gaming paradigms: games of emergence and games of progression. The first category relates to most traditional games and also some videogames. It encompasses games that, starting from a relatively small set of elements and rules, can have a multiplicity of possible outcomes. The second one refers to games in which the player needs to complete a series of predefined tasks in order to advance. This is the case of some traditional games, such as treasure hunt, but also of the majority of existing videogames in which solving puzzles and quests, or overcoming obstacles organized in screens or levels are the basis for progressing in them. This article argues that the use of emergence in games research falls short in capturing the full potential of such an intricate concept. Clarifying the term and its use can help expand this theorization of games, separating what refers to -- or is a result of -- an open space of possibilities and what is indeed emergence in a strict sense. Distinguishing the notions of open and emergent widens the discourse on game design. It will allow game scholars to account for self-organizing phenomena in digital games on one hand, and for the appearance of emergent novelty on the other, both during the design process and in respect to the model player of the game.
... La naturaleza participativa del nuevo arte activista y comunitario se ve enormemente potenciada por Internet. La emergencia del llamado "net art" se da en el contexto de una dinámica más amplia que pone en jaque las estructuras y jerarquías tradicionales de la producción cultural convencional (Yúdice, 2002, Soler-Adillon, 2015. Las plataformas y modos de organización y distribución cultural que ha posibilitado internet podrían interpretarse como una concreción contemporánea del deseo benjaminiano de contar con los "medios de producción" para dotarlos de una "función organizadora". ...
Full-text available
La educación para el compromiso social es sin duda un factor esencial para una sociedad deliberativa y democrática. Las aportaciones de la pedagogía y las ciencias sociales a tal cometido son conocidas y objeto de un amplio y rico debate. Sin embargo, también las artes cuentan con un enorme potencial frete al reto de educar para el compromiso. En este artículo analizaremos las aportaciones del arte a la idea de compromiso en la cultura contemporánea. Aportaciones que tienen hoy un amplio eco en la creatividad visual anónima que se difunde en las redes, y que es enormemente popular entre los más jóvenes. Como consecuencia, consideraremos las propuestas recientes de educación artística orientada hacia el contenido social y/o activista y recapitularemos algunas de las demandas que estos nuevos posicionamientos proyectan sobre la formación del profesorado. En la última parte de este texto, daremos cuenta de una investigación sobre un caso de estudio en el contexto de la formación inicial de los maestros en que tratamos de sopesar el sentido y alcance que tuvo para los participantes la realización de una experiencia de educación artística orientada al compromiso social. Constatamos que utilizaron de forma efectiva las artes como vehículo para la indagación y la consciencia social, superando concepciones inerciales de la educación artística, y que relacionaron la experiencia con aspectos relevantes de la formación docente
... Interactivity is explained by the scholar and artist Joan Soler-Adillon as an artificial situation in which one or more agents carry out a series of actions in order to gain responses to their specific behavior, which is synchronized with their previous actions in a space. 53 Interactivity itself might not be seen as performance, but could be said to facilitate performance when it interacts with one or more bodies. ...
Full-text available
This research analyzes performance art that uses biometric data, based on two concept perspectives – inhuman interconnections and transcorporeality – applied to examples of European performance art from Lithuania, Finland, Poland, and Denmark. The term performance art theoretically refers to all art that involves the human body, human biometric data, inhuman interconnections, transcorporeality, and liminal space. This study examines the differences between wide-scope interactive art and design and performance art involving biometric data created through the application of recent developments in consumer technology for live events. This research examined three case projects through the method of autoethnography. The data presented in the article was either collected during the author’s performance art events or at performances the author was attending as a visitor. The cases are analyzed by means of qualitative data analysis, utilizing terms representing human biometric data and interactivity adopted from research within the fields of interactive art and interactive design. The study examines the combination of biometric data and art to explain the phenomenon of humans meeting technology, revealed through data collected from a body and transmitted to an audience during a performance art event. The conclusion revisits the key terms – performance art, biometric data, inhuman interconnections and transcorporeality – as applied to artistic practices, where performance art and biometric data meet.
... Clearly, visitors are an essential part of Ancestral Symbol, as much as in many present-day interactive installations [7,8]. Besides, it is important to mention that an interactive installation is not an incomplete work just because it is interactive [9]. ...
Full-text available
This text explains how the visitors’ unpredictable movements are transformed into a consistent narrative musical form by using electronic devices with no central software to control them in the interactive installation Ancestral Symbol, created by Edson Zampronha. The installation uses a paleolithic cave sign as the basis for the connection of sounds and images through the visitors’ movements in the installation room, thus connecting art, archaeology and interactivity. This text explains the technology involved, the sounds and visual materials used, how the visual elements work as loudspeakers, and the strategic spatial distribution of all the elements to organize the installation.
... En casos extremos, como en juegos en línea de multijugador, u obras colaborativas, el objeto incorpora potencial de dinámicas sociales más allá de lo que los autores hubiesen anticipado. En consecuencia, puede afirmase que la indeterminación del texto digital permite la incorporación de la noción de emergencia como potencialidad (Soler-Adillon, 2015). Ciertamente, en los mundos virtuales y en el análisis de complejas obras transmedia, para algunos autores la lectura detallada a menudo se asemeja a las prácticas de la auto-etnografía y la observación participante. ...
Full-text available
[ENGLISH BELOW] //// Desde finales de la década de 1990, autores de ámbitos diversos han producido obras en las que el diálogo interactivo, la multimedialidad y la participación del usuario han permitido explorar las posibilidades narrativas del ecosistema digital. Se trata de textos que ahondan, entre otros, en aspectos como la hipertextualidad, la hipermedialidad y la comunicación bidireccional. Trabajos en los que al usuario se le propone una lectura activa, a menudo complementada con la aportación, creación o co-creación de contenido. La lectura y comprensión de estos textos precisa de procesos de aprendizaje específicos que, lamentablemente, no se recogen en los programas de alfabetización mediática de la enseñanza reglada, ni en la escuela primaria ni en la secundaria. En este artículo se propone una reflexión sobre la lectura detallada de textos audiovisuales interactivos, y se proponen estrategias y herramientas para su comprensión y análisis. *** //// ENGLISH //// Since the end of the 1990s, authors from diverse areas have produced works in which interactive dialogues, multimedia and user participation have allowed us to explore the possibilities that digital media and networks offer. These texts deepen into aspects such as hypertextuality, hypermediality and bidirectional communication. In these works, the user is offered an active reading, often implemented with the possibility of content contribution, creation or co-creation. The reading and comprehension of these texts requires specific learning processes that, unfortunately, are not included in the media literacy programs of regulated training, nor in primary schools and high schools. This article invites a reflection on close reading of interactive digital media texts and proposes strategies and tools for their understanding and analysis.
Full-text available A música e a tecnologia têm experienciado, desde há séculos, uma coabitação e um desenvolvimento mútuo e constante na nossa sociedade, o qual tem possibilitado aos músicos instrumentistas, não só a oportunidade de evoluírem os seus instrumentos, como também a sua performance. No que concerne à música em específico, Irving Godt (2005) afirma que não há conhecimento de sociedade alguma que tenha existido, ou que exista, sem música. Tal afirmação espelha bem a importância desta nas sociedades, desempenhando um papel de relevância sobre vários aspetos, sejam eles de cariz artístico, festivo, fúnebre, ou noutros enquadramentos. Na realidade, a música e a tecnologia vivem numa simbiose e com uma dinâmica particular, na qual a tecnologia fornece meios para que a música evolua, mas, por sua vez, a música também vai criando necessidades e fomentando o desenvolvimento tecnológico. Inerentemente a esta evolução conjunta e consoante os espaços temporais em que acontecem, o papel dos músicos vai-se alterando espontaneamente, por adaptação. É uma destas alterações, num destes timings, que é aqui investigada, nomeadamente a influência que a média-arte digital exerceu/exerce na música, sob o ponto de vista do músico instrumentista erudito de tradição ocidental escrita e, em particular, o ponto de vista do próprio autor enquanto clarinetista. Este, que vinha já observando ampliações nas suas responsabilidades enquanto intérprete - mais propriamente por intermédio de adições cénicas, físicas, acústicas, entre outras – viu-as serem aumentadas de forma mais acentuada através da junção com os canais média digitais. Sumariando, o presente documento é o resultado investigativo da forma e das implicações que este desenvolvimento relacional trouxe aos instrumentistas. Para o efeito, foi observada e identificada a performance musical instrumental tradicional, a sua evolução mediante diversos pontos de vista entre os agentes envolvidos, a evolução notacional, a evolução conceptual, a adição tecnológica, entre outros aspetos, com o objetivo de identificar e diferenciar influências. Foram também elaborados três artefactos que explorassem ambas as áreas – média-arte digital e música –, aproveitando a formação e experiência profissional do autor enquanto instrumentista e como forma geradora de dados para investigação proposta.
This article presents In Pieces VR, a VR-based artwork and experimental documentary on political prison, and discusses its main design challenges, goals, and creative approach. This project aims at creating a documentary and artistic experience that departs from conventional immersive journalism by presenting to its viewers a story made out of very small narrative units, and with subjects presented in the form of virtual sculptures devoid of any specific identity. The idea is that by leaving much of the making sense left open, viewers will have to fill in the gaps. The working hypothesis is that this will help create a different emotional, intellectual and political connection with the piece than that a conventional documentary would achieve, particularly to an audience unfamiliar with, or even politically alien to its specific context. Public exhibition of the work and user evaluation showed that the piece was successful in creating such connections.
This paper presents the research around the concept and idea of agency in the context of human-computer interaction (HCI) design. The notion of agency is made up of several elements and therefore should not be considered as a rigid concept. This work proposes a framework for understanding agency in HCI that considers the main elements that must be taken into account in the design and evaluation of interactive artifacts. The framework is composed of four main elements: mental models, affordances, reciprocity and responsiveness. Two elements related to the cognitive processes that allow people to perceive and make decisions and two elements related to the response processes generated by technological devices to people’s actions. Each of these four main elements is specified by three attributes organized in levels. The proposed framework has been evaluated through focus groups, involving design practitioners. The results show that the concept of agency not well known in interaction design practice and that a framework can facilitate its implementation in design processes and projects.
This article is devoted to the problem of novelty in embedded art. Exciting possibilities of microelectronics for visual art are considered. Implementations of available microcontrollers, sensors and actuators are given. A new kind of interactions between spectators and artworks with embedded electronics was investigated. Special attention is paid to the effect of emergence, which is generated by this interaction. Computational artwork is being considered in a new unconventional context. An aesthetic difference between the deterministic and nondeterministic algorithms was shown. The combination of acrylic paintings with light-emitting diodes (LEDs) was systematically studied to give useful recommendations for an artistic community. Produced artifacts were demonstrated at exhibitions and were used to teach students at the university and VHS Freiburg, Germany. The results showed a great public interest in embedded electronic systems and a tremendous theoretical and practical perspective for the permanently developing contemporary art.
Full-text available
Contemporary classics on the the major approaches to emergence found in contemporary philosophy and science, with chapters by such prominent scholars as John Searle, Stephen Weinberg, William Wimsatt, Thomas Schelling, Jaegwon Kim, Daniel Dennett, Herbert Simon, Stephen Wolfram, Jerry Fodor, Philip Anderson, David Chalmers, and others. Emergence, largely ignored just thirty years ago, has become one of the liveliest areas of research in both philosophy and science. Fueled by advances in complexity theory, artificial life, physics, psychology, sociology, and biology and by the parallel development of new conceptual tools in philosophy, the idea of emergence offers a way to understand a wide variety of complex phenomena in ways that are intriguingly different from more traditional approaches. This reader collects for the first time in one easily accessible place classic writings on emergence from contemporary philosophy and science. The chapters, by such prominent scholars as John Searle, Stephen Weinberg, William Wimsatt, Thomas Schelling, Jaegwon Kim, Robert Laughlin, Daniel Dennett, Herbert Simon, Stephen Wolfram, Jerry Fodor, Philip Anderson, and David Chalmers, cover the major approaches to emergence. Each of the three sections ("Philosophical Perspectives," "Scientific Perspectives," and "Background and Polemics") begins with an introduction putting the chapters into context and posing key questions for further exploration. A bibliography lists more specialized material, and an associated website ( links to downloadable software and to other sites and publications about emergence. ContributorsP. W. Anderson, Andrew Assad, Nils A. Baas, Mark A. Bedau, Mathieu S. Capcarrère, David Chalmers, James P. Crutchfield, Daniel C. Dennett, J. Doyne Farmer, Jerry Fodor, Carl Hempel, Paul Humphreys, Jaegwon Kim, Robert B. Laughlin, Bernd Mayer, Brian P. McLaughlin, Ernest Nagel, Martin Nillson, Paul Oppenheim, Norman H. Packard, David Pines, Steen Rasmussen, Edmund M. A. Ronald, Thomas Schelling, John Searle, Robert S. Shaw, Herbert Simon, Moshe Sipper, Stephen Weinberg, William Wimsatt, and Stephen Wolfram Bradford Books imprint
Full-text available
forthcoming in Robots and Art: Exploring an Unlikely Symbiosis Eds: Damith Herath Christian Kroos and Stelarc Pub: Springer.
The subject of study of this investigation is the concept of emergence and its viability as a driving force for the creation of interactive artwork. Emergence appears in the literature as related to self-organization and novelty. For many authors it is the result of multiple local interactions among agents within a system, which generate phenomena that could not be understood, nor anticipated, through the analysis of the elements and their behaviors in isolation. For others, emergent phenomena are related to fundamental novelty and, thus, to creativity. These two formulations of emergence are reviewed in this thesis in the context of interactive art and communication. The focus of the study is on the interactive works of the artistic field known as Artificial Life Art. After reviewing the foundational concepts and proposing an approach to both emergence and interactivity, an analytical framework is presented in order to assess the presence of emergence in interactive systems, which in turn shall serve as a first step to designing systems that aim at generating interactive emergent behaviors.
This paper places contemporary modalities of digital interaction in an historical context of sixty years of intersections between technological development and artistic experimentation. Specific technological developments are identified as context-defining historical markers and specific works are discussed as exemplars of significant milestones in the engineering and the aesthetics of interaction. The shortage of theorisation of non-instrumental interaction is lamented. The process of naturalisation to increasingly sophisticated digital tools and appliances in the current period of ubiquitous computing is noted. A number of theoretical issues are drawn out and discussed in terms of cognitive and sensorimotor dynamics. Woven through the discussion is the proposal that a synthesis of performance theory and neuro-cognitive studies might provide a basis for a performative ontology around which an aesthetics of interaction might be constructed. As the paper progresses a theoretical framework for an ontologically performative aesthetics of interaction and ubiquity is formulated.
Human Cognition and Social Agent Technology is written for readers who are curious about what human (social) cognition is, and whether and how advanced software programs or robots can become social agents. Topics addressed in 16 peer-reviewed chapters by researchers at the forefront of agent research include: Narrative intelligence and implementations of story-telling systems, socially situated avatars and conscious software agents, cognitive architectures for socially intelligent agents, agents with emotions, design issues for interactive systems, artificial life agents, contributions to agent design from artistic practice, and a Cognitive Technology view on living with socially intelligent agents. The book addresses both software and robotic agents. On the one hand justice is done to the scientific and technical aspects, and on the other hand the reader will learn about pioneering technological developments which are necessary for a public discourse and critical evaluation on where social agent technology is leading us and how such a development can be shaped in order to meet the social, cultural and cognitive needs of humans. The book is suitable for students, researchers, and everyone interested in this emerging and quickly growing field, it does not require any specialist background knowledge. (Series B)