Bau, Centre Universitari de Disseny
Teresa Martínez Figuerola
Jorge Luis Marzo
Cantidad de ejemplares 150
Diseño y maquetación:
ISBN (Ed. Impresa): 978-84-617-5132-7
Depósito Legal: DL B 20848-2016
Copyrights de la publicación:
Bau, Centro Universitario de Diseño
Copyrights de los textos:
Todos los autores
El contenido de los artículos es de absoluta responsabilidad de los autores.
Interfacing with the unexpected: notes towards a design
theory for emergent interaction
is paper presents a rst approach to the design of emergent behaviors
and interfaces in the context of interactive arts. e main conceptual issues
involved are presented, along with the metadesign challenge implied in
designing something that, by denition, cannot be designed per se. Key
examples are discussed and used as examples of successful attempts to gene-
rate emergence in interactive art. Whilst the issue remains largely unsolved,
these examples and the proposed theoretical framework are presented as
a starting point towards the design of interactivity that can be regarded as
Keywords: Emergence, interactivity, design, behavior, interface
INTERFACE POLITICS. 1st International Confrence
1. Introduction: Emergence and Digital Art
Emergence is often referred to as the idea of the whole being more than
the sum of its parts, or of being order out of chaos. In this sense, it refers to
the idea of complex behaviors arising from an aggregate of relatively simple
elements with relatively simple behaviors (Langton, 1988; Holland, 1998;
Bedau, 2008). In other accounts, complementary of those in occasions, it
is related to fundamental novelty (i.e. the appearance of novel and unex-
pected behaviors) (Cariani, 1992; 2012; Steels, 1992). It can be argued that
emergence refers, in fact, to either (or both) a self-organization process or
the appearance of novelty (Nagel, 1961; Soler-Adillon and Penny, 2014;
e concept found a fertile ground in Complexity Sciences, where it
became a central concern, especially in the eld of Articial Life (ALife), a
discipline initiated by Christopher Langton in 1987 (Waldrop, 1994). It was
through the use of ALife techniques and themes in art –which constituted the
discipline known as ALife Art– that emergence permeated digital art mostly as
related to the idea of the search for the unexpected result (the surprise).
e early years of interactive and digital art were coincidental in time
with the early years of Complexity Sciences. e personal computer faci-
litated the former while computer simulations enabled the latter. When
experimenting with them, computers were capable of producing unexpected
results and computational emergence became one of the ways to explain
these phenomena. In a nutshell, when a set of relatively simple rules produ-
ced complex and varied results, these were interpreted as emergence.
Conway’s Game of Life was an iconic example of his. is cellular auto-
maton brilliantly shows how a simple set of rules can generate surprisingly
complex dynamic results. Another iconic example is Craig Reynolds’ ‘boids’,
a simulation of a ock of birds or school of sh using three very simple
and local rules that generate a group behavior strikingly ressemblant of the
modeled system. Finally, genetic algorithms, developed by John Holland,
constituted a very fertile ground of experimentation.
In any case, in the formation years of interactive art, emergence became
an idea linked to the possibility of escaping the pre-specication that
computer programming forces the artists into. e idea was that, instead of
predening behaviors, one should be able create agents that would exhibit
interactive behavior that was emergent. It was presented in contraposition
to the reductionist approach, which implies that the complex can always be
accounted for by breaking it apart and analyzing its (simpler) parts:
[Reductionism] is premised on the assumption that to understand
a complex object, one breaks it into component parts and examines
those parts in controlled settings, then adds the results of those
examinations together. e basic principle of emergence is that
organization (behavior/order/meaning) can arise from the agglo
meration of small component units which do not individually
exhibit those characteristics. Emergent order implies that the
whole is indeed greater than the sum of its parts, that higher level
behaviors cannot be disassembled into their component lower level
building blocks. (Penny, 1996).
2. The Design Challenge
Emergent interactive behavior, as formulated by Penny in (1996), becomes
in this context a design endeavor, and quite a challenging one. If taken lite-
rally, the idea is to be able to create artistic devices that behave emergently.
Accordingly to the formulation proposed here, this means either generating
a self-organization process or generating novel and unexpected behaviors.
But emergence, by denition, cannot be designed per se. us, from the
design point of view, designing emergent behavior is a paradox, at least up
to some degree. If we focus on emergence as self-organization, we cannot
design the emergent behavior but only the agents and their local behaviors
and interactions. It is through these local interactions that something emer-
gent might appear. Similarly, emergence as novelty cannot be designed: if
we design a behavior, it will not be unpredictable in the sense that emergent
novelty is. Of course here the ‘novel to whom’ question arises. Cariani’s
emergence-relative-to-a-model is aimed at answering this and also the ‘novel
in respect to what exactly’ question (Cariani, 1992; 2011; 2012).
e task of designing emergent interactive behavior, therefore, becomes
a metadesign eort. at is, we can only design the conditions for emer-
gence to appear, but not the emergent phenomena itself. When aiming to
create self-organizing phenomena, as said, this means creating agents that
have local rules and interact locally. No central control or collective goal can
exist in order for it to be a genuine example of self-organization. e above
mentioned boids, or the robotic installation Sympathetic Sentience (see
below) are successful examples of this, although the rst is not interactive in
the sense that it doesn’t respond to an interactor in any way. On the other
hand, in order to generate novelty, the interactive systems and devices have
to be open in a way that can facilitate this change at one point or another.
Cariani’s emergence-relative-to-a-model is a theoretical framework that
INTERFACE POLITICS. 1st International Confrence
thoroughly explains how this can be described and what types of emergent
novelty can be assessed. Elaborating on it, the SOE/GNO Framework (So-
ler-Adillon, 2015) is an attempt to put this framework in practice in terms
of analyzing particular systems, under the umbrella of the dual understan-
ding of emergence mentioned above. Arguably, this latter framework can,
in turn, serve as the basis for the elaboration of the guidelines for designing
interactive behavior that is emergent.
3. Emergent Interactivity
When designing interactive artistic artifacts, the approach to interactivity is
not that of functional interaction, but what can be labeled as poetic interac-
tion (Penny, 2011). Within this paradigm, the fundamentals of traditional
Human-Computer Interaction, Design and Digital Creativity conate in an
eort to create communicational interactive experiences that go beyond the
instrumental (Soler-Adillon et al., 2016).
In this context, there are two main aspects in which the designer needs
to focus, above anything else: behavior and interface. e rst refers to
how the artifact responds to its environment and to the interactor, and the
second is the means of establishing such relationship. Elaborating on this
idea, this section looks at examples of emergence in both.
3.1 Emergent behavior
Sympathetic Sentience in an interactive sound installation by Simon Penny
rst presented in 1995. It consists of a group of twelve robots that com-
municate to one another sequentially. Each of the robots is a relatively
simple electronic device, capable of producing one chirp each minute with
a particular rhythm and to pass along to the next an infrared signal indica-
ting it. en, this receiving unit combines its own rhythm with the received
information, and passes the new rhythmic pattern along. As the process
advances, the complexity of the rhythms increases as they cycle around
the group. e system is self-organizing or, in Penny’s words, self-gover-
ning (Penny, 2000). e sound pattern that the visitors to the installation
perceive is neither predesigned, nor directed by any one of the robots or an
external entity. As the robots communicate locally –each one to the next
in circle– the overall process is formed. In this respect, it is an example of
emergence as self-organization.
Interactivity is implemented in the piece in an unconventional way.
After an initial build-up period, the system works on its own as rhythmic
patterns continually evolve. It never becomes fully saturated, nor does it
become fully silent if it is not interfered with. However, this equilibrium
can be disrupted, often unintentionally, by the presence of the installation
visitors whom, by moving around the space, interrupt the infrared transmis-
sions of the units. If these interruptions are short in time, lapses of silence
will inltrate the rhythm patterns of Sympathetic Sentience. If long, the
whole system can be forced into complete silence. When this happens, once
the interruption of transmissions ends, the build-up process will have to
start all over again (Penny, 2008).
However, a point can be made here to whether or not in this piece we
have an actual example of emergent interactive behavior or, rather, the
juxtaposition of an interactive behavior (the group symphony) with the in-
teractivity towards the visitor (his or her ability to interrupt the symphony).
Two more examples should be of use here in making this point: Ruairi
Glynn’s Performative Ecologies (2008) and my own Digital Babylon (2005).
Since in terms of analyzing emergent interactive behavior they both use the
same technique and oer comparable results, these two pieces are analyzed
here together. Details on the pieces can be found in (Glynn 2008) and in
(Soler-Adillon, 2011) respectively.
In short, Digital Babylon presents the user with a virtual ecosystem in
which he or she can interact with one of the species. is species evolves
through genetic algorithms, and one of the characteristics that change over
time is the likelihood that its individuals will pay attention to the interactor
or not. As a result, if the interactor helps the friendly individuals survive,
the species as a whole becomes friendlier. If he or she harms them, they will
be less friendly for future interactions. In Performative Ecologies, a series of
robots dance in front of the users and check, through the visitor’s gaze, what
dance moves are capable of maintaining the attention. en, the successful
moves are recombined trough genetic algorithms to create new dances.
us, in both cases what changes is actually the way in which the piece
interacts with the visitor. at is, the piece modies itself over time and
will not respond the same way to interactors at dierent moments of its
existence. In this respect, the door is open for emergent interactive novelty
to appear in their behavior, since the conditions for emergence are set. If
unanticipated behaviors appear after the system has been initially dened
(according to emergence-relative-to-a-model or the SOE/GNE Framework),
then the new behavior is emergent (for an example, see (Soler-Adillon,
INTERFACE POLITICS. 1st International Confrence
3.2 Emergent interfaces
If creating emergent behavior is a dicult task, emergent interfaces are to be
found one step closer to impossibility. For an interface to be emergent, it has
to be spontaneously formed. at is, it cannot be previously designed. As Ca-
riani indicates when discussing creative emergence, this is feasible in natural
system (through the course of evolution), but extremely rare in articial sys-
tems. In fact, he identies only one single example in the literature: Gordon
Pask’s electrochemical ‘ear’. is somewhat obscure device, developed in the
1950s, was capable of evolving its own sensors in order to choose those as-
pects of its external environment to which it would react, and it would to so
independently of its designer (Cariani, 1993). As Pask presented it in 1958, it
could either be trained to recognize magnetic elds or sound. In about half a
day, it was capable of adaptively grow its own connections in order to do so.
In the case of sound, once this was done it could also rapidly gain the ability
to distinguish between two dierent frequencies (Pask, 1959).
A similar example, presented fty years after Pask’s, is the evolved radio
developed at the University of Sussex by Jon Bird and colleagues (Bird et
al., 2003). is group of researchers, inspired by Cariani’s taxonomy of
robotic devices, pursued the creation of epistemically autonomous hard-
ware, in what they called the ‘unconstrained intrinsic hardware evolution,’
a design method with which they evaluated hardware by instantiating it.
With this approach, they were able to create and evolved radio. After some
experimentation with oscillators, they found that some circuits, which had
achieved good tness according to the predened criteria, were however not
oscillating in a stable manner. Upon further examination, they found that
these circuits had evolved to pick up radio frequencies that were present in
the physical environment were the prototypes were being tested. To do so,
the circuits had evolved to use some of its components as antennas. In this
particular instance, besides the transistors of the evolvable motherboard
on which the circuits were constructed, the circuits also utilized the analog
switches and the printed circuit boards (Bird et al., 2003).
e authors claim that this is “the second experimental system ever to
construct novel sensors through a process of creative emergence”–the rst
being Pask’s device. However, such systems have a fundamental problem,
they argue: since these circuits sometimes utilize environmental conditions
and component properties that are very particular of a given implementa-
tion, they do not always generalize well. But if, to avoid this issue, the evo-
lutionary process is constraint, so that the circuits are more robust, then the
possible advantages of unconventional design, in terms of exibility, are lost.
Twenty years after its conceptualization, emergent interactive behavior is
still a challenging goal in terms of design. Self-organization can be sought
by setting up a series of agents that interact with one another locally, and
techniques such as genetic algorithms can facilitate the recombination of
computations in order to achieve results that were unexpected (or at least
no specically pre-programmed). But translating that into the interactive
behavior, in the sense of aecting how the artifact relates to its environment
and visitors, remains a dicult task. In terms of physical interface design,
the complexity of the challenge escalates, since achieving interfaces as the
result of emergence implies enormous diculties as seen in the last section
(provided that the approach to emergence proposed here is accepted). e
presented examples shade some light into this issue. e clear separation, in
the design process, of behavior and interface, and a thorough understanding
of emergence, along with the further development of theoretical tools such
as the SOE/GNE Framework, might become key elements in resolving this.
INTERFACE POLITICS. 1st International Confrence
Bedau, M. A. (2008). Downward Causation and Autonomy in Weak Emergence. In M. A.
Bedau & P. Humphreys (Eds.), Emergence. Contemporary Readings in Philosophy and Science
(pp. 155–188). Cambridge, Massachussetts: MIT Press.
Bird, J., Layzell, P., Webster, A., & Husbands, P. (2003). Towards Epistemically Autonomous
Robots: Exploiting the Potential of Physical Systems. Leonardo, 36(2), 109–114.
Cariani, P. (1992). Emergence and articial life. In C. G. Langton, C. Taylor, J. D. Farmer,
& S. Rasmussen (Eds.), Articial Life II (pp. 775–797). Reading, Massachussetts:
Addison-Wesley Publishing Company.
Cariani, P. (1993). To evolve an ear: epistemological implications of Gordon Pask’s
electrochemical devices. Systems Research, 10 (3), 19–33.
Cariani, P. (2011). e Semiotics of Percept-Action Systems. International Journal of Signs
and Semiotic Systems, 1(1), 1–17.
Cariani, P. (2012). Creating New Informational Primitives in Minds and Machines. In J.
McCormack & M. D’Inverno (Eds.), Computers and Creativity (pp. 395–430).
New York: Springer.
Glynn, R. (2008). Conversational Environments Revisted. In 19th European Meeting of
Cybernetics & Systems Research. Viena.
Holland, J. (1998). Emergence. From Chaos to Order. New York: Basic Books.
Langton, C. G. (1988). Articial life. In C. G. Langton (Ed.), Articial Life: Proceedings of
an Interdisciplinary Workshop on the Synthesis and Simulation of Living Systems (pp. 1–47).
Boston: Addison-Wesley Publishing Company.
Retrieved from http://library.iscpif.fr/les/CSSS2010-20100805-Beslon-lecture4.pdf
Nagel, E. (1961). e Structure of Science: Problems in the Logic of Scientic Explanation.
New York: Harcourt, Brace & World, Inc.
Pask, G. (1959). Physical Analogues to the Growth of a Concept. In A. Uttley (Ed.),
Mechanisation of ought Processes (pp. 877–922). London: HMSO.
Penny, S. (1996). e Darwin Machine: Articial Life and Interactive Art.
New Formations UK, 29, 59–68.
Penny, S. (2000). Agents as Artworks: and Agent Design as Artistic Practice. In K. Dauthen-
hahn (Ed.), Human Cognition and Social Agent Technology (pp. 395–414).
Amsterdam, Netherlands: John Benjamins Publishing Company.
Penny, S. (2008). Robotic art and interactive installation 1988-1997. DVD documentation.
Pen ny, S . (2 011 ). Towa rds a Perfo rmat ive Aes the tics of Int era ctiv ity. Fiberculture, 19, 72–108.
Soler-Adillon, J. (2011). Playing with Pixels. In E. Puig, A. Vela, & A. Vilà (Eds.),
Impresión Expandida / Expanded Print (pp. 161–198). Barcelona: Universitat de Barcelona.
Soler-Adillon, J. (2015). Emergence as Self-Organization and as Generation of Novelty. A Fra-
mework for Understanding Emergence in the Context of Interactive Art.
Universitat Pompeu Fabra.
Soler-Adillon, J., & Penny, S. (2014). Self-organization and novelty: pre-congurations of
emergence in early British Cybernetics. In Norbert Wiener in the 21st Century (21CW), 2014
IEEE Conference on (pp. 1–8). Boston, MA: IEEE.
Soler-Adillon, J., Sora, C., Freixa, P., & Ribas, J. I. (2016). El perl del profesional de la
comunicación interactiva: fundamentos, actualidad y perspectivas.
El Profesional de La Información, 25(2).
Steels, L. (1995). e Articial Life Roots of Articial Intelligence. In C. G. Langton (Ed.),
Articial life. An Overview1 (pp. 75–110). Cambridge, Massachussetts: MIT Press.
Waldrop, M. M. (1992). Complexity. e Emerging Science at the Edge of Order and Chaos.
New York: Simon and Shuster.