Conference PaperPDF Available

Interfacing with the unexpected: notes towards a design theory for emergent interaction



This paper presents a first approach to the design of emergent behaviors and interfaces in the context of interactive arts. The main conceptual issues involved are presented, along with the metadesign challenge implied in designing something that, by definition, cannot be designed per se. Key examples are discussed and used as examples of successful attempts to gene- rate emergence in interactive art. Whilst the issue remains largely unsolved, these examples and the proposed theoretical framework are presented as starting point towards the design of interactivity that can be regarded as being emergent.
Bau, Centre Universitari de Disseny
de Barcelona
Teresa Martínez Figuerola
Jorge Luis Marzo
Comité Cientíco:
1ª Edición.
Cantidad de ejemplares 150
Barcelona, España.
Septiembre 2016
9. disseny
Diseño y maquetación:
Best Boy
ISBN (Ed. Impresa): 978-84-617-5132-7
Depósito Legal: DL B 20848-2016
Copyrights de la publicación:
Bau, Centro Universitario de Diseño
Copyrights de los textos:
Todos los autores
El contenido de los artículos es de absoluta responsabilidad de los autores.
Interacciones inesperadas
Joan Soler-Adillon
Interfacing with the unexpected: notes towards a design
theory for emergent interaction
is paper presents a rst approach to the design of emergent behaviors
and interfaces in the context of interactive arts. e main conceptual issues
involved are presented, along with the metadesign challenge implied in
designing something that, by denition, cannot be designed per se. Key
examples are discussed and used as examples of successful attempts to gene-
rate emergence in interactive art. Whilst the issue remains largely unsolved,
these examples and the proposed theoretical framework are presented as
a starting point towards the design of interactivity that can be regarded as
being emergent.
Keywords: Emergence, interactivity, design, behavior, interface
INTERFACE POLITICS. 1st International Confrence
1. Introduction: Emergence and Digital Art
Emergence is often referred to as the idea of the whole being more than
the sum of its parts, or of being order out of chaos. In this sense, it refers to
the idea of complex behaviors arising from an aggregate of relatively simple
elements with relatively simple behaviors (Langton, 1988; Holland, 1998;
Bedau, 2008). In other accounts, complementary of those in occasions, it
is related to fundamental novelty (i.e. the appearance of novel and unex-
pected behaviors) (Cariani, 1992; 2012; Steels, 1992). It can be argued that
emergence refers, in fact, to either (or both) a self-organization process or
the appearance of novelty (Nagel, 1961; Soler-Adillon and Penny, 2014;
Soler-Adillon, 2015).
e concept found a fertile ground in Complexity Sciences, where it
became a central concern, especially in the eld of Articial Life (ALife), a
discipline initiated by Christopher Langton in 1987 (Waldrop, 1994). It was
through the use of ALife techniques and themes in art –which constituted the
discipline known as ALife Art– that emergence permeated digital art mostly as
related to the idea of the search for the unexpected result (the surprise).
e early years of interactive and digital art were coincidental in time
with the early years of Complexity Sciences. e personal computer faci-
litated the former while computer simulations enabled the latter. When
experimenting with them, computers were capable of producing unexpected
results and computational emergence became one of the ways to explain
these phenomena. In a nutshell, when a set of relatively simple rules produ-
ced complex and varied results, these were interpreted as emergence.
Conway’s Game of Life was an iconic example of his. is cellular auto-
maton brilliantly shows how a simple set of rules can generate surprisingly
complex dynamic results. Another iconic example is Craig Reynolds’ ‘boids’,
a simulation of a ock of birds or school of sh using three very simple
and local rules that generate a group behavior strikingly ressemblant of the
modeled system. Finally, genetic algorithms, developed by John Holland,
constituted a very fertile ground of experimentation.
In any case, in the formation years of interactive art, emergence became
an idea linked to the possibility of escaping the pre-specication that
computer programming forces the artists into. e idea was that, instead of
predening behaviors, one should be able create agents that would exhibit
interactive behavior that was emergent. It was presented in contraposition
to the reductionist approach, which implies that the complex can always be
accounted for by breaking it apart and analyzing its (simpler) parts:
Interacciones inesperadas
[Reductionism] is premised on the assumption that to understand
a complex object, one breaks it into component parts and examines
those parts in controlled settings, then adds the results of those
examinations together. e basic principle of emergence is that
organization (behavior/order/meaning) can arise from the agglo
meration of small component units which do not individually
exhibit those characteristics. Emergent order implies that the
whole is indeed greater than the sum of its parts, that higher level
behaviors cannot be disassembled into their component lower level
building blocks. (Penny, 1996).
2. The Design Challenge
Emergent interactive behavior, as formulated by Penny in (1996), becomes
in this context a design endeavor, and quite a challenging one. If taken lite-
rally, the idea is to be able to create artistic devices that behave emergently.
Accordingly to the formulation proposed here, this means either generating
a self-organization process or generating novel and unexpected behaviors.
But emergence, by denition, cannot be designed per se. us, from the
design point of view, designing emergent behavior is a paradox, at least up
to some degree. If we focus on emergence as self-organization, we cannot
design the emergent behavior but only the agents and their local behaviors
and interactions. It is through these local interactions that something emer-
gent might appear. Similarly, emergence as novelty cannot be designed: if
we design a behavior, it will not be unpredictable in the sense that emergent
novelty is. Of course here the ‘novel to whom’ question arises. Cariani’s
emergence-relative-to-a-model is aimed at answering this and also the ‘novel
in respect to what exactly’ question (Cariani, 1992; 2011; 2012).
e task of designing emergent interactive behavior, therefore, becomes
a metadesign eort. at is, we can only design the conditions for emer-
gence to appear, but not the emergent phenomena itself. When aiming to
create self-organizing phenomena, as said, this means creating agents that
have local rules and interact locally. No central control or collective goal can
exist in order for it to be a genuine example of self-organization. e above
mentioned boids, or the robotic installation Sympathetic Sentience (see
below) are successful examples of this, although the rst is not interactive in
the sense that it doesnt respond to an interactor in any way. On the other
hand, in order to generate novelty, the interactive systems and devices have
to be open in a way that can facilitate this change at one point or another.
Cariani’s emergence-relative-to-a-model is a theoretical framework that
INTERFACE POLITICS. 1st International Confrence
thoroughly explains how this can be described and what types of emergent
novelty can be assessed. Elaborating on it, the SOE/GNO Framework (So-
ler-Adillon, 2015) is an attempt to put this framework in practice in terms
of analyzing particular systems, under the umbrella of the dual understan-
ding of emergence mentioned above. Arguably, this latter framework can,
in turn, serve as the basis for the elaboration of the guidelines for designing
interactive behavior that is emergent.
3. Emergent Interactivity
When designing interactive artistic artifacts, the approach to interactivity is
not that of functional interaction, but what can be labeled as poetic interac-
tion (Penny, 2011). Within this paradigm, the fundamentals of traditional
Human-Computer Interaction, Design and Digital Creativity conate in an
eort to create communicational interactive experiences that go beyond the
instrumental (Soler-Adillon et al., 2016).
In this context, there are two main aspects in which the designer needs
to focus, above anything else: behavior and interface. e rst refers to
how the artifact responds to its environment and to the interactor, and the
second is the means of establishing such relationship. Elaborating on this
idea, this section looks at examples of emergence in both.
3.1 Emergent behavior
Sympathetic Sentience in an interactive sound installation by Simon Penny
rst presented in 1995. It consists of a group of twelve robots that com-
municate to one another sequentially. Each of the robots is a relatively
simple electronic device, capable of producing one chirp each minute with
a particular rhythm and to pass along to the next an infrared signal indica-
ting it. en, this receiving unit combines its own rhythm with the received
information, and passes the new rhythmic pattern along. As the process
advances, the complexity of the rhythms increases as they cycle around
the group. e system is self-organizing or, in Penny’s words, self-gover-
ning (Penny, 2000). e sound pattern that the visitors to the installation
perceive is neither predesigned, nor directed by any one of the robots or an
external entity. As the robots communicate locally –each one to the next
in circle– the overall process is formed. In this respect, it is an example of
emergence as self-organization.
Interactivity is implemented in the piece in an unconventional way.
After an initial build-up period, the system works on its own as rhythmic
patterns continually evolve. It never becomes fully saturated, nor does it
Interacciones inesperadas
become fully silent if it is not interfered with. However, this equilibrium
can be disrupted, often unintentionally, by the presence of the installation
visitors whom, by moving around the space, interrupt the infrared transmis-
sions of the units. If these interruptions are short in time, lapses of silence
will inltrate the rhythm patterns of Sympathetic Sentience. If long, the
whole system can be forced into complete silence. When this happens, once
the interruption of transmissions ends, the build-up process will have to
start all over again (Penny, 2008).
However, a point can be made here to whether or not in this piece we
have an actual example of emergent interactive behavior or, rather, the
juxtaposition of an interactive behavior (the group symphony) with the in-
teractivity towards the visitor (his or her ability to interrupt the symphony).
Two more examples should be of use here in making this point: Ruairi
Glynn’s Performative Ecologies (2008) and my own Digital Babylon (2005).
Since in terms of analyzing emergent interactive behavior they both use the
same technique and oer comparable results, these two pieces are analyzed
here together. Details on the pieces can be found in (Glynn 2008) and in
(Soler-Adillon, 2011) respectively.
In short, Digital Babylon presents the user with a virtual ecosystem in
which he or she can interact with one of the species. is species evolves
through genetic algorithms, and one of the characteristics that change over
time is the likelihood that its individuals will pay attention to the interactor
or not. As a result, if the interactor helps the friendly individuals survive,
the species as a whole becomes friendlier. If he or she harms them, they will
be less friendly for future interactions. In Performative Ecologies, a series of
robots dance in front of the users and check, through the visitor’s gaze, what
dance moves are capable of maintaining the attention. en, the successful
moves are recombined trough genetic algorithms to create new dances.
us, in both cases what changes is actually the way in which the piece
interacts with the visitor. at is, the piece modies itself over time and
will not respond the same way to interactors at dierent moments of its
existence. In this respect, the door is open for emergent interactive novelty
to appear in their behavior, since the conditions for emergence are set. If
unanticipated behaviors appear after the system has been initially dened
(according to emergence-relative-to-a-model or the SOE/GNE Framework),
then the new behavior is emergent (for an example, see (Soler-Adillon,
2015: 324)).
INTERFACE POLITICS. 1st International Confrence
3.2 Emergent interfaces
If creating emergent behavior is a dicult task, emergent interfaces are to be
found one step closer to impossibility. For an interface to be emergent, it has
to be spontaneously formed. at is, it cannot be previously designed. As Ca-
riani indicates when discussing creative emergence, this is feasible in natural
system (through the course of evolution), but extremely rare in articial sys-
tems. In fact, he identies only one single example in the literature: Gordon
Pask’s electrochemical ‘ear’. is somewhat obscure device, developed in the
1950s, was capable of evolving its own sensors in order to choose those as-
pects of its external environment to which it would react, and it would to so
independently of its designer (Cariani, 1993). As Pask presented it in 1958, it
could either be trained to recognize magnetic elds or sound. In about half a
day, it was capable of adaptively grow its own connections in order to do so.
In the case of sound, once this was done it could also rapidly gain the ability
to distinguish between two dierent frequencies (Pask, 1959).
A similar example, presented fty years after Pask’s, is the evolved radio
developed at the University of Sussex by Jon Bird and colleagues (Bird et
al., 2003). is group of researchers, inspired by Cariani’s taxonomy of
robotic devices, pursued the creation of epistemically autonomous hard-
ware, in what they called the ‘unconstrained intrinsic hardware evolution,
a design method with which they evaluated hardware by instantiating it.
With this approach, they were able to create and evolved radio. After some
experimentation with oscillators, they found that some circuits, which had
achieved good tness according to the predened criteria, were however not
oscillating in a stable manner. Upon further examination, they found that
these circuits had evolved to pick up radio frequencies that were present in
the physical environment were the prototypes were being tested. To do so,
the circuits had evolved to use some of its components as antennas. In this
particular instance, besides the transistors of the evolvable motherboard
on which the circuits were constructed, the circuits also utilized the analog
switches and the printed circuit boards (Bird et al., 2003).
e authors claim that this is “the second experimental system ever to
construct novel sensors through a process of creative emergence”–the rst
being Pask’s device. However, such systems have a fundamental problem,
they argue: since these circuits sometimes utilize environmental conditions
and component properties that are very particular of a given implementa-
tion, they do not always generalize well. But if, to avoid this issue, the evo-
lutionary process is constraint, so that the circuits are more robust, then the
possible advantages of unconventional design, in terms of exibility, are lost.
Interacciones inesperadas
4. Conclusions
Twenty years after its conceptualization, emergent interactive behavior is
still a challenging goal in terms of design. Self-organization can be sought
by setting up a series of agents that interact with one another locally, and
techniques such as genetic algorithms can facilitate the recombination of
computations in order to achieve results that were unexpected (or at least
no specically pre-programmed). But translating that into the interactive
behavior, in the sense of aecting how the artifact relates to its environment
and visitors, remains a dicult task. In terms of physical interface design,
the complexity of the challenge escalates, since achieving interfaces as the
result of emergence implies enormous diculties as seen in the last section
(provided that the approach to emergence proposed here is accepted). e
presented examples shade some light into this issue. e clear separation, in
the design process, of behavior and interface, and a thorough understanding
of emergence, along with the further development of theoretical tools such
as the SOE/GNE Framework, might become key elements in resolving this.
INTERFACE POLITICS. 1st International Confrence
Bedau, M. A. (2008). Downward Causation and Autonomy in Weak Emergence. In M. A.
Bedau & P. Humphreys (Eds.), Emergence. Contemporary Readings in Philosophy and Science
(pp. 155–188). Cambridge, Massachussetts: MIT Press.
Bird, J., Layzell, P., Webster, A., & Husbands, P. (2003). Towards Epistemically Autonomous
Robots: Exploiting the Potential of Physical Systems. Leonardo, 36(2), 109–114.
Cariani, P. (1992). Emergence and articial life. In C. G. Langton, C. Taylor, J. D. Farmer,
& S. Rasmussen (Eds.), Articial Life II (pp. 775–797). Reading, Massachussetts:
Addison-Wesley Publishing Company.
Cariani, P. (1993). To evolve an ear: epistemological implications of Gordon Pask’s
electrochemical devices. Systems Research, 10 (3), 19–33.
Cariani, P. (2011). e Semiotics of Percept-Action Systems. International Journal of Signs
and Semiotic Systems, 1(1), 1–17.
Cariani, P. (2012). Creating New Informational Primitives in Minds and Machines. In J.
McCormack & M. D’Inverno (Eds.), Computers and Creativity (pp. 395–430).
New York: Springer.
Glynn, R. (2008). Conversational Environments Revisted. In 19th European Meeting of
Cybernetics & Systems Research. Viena.
Holland, J. (1998). Emergence. From Chaos to Order. New York: Basic Books.
Langton, C. G. (1988). Articial life. In C. G. Langton (Ed.), Articial Life: Proceedings of
an Interdisciplinary Workshop on the Synthesis and Simulation of Living Systems (pp. 1–47).
Boston: Addison-Wesley Publishing Company.
Retrieved from
Nagel, E. (1961). e Structure of Science: Problems in the Logic of Scientic Explanation.
New York: Harcourt, Brace & World, Inc.
Pask, G. (1959). Physical Analogues to the Growth of a Concept. In A. Uttley (Ed.),
Mechanisation of ought Processes (pp. 877–922). London: HMSO.
Penny, S. (1996). e Darwin Machine: Articial Life and Interactive Art.
New Formations UK, 29, 59–68.
Penny, S. (2000). Agents as Artworks: and Agent Design as Artistic Practice. In K. Dauthen-
hahn (Ed.), Human Cognition and Social Agent Technology (pp. 395–414).
Amsterdam, Netherlands: John Benjamins Publishing Company.
Interacciones inesperadas
Penny, S. (2008). Robotic art and interactive installation 1988-1997. DVD documentation.
Pen ny, S . (2 011 ). Towa rds a Perfo rmat ive Aes the tics of Int era ctiv ity. Fiberculture, 19, 72–108.
Soler-Adillon, J. (2011). Playing with Pixels. In E. Puig, A. Vela, & A. Vilà (Eds.),
Impresión Expandida / Expanded Print (pp. 161–198). Barcelona: Universitat de Barcelona.
Soler-Adillon, J. (2015). Emergence as Self-Organization and as Generation of Novelty. A Fra-
mework for Understanding Emergence in the Context of Interactive Art.
Universitat Pompeu Fabra.
Soler-Adillon, J., & Penny, S. (2014). Self-organization and novelty: pre-congurations of
emergence in early British Cybernetics. In Norbert Wiener in the 21st Century (21CW), 2014
IEEE Conference on (pp. 1–8). Boston, MA: IEEE.
Soler-Adillon, J., Sora, C., Freixa, P., & Ribas, J. I. (2016). El perl del profesional de la
comunicación interactiva: fundamentos, actualidad y perspectivas.
El Profesional de La Información, 25(2).
Steels, L. (1995). e Articial Life Roots of Articial Intelligence. In C. G. Langton (Ed.),
Articial life. An Overview1 (pp. 75–110). Cambridge, Massachussetts: MIT Press.
Waldrop, M. M. (1992). Complexity. e Emerging Science at the Edge of Order and Chaos.
New York: Simon and Shuster.
Related links
Digital Babylon
Performative Ecologies
Sympathetic Sentience
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
Durante los últimos años, el rol del profesional de la comunicación interactiva ha ido cambiando paralelamente a las innovaciones tecnológicas y a la introducción de nuevas prácticas comunicativas. La irrupción de las tecnologías digitales propició la aparición del diseñador de interacción, una figura presente en muchos equipos de producción y redacciones digitales pero que continúa sin estar bien definida. En este artículo se presenta una aproximación a las características que definen este perfil profesional a partir de tres aportaciones: las competencias necesarias para desarrollar la tarea del diseñador de interacción, las salidas profesionales de los estudios universitarios centrados en esta figura y un estudio de casos en el ámbito del documental interactivo. La confrontación de los datos obtenidos mediante los tres métodos de análisis utilizados (estudio bibliográfico, análisis comparative y estudio de casos) permite trazar un amplio panorama que abarca las distintas percepciones que, de forma simultánea, se tienen de esta figura. Los diferentes enfoques ponen de relieve tanto la falta de consenso, como el eclecticismo y la inmadurez de los estudios sobre un ámbito profesional que sigue en proceso de evolución, adaptación y reafirmación veinte años después de su aparición.
Full-text available
Contemporary classics on the the major approaches to emergence found in contemporary philosophy and science, with chapters by such prominent scholars as John Searle, Stephen Weinberg, William Wimsatt, Thomas Schelling, Jaegwon Kim, Daniel Dennett, Herbert Simon, Stephen Wolfram, Jerry Fodor, Philip Anderson, David Chalmers, and others. Emergence, largely ignored just thirty years ago, has become one of the liveliest areas of research in both philosophy and science. Fueled by advances in complexity theory, artificial life, physics, psychology, sociology, and biology and by the parallel development of new conceptual tools in philosophy, the idea of emergence offers a way to understand a wide variety of complex phenomena in ways that are intriguingly different from more traditional approaches. This reader collects for the first time in one easily accessible place classic writings on emergence from contemporary philosophy and science. The chapters, by such prominent scholars as John Searle, Stephen Weinberg, William Wimsatt, Thomas Schelling, Jaegwon Kim, Robert Laughlin, Daniel Dennett, Herbert Simon, Stephen Wolfram, Jerry Fodor, Philip Anderson, and David Chalmers, cover the major approaches to emergence. Each of the three sections ("Philosophical Perspectives," "Scientific Perspectives," and "Background and Polemics") begins with an introduction putting the chapters into context and posing key questions for further exploration. A bibliography lists more specialized material, and an associated website ( links to downloadable software and to other sites and publications about emergence. ContributorsP. W. Anderson, Andrew Assad, Nils A. Baas, Mark A. Bedau, Mathieu S. Capcarrère, David Chalmers, James P. Crutchfield, Daniel C. Dennett, J. Doyne Farmer, Jerry Fodor, Carl Hempel, Paul Humphreys, Jaegwon Kim, Robert B. Laughlin, Bernd Mayer, Brian P. McLaughlin, Ernest Nagel, Martin Nillson, Paul Oppenheim, Norman H. Packard, David Pines, Steen Rasmussen, Edmund M. A. Ronald, Thomas Schelling, John Searle, Robert S. Shaw, Herbert Simon, Moshe Sipper, Stephen Weinberg, William Wimsatt, and Stephen Wolfram Bradford Books imprint
Conference Paper
Full-text available
Emergence appears in the literature as related to self-organization and novelty. For many authors it is the result of multiple interactions among agents within a system, which generate phenomena that could not be understood, nor anticipated, through the analysis of the elements and their behaviors in isolation. For others, emergent phenomena are related to fundamental novelty and, thus, to creativity. These two formulations of emergence can be traced back to the experimental work of some key early cybernetic experimental devices by Ross Ashby, Grey Walter and Gordon Pask. As a group, the devices illustrate the potential of both formulations of emergence and of its combination. As such, they can help with the elaboration of a framework to understand emergence in the context of interactive art and communication, both to analyze its presence in interactive systems and to design systems that aim to generate them.
Full-text available
Conference Paper
Designers do not require the use of the most advanced electrical, mechanical & computational engineering to build highly sophisticated interactive devices, art & architecture. Instead what is crucial is that the conceptual framework within which they are conceived has a thorough understanding of interaction. A common current misconception fostered by so called 'human computer interaction' design is that by giving functional devices, or aesthetic works of art & architecture, the ability to 'React' in some way to the stimuli of a 'User', qualifies this artefact as 'Interactive'. Such an over simplification trivialises the powerful & productive quality of interaction. This paper insists on Interactivity as a more productive & advanced form of Reactivity, & explains the process of interaction as a conversational activity between participants. The conversational model is examined through this authors recent art installation & reflects on the powerful precedent Gordon Pask provides for understanding interaction through the experimental machines that he built to embody his Conversation Theories.
The subject of study of this investigation is the concept of emergence and its viability as a driving force for the creation of interactive artwork. Emergence appears in the literature as related to self-organization and novelty. For many authors it is the result of multiple local interactions among agents within a system, which generate phenomena that could not be understood, nor anticipated, through the analysis of the elements and their behaviors in isolation. For others, emergent phenomena are related to fundamental novelty and, thus, to creativity. These two formulations of emergence are reviewed in this thesis in the context of interactive art and communication. The focus of the study is on the interactive works of the artistic field known as Artificial Life Art. After reviewing the foundational concepts and proposing an approach to both emergence and interactivity, an analytical framework is presented in order to assess the presence of emergence in interactive systems, which in turn shall serve as a first step to designing systems that aim at generating interactive emergent behaviors.
This paper places contemporary modalities of digital interaction in an historical context of sixty years of intersections between technological development and artistic experimentation. Specific technological developments are identified as context-defining historical markers and specific works are discussed as exemplars of significant milestones in the engineering and the aesthetics of interaction. The shortage of theorisation of non-instrumental interaction is lamented. The process of naturalisation to increasingly sophisticated digital tools and appliances in the current period of ubiquitous computing is noted. A number of theoretical issues are drawn out and discussed in terms of cognitive and sensorimotor dynamics. Woven through the discussion is the proposal that a synthesis of performance theory and neuro-cognitive studies might provide a basis for a performative ontology around which an aesthetics of interaction might be constructed. As the paper progresses a theoretical framework for an ontologically performative aesthetics of interaction and ubiquity is formulated.
Human Cognition and Social Agent Technology is written for readers who are curious about what human (social) cognition is, and whether and how advanced software programs or robots can become social agents. Topics addressed in 16 peer-reviewed chapters by researchers at the forefront of agent research include: Narrative intelligence and implementations of story-telling systems, socially situated avatars and conscious software agents, cognitive architectures for socially intelligent agents, agents with emotions, design issues for interactive systems, artificial life agents, contributions to agent design from artistic practice, and a Cognitive Technology view on living with socially intelligent agents. The book addresses both software and robotic agents. On the one hand justice is done to the scientific and technical aspects, and on the other hand the reader will learn about pioneering technological developments which are necessary for a public discourse and critical evaluation on where social agent technology is leading us and how such a development can be shaped in order to meet the social, cultural and cognitive needs of humans. The book is suitable for students, researchers, and everyone interested in this emerging and quickly growing field, it does not require any specialist background knowledge. (Series B)