Quantitative Emergence

Abstract

Emergence can be defined as the formation of order from disorder based on self-organisation. Humans—by looking at a self-organising
system—can decide intuitively whether emergence was taking place or not. To build self-organising technical systems we need
to automate the recognition of emergent behaviour. In this paper we try to give a quantitative and practically usable definition
of emergence. The presented theoretical approach is applied to an experimental environment, which shows emergent behaviour.
An Observer/Controller architecture with emergence detectors is introduced. The proposed definition of emergence is discussed
in comparison with Shannon’s information theoretical approach.

KeywordsEmergence-Self-organisation-Order-Entropy-Observer/controller architecture

Full text

1-4244-0166-6/06/$20.00 ©2006 IEEE.
Quantitative Emergence
M. Mnif, C. Müller-Schloer
Institute of Systems Engineering –
System and Computer Architecture
Hannover, Germany
E-mail: {mnif, cms}@sra.uni-hannover.de
Phone: +49-511-762 19730, Fax: +49-511-762 19733
Abstract —Emergence can be defined as the formation of
order from disorder based on self-organization. Humans -
by looking at a self-organizing system - can decide intuitively
whether emergence was taking place or not. To build self-
organizing technical systems we need to automate the recog-
nition of emergent behavior. In this paper we try to give a
quantitative and practically usable definition of emergence.
The presented theoretical approach is applied to an experi-
mental environment, which shows emergent behavior. An
observer/controller architecture with emergence-detectors is
introduced. The proposed definition of emergence is dis-
cussed in comparison with Shannon’s information theoreti-
cal approach.
I. INTRODUCTION
Organ ic Computing (OC) has become a major research
activity in Germany and worldwide [1]. Its goal is the
technical utilization of emergence and self-organization
as observed in natural systems. Emergent and self-
organizing behavior has been observed in nature, demon-
strated in a variety of computer simulated systems in arti-
ficial life research, and it has occurred also in highly
complex technical systems (like the internet) where it has
led to unexpected global functionality. Despite the impor-
tance of a rigorous description of these phenomena, the
quantitative analysis of technical self-organizing systems
is still a rather unexplored area.
Emergence and self-organization have been discussed
by a variety of authors for many years. The most com-
monly accepted definition is that the whole is more than
the sum of the parts. We want to avoid a new verbal defi-
nition and refer instead to a few excellent papers and
books [2, 3, 4, and 5].
There seem to be certain necessary ingredients for an
observed phenomenon to be called “emergent”: A large
population of interacting elements (or agents) without
central control and hence based only on local rules leads
to a macroscopic behavior which displays new properties
not existent on the element-level. This macroscopic pat-
tern is perceived as structure or order. Although the re-
sulting order is a necessary precondition for emergence, it
is not sufficient. We require that this order has been de-
veloped without external intervention – i.e. self-
organized. Hence we define emergence as self-organized
order1. An attempt to measure emergence quantitatively
_____________________________
1 We appreciate the discussion of the possible separation of emergence
and self-organization by T. De Wolf but claim that for practically inter-
should therefore rely on a well-known metric for order,
i.e. entropy.
Section II of this paper proposes a method to determine
the entropy of an arbitrary system based on Shannon’s en-
tropy definition. This definition relies on the selection of
observable attributes, which means a subjective influence
on the measured values. Section III introduces the notion
of an observation model, which subsumes these subjec-
tive decisions. Emergence is not the same as entropy. In
section IV we derive an emergence measure based on en-
tropy and discuss its implications in comparison to Shan-
non’s redundancy. Section V discusses the limitations of
our approach, the relationship of redundancy and emer-
gence, and the term “pragmatic information” as intro-
duced by von Weizsäcker, Section VI proposes an ob-
server/controller architecture we are presently implement-
ing, which includes detectors for the measurement of
emergence. Section VII presents first experimental results
of emergence measurements.
II. THE MEASUREMENT OF ORDER
The meaning of order as perceived by a human2 ob-
server is not clear without ambiguity. A homogeneous
mixture of two liquids can be regarded as “orderly” (Fig.
1, right). Applying the thermodynamic entropy, however,
will result in lower entropy (i.e. higher order) for the left
example of figure 1. Apparently, order depends on the se-
lection of certain attributes by the (human) observer. If
we are interested in the spatial structure we have to base
our measurement on the positions of the molecules (Fig. 1
left), if we are interested in homogeneity we can use the
relative distances between the molecules (Fig. 1 right).
The emergence definition presented in this paper is based
on the statistical definition of entropy (which essentially
can be explained as counting events or occurrences).
esting phenomena emergence always implies self-organization.
2 Currently the only observers who ma ke the se decisions are huma n de-
signer s and researchers, but eventually one could i n fact imagine a sys-
tem that could make these decisions based on knowledge bases and ex-
periments with a target system (e.g. trying out a set of likely candidate
attributes etc.) The theoretical approach of this section is the basis for
our observer/controller architectures discussed later in this article.

Fig. 1. Order perception: Both pictures could be perceived as high order
(left: more structure, right: more homogeneity) depending on the objec-
tive of the observer.
The computation of the entropy of a system S with N
elements ei is done as follows:
1. Select an attribute A of the system elements of S with
discrete, enumerable values aj.
2. Observe all elements ei and assign a value aj to each ei.
This step corresponds to a quantization.
3. Transform into a probability distribution (by consider-
ing the relative frequency as a probability) over the at-
tribute values aj (i.e. a histogram) with pj being the prob-
ability of occurrence of attribute aj in the ensemble of ele-
ments ei.
4. Compute the entropy according to Shannon’s definition
∑
−
=
−=
1N
0j j
pld
j
p
A
H (1)
If the attribute values are equally distributed (all pj
equal) we will obtain the maximum entropy. Any devia-
tion from the equal distribution will result in lower en-
tropy values (i.e. higher order ). In other words: The more
structure is present (unequal distribution), the more order
is measured. The unit of measurement is bit/element. So
the entropy value can be interpreted as the information
content necessary to describe the given system S with re-
gard to attribute A. A highly ordered system requires a
simpler description than a chaotic one3.
III. OBSERVATION MODEL
The resulting entropy value depends on two decisions
of the observer: (1) Which attribute A is measured? and
(2) With what resolution (or quantization) is A measured?
The quantization determines the information content of
the system description but it is not a property of the sys-
tem. Neither is the selection of a certain attribute A a sys-
tem property. This means that a measured entropy value
is only meaningful if we know the exact observation con-
text. This context is subsumed by the observation model.
This reflects the fact that order is not an intrinsic prop-
erty of the system. Rather order depends on subjective
decisions or capabilities of the observer. In living systems
the sensory equipment limits the selection of observable
attributes and the resolution of the measurement. In addi-
tion, the brain directs attention to certain observables,
which are relevant in the present situation, and masks
other attributes or registers them with lower effort, i.e.
lower resolution. Hence, order results from an interaction
_____________________________
3 This reminds of the definition of Kolmogorov complexity [8].
between the observer and the observed system guided b y
the observation model. The observation model depends
on the capabilities of the sensory equipment and the util-
ity of certain observations with regard to the purpose.
An observer might be interested in more than one at-
tribute. In this case, we obtain a vector of entropy values
(HA, HB, HC…) with respect to attributes A, B, C… We
could add them into a total system entropy HS. HS denotes
the information content of the total system description
under the given observation models. It has the drawback
of hiding or averaging the single attribute entropies.
Therefore we prefer the non-added entropy values.
IV. EMERGENCE
Entropy is not the same as emergence. Entropy de-
creases with increasing order while emergence should in-
crease with order. As a first try we define emergence as
the difference ∆H between the entropy at the beginning of
some process and at the end:
∆H = Hsta rt – Hend (2)
In case of an increase of order this results in a positive
value of ∆H. A process is called emergent if (1) ∆H > 0
and (2) the process is self-organized. This definition has
two problems:
1. The measurement of H depends on th e observation
model (or abstraction level). An observation on a higher
abstraction level will lead to a lower entropy value H
even when there is no change of S in terms of self-
organized order.
2. Since the start condition of the system is arbitrary,
∆H represents only a relative value for the increase of or-
der. It would be preferable to have a normalized emer-
gence measure.
The first problem can be solved by determining the
portion of ∆H, which is due to a change of abstraction
level (∆Hview) and subtracting it.
view
H
emergence
HH ∆+∆=∆ (3)
view
HH
emergence
H∆−∆=∆ (4)
In other words: (1) holds only if we have not changed
the abstraction level or if ∆Hview can be determined and
subtracted.
The second problem is solved by definition of an abso-
lute reference as starting condition. Obviously this start-
ing condition could be the state of maximum disorder
with an entropy of Hmax. Hmax corresponds to the equal
probability distribution.
This leads to the following definition:
Emergence view
HH
max
H
emergence
HM ∆−−=∆= (5)
Absolute emergence is the increase of order due to self-
organized processes between the elements of a system S
in relation to a starting condition of maximal disorder.

Fig. 2. Visualization of the emergence fingerprint
The observation model used for both observations must
be the same (∆Hview = 0) or ∆Hview must be determined and
subtracted.
We can also define the relative emergence m as
m=
H
max −
H
Hmax
(if ∆Hview = 0) (6)
m has a value between 0 and 1. m=0 means high disorder
and m=1 high order.
Of course we can define – in analogy to the above dis-
cussion – attribute-specific emergence-values MA, MB,
MC or mA, mB, mC. The vector of Mk or mk (with k denot-
ing the attributes of the elements of a given system) con-
stitutes a so-called emergence fingerprint. An emergence
fingerprint can be visualized e.g. as a 6-dimensional
Kiviat graph (Fig. 2). It represents the “order pattern”
with regard to the chosen attributes. In the example of fig
2, the 6 attributes x and y coordinate, status, intention (of
an animat), direction of movement and color have been
chosen. The fingerprint changes over time (t0, t1, t2) and
thus represents the development of order in the different
dimensions (attributes).
V. DISCUSSION
A. Limitations
The definition is not applicable if ∆Hview cannot be
made zero or determined quantitatively. This is always
the case if the macro phenomenon is totally different from
the micro behaviors as seemingly in the case of the reso-
nance frequency as an emergent property resulting from
the interaction of a capacitor and an inductivity. Our
quantitative definition of emergence is based on the as-
sumption that emergent phenomena can always be ob-
served in terms of patterns (space and/or time) consisting
of large ensembles of elements. The resonance frequency
of a resonant circuit does not constitute the emergent pat-
tern but is rather a property of such a pattern. Order can
also be determined in the time or frequency domain.
Therefore, we can apply our emergence definition to the
resonance frequency example if we observe the system
behavior after a Fourier analysis. This extends the above
definition of the observer model: Any type of preprocess-
ing can also be a part of the observer model. This corre-
sponds quite well to the operation of the animal (and hu-
man) perception4.
We admit that our model does not cover the so-called
‘strong emergence’ definition, which demands that emer-
gence is a phenomenon principally unexplainable. But
this is a quite unscientific argumentation, which we reject.
To the contrary, we would like to propose, that only a
quantifiable phenomenon resulting in a (self-organized)
increase of order, deserves to be accepted as emergence.
If this definition is too restrictive, excluding some unex-
plainable emergent effects, we could accept that what we
measure with our method is “quantitative emergence” and
constitutes a certain form of emergence meaningful in
technical systems. This definition leaves room for wider
definitions of emergence in a more general meaning.
B. Redundancy and emergence
The reader familiar with Shannon’s information theory
might have recognized that our definition of emergence is
formally equivalent to Shannon’s redundancy since
Redundancy H
max
HR −= (7)
and relative redundancy
max
H
H
max
H
r
−
= (8)
Redundancy is a property of a message source and a
code, which should be reduced as far as possible in order
to utilize a given channel optimally. Emergence on the
other hand is a measure of order in a system, which in
many cases is highly desirable. At least, as we have seen,
a high emergence value means that the description com-
plexity of the system is low. The explanation of the ap-
parent contradiction lies in the different points of view of
a communication engineer (Shannon) and a systems engi-
neer.
A channel is utilized optimally if it transports only infor-
mation, which is new to the receiver. Shannon defines
predictable information as useless or redundant. But the
notion that predictable information is useless contradicts
both our intuition and biological research. It is justified
only in communications engineering. Shannon assumes
that sender and receiver have the same semantic frame-
work, the receiver has a-priori knowledge, he can match a
received message against a known set of messages. A re-
ceived message has a value only within this pre-defined
context. The common context in a traditional technical
system is given by the fact that both, sender and receiver
(and the communication system in between), have been
designed by the “same” engineer.
This is not true for adaptive systems (living systems or
organic computing systems). Communicating adaptive
systems have (1) to maintain a valid context, which might
change over time, and (2) to validate new information
within this context. The maintenance of context requires
regular affirmation and a certain degree of modification.
_____________________________
4 In the cochlea, the sound mo ves hair bundles, which respond to c ertain
frequencies. The brain therefore reacts to preprocessed signals [9].

The more context the receiver has accumulated in his
memory (be it genetic or acquired), the less new or cur-
rent information he needs in order to recognize a certain
situation and act accordingly.
This means that each animal (or animat5) will strive to
build and maintain a dependable and stable context “data-
base”, which allows it to reduce the amount of new in-
formation to be transferred in a possibly dangerous
situation and hence save valuable time and energy. This
works fine as long as the environment is stable and in
agreement with the memorized context database. In
changing situations this agreement will be disturbed, and
erroneous decisions will follow. The context information
has to be updated as fast as possible.
The above discussion suggests a possible approach to
separate information into two parts: affirmation informa-
tion and newness information. The first part is used to af-
firm and/or modify the context database, the second se-
lects between several contexts. But it is more realistic to
assume that each received message is used as a whole for
both purposes: It is compared to the stored contexts and
selects one of them in case there is a sufficient match. A
successful match results in an action (stimulus–response
relationship). If there is no match or if there is a wrong
match, a detrimental action will result leading to the ne-
cessity to update the context database. An update phase
will require the temporary collection of more information
until the animat can return to business as usual.
An animat (animal) prefers a low entropy environment
(orderly and predictable, with high emergence)! On the
other hand, instability (caused by random processes like
mutation) is the necessary precondition for the explora-
tion of unknown areas of the configuration space.
C. Pragmatic Information
The term pragmatic information has been introduced
by Christine and Ernst von Weizsäcker, cited by Küppers
in [10]. Pragmatic information means information, which
has an effect on the receiver. This could be a structural
change (leading to some kind of action) or the readiness
of the receiver for some action ([10] p. 85). Pragmatic in-
formation is determined by two aspects: Newness (Ger-
man: Erstmaligkeit) and affirmation (German: Bestäti-
gung). A qualitative curve proposed by Christine and
Ernst von Weizsäcker (Fig. 3) claims that the Shannon
part of the information measures newness. It is zero for a
known (i.e. predicted) event and increases with increasing
newness. On the other hand, for a message to be recog-
nized within a context there must be a-priori knowledge
of this context. Therefore, pragmatic information is also
zero if there has been no prior communication between
sender and receiver. In this case, newness = 100%. Affir-
mation is
_____________________________
5 Animats are artificial animals, e.g. robots equipped with sensors, ac-
tuators and some decision mechanism.
Affirmation
Newness
0 %
100 %
100 %
0 %
I
“Shannon”
?
Fig. 3. Newness and affirmation [10]
complementary to newness: It is zero when a message is
totally new, and 100% when a message is known in ad-
vance. If there are two zero values of pragmatic informa-
tion, there must be at least one maximum in between. Von
Weizsäcker concludes that to achieve a maximum of
pragmatic information, there must be a certain optimal
combination of newness and affirmation in the communi-
cation relation between two partners.
A highly predictable system as message source (i.e. a
system displaying high order = high emergence) requires
a channel with low bandwidth because the transmitted in-
formation serves essentially as affirmation, which is
needed less frequently. The receiver compares these mes-
sages to his context database, finds matches with a high
probability and initiates corresponding actions. If affirma-
tion is sent too frequently, it becomes useless: the channel
transports redundant information (already known by the
receiver). As soon as the message source changes its be-
havior, the receiver needs more frequent update informa-
tion in order to change his context database. The newness
aspect of the messages becomes more important.
Technically speaking, there are two mechanisms in the
receiver working in parallel on all received messages. On
the lower „working“ level, messages are compared
against known probability distributions and mapped to ac-
tions. On a higher semantic level, the newness of all mes-
sages and the effectiveness of the corresponding actions
are monitored. In case of inadequate results, the structure
of the receiver has to be adapted by changing/extending
the context database and adding new actions. This higher
level is realized by (possibly multi-level) observer-
controller architectures as shown in the next section. The
lower level corresponds to the production system.
VI. OBSERVER/CONTROLLER ARCHITECTURE
The objective of our work is to make emergent phe-
nomena accessible to a practical measurement process.
This is important in technical applications that have to de-
tect emergent phenomena in order to support or to sup-
press them. In other projects presently run by the authors
the collective behavior of chicken in a chicken farm [12],
the behavior of cars in th e environment of an intersection
[6] or the synchronization of elevators (so-called

Observer
Observer Controller
Controller
observes changes
reports
selects obs. model goals
Production System
Fig. 4. Observer/controller architecture.
Fig. 5. Observer architecture.
Bunching effect [12]) is of interest. We propose a gener-
alized observer/controller architecture [7] (Fig. 4). The
observer collects and aggregates information about the
production system. The aggregated values (system indica-
tors) are reported to the controller who takes appropriate
actions to influence the production system. The observer
contains several specialized detectors to calculate the sys-
tem indicators from the observed raw data (Fig. 5). We
are presently building emergence detectors specialized for
certain attributes. The collection of attribute emergence
values (the emergence fingerprint) is a part of the obser-
vation result as determined by the observer. The observer
model influences the observation procedure, e.g. by se-
lecting certain detectors or certain attributes of interest.
The feedback from the controller to the observer directs
attention to certain observables of interest in the current
context.
VII. EXPERIMENTAL RESULTS
In this section we present first experimental results of the
emergence fingerprint. The results discussed here could
be part of the observer in the observer/controller architec-
ture presented in section VI.
A. Experimental environment
One of the experimental environments is a chicken
simulator, whose goal is to explain the collective canni-
balistic behavior of densely packed chicken in cages (co-
operation with the University of Veterinary Medicine
Hannover). This behavior is frequently observed when a
chicken is injured, and leads to a major loss of animals.
The described reaction occurs only on the basis of an op-
tical stimulus. That means the reaction exists as long as
the stimulus is apparent. While simulating this behavior,
order patterns emerge in form of chicken swarms. These
patterns are at present interpreted by human experts. It
should be possible to classify them automatically. The
emergent behavior in this scenario is spatial, but swarms
move over time. This is a case of “negative”, i.e. un-
wanted, emergence, since the global goal is to reduce
chicken death rate. The controller has to react with ac-
tions
to disperse the swarms.
B. Results
Figures 6, 7 and 8 show three typical sates of emergent
clustering behaviors (taken from our simulations). State 1
shows no recognizable clustering. In state 2, a chicken is
wounded, and a small group of aggressing chicken has
Fig. 6. Emergence fingerprint of state 1: no cluster; mx = 0.181, my =
0.177, md = 0.091 (uninjured chicken: white, feeding troughs: hexagons).
Fig. 7. Emergence fingerprint of state 2: small cluster; mx = 0.226, my =
0.237, md = 0.046.
Fig. 8. Emergence fingerprint of state 3: one big cluster; mx = 0.359, my
= 0.328, md = 0.041.

Fig. 9. Overlay of 3 fingerprints (state 1, state 2 and state 3).
already clustered around it. In state 3, all the chicken of
the cage have realized the injury and are participating in
the attack. The Kiviat graph next to the state pictures has
three dimensions: x- and y-coordinate and the direction
(of view). Only the emergence values for the x- and the y-
coordinate, mx and my, show a significant increase (as ex-
pected). The heading direction plays no role in the chas-
ing behavior. The corresponding emergence md stays very
small. Fig. 9 shows the overlay of the 3 states and their
development over time.
C. PREDICTION
We are especially interested in the prediction of future
emergent behavior in order to be able to prevent un-
wanted behavior in time. To deal with this goal, it must
be possible to predict the positions of the chicken. This
can be done by extrapolating a trajectory. We measure the
position of the chicken at two consecutive points in time.
Based on these two points the trajectory of the chicken is
computed by extending the line between them. Only those
positions are of practical interest that can be reached
within a certain prediction time ∆tprediction. Using the pre-
sent speed of the animals vaverage, we determine a critical
distance d = vaverage* ∆tprediction. We compute the intersec-
tion points of all trajectories within the critical distance d
(Fig 10). An accumulation of these points means that the
chicken head on a point in the area, which might indicate
the existence of an injured chicken (Fig 11). This point
accumulation in space can also be measured by using the
emergence indicator applied to their x- and y-coordinates.
Fig 12 shows the emergence values of the x-coordinate of
the chicken positions and the ones of the intersection
points of the trajectories. The emergence of the intersec-
tion points grows before the emergence of the actual
chicken positions and can therefore be used as an early
warning indicator for chicken clustering. We are currently
experimenting to increase the prediction time and to re-
duce the effect of noise.
Fig. 10. Trajectory-based prediction method of chicken positions.
Fig. 11. Cluster prediction
Fig. 12. Prediction: The emergence mx prediciton indicates a future cluster
(mx), which emerges about 10 time units later.
VIII. CONCLUSION AND OUTLOOK
We have proposed a quantitative measure of emer-
gence based on the statistical definition of entropy and
discussed it in comparison with Shannon’s information
theory. While proponents of a so-called strong definition
of emergence might argue that “true” emergent effects
must always represent something totally new and unex-
pected, we claim that with our emergence definition we
can measure at least some effects of the generation of or-
der. Emergence definitions going beyond might have to
live with the flaw that they principally do not lend them-
selves to a quantitative approach.
It is the objective of our work to make emergent effects
quantitatively treatable in a technical environment. We
have proposed an observer/controller architecture with
special detectors for determining attribute emergence val-
ues. First experimental results obtained from a chicken
simulation show the viability of the approach.
We plan to extend these detect ors by preprocessing
steps, using perhaps a Fourier analysis to make regular
patterns like a crystal lattice treatable. The method will be
applied to more technical problems like self-

synchronizing elevators (bunching effect) and traffic
simulations.
Acknowledgment
This work has been done in close co-operation with
Hartmut Schmeck, Jürgen Branke, Urban Richter, Holger
Prothmann (University of Karlsruhe) and Fabian Rochner
(University of Hannover) within the DFG Priority Pro-
gram Organic Computing. We are especially indebted to
Kirstie Bellman, The Aerospace Corporation, Los Ange-
les, U.S.A., for reviewing the manuscript and making
valuable suggestions for improvement.
References
[1] C. Müller-Schloer, “Organic Computing – On the Feasibility of
Controlled Emergence”, CODES + ISSS 2004 Proceedings, pp. 2-
5, 2004.
[2] Gershe nson C. and F. H eylig hen, "When Can we Call a System
Self-organizing?", Advances in Artificial Life, 7th European Con-
ference, ECAL 2003, (Springer, LNAI 2801.), p. 606-614.
[3] T. De Wolf, and T. Holvoet, “Emergence Versus Self-
Organi sation: Differ ent Concepts bu t Promising W hen Combined,
Engineering Self Organising Systems: Met hodologies and Appli -
cation s”, Lecture Notes in Computer Scien ce, Volume 3464, May
2005, pp. 1-15.
[4] A. Stephan, “Varieties of Emergence in Artificial a nd Natural S ys-
tems”, Zeitschrift für Naturforschung 53c (A Journal of Biosci-
ences), pp. 639-656, 1998.
[5] J. Fromm: “The Emergence of Complexity”, Kassel university
press, 2004, ISBN 3899580699.
[6] F. Rochner, C. Müller-Schloer, „Adaptive Decentralized and Col-
laborative Control of Traffic Lights“, GI-Tagung Informatik 2004,
Bd. 2, pp. 595 – 599, ISBN 3-88579-380-6
[7] T. Schöler, C. Müller-S chloer, “An Observer/Controller Archit ec-
ture for Adaptive Reconfigurable Stacks”, Proceedings ARCS 05,
ISBN 3-540-25273-8, pp. 139-153
[8] M. Li and P. Vitányi, “An Introduction to Kolmogorv Complexity
and Its Applications”, Springer, 1997.
[9] Wikipedia, http://en.wikipedia.org/wiki/Cochlea
[10] B.O. Kü ppers, „Der Ursprung bi ologischer Information“, Piper
1990.
[11] T. Bartz-Beielstein, “New Experimentalism Applied to Evolution-
ary Computation”, eldorado, Universität Dortmund, 2005.
[12] LJ. Keeling, “Feather peckin g and cannibalism in lay ers”. Poult.
Internat. 6:46–50. 1995.