ArticlePDF Available

Dynamis review: An overview about applications of the Dynamis approach in cognitive psychology


Abstract and Figures

The Dynamis review has been designed to provide an overview about applica-tions of the Dynamis approach in cognitive psychology. Since the development of computer-based scenarios Dynamis, based on linear structural equation systems, has become increasingly popular as a tool for analysing decision making and com-plex problem solving. Beginning with the role of system size, connectivity, and types of relations in a system (e. g, eigendynamics and side effects) it has been discussed how formal system characteristics may influence the process of complex problem solving. In a second part, essential task demands, i. e. knowledge ac-quisition and knowledge application, have been specified further with particular respect to their interdependency. Supporting influences as to effective presentation of information and task demands have been suggested in this context: graphical presentation, semantic embedding, structural diagrams and other tutorials, in-structed strategies, specific goals, and hypotheses. Research on the impact of problem solvers' habitual intellectual and strategic abilities as well as motivation completed the presentation of the extensive contexts in which Dynamis systems have been applied and possibly will be applied to in future research.
Content may be subject to copyright.
Dynamis review: An overview about applications of
the Dynamis approach in cognitive psychology
Christine Blech
Joachim Funke
August 24, 2005
The Dynamis review has been designed to provide an overview about applica-
tions of the Dynamis approach in cognitive psychology. Since the development of
computer-based scenarios Dynamis, based on linear structural equation systems,
has become increasingly popular as a tool for analysing decision making and com-
plex problem solving. Beginning with the role of system size, connectivity, and
types of relations in a system (e. g, eigendynamics and side effects) it has been
discussed how formal system characteristics may influence the process of complex
problem solving. In a second part, essential task demands, i. e. knowledge ac-
quisition and knowledge application, have been specified further with particular
respect to their interdependency. Supporting influences as to effective presentation
of information and task demands have been suggested in this context: graphical
presentation, semantic embedding, structural diagrams and other tutorials, in-
structed strategies, specific goals, and hypotheses. Research on the impact of
problem solvers’ habitual intellectual and strategic abilities as well as motivation
completed the presentation of the extensive contexts in which Dynamis systems
have been applied and possibly will be applied to in future research.
Correspondence concerning this report should be addressed to Joachim Funke, Department of
Psychology, Universit¨at Heidelberg, Hauptstraße 47-51, D-69117 Heidelberg, Germany.
E-mail: (
1 General introduction 3
2 The Dynamis approach 3
3 Psychological research based on Dynamis 5
3.1 Systemcharacteristics ............................ 6
3.1.1 The number of variables . . . . . . . . . . . . . . . . . . . . . . . 6
3.1.2 The number of relations . . . . . . . . . . . . . . . . . . . . . . . 9
3.1.3 The quality of relations . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 Taskcharacteristics.............................. 15
3.2.1 Format of presentation . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2.2 Taskdemands............................. 20
3.3 Personcharacteristics............................. 36
3.3.1 Strategies and systematic learning . . . . . . . . . . . . . . . . . . 37
3.3.2 Motivation............................... 39
3.3.3 Intelligence .............................. 44
References 47
1 General introduction
The use of computer-simulated scenarios in problem solving research has become in-
creasingly popular during the last 25 years (for a representative collection of papers see,
e. g., the two editions from Sternberg & Frensch, 1991, and Frensch & Funke, 1995).
This new approach to problem solving seems attractive for several reasons. In con-
trast to static problems, computer-simulated scenarios provide the unique opportunity
to study human problem solving and decision making behaviour when the task envi-
ronment changes concurrently to subjects’ actions. Subjects can manipulate a specific
scenario via a number of input variables (typically ranging from 2 to 20, in some ex-
ceptional instances even up to 2000), and they observe the system’s state changes in a
number of output variables. In exploring and/or controlling a system, subjects have to
continuously acquire and use knowledge about the internal structure of the system.
Research on dynamic systems was motivated partly because traditional IQ tests turned
out to be weak predictors in non-academic environments (see Rigas & Brehmer, 1999, p.
45). According to their proponents, computer-simulated “microworlds” seem to possess
what is called “ecological validity”. Simulations of (simplified) industrial production
(e. g., Moray, Lootsteen, & Pajak, 1986), medical systems (e. g., Gardner & Berry, 1995),
or political processes (e. g., D¨orner, 1987) have the appeal of bringing “real world tasks”
to the laboratory. Brehmer and D¨orner (1993) argue that these scenarios escape both
the narrow straits of the laboratory and the deep blue sea of the field study because
scenarios would allow for a high degree of fidelity with respect to reality and at the same
time allow for systematic control of influential factors.
2 The Dynamis approach
In everyday life, a number of activities require the regulation and control of processes
which consist of quantitative variables (e.g., driving a car, controlling a CAD-machine).
Not only technical but also economic and ecological situations require that we first have
to understand the system before goal-oriented action is possible. In many sciences,
systems with quantitative variables are represented successfully by means of the general
linear model (cf. Stevens, 1992). The use of linear structural equation systems as a
tool for problem solving research has been introduced by Funke (1985) under the name
of Dynamis (which was the name of the first software shell for realising this type of
How can such a linear Dynamis model be used as a tool for analysing decision making
and problem solving? The subject is instructed that she or he has to deal with a system
that consists of some exogenous and endogenous variables. The exogenous ones can be
directly manipulated by the subject and, thus, can influence the endogenous variables
which can not be manipulated directly. The general task is (a) to find out how the
exo- and endogenous variables are related to each other, and (b) to control the variables
in the system so that they reach certain goal values. Normally, these two subtasks of
system identification and system control are separated experimentally as two steps of
the whole task (see Funke, 1993).
The basic structure of a simple linear Dynamis system, for example, consisting of four
Figure 1: Structure of the simple linear Dynamis system with two input variables A and B as
well as two output variables Y and Z. Variables are represented as boxes, connections between
them are marked by weighted arrows.
variables is shown in Figure 1 (adopted from Vollmeyer & Funke, 1999). Instead of
labelling the variables semantically, abstract letters are used. A system which contains
semantics from biology can be found in Vollmeyer, Burns, and Holyoak (1996).
In the example system from Figure 1, the variables A and B represent the exogenous
variables which have an effect on the endogenous variables Y and Z. The numbers on
the arrows represent the weight with which the respective exogenous variables affect
the endogenous ones. The system is described formally by two equations (one for each
endogenous variable):
Yt+1 = 2 At(1)
Zt+1 = 3 At2Bt+ 0.5Yt+ 0.9Zt(2)
In these equations, the indices tand t+ 1 represent the actual state of the system which
itself goes on in discrete steps (= periods) on the time axis. From equation (1) it turns
out that the value of Y at period t+ 1 can be calculated from the value of A at period
t, times two. Similar, in equation (2) the value of Z at t+ 1 can be calculated from the
exogenous variables A and B at period t(with weight 3 and 2), from the value of Y
at that time (weight 0.5), and from its own value at period ttimes 0.9. Normally, such
a system is presented on a screen where all the variables are shown together with the
system’s history (for a certain period of time). What is not shown to the subjects is the
structure of the system because it has to be discovered by them during the exploration
In some systems, the endogenous variables have effects on other endogenous ones (in
Figure 1, the effect from Y to Z), an effect which one might label as “indirect effect”which
shows up only in case of manipulating the exogenous variable A. This variable A has
itself two effects, one being larger (“main effect” on Z), one being smaller (“side effect”
on Y). Also, endogenous variables can influence themselves (in the example shown with
variable Z), thus representing an effect one might call “eigendynamic” because of the
constant increase or decrease of this variable independent of other influences. As the
reader might imagine, there are many possibilities to construct linear systems with a
full range of effects of the kind described above and, thus, making identification and
control of such Dynamis systems to a hard problem. The two main task demands are
knowledge acquisition and knowledge application.
Task Demand 1: Knowledge Acquisition. The term “knowledge acquisition” (sys-
tem identification) describes a complex learning situation during which the subject has
to find out details about the connectivity of the variables and their dynamics. The
structural aspects of the system (= connectivity) cannot easily be separated from the
dynamic aspects because the system itself can only be analysed interactively over the
time course.
In the Dynamis situation, this identification problem requires an identification strategy,
that is, a certain way of manipulating the exogenous variables so that you can derive from
the consequences (in terms of values of the endogenous variables) the causal structure
of the system or at least to come to hypotheses about this structure which could be
tested subsequently. Identification of system relation can occur in different levels: (a)
as identification of the existence or non-existence of a relation, (b) as specification of a
direction, (c) as specification of qualitative aspects of this (either positive or negative)
relation, and (d) finally as the exact quantitative specification of the weight of this
Task Demand 2: Knowledge Application The term “knowledge application” (sys-
tem control) describes the situation of applying previously acquired knowledge in order
to reach a certain goal state within the system. The goal specifications are normally
given by the experimenter.
In the Dynamis situation, knowledge application requires two subgoals: first, to trans-
form a given state of the endogenous variables by means of an input vector into the vector
of goal values, and second, to keep this goal state on a stable level because in a dynamic
system the goal state – once reached – may disappear quickly due to “eigendynamics”.
3 Psychological research based on Dynamis
The main aim of the present report is to provide a summary of various applications
and research contexts to which computer scenarios of the Dynamis family have already
been applied or possibly could be of use in the future. Central results and interesting
methodology especially of recent Dynamis studies are presented to give an introduction
to the most relevant findings in the extensive domain of Dynamis research.
The overview consists of three parts with subchapters: Part one deals with basic terms
and formal characteristics of Dynamis systems, i. e. the Dynamis algorithm. Experi-
mental studies examining the effects these characteristics have on the process of complex
problem solving (CPS) will be discussed. Part two presents findings and theories con-
cerning task characteristics some of which may be implemented in Dynamis, other of
which refer to the general experimental setting. The third and final part focuses on
the person interacting with the Dynamis task. Findings on some selected influences of
personality such as motivation and intelligence will be discussed in the context of CPS.
3.1 System characteristics
As noted above, system characteristics in Dynamis are integrated in the formal algo-
rithm underlying a problem solving scenario and can therefore be termed the “core” of
any Dynamis program. A very close connection exists between these formal charac-
teristics and the complexity of a system. Asked to imagine a complex system, in the
first place one would probably think of a large number of different variables, an even
larger number of relations between these variables, the interconnections being hardly
detectable and not at all transparent. The more possibilities to interact with a system,
the more difficult it will be handled, one might expect. Yet experimental findings about
such features are a bit more intricate than just confirming the basic notion that high
complexity means low performance.
Three aspects of system complexity will be considered in the following sections: the
number of variables in a system, the number of relations and the quality of relations
including special types of relation such as parallel or side effects, eigendynamics and
effect delays.
3.1.1 The number of variables
The most obvious feature of a CPS scenario possibly is the number of the system’s
variables. This will also be referred to as the system’s size. Numbers of variables range
from as few as two variables in the smallest systems to more than 2000 variables in the
largest scenarios such as the well-known Lohhausen scenario developed by D¨orner et al.
(for a brief survey of CPS scenarios in psychological research see Funke, 1992b, p. 8-9).
CPS scenarios intended to be more “naturalistic”, i. e. closer to problems in real life,
usually employ larger numbers of variables whereas in Dynamis research, less variables
are common. Most of the studies discussed in the present paper applied Dynamis
systems containing six, seven or eight variables.
Systematic attempts to find out how problem solving is affected by different sizes of
Dynamis scenarios have been undertaken by Funke (1992a, 1992b). In a study based
on the data of 40 participants (students at the University of Bonn) Funke compared
problem solving under two conditions: a “low task difficulty” scenario with only two
goal variables to be controlled versus a “high task difficulty” scenario with four goal
variables to be controlled. The system presented to subjects of the two experimental
groups was, in fact, the same, but the task was different: In the “low complexity task”
two out of four goal variables simply had not to be considered as goal variables.
The complete scenario consisted of eight variables, four exogenous, four endogenous
variables. The employed system called Alt¨
ol (used oil) semantically refers to pollution
by used oil including exogenous variables such as “control procedures” or “price of crude
oil” and endogenous variables such as “maritime pollution” and “air pollution” (Funke,
1992b, S. 123).
Further experimental variation complemented the design: It was tested for effects of
prior knowledge, effects of different degrees of controllability and effects of presentation
format, i. e. whether input and output information were displayed as either numbers
(numerical type of presentation) or in charts (graphical type of presentation).1
With respect to dependent variables two types of measures appear relevant in this
context, claiming a brief explanation: the concept of control performance and the quality
of prediction.
Control performance provides a measure of how well problem solvers are able to reach
and maintain the goal states they aim at. At least since the 1990s, achievements in
control performance are usually computed as the mean deviations of the output values
actually observed from their corresponding goal values.
Quality of prediction is a less frequently employed diagnostic tool which, in Dynamis
research papers, has first been mentioned by Funke and M¨uller (1988). It is a measure of
how well participants are able to predict future states of a CPS system, once the subjects
know the current state and following interventions. Quality of prediction is computed
in a similar manner as control performance: by the mean deviation of predicted values
from the values that would – according to the system’s underlying algorithm – actually
have followed the current status if the intervention had taken place.
Exactly as expected, Funke’s results showed a significantly lower control performance
for the group dealing with the “high task difficulty” scenario. Obviously, it was easier
to control only two rather than four variables. However, this finding was confined
to problem solving under the condition of a numerical type of presentation. When
input and output values were displayed graphically, the number of goal variables had no
effect on control performance. The author only briefly points at a possible explanation:
Perhaps only for numerical presentation, participants had the chance to analyse changes
in states of input and output variables thoroughly enough so that different levels of
control performance could occur.
Secondly and not expected, but in line with the first result, it was found that subjects
predicted future system states significantly better when task difficulty was low (two goal
variables) than when it was high (four goal variables). On the whole, the experiment
supports the assumption that on several dimensions smaller systems, i. e. systems con-
taining fewer variables, are easier to deal with than larger systems with more variables.
This general interpretation has yet given rise to more specific research. An experiment
conducted by Preußler (1997) suggests that in addition to a system’s mere number
of variables, the variables’ quality may be just as important or even more important.
Preußler’s experiment showed that under certain conditions subjects controlling a dy-
namic system in fact perform better if the system contains more variables.
60 students at the University of Bayreuth took part in the study. Their task was to
control a Dynamis system named Linas (linear additive system). As in a typical
CPS experimental paradigm relations and outcomes of interventions were unknown a
priori. Since Linas variables have fancy names (e. g, Bulmin, Ordal, Trimol) no prior
semantic knowledge can be expected to play a role in this task. Two different versions
of Linas were presented to the subjects: Half of the students dealt with a simplified
version which involved a set of four input variables and three output variables (control
group). The other half (experimental group) learned to control a full version of Linas.
1These further experimental factors, however, will be discussed below in more detail.
Four additional variables were included, but – though participants were not informed
about this fact – the extra variables were irrelevant to system control. Subjects in both
conditions controlled Linas for eight rounds with eight trials each. After that, they
accomplished a decision task which was intended to measure the structural knowledge
subjects had found out about Linas. Pairs of variable names were presented, and
participants had to decide whether there had been a relation between these two variables
or not.
The major results concentrate on both control performance and the amount of knowledge
acquired under the two conditions: Control performance, at least at the end of round
eight, was significantly better in the experimental group as compared to the control
group. Yet with respect to the acquisition of knowledge, the pattern was reversed, i. e.
subjects controlling the simplified version outperformed those controlling the extended
system; they had found out more about the relations within Linas.
What might explain these findings? To Preußler, the results are exactly as predicted.
Larger systems with irrelevant additional variables provide redundant information which
requires cognitive capacity and hence means extra effort to subjects. That is why, ac-
cording to Preußler, the experimental group did not show any superior performance in
the initial rounds. At the end of the task, however, the benefits of redundancy outweigh
its advantages. The larger system provides a “multiple learning context”, learning con-
ditions which could have allowed participants to try different approaches to the same
problem. Just as with transfer tasks in traditional learning psychology, multiple contexts
facilitate the control of the dynamic system. Preußler concludes that “learning contexts
requiring additional cognitive resources for problem solving can have differential effects
on the acquisition of structural knowledge and the acquisition of control knowledge” (p.
48). Thus, cognitive load due to redundant information obviously can enhance control
performance and at the same time impair the acquisition of structural knowledge.
Another complementing contribution as to the impact of the number of variables in
Dynamis has been proposed by Strauß (1993) in experimental studies and theoretical
reflections. Strauß argues that in problem solving scenarios the number of variables
per se is less decisive as to controlling and gaining knowledge than the question of how
easily performance goals could be accomplished even if subjects were acting totally at
random. In formal terms: Compared to all possible solutions, which is the amount of
interventions leading to a certain goal state within a defined period of time? Strauß has
termed this ratio the “share of correct solutions” (Strauß, 1993, p. 60). Using vector
algebra the share of correct solutions can either be computed exactly from the full set
of possible solutions or it can be estimated from samples.
In experimental studies Strauß has shown that performance in CPS scenarios can indeed
be affected by this formal system characteristic. He compared subjects dealing with
problem solving systems that were identical but for the share of correct solutions. In
line with Strauß’s assumptions, higher shares of correct solutions were associated with
higher control performance whereas lower shares resulted in lower performance.
A second experiment revealed that even if two treatment conditions differed in the
number of variables included, the share of correct solutions kept constant, regarding
control performance no marked differences between the groups were found. Accordingly,
for successful controlling of a dynamic system the system’s size hardly matters as long
as it is easy enough to reach defined goal states when merely guessing.
Strauß pleads for the share of correct solutions to be treated as a system’s formal feature
which supplements the number of variables. Neither the number of variables nor the
share of correct solution considered on its own is convincing; instead, both concepts seem
confounded with one another. For full report, it should yet be mentioned that Strauß’s
findings proved to be valid for control performance, but that to CPS processes there
are more determining factors than just this. With respect to the amount of knowledge
acquired the share of correct solutions had no such effects in Strauß’s experiments.
3.1.2 The number of relations
A second factor which determines the formal complexity of a system is the number of
relations between variables. Even large systems with many variables may appear rather
simple if there are only few interconnections between these variables. On the other hand,
comparatively small systems can be constructed, involving, say, as few as six variables
and yet being most intricate to solve since almost all variables affect one another in an
unknown way. In research literature, this concept has been named the connectivity (“
Vernetztheit”) of a complex system.
Effects of connectivity as an experimental treatment factor have been investigated as
early as 1985 by Funke (1985). The study involved a Dynamis CPS system named
Okosystem (ecosystem) whose cover story refers to gardening in an ecological context
(the variables being labelled as “poison”, “varmints”, “fertiliser”, “beetles”, “water
pollution”, “number of leaves”). In ¨
Okosystem there are three exogenous and three
endogenous variables, i.e. a total number of six variables. For experimental study, the
number of relations was manipulated to provide three different conditions: four relations
in the condition of low connectivity, six relations for medium connectivity, eight relations
for high connectivity. A further experimental condition concerning the presentation
format resulted in a design with six groups, ten subjects per group. Participants task
was to control goal parameters in a fictive ecosystem for five rounds with seven trials
In addition to control performance Funke measured the amount of knowledge subjects
gained while exploring ¨
Okosystem. Unlike in Preußler’s study employing the “pair-
task”, Funke used the method of causal diagrams. In this paradigm, problem solvers
are asked to formally express their findings about causal relationships with the help of
arrows and symbols in diagrams. They draw arrows from one rectangle displaying a vari-
able’s name to another once they know that any relation exists between these variables
(knowledge of relation). If they have found out that increments in one variable increase
the outcome of another variable or that by increments in one variable another variable
will decrease they add a plus or minus sign, respectively (knowledge of direction). In
case they even know the exact numerical factor underlying a relation they put down a
number (knowledge of strength). Hence, knowledge can be analysed on three increasing
levels of accuracy. For each type of knowledge the quality of system identification is
calculated from a formula involving the number of correctly identified items relative to
the maximum detectable number of items and the number of mistakes.
The experiment supports the author’s expectation that higher connectivity yields both
lower control performance and less causal knowledge than low connectivity conditions.
Significant differences between the connectivity conditions were found for all three grad-
uations of knowledge with knowledge of relation being most extensive, knowledge of
direction having an intermediate state and knowledge of strength being relatively sparse
in all three experimental groups.2More recently, Kluge (2003) reported comparable
effects as to the impact of connectivity on CPS.
The interpretation that connectivity in a problem solving scenario can serve as a measure
of task complexity appears to be obvious. It was Strauß (1993) again who pointed
at a further explanation. Re-analysing Funke’s experiment Strauß demonstrated that
connectivity, the number of relations in ¨
Okosystem, has been confounded with the
share of correct solutions as defined above: the less relations, the larger the share of
correct solutions, the more relations, the smaller the share of correct solutions. As Strauß
illustrates, controlling a system with only four relations between variables would be more
successful than controlling an eight-relations-system even for random interactions with
the system. These limitations may be taken into account even though one does not wish
to fully abandon the first straightforward interpretation.
3.1.3 The quality of relations
The concept of connectivity can be specified further according to the types of relations
between variables. For example, it may occur that input variables affect more than but
one output variable each. In turn, output or endogenous variables can depend on a
set of several input or exogenous variables. There may be parallel effects. Just as well
it is possible that endogenous variables show influence on other endogenous variables,
indirect effects which have been referred to as side effects. A third possibility concerning
the quality of relations is the effect of an endogenous variable on itself. Challenged by
problems in real life where systems’ states often change without active interventions,
without noticeable influences from known outside causes, in complex problem research,
too, eigendynamics have become an established concept and experimental paradigm. A
formal definition of eigendynamic (“ Eigendynamik”) is provided by Funke: “Eigendy-
namik means that an endogenous variable at time thas an effect on its own state at
time t+ 1 independent of exogenous influences which might add to the effect” (Funke,
1993, p. 322).
Parallel effects. Parallel effects have been studied with regard to two questions:
Given the fact that in a dynamic system one input variable affects more than one output
variable, what is the impact of different numerical weights shaping the relations between
input and output variables? Will the strongest relation, i. e. the relation formed by the
largest numerical weight, be detected and controlled more easily than other relations?
Secondly one may ask under which conditions a system will be identified and controlled
more successfully – if there is one input variable with parallel effects on several output
2The finding reported here applies to experimental conditions in which interactions with the system
had immediate effects on the outcome states of the next trials (immediate feedback). In systems with
so-called delayed feedback effects of connectivity on knowledge and performance were not quite as
distinct. The influence of feedback delays will be described in a separate section at the end of part one.
variables or if there are several input variables that add to the outcome state of a single
output variable.
The impact of parallel effects of different strength has been studied by Funke (1992b).
As part of a larger series of studies examining side effects and eigendynamics, on the base
of causal diagrams drawn by forty-eight subjects Funke analysed how well participants
detected relations in a dynamic system called Sinus. The two relevant relations to be
compared were an input-output-relation with a large numerical factor of 3 (dominant
effect) and an input-output-relation with a much smaller numerical factor of only 0.5
(subdominant effect). The input variable was the same to both relations, the two output
variables were different.
Frequency analyses were conducted in order to know in which category most of cases
would be found: subjects who noticed the dominant effect at first and then the sub-
dominant effect, subjects who noticed the subdominant effect earlier than the dominant
effect, subjects who noticed both effects at the same time or subjects who noticed neither
the dominant nor the subdominant relation. Since the majority of participants identified
the dominant relation earlier than the subdominant relation, it is obviously easier to
detect relations with larger impact on other variables although we do not know to what
extent identification of a system is facilitated by increasing dominance. Neither there
have been findings relating dominance or subdominance to successful or less successful
control performance so far.
Another experiment described in the same publication (Funke, 1992b) dealt with the
second question risen above, the impact different patterns of parallel effects have on
solving a complex problem. In this context a formal concept from cybernetics (Ashby,
1958) has been introduced into the domain of CPS: the degree of controllability. Con-
trollability in technical terms means the ratio of two numbers of variable classes: the
number of variables that influence or control other variables divided by the number of
variables that are influenced or controlled. In Dynamis scenarios a system’s controlla-
bility is thus determined by the numbers of exogenous and endogenous variables. The
more exogenous variables as compared to endogenous variables, the more easily the sys-
tem should be controlled. Funke’s idea was to figure out whether this theoretical and
rather technical notion fits with control performance shown by human problem solvers.
The design of the study has already been described in a previous section when dealing
with system complexity due to the number of variables. Besides the number of vari-
ables and two further experimental variations concerning the Dynamis scenario Alt¨
Funke’s experiment comprised three different degrees of controllability as a within sub-
ject factor. Within the whole of eight variables three independent subsystems could be
separated: One exogenous variable affecting two endogenous variables (1:2 relation or
low controllability), one exogenous variable affecting one endogenous variable (1:1 rela-
tion or medium controllability) and two exogenous variables affecting one endogenous
variable (2:1 relation or high controllability). For statistical analysis data of 40 students
were available.
The results showed effects contrary to the expectations. In fact, significant differences as
to control performance for the three subsets occurred, but performance was best when
the ratio of exogenous and endogenous variables was 1:2, i. e. for the subsystem which
had been named the low controllability condition. In turn, it was lowest for the reverse
relation, which should have facilitated control performance according to the author’s
hypothesis. Medium effects were found for the 1:1 relation. In this condition high
performance resulted if the variables’ states had been presented in numbers whereas for
graphical display performance was just as low as in the so-called high controllability
Reflecting on these findings Funke has proposed methodological problems as alternative
explanations, i. e. lacking independence of the subsystems from semantic features. So it
remains unclear whether the psychological notion of controllability corresponds to the
technical concept or not.
Side effects. Side effects in the context of CPS, too, have been studied by Funke
(1992a, 1992b, 1993). For the study on side effects a Dynamis system named Sinus
was chosen. Sinus comprises a total number of six variables, three endogenous variables,
three exogenous variables, these being linked by four basic relations. In a standard ver-
sion, two additional relations are implemented, one an instance of eigendynamics, the
other a side effect of one endogenous variable on another. When dealing with Sinus par-
ticipants are asked to control population numbers of six different tribes on a fictive planet
in a solar system different from ours. The tribes are named “Gaseln”, “Schmorken” and
“Sisen”, the endogenous variables or population numbers that have to be controlled,
and “Olschen”, “Mukern”, “Raskeln”, the exogenous variables or population numbers
that can be manipulated by direct interventions. Relations between the six tribes on
the fictive planet have to be explored and controlled for five rounds consisting of seven
trials or simulated weeks. According to the standard procedure as employed in Funke’s
experiment rounds one to four are intended to serve knowledge acquisition due to the
exploration of Sinus. Specific goal values to be aimed at are only given as late as round
five when the main task demand is applying knowledge gained in earlier rounds.
In order to examine the impact of side effects on knowledge and performance Funke mod-
ified the system structure so that three increasing degrees of side effects were realised:
One experimental condition without any side effect at all, a second condition imple-
menting one small side effect with a numerical magnitude of 0.2 and a third condition
implementing two side effects of 0.2 and 0.5.
Statistical analysis was based on scores of control performance and causal diagrams from
24 students, eight subjects per group. It was found that increasing side effects led to a
linear decrease in the amount and quality of knowledge just as assumed prior to testing.
The same effect occurred as to control performance which also decreased for systems with
growing numbers of side effects. This fact has been interpreted as a logical consequence
of limited knowledge acquisition associated with the side effects. Interestingly, as more
detailed analysis revealed, in identifying the system’s formal structure, subjects tended
to compensate for mistakes in a rather systematic way: In case side effects had not been
detected participants typically assumed additional relations between variables which, in
fact, had not been manipulated at all.
Eigendynamics. Concerning the effects of eigendynamics an experiment analogous to
the study on side effects was conducted (Funke 1992a, Funke 1992b, Funke 1993). Again
24 students worked on the dynamic system Sinus, following the procedure described
above. This time different degrees of eigendynamics were implemented in Sinus (while
the number of side effects was kept constant): One version without any eigendynamics
was provided, another version including one instance of eigendynamic with a magnitude
of 0.9 and a third version including two instances of eigendynamic, the magnitudes being
0.9 and 1.1.
The magnitude of eigendynamics represents a numerical factor by which a variable’s
outcome value is multiplied from trial to trial. In case no eigendynamic contributes
to the outcome state, this factor is set to the value of 1 (which is the default value in
Dynamis). Accordingly, eigendynamics defined by the factor 0.9 lead to a ten percent
decrease of the former state in each following trial. Speaking in terms of the Sinus cover
story: Sisen, the Sinus tribe associated with eigendynamics of 0.9, will lose ten percent
of its population each week and gradually die out unless appropriate interactions are
undertaken. The reverse effect on Gasen, the Sinus tribe associated with eigendynamics
of 1.1 will lead to a ten percent increase in population number if no counteracting
measures take place.
Just like for growing degrees of side effects the author assumed that increasing levels
of eigendynamic, too, would deteriorate both the quality of control performance and
the quality of system identification, i. e. the amount and type of knowledge. The
expectation was confirmed as to the quality of control performance since the three
groups significantly differed in the predicted manner. The condition containing two
instances of eigendynamic turned out to be especially hard to control. There was,
however, no corresponding effect on the quality of identification. According to Funke,
“causal dependencies were detected equally well under all three conditions” (Funke,
1993, p. 323).3He concludes that acquisition of knowledge and its application in the
control task are different tasks requiring different abilities. Dissociations may therefore
Perhaps still more interesting than considering side effects and eigendynamics each on
their own it is to compare and combine the findings. The above results suggest that
both side effects and eigendynamics, adding complexity to a problem solving task, make
scenarios such as Sinus more difficult to handle. While successful identification of the
system’s structure obviously remains unaffected by increasing levels of eigendynamics,
side effects impair the gaining of knowledge as well.
In order to corroborate the idea that in CPS systems side effects are harder to detect than
eigendynamics, Funke applied combined frequency analyses on comparable data from
three experiments of equal structure (the two experiments described above plus a third
analogous study). Causal diagrams were selected from 48 participants who had worked
on a Sinus version including one instance of eigendynamics and one side effect. The
subjects were classified according to four exclusive categories: subjects who detected
eigendynamics prior to the side effect, subjects who detected the side effect prior to
eigendynamics, subjects who detected both eigendynamics and the side effect in the
same round and subjects who detected neither of these relations. The first category was
considered as in line with the expectations, the second category directly opposed it, the
3In an earlier publication Funke (1992b) notes that older measures of control performance and
knowledge yielded effects for quality of identification as well. Obviously, however, the more recent
measures should be preferred.
other two categories were deemed neutral or irrelevant to the expectations.
From a statistically significant ratio of 20 cases supporting the hypothesis and only ten
cases opposing it, it was concluded that in Sinus eigendynamics indeed were easier
to identify than side effects. A closer look at typical faults in the identification of the
system revealed that often side effects which had remained unnoticed had been mistaken
for additional eigendynamics leading to about the same outcome values.
Why could eigendynamics in Sinus be detected with relatively little effort? As Funke
(1993, p. 323) points out, there is a rather simple strategy to notice eigendynamics in
a CPS scenario: leaving the system to itself without any active interventions for one
or more trials. This strategy can be applied in a very appropriate manner especially
during the first four rounds of the experiment, i. e. when no specific goal values are pro-
vided. Under these conditions free exploration is both possible and desired. Unfortunate
outcomes of interventions or omitted interventions do not matter as long as relations
within the system are well identified. Certainly, in real life problems many people would
rather refrain from strategies named “just wait and see”. That is why from our studies
we cannot necessarily conclude that eigendynamics are less relevant to problem solving
than, e. g, side effects or parallel effects.
Feedback delays. Another feature finally appears to be typical of complex problems
in the outside world beyond science laboratories. In reality, interventions rarely yield
effects that can be noticed immediately after an operation has been carried out. Some
effects may occur at once, others only occur after some time if at all.
In Dynamis effects in the long term have been modelled by means of feedback delays.
In the Dynamis systems presented so far changes in a variable at a defined time thad
an impact on outcome states of the trial following immediately, i. e. time t+ 1. This case
shall be termed immediate feedback. Yet changes at tdo not necessarily have to affect
outcomes in t+ 1; instead they can “skip” one trial and affect, say, outcomes in t+2 or
in times even further in the future. In this case the system’s reaction to an intervention
is deferred.
The already mentioned study by Funke (1985) employing ¨
Okosystem was conducted
to examine the role of feedback delays in CPS besides the impact of connectivity (see
above). Including three different levels of complexity due to connectivity two different
types of systems were constructed, one which contained immediate feedback exclusively
and another in which part of immediate feedback relations were replaced by correspond-
ing relations with delayed feedback. Half of the 60 participants were assigned to work
on each system.
It was expected that feedback delays should be more difficult to identify and to control
than immediate feedback. Indeed, the results showed more and better knowledge gained
by participants who had been given immediate feedback instead of delayed feedback.
Significant effects appeared for three distinct types of knowledge: knowledge of rela-
tion, knowledge of direction, and knowledge of strength. With regard to interactions
between feedback delay and the second experimental manipulation, the complexity due
to connectivity, the feedback manipulation seemed to give rise to differential effects of
connectivity. While for subjects solely confronted with immediate feedback increasing
connectivity consistently resulted in decreasing accuracy of knowledge, in the condition
of delayed feedback it was only the medium connectivity group who identified less rela-
tions than the low connectivity group; participants in the condition of high connectivity
identified the system almost as well as the low connectivity group. Funke has attributed
this unexpected finding to enhanced effort and extra time spent on the task by subjects
dealing with both delayed feedback and high connectivity. Effects of feedback delays
on control performance, unfortunately, turned out to be unsuitable to statistical test-
ing. Since in recent Dynamis studies feedback delays have hardly been examined we
can only guess that the immediate feedback should rather facilitate control performance
than systems including feedback delays.
3.2 Task characteristics
Occasionally recurring to the basics of part one, in the following sections we will consider
broader applications of CPS scenarios. Leaving the mere formal attributes of Dynamis
systems we will shift the focus to the interaction between the person, i. e. the problem
solver, and the system.
How should information be presented in order to facilitate working on a complex system?
What are relevant and appropriate task demands providing an insight into the process
of CPS, and how are differential demands – such as the acquisition and application of
knowledge – interrelated? Finally, which methods can be employed in terms of training
or task instructions to support successful interactions with the system? These may be
the leading questions to the section about task characteristics.
3.2.1 Format of presentation
The first question risen above was: How should information be presented in order to
facilitate working on a complex system? Although it seems nearly impossible to find a
definite answer applying to all instances of CPS we will have a closer look at the impact
two prominent task characteristics have on different facets and demands in problem
solving: Numerical vs. graphical format of presentation and semantic embedding.
Numerical versus graphical format of presentation. In the sections on system
characteristics it has been briefly pointed at the fact that in Dynamis scenarios infor-
mation as to the state of input and output variables can be displayed either in numbers
or in diagrams, which implies two different modes of the user’s interface. All of the
studies discussed above – unless it has been noted explicitly – dealt with systems of the
numerical format. According to Beckmann (1994, p. 65) this mode of presentation is
far more common than graphical interfaces. Recent scenarios of the numerical mode
show variables’ states as numerical values in tables. For interventions numbers have to
be entered by using the keybord.
In his own studies Beckmann employed two Dynamis systems of the graphical type,
Kirschbaum (cherry tree) an Maschine (machine).4In these scenarios exogenous
variables can be manipulated by means of three different mouse buttons on a control
4In fact, the systems are identical in their formal structure, but different in semantic embedding.
This will be considered more closely in the following section.
panel: one button for zero interventions, another button for increasing the value of
a chosen exogenous variable, the third button for decreasing the value of a chosen
exogenous variable. The relative size of the values can be inferred from the length
of a vertical bar: the longer the bar, the higher the corresponding numerical value.
Positive or negative values are symbolised by the bars position above or below a “zero
intervention line”, respectively. Line diagrams, on the other hand, present the course of
previous states and actual states relative to top and bottom limitations and relative to
goal values marked by a straight horizontal line.
Another descriptive example of graphical features in Dynamis is given by the dynamic
system MultiFlux (Kr¨oner, 2001). MultiFlux has been developed as the simulation
of a fictive machine. The machine consists of four regulators or control devices, i. e.
the exogenous variables, and four instruments depending on the regulators, i. e. the
endogenous variables. Variables are labelled control devices 1 to 4 and instruments
1 to 4. To problem solvers the task is to adjust regulators so that the instruments
will reach defined goal adjustments. In doing so, they are confronted with a graphical
display optically similar to the control buttons in real technical instruments such as hi-fi
While in Beckmann’s and Kr¨oner’s studies it has not been the primary aim to examine
impacts of graphical presentation on CPS, Funke (1992a, 1992b) conducted experimen-
tal comparisons of the two different modes of presentation. In studies based on the
data of 80 students he analysed problem solving in a graphical vs. a numerical ver-
sion of the already mentioned eight variable system Alt¨
ol. Almost as in Beckmann’s
Kirschbaum and Maschine the graphical interface displayed the course of outcome
values in line diagrams. An additional horizontal line indicated the level of the corre-
sponding goal value. Type of presentation was one experimental factor among others
such as task difficulty, degree of controllability and a manipulation referring to the role
of prior knowledge.
Former studies dealing with scenarios other than Dynamis systems had suggested that
control performance would be facilitated if information were displayed in diagrams as
compared to numerical tables (see H¨ubner, 1987, 1988). Opposing that view, Funke
(1992a) hypothesised that subjects dealing with numerical interfaces would outperform
the graphical presentation group in control performance, but that the identification
of the system would be better if graphical information were provided. Yet neither of
the these expectations could be supported (nor could be the opposite). Only when
additional factors such as task difficulty were taken into account significant effects of
interaction appeared: Provided that participants controlled a system with only two goal
variables (low task difficulty) better identification of the system’s relations was found
for the numerical than for the graphical condition. In turn, if four goal variables had
to be controlled (high task difficulty), participants in the graphical condition identified
the system’s structure more easily. Funke concludes that “presentation format per se is
not a critical factor. However, it is obvious that depending on the nature of the task,
differential effects occur: In order to cope with the more complex task the graphical
presentation which is less precise in presenting system information yields better results
(Funke, 1992a, p. 37)”.
The same notion is supported in another analysis by Funke (1992b). In this paper
the author reports a study which revealed no significant differences as to the quality
of system control and the quality of system identification. Interestingly, the format of
presentation seemed to affect some motivational factors which had been recorded by
means of questionnaires for the purpose of experimental control. At the beginning of
the task subjects confronted with the numerical presentation reported higher “failure
motivation”, i. e. higher anxiety to fail on the task, and lower readiness to make an
effort. Fortunately, however, at least in this context, impaired motivation had no strik-
ing detrimental effects on the overall performance and identification. Only for a third
diagnostic measure, the quality of prediction, numerical presentation as compared to the
graphical mode seemed to facilitate the task demands. However, since in both conditions
subjects had to predict future outcomes in numbers (instead of graphical predictions)
differential experience with the numerical mode might be regarded convincing enough
to offer an alternative explanation.
Semantic embedding and the role of prior knowledge. Thinking of abilities
and skills relevant to CPS, personality factors such as intelligence and a “feeling” for
successful strategic approaches (compare D¨orner’s concept of operative intelligence, e. g,
orner, 1986) probably come to our mind in the first place. Yet just as well we might
think of practice and the knowledge a problem solver has gained prior to the task in con-
texts associated with the task demands. This, too, should affect a person’s interaction
with a given dynamic system.
With regard to the role of prior knowledge the Dynamis scenarios discussed in the
above sections can be assigned to two classes. Scenarios such as Sinus (the fictive
planet), Linas (fancy names) and Kr¨oner’s fictive machine MultiFlux have been
constructed to be relatively neutral to prior knowledge. Although problem solvers may
be familiar with the concept of a machine, when confronted with MultiFlux, they will
not know whether and how control device number 1 is likely to affect instrument A, for
instance. Systems of these properties will be referred to as systems without semantic
embedding. In contrast, semantically embedded Dynamis scenarios like Alt¨
ol or
Okosystem refer to rather familiar contexts. Dealing with ¨
Okosystem anyone with a
basic idea of gardening and ecology can make up his or her own expectations once he or
she has but realised the variables’ names: “poison”, “varmints”, “fertiliser”, “beetles”,
“water pollution”, “number of leaves”. Will poison lead to increasing water pollution?
Will fertiliser, brought into play, induce growth and increase the number of leaves?
Hypotheses such as these may be formed immediately unless there is no prior knowledge
at all.
In order to elucidate effects of prior knowledge on CPS one common approach is to ex-
amine how “experts”, i. e. problem solvers with sufficient relevant prior knowledge, cope
with dynamic systems, as compared to “novices” without relevant prior knowledge (see,
e. g, Reither, 1981). This, however, does not concern Dynamis systems in particular
and will not be the subject of the present paper. A second approach which is more
closely related to Dynamis is found by comparing problem solving in semantically em-
bedded systems to problem solving in systems without semantic embedding. For valid
comparisons, of course, systems have to be identical in structure and user’s interface
but for the different names of exogenous and endogenous variables.
This is exactly the essence of Beckmann’s (1994) research on semantically embedded sys-
tems. For experimental studies Beckmann employed the semantically embedded system
Kirschbaum (cherry tree) which contains three exogenous variables labelled “light”,
“water supply”, “warmth” and three exogenous variables named “cherries”, “leaves”,
“beetles”, hence appealing to general knowledge. According to the cover story, partic-
ipants explore and influence growth and growth conditions of a hardly known type of
cherry tree. In Maschine, the equivalent version without semantic embedding, exoge-
nous variables are represented as control devices, endogenous variables are represented
as instrumental displays. Preliminary examinations based on a sample of pupils made
Beckmann assume that semantic embedding as a system’s property had an effect on
an identification task, but seemingly not on control performance. In order to replicate
the quasi-experimental findings he realised a first experiment which involved 40 partici-
pants, students at the University of Bonn. Half of the students explored the semantically
embedded system Kirschbaum for two rounds with seven intervention trials and after-
wards controlled the system for one round in order to accomplish defined goal values.
The other half did the same task on Maschine.
As expected, subjects in the condition without semantic embedding identified the sys-
tem’s structure better than those dealing with the semantically embedded system. A
likely interpretation to this finding is that the semantic context in Kirschbaum triggers
participants’ prior knowledge and their assumptions about relations, which, in fact, are
incompatible with the relations specified in the Dynamis linear equations (hypothesis
of knowledge incompatibility). However, more astonishing, though also expected from
the preliminary analysis, control performance was about equal in the two groups; in
spite of less knowledge gained the subjects in the semantically embedded condition did
not control any worse.
If knowledge incompatibility is indeed a crucial determinant to differential achievements
in problem solving, why should control performance remain unaffected? Beckmann ar-
gues that even with regard to knowledge acquisition “explicit knowledge incompatibility”
offers an at least incomplete explanation (see p. 172). Relations between variables in the
Kirschbaum scenario implied no serious incompatibilities to subject’s prior knowledge.
More noteworthy, a closer look at identification patterns revealed that the relations sub-
jects detected well or not so well in the semantically embedded system coincided with
the equivalent relations in the scenario free of semantics.
Instead of knowledge incompatibility, Beckmann pleads for two processes of knowledge
acquisition which are different in quality, not but in the quantity of resulting gained
knowledge. Concerning systems without semantic embedding a constructive process of
knowledge acquisition is proposed. Since, due to lacking semantics, no prior knowledge
is “activated” and no expectations are available, the default assumption about possible
relations between any two variables is that there is no relation. Only after interactions
with the system, an internal model about structural relations will be built up, succes-
sively extended and refined. The reverse process should be the case for people dealing
with semantically embedded systems. According to Beckmann, semantic contexts as
in Kirschbaum give rise to the default assumption that each variable depends on all
other variables. The structural model of knowledge is formed and refined in a reductive
process, gradually decomposing the model of general interconnection. As the reduc-
tive process takes more cognitive effort less knowledge should be gained in semantically
embedded scenarios. Beckmann’s data obviously fit well with the assumption of two
different models of processes.
But what about control performance? What can explain the fact that control perfor-
mance did not seem impaired in a semantically embedded system? Beckmann’s idea
is that there are at least two different ways of controlling dynamic systems, though
probably not equally useful. The first process is control performance based on gained
knowledge. This is, of course, assumed to be the primary process guiding participants
who cope with Maschine and do well on both the identification task and the control
task. But even for participants dealing with Kirschbaum episodes of good identifica-
tion preceded or coincided with episodes of good control performance. Only if control
cannot – because of insufficient knowledge – be based on knowledge a second process
named “ad hoc” control comes into play: basic rules or heuristics which do not apply to
a certain dynamic system in particular (“situationally unspecific heuristics”). By means
of “ad hoc” control subjects who controlled the semantically embedded version might
have compensated for limited structural knowledge.
To validate this notion, Beckmann designed a second experiment involving control con-
ditions which should only enable “ad hoc” control. Knowledge acquisition was said to
be prevented by leaving out the former two exploration phases and replacing them by
control phases with specific goal values to be accomplished. It was expected that as to
control performance no marked effects of semantics should occur then. With regard to
scenarios and general procedure, the second experiment equalled the first one. 51 school
students from Leipzig area participated.
Not fully consistent with the expectations, in experiment two results indicated some in-
fluence of semantic embedding on control performance although not all effects turned out
to be statistically significant. For both experimental conditions, performance increased
in the course of the three rounds, yet for subjects in the Kirschbaum condition the
effect was more prominent: The pupils working on Kirschbaum started with a lower
performance in the first round as compared to the those working on Maschine, in the
second round the performance of the two groups was equal and in the final third round
subjects of the semantically embedded condition even tended to control better.
Beckmann hence answers the second question as follows: Although control performance
with semantic contexts appears to be the same level as with contexts free of semantics
performance is indeed impaired. Consequences of the deteriorating effect of semantic
embedding, however, are only obvious and determinant at the beginning of the process
of CPS. Early phases of exploration or additional rounds of control, fostering successful
“ad hoc” control will make the effect vanish. As Beckmann illustrates impaired control
performance due to semantics, “it is obviously easier to understand and control a system
if ‘you know that you know nothing’ (Maschine) than if ‘you think you know something’
(Kirschbaum)” (Beckmann, 1994, p. 199). Further research on semantic embedding
and the role of intelligence in CPS is provided by Beckmann and Guthke (1995, see
Besides comparing experts vs. novices and experiments on semantic embedding there is
a third approach to study impacts of prior knowledge on problem solving. While in the
experimental designs described above systems with exactly equal structural relations,
but a different “surface” have been employed, it is just as possible to compare two sys-
tems, both semantically embedded, with the same number and names of variables; these
will look the same in superficial characteristics, but the underlying structure may be dif-
ferent. In research done by Funke (1992a, 1992b) two such systems, different versions of
the Alt¨
ol scenario were constructed. They were even identical in the general structure
but for the signs of parameters in linear equation models. One version, the matching
condition, was assumed to coincide with participants’ prior knowledge while the other
version, the mismatching condition which contained inverted signs of parameters, dis-
agreed with expected knowledge. It was statistically tested for effects of knowledge
compatibility on the three measures quality of system control, quality of system identi-
fication and quality of prediction. Each of these variables turned out to be significantly
impaired for the mismatching condition, the Alt¨
ol scenario containing counterintuitive
relations. The results, reflecting straightforward expectations, once again confirm that
prior knowledge definitely requires consideration in cognitive research, even though in
most cases little is known about knowledge specific persons have about specific tasks
(compare Funke, 1992b, p. 139).
3.2.2 Task demands
As the introduction and preceding chapters have underlined already, there are at least
two essential demands to a person dealing with a complex dynamic system. Firstly,
he or she has to find out how the system works, secondly, he or she is asked to reach
and maintain a certain goal state of the system. If psychological research wishes to
get a functional insight into processes of CPS, it needs to focus on both: on subjects’
performance in identifying the system, i. e. his or her acquiring knowledge, as well as on
subjects’ control performance, often termed as knowledge application. In this section we
will try to elucidate how acquisition and application of knowledge depend on each other.
Prior to that, however, let us briefly recapitulate that in experimental practice several
indicators exist to record the type and amount of gained knowledge and the quality of
control performance.
Measuring acquisition of knowledge. Regarding acquisition of knowledge, the far
most common experimental indicator is the structural score gained from the method of
causal diagrams in the paper and pencil format (see, e. g, Funke, 1985). Computer-based
modifications of this method exist in some rather exceptional cases. In studies, e. g, by
Beckmann (1994) or Schulz (2003) the analogous format of arrows in diagrams is ei-
ther complemented or replaced by verbal and numerical elements. Beckmann recorded
participants’ structural knowledge by means of sequential verbal questions after each
experimental trial. Subjects were confronted with very general questions first (such as
the question whether they had found out anything new about any relation in the dy-
namic system) and later – in case subjects had answered accordingly – more specific
questions followed (such as the question concerning the strength of the relation between
two defined variables). Hence, the level of specifity was adapted to participants’ indi-
vidual knowledge. Verbal answers of the question sequence then were translated into
an analogous format, i. e. into arrows and symbols between exogenous and endogenous
variables implemented in the system’s graphical user’s interface (see Beckmann, 1994, p.
72). In Schulz’s diploma thesis (2003), the method of causal diagrams was completely
transformed into a tabular display leaving room for the same information as can be
specified in a graphical causal diagram.5
Although the method of causal diagrams has been criticised due to its potential reactiv-
ity, i. e. the fact that the way of recording knowledge draws subjects’ attention to causal
relations to an unnatural, exaggerated extent (see Kluwe, 1988, p. 370), there are two
major advantages of this method. Firstly, causal diagrams allow differentiated analyses
of three levels of system identification: the identification of the existence or non-existence
of a relation, the identification of a relation’s direction and the identification of a quan-
titative weight indicating the strength and direction of a relation. Secondly, measures
developed on the base of causal diagrams have proved highly reliable, especially the
indicator quality of knowledge acquisition, which has been validated by M¨uller (1993)
in extensive studies. Quality of knowledge acquisition defines a weighted difference of
ratios of correct and false answers relative to the maximum numbers of correct and
false items (see also Funke, 1992b, p. 81 ff.). Another approach to measure structural
knowledge is found with Preußler’s “pair-task” (Preußler, 1996, 1997, 1998). In the
“pair-task” pairs of two variables’ names are presented to subjects who decide whether
a relation between these variables exists or not. According to Preußler, the pair-task
is considered less reactive than the method of causal diagrams, yet information about
gained knowledge is confined to the mere knowledge of existent or non-existent relations.
While causal diagrams and the “pair-task” aim at recording abstract structural knowl-
edge about causal relations in a system, in terms of acquired knowledge it is also possible
to examine knowledge on a more concrete level, closer to the application of knowl-
edge. Schoppek (2002), e. g, emphasises the distinction between structural knowledge
and input-output knowledge which “represents specific input values together with the
corresponding output values”. Accordingly, in order to consider this type of specific
knowledge as well, participants may be asked which will be the resulting outcome states
provided certain input states and interventions. The measure calculated from such pre-
dictions and the actually resulting outcome states is named the quality of prediction
(see Funke, 1992b, Funke & M¨uller, 1988). Another way to infer input-output knowl-
edge is to present specific system’s states which either correspond to dynamic situations
actually occurred (target situations) or to additionally constructed distractor situations
which resemble target situations but have never been encountered by subjects (Preußler,
2001). Subjects are requested to assess whether a presented situation is “old”, i. e. a
target, or whether it is completely “new” to them, i. e. a distractor. On the base of
correct and false recognition answers knowledge scores can be calculated.
Even though the latter measures can possibly be regarded as less reactive than causal
diagrams, problem solvers confronted with these demands still know that they are asked
about explicit structural knowledge they have gained. It was Preußler (1996) who sug-
gested and conducted an attempt of recording rather implicit knowledge. In Preußler’s
lexical decision task participants are interrupted in the actual task of CPS just as with
causal diagrams or the “pair-task”, but they do not know that the intermittent task
5The reason for this practice were not general objections against graphical causal diagrams, but a
specific experimental manipulation which might have interfered with graphical display of the system’s
intends to gain an insight into their knowledge. Subjects in the experimental setting are
simply asked to decide as quickly as they can whether a variable’s name presented in a
temporal sequence on the screen refers to a “real” variable or to a similar, non-existent
variable’s name. In diagnosing knowledge the decisive measure is provided by records
of reaction time rather than by the mere distinction of correct or false reactions. As
common in research on associative priming (compare Goshen-Gottstein & Moscovitch,
1995) it is assumed that problem solvers will react to a correct name more quickly if the
preceding item represents the name of a variable which is – within the examined com-
plex system – indeed related to the other variable. The approach seems both appealing
for its comparable non-reactivity and prone to criticism: Certainly it is a daring busi-
ness to infer associations from reaction times and structural knowledge from superficial
associations (compare Funke, 2003, p.166).
Measuring control performance. Control performance on the other hand is mea-
sured almost consistently throughout recent experimental studies by means of the quality
of system control. This measure calculates mean logarithm deviations of systems’ ob-
served states from defined goal states (see, e. g, M¨uller, 1993, p. 61). Shown by M¨uller’s
validating analyses, quality of system control as an experimental tool is just as reliable
as the quality of system identification.
A measure found in older studies is given by mean absolute deviations from goal values
(e. g, Funke & M¨uller, 1988). In the exceptional case of a special paper and pencil
Dynamis adaptation (G¨orn, Vollmeyer & Rheinberg, 2001) merely counting the number
of accomplished goals can be an indicator of control performance as well.
Knowledge acquisition: active exploration vs. passive observation. So far
we have considered processes of knowledge acquisition as a result of subject’s specific
interventions and experiences with dynamic systems. Phases of system control and
active exploration have been interwoven with the formation of structural knowledge. In
reality, however, this is not necessarily the case: In some situations we better learn from
pure observation. Maybe this is true for some aspects of CPS as well?
This question was risen and investigated in an study by Funke and M¨uller (1988; see also
Funke 1992a, 1992b, 1993) based on the data of 32 college students. Using Sinus, the
authors implemented a new experimental design which is suitable to comparing effects of
active intervention vs. passive observation on problem solving. Active intervention was
realised as in traditional experiments whereas participants in the condition of passive ob-
servation received the intervention data from an “experimental twin” of the intervention
group (yoked control-design). During four initial exploration phases they but observed
input states, the twin’s intervention values and corresponding outcome states. Like the
intervention group they filled out causal diagrams after each round. These blocks of
either active or passive exploration were followed by one round requiring active control
of the dynamic system from all participants.
Prior to statistical testing Funke and M¨uller expected a general superiority of the inter-
vention group with regard to both the amount of knowledge and the quality of system
control. Their assumption was only partly supported. As path-analytical expectation re-
vealed, subjects who actively explored Sinus showed indeed better control performance
than the passive observers. The observers, however, had gained significantly more and
better knowledge about the system, especially with regard to exact numerical param-
eters. An explanation proposed by the authors points at differential perception of the
experimental demands: While subjects asked to actively explore and control the system
may consider the final control task as their primary task demand, to their experimental
twins in the observers’ condition, knowledge acquisition and producing valid causal dia-
grams, being their only active contribution during the exploration blocks, should appear
far more important.
Partially differing results and interpretations are found in Beckmann’s (1994) research
which is dedicated to learning in traditional learning tests as compared to CPS in the
first place, but which additionally deals with effects of active vs. passive exploration on
problem solving. Beckmann’s terms active and passive exploration correspond to Funke’s
and M¨ullers concepts of active intervention vs. passive observation, respectively. Only
unlike in the above experiment, passive exploration in Beckmann’s practical realisation
implies observing system’s states which have been prepared by the experimenters in
order to represent information resulting from possible optimal exploration strategies.
The yoked-control design is replaced by displaying informative data identical to all
observers. With regard to the scenario employed (Maschine, a six variable system
free of semantic embedding), with regard to general procedure, and the number of
participants (40 students) Beckmann’s experiment seems comparable to Funke’s and
uller’s study.
The results, however, are different. Beckmann could not replicate the finding that sub-
jects in the observers’ condition gained significantly more structural knowledge than
subjects who actively intervened. The observers even tended to identify less precise al-
though this effect failed statistical significance. Concerning control performance, Beck-
mann’s participants, too, performed better if they had actively explored the dynamic
system. This finding has been interpreted in terms of different procedural skills that
arise from different task demands. Beckmann assumes that when actively controlling a
system former observers can only rely on the declarative knowledge they have acquired
during observation while active explorers on the other hand gain and make use of both
declarative and procedural, often implicit knowledge about the system’s reactions.
For the subsequent sections on knowledge acquisition and knowledge application we can
at least keep in mind that knowledge can be gained both from participants’ active in-
tervention as well as from passive observation. As we will see with studies providing
additional structural knowledge to subjects besides the knowledge from active explo-
ration both active and passive ways of knowledge acquisition can be combined.
A closer look at the relation between knowledge acquisition and control per-
formance. It is commonly assumed that acquiring knowledge represents a necessary
precondition for controlling a dynamic system. As in the above text, too, most scien-
tists interpret system control in terms of applying gained knowledge.6That is why in
6Interestingly, the reverse path, i. e. whether and how control performance in turn affects knowledge
acquisition has been examined far less frequently. Assumptions as to the question whether successful
control performance encourages increased knowledge acquisition appear to be controversial. Preußler’s
(2001) research on the base of a Dynamis scenario suggests that at least gaining specific input-output
experimental practice, phases of free exploration, aiming at knowledge acquisition, usu-
ally precede phases of controlling a dynamic system with respect to defined goal values,
aiming at knowledge acquisition.
Although in general there is empirical evidence for this theoretical notion, we will see
that findings on the relation between knowledge acquisition and control performance are
not quite as distinct for several reasons: Knowledge relevant to control demands can be
explicit and abstract or rather implicit and concrete to differing degrees; furthermore,
mediating factors exist that make it more or less probable that problem solvers in control
tasks actually make use of acquired knowledge.
Correlation analyses. A straightforward approach to investigate the relation be-
tween knowledge acquisition and control performance is to analyse statistical (a posteri-
ori) correlations between these two indicators in CPS. Funke and M¨uller (1988) already
found positive statistical correlations (r= 0.41 or – for revised diagnostic measures in
Funke, 1992b – r= 0.54) hence indicating that the degree of knowledge acquisition will
determine the level of control performance. The possibly most prominent demonstra-
tion of this relation is found with M¨uller’s (1993) analyses based on the theory of latent
state-trait models (e. g Steyer, Schmitt, & Eid, 1999). LISREL analyses, the practical
implementations of this model, enable separating manifest variables, i.e. variables that
are empirically observed, from latent variables, i. e. variables assumed to form an un-
derlying, non-observable psychological construct. With respect to CPS it was M¨uller’s
idea to distinguish manifest knowledge and control performance in specific Dynamis
situations from corresponding latent traits of system identification and system control
as problem solvers would reveal irrespective of situational influences.
For empirical validation 143 school students worked on Sab, the Dynamis computer
simulation of an abstract machine (a precursor of Kr¨oner’s MultiFlux system). For
LISREL requirements two sessions with two equivalent Sab versions were conducted,
unfortunately reducing the number of complete data to 78 participants. On the base
of the LISREL procedure M¨uller showed that for the latent variables corresponding to
manifest system identification and system control correlations were significantly posi-
tive (r= 0.83 for the first session, r= 0.86 for the second session; for a brief summary
of M¨uller’s procedure and results see also Funke, 2003, p. 163-165). M¨uller’s analy-
sis furthermore suggests that the trait of system identification can be regarded as a
one-dimensional construct, dismissing any implicit or procedural knowledge specific to
control performance. According to M¨uller, control performance can fully be predicted
on the base of subjects’ abstract knowledge about causal relations in the system (p.208).
Another interesting aspect of M¨uller’s work provides one possible explanation as to em-
pirical findings inconsistent with M¨uller’s positive correlations between identification
and control of a complex system. In a small experimental sample of 20 students M¨uller
showed that the amount of knowledge gained obviously depends on some facet of pre-
senting information about the system’s state. Students who were – during a problem
solving task – confronted with a simultaneous presentation of actual and past system’s
knowledge is not systematically related to control achievements. Yet in this section the discussion of
relevant Dynamis studies will focus on the more obvious relation: how control performance depends
on acquired knowledge.
states gained more and better knowledge than those who viewed but one system’s state
at a time, the actual system’s state which was replaced by the subsequent state in the
following trial (sequential presentation). Given simultaneous presentation participants
seemed to benefit from extra time spent on the task whereas in the condition of sequen-
tial presentation this was not observed. The author concludes that simultaneous display
provides the possibility of analysing the course of the system’s development, a necessary
condition for acquiring knowledge. In turn, since only with simultaneous display sub-
jects acquired sufficient knowledge, only in this condition positive correlations between
system identification and system control may occur whereas for sequential presentation
correlations are – or rather seem – insignificant.
The notion that acquired structural knowledge can predict goal attainment is also sup-
ported within more comprising path-analytical models by Vollmeyer and Rheinberg
(1998), Beckmann and Guthke (1995) as well as Kr¨oner (2001). Correlation coefficients
range from 0.51 with Beckmann and Guthke to 0.65 in Kr¨oner’s experiment or 0.59 when
it was statistically controlled for the impact of test intelligence. Similarly, Vollmeyer,
Burns, and Holyoak (1996) found that for both an initial control task as well as for
a transfer control task structural knowledge measured by a structural score revealed
significantly negative correlation coefficients when correlated with the solution error,
the absolute difference between a target value and the observed value (r=0.57 for
the initial control task, r=0.65 for a transfer control task). All of the correlations
reached significance or even high significance making a general dissociation between
system identification and system control appear highly improbable.
Yet it should be mentioned that the obvious relation between gaining and applying
knowledge may be mediated by various influences. One such influence has already
been discussed with Beckmann’s (1994) study on the impact of semantic embedding in
complex dynamic systems. Positive correlations between achievement on the two task
demands rather occurred when the system was abstract than semantically embedded. In
Kluge’s (2003) work task difficulty has proved to be another mediating factor. In a large-
scale experiment involving 496 subjects working on the Dynamis system ColorSim
(see below), among other factors formal task complexity was manipulated by three levels
of increasing connectivity. Correlation coefficients as to acquired knowledge and control
performance were found to be high for a low or medium degree of connectivity (r= 0.70
and r= 0.68, respectively), but it was considerably lower for subjects who had dealt
with the most complex version (r= 0.44). Possibly in the latter case additional factors,
e. g, the amount of time spent on the task became more influential, hence lowering the
statistical correlation.
Manipulating structural knowledge: the impact of transparency. The a pos-
teriori correlations described in the preceding section obviously provide an appropriate
tool to detect that there is, in fact, a positive relation between the two task demands in
CPS and that this relation is relatively powerful. Yet the results remain confined to a
descriptive level because no causal relation is implied by correlation analyses. In order to
find out more about the mechanisms which determine how knowledge is translated into
control performance experimental manipulations in this context are necessary. If the
researcher is interested in the effects of structural knowledge on control performance, he
or she can try to systematically promote or impair subjects’ structural knowledge and
observe whether and how control performance will be affected. The first methodolog-
ical approach has been realised in a number of experimental studies. Since structural
knowledge is fostered by providing diagrams which present the system’s structure, mak-
ing the system “transparent” to a problem solver, the term transparency will be applied
to systems whose structure is disclosed to subjects.
In a sample of 50 university students Putz-Osterloh (1993a, 1993b) conducted an ex-
periment on possible effects of transparency on problem solving in order to examine the
relation between knowledge acquisition and system control. Participants dealt with a
version of Linas which involves four input variables (named A, B, C, D), seven output
variables (having fancy names) and 15 relations, some of these characterised by delayed
effects. In a first part of the experiment subjects’ exploring and controlling the sys-
tem was required. The task was identical for all subjects but for a manipulation on
structural knowledge: Subjects of the experimental group (25 students) were given a
structural diagram displaying the relations within Linas; the diagram was explained
by the experimenter, and by standardised questions it was ensured that subjects had
understood relevant information. Subjects of the control group (25 students) received
no structural information.
Putz-Osterloh assumed that students of the experimental group would benefit from the
extra knowledge due to transparency and would perform more successfully in controlling
Linas. Yet the expectation could not be confirmed. As to control performance no
systematic differences between the two transparency conditions were found although
subjects of the experimental group indeed seemed to have more structural knowledge
than those of the control group.7A first conclusion therefore contradicts the above
suggested coincidence of knowledge acquisition and control performance. On the base
of the experiment’s first part one might rather suppose that these are two independent
processes in CPS. According to Putz-Osterloh and similar to Beckmann’s (1994) notion
of ad hoc control, it might be possible to successfully control a system despite little
knowledge, acting almost in terms of trial and error.
When, however, can structural knowledge still support performance in a complex dy-
namic system? A second part of the experiment suggests that structural knowledge
provided by diagrams can, although not applicable immediately after its acquisition,
enhance control performance in a transfer task. Half a year after the first session, 38
students of both the experimental and control group from Putz-Osterloh’s original sam-
ple worked on a modified or “flawed” version of Linas in which the effect of one input
variable had been omitted. Participants were informed about this fact, yet they did
not know which variable had been changed. They were requested to name the deficient
variable after controlling the dynamic system.
As expected, this time subjects formerly supported by structural knowledge outper-
formed subjects of the initial control group, again applying more efficient strategies.
They also proved to be more successful in diagnosing the modification. Interestingly,
for participants of the former experimental group a positive correlation was found con-
cerning the amount of reproduced structural knowledge (as could be inferred from the
7Especially the finding that participants supported by transparency adopted more efficient strategies
made the author assume that these subjects indeed made use of their enhanced structural knowledge.
diagnosis task) and control performance. Subjects of the former control group whose
knowledge acquisition had not been systematically promoted on the other hand revealed
no such systematic correlation. Instead, Putz-Osterloh states a dissociation: Problem
solvers who easily detected the system’s modification tended to do relatively badly on the
control task, not adopting their strategies to the modified system’s structure, whereas
other subjects seemed to cope successfully with the changed version although they did
not detect which variable had been manipulated.
Possibly, Putz-Osterloh argues, in the initial task subjects confronted with abstract
knowledge indeed acquired helpful, comprising structural knowledge, but the demands
of analysing and interpreting the graphical structural diagram occupied their cognitive
capacities almost completely so that they could not apply their comprising knowledge
right from the beginning. Supporting effects of structural knowledge on control perfor-
mance only became obvious in the transfer task after the process of knowledge acquisition
had been completed. With respect to the relation of acquiring structural knowledge and
accomplishing the control task Putz-Osterloh, partly disagreeing, e. g., with M¨uller’s
(1993) interpretation, assumes that both demands can still be conceived as two separate
goals. As shown by the dissociation found in Putz-Osterloh’s control group the goals
are not necessarily coordinated, but they well may be if problem solvers’ structural
knowledge reaches a sufficient extent.
Preußler (1996) as well examined the role of structural knowledge and control perfor-
mance in Dynamis situations and adapted the view that there might not necessarily be a
direct translation of structural knowledge into applied knowledge in control performance.
Two successive experiments emphasise that in actually applying gained knowledge at
least two conditions need to be accomplished: Firstly, structural knowledge has to be
gained in the context of application instead of abstract “inert” knowledge, secondly,
knowledge acquisition should take place prior to application.
With respect to structural knowledge, Preußler did not but analyse explicit verbalised
knowledge recorded by the “pair-task”, but also implicit associative knowledge about
structural relations which was inferred from a lexical decision task. Both qualities
of knowledge were to be considered to examine whether implicit rather than explicit
knowledge was associated with successful control performance as Berry and Broadbent
(1984) had suggested.
Similar to Putz-Osterloh, yet closer to knowledge application, structural knowledge
in the transparency condition was promoted by specific examples and explanations.
Within a total sample of 50 students (studying at the University of Bayreuth) half
of the students, the experimental group, passed through a practice phase before the
control task: The experimenter told them – according to standardised examples – how to
manipulate certain input variables and explained subsequent effects on output variables
to them. The remaining subjects forming the control group explored the dynamic system
without requests and explanations before they worked on the same system for eight
rounds with eight trials each. The Dynamis system chosen for this task was a version
of Linas.
Three experimental hypotheses were to be tested for statistically. Firstly, the experimen-
tal group systematically supported in gaining structural knowledge should reveal more
knowledge as to both the explicit and implicit measure. Secondly, as a consequence of
enhanced knowledge acquisition the experimental group should outperform the control
group in controlling Linas. Thirdly, the amount of gained knowledge, both implicit
and explicit, should be a predictor of control performance. The first experiment, how-
ever, supported neither of these assumptions. Only marginal support was found with
regard to enhanced implicit knowledge in the experimental group (first hypothesis) and
a positive relation between knowledge and system control (third hypothesis).
It was the second experiment involving another 48 university students for participation
which confirmed that subjects systematically promoted in knowledge acquisition dis-
played indeed higher levels of implicit knowledge as compared to those of the control
group. Control performance, too, tended to be higher within the experimental group.
What had been changed in the second experiment relative to the first experiment? Ac-
cording to Preußler, the procedure of instructing knowledge had been extended to more
systematic analysis in the practice phase. This fact emphasises the importance of in-
structing knowledge appropriately, ready to application in controlling the system. As
expected, in the second experiment, too, when acquiring knowledge had been success-
fully promoted control performance turned out to be positively related to the amount
of implicit knowledge (at least for Linas relations which could be detected quite eas-
ily). Since comparable positive correlations also occurred with knowledge acquisition
and control performance in the early practice phase, Preußler corroborates the inter-
pretation from correlation analyses (e. g, M¨uller, 1993, Funke, 1992b) that knowledge
precedes and determines subjects’ coping with the control task. She yet emphasises the
importance of effective knowledge instruction. To a considerable extent effectiveness
appears to depend on the time of instruction. Obviously, it is suitable to promote struc-
tural knowledge prior to its application in a control task so that the two processes of
analysing and controlling will less interfere with one another.
A following study by Preußler (1998) attempted more detailed specifications of the
suggested conditions under which induced structural knowledge will be predictive of
success in a control task. Unlike in the previous experiment, Preußler was especially
interested in the role of explicit structural knowledge; explicit knowledge was deemed
more determinant than implicit knowledge because it enables subjects’ making up and
testing explicit hypothesis which might be helpful in control tasks. In accordance with
the earlier findings emphasis was put firstly on early instruction of explicit knowledge
(before the control task) and on even more application-oriented instructions.
Application-oriented promotion of structural knowledge in an experimental group meant
a practice phase involving a graphical structural diagram for transparency combined
with practice tasks: Goal values had to be attained by particular manipulations; the
task lasted until participants themselves had found out correct solutions. In a control
group by contrast structural diagrams were omitted and the same practice tasks could be
finished without finding the correct solutions. The experimental procedure comprised
two subsequent control tasks, one learning phase after the practice phase, then after
some delay a second control task, the transfer phase. In total, 56 Bayreuth University
students took part in a first experiment and another 54 students in a second experiment.
All of them worked on different Linas implementations.
Results from the first experiment suggested supporting effects of the application-oriented
treatment on structural knowledge (measured by the pair task) and performance: Struc-
tural knowledge of the experimental group, after the first control task already, was found
to be significantly higher than with subjects of the control group. Interpreted as a conse-
quence of enhanced and successfully applied structural knowledge, in both the learning
phase and the transfer phase the experimental group outperformed the control group.
Unlike the control group, the experimental group displayed “positive transfer”, i. e. in
the transfer control task right from the beginning they performed equally well as at
the end of the first learning control task. Concerning the relation between structural
knowledge and control performance, as expected, positive correlations were revealed, yet
only for problem solving in the transfer task. The partly deviation from expectations
as to the learning task cannot fully be interpreted. However, unlike in Putz-Osterloh’s
experiment (1993a, 1993b; see above) there is at least evidence that subjects’ control
performance will benefit from induced structural knowledge in the first control task
already instead of deferred benefits which are revealed but on a transfer task.
Preußler’s second experiment dealt with the question whether the enhanced control
performance of the experimental group results from the mere fact that (some kind
of) knowledge is instructed or whether successful control performance is promoted by
the particular structural knowledge implied in the transparency condition. For this
purpose the experimental group (receiving an almost identical treatment as in the first
experiment) was compared to a control group receiving a training on specific, more
concrete knowledge about isolated effects of intervention in the practice phase.
Obviously, confirming the author’s expectations, structural knowledge proved to be
superior in promoting control performance. Although specific interventional knowledge
might, of course, not be regarded as futile to control performance, evidence occurred
that the practice on reaching isolated goals in the practice phase might have interfered
with the successful coordination of these goals as required to system control. Additional
analyses showed that less time was required to pass through the training on abstract
knowledge (experimental group) as to the training of specific interventional knowledge
(control group), suggesting that the application of structural knowledge is not merely
more successful, but also more economic.
To summarise, the later findings of Preußler (1998) partly reconcile experimental in-
vestigation with the findings of correlation analyses claiming positive relations between
knowledge application and knowledge acquisition. Provided that knowledge is acquired
prior to system control in an application-oriented context, the equivalent positive cor-
relations as in a posteriori analyses will appear.
The quality of knowledge and knowledge instruction. Studies on the impact
of transparency made obvious that in general appropriate knowledge and appropriate
ways of instructing knowledge will enhance the application of this knowledge in con-
trol tasks. Some questions, however, remain: With respect to the type of knowledge
Preußler (1998) suggested that promoting structural knowledge is most helpful in prob-
lem solving whereas specific interventional knowledge has less beneficial effects. Is this
necessarily the case? Additionally, one might consider the way in which the acquisition
of (structural) knowledge is supported. Early application-oriented learning seems to
facilitate later control performance (Preußler, 1998) – but maybe there is more to an
effective instruction or tutorial?
For systematically comparing different modes of knowledge instruction Kluge (2003)
designed an experiment involving the Dynamis system ColorSim.ColorSim, a sys-
tem of the graphical presentation format, is made up of three abstract input variables
(X, Y, Z) and three output or goal variables representing the names of colours (“yel-
low”, “black”, “green”). In the most simple realisation output variables are affected
by four basic relations and one instance of eigendynamic; the most complex realisation
includes two instances of eigendynamic as well as two side effects in addition to the
four basic relations. In Kluge’s experiment three degrees of connectivity – manipulated
between subjects – gave rise to three experimental conditions: low, medium and high
task difficulty.
Knowledge instruction as well was realised by means of three treatment conditions:
explanatory screen mask, guided exploration and a combination of the two former in-
structional types. The main difference between the explanatory screen and guided ex-
ploration is the extent to which subjects become actively involved in the process of
acquiring structural knowledge. While guided exploration requires active exploration,
testing self-constructed hypotheses and independent reasoning,8the explanatory screen
mask but presents and explains information to be received and analysed, yet very com-
prising information; besides graphical displays of structural diagrams verbal descriptions
and corresponding numerical equations are provided so that the system’s algorithm can
be explained in successive steps.
Based on the statistical results from a sample of as many as 496 subjects Kluge concluded
that none of the three instructional conditions generally proved superior to another. Ef-
fectiveness rather seemed to depend on the level of task difficulty. When the scenario
had been classified as simple subjects of all three instructional conditions controlled
ColorSim equally well; only knowledge acquisition was reduced for participants con-
fronted with guided exploration as compared to control subjects. Regarding medium
task difficulty the combination of guided exploration and the explanatory screen turned
out to promote both knowledge acquisition and control performance in a more effective
way than guided exploration without additional explanatory elements. As the author
assumes, when dealing with simple problems self-employed learning might be helpful
and successful, but with more complex problems the advantages of self-instruction de-
cline relative to the benefits of objective explanations. For high task difficulty each of
the three instructional modes revealed deficits that might, according to Kluge, only be
compensated for by further support such as leaving more time to cope with the complex
The question whether structural knowledge in CPS is most important relative to other
kinds of knowledge has been studied by Schoppek (2002). Schoppek promotes the theory
that in general both abstract causal knowledge, i. e. structural knowledge, as well as
specific interventional knowledge named input-output knowledge may be relevant to
CPS. Yet the size of the complex system should determine which type of knowledge
plays a dominant role. Analysing previous studies (involving both scenarios of the
Dynamis approach and other scenarios) Schoppek found that in scenarios implying but
a small number of possible solutions problem solvers seemed to acquire input-output
8Guided exploration is about equivalent to the instruction of testing hypothesis in studies on the
effects of systematic rule-induction (see Vollmeyer, Burns, & Holyoak, 1996).
knowledge spontaneously. This kind of interventional knowledge obviously sufficed the
demand of controlling the system. In contrast, for large systems making it impossible
to store isolated combinations of input and output values to a sufficient degree subjects
tended to develop structural causal knowledge even if they had not been instructed to
do this.
Schoppek’s argumentation, however, goes further and – as the conclusion of his experi-
mental investigation – states that beyond structural knowledge and specific knowledge
of input-output relations a third type of knowledge might come into play, operative or
strategic knowledge. This kind of knowledge is defined as “knowledge about how to
proceed in order to accomplish a task”. Strategic knowledge can imply aspects of proce-
dural input-output knowledge, but it may as well comprise quite abstract representations
about a sequence of operator steps.
Which experimental evidence supports Schoppek’s concept? With respect to hypothe-
ses and general procedure the study Schoppek conducted on the impact of structural
knowledge resembles previous studies on transparency (Putz-Osterloh, 1993a, 1993b)
and enriched tutorials in promoting structural knowledge (Preußler, 1996, 1998). In a
between subjects design 80 students studying at the University of Bayreuth controlled
Linas, preceded by a learning phase of either one of two conditions: Subjects of the
experimental group passed through a computer tutorial which guided them to analyse
effects of input variables; subjects of the control group explored the system and tried to
reach goal values.
In accordance with Putz-Osterloh (1993a, 1993b) it was found that on the average par-
ticipants of the two conditions controlled the system equally well although subjects of
the experimental group had gained more structural knowledge. Following Preußler’s ear-
lier interpretation (1996, 1998; see above) one should assume that enhanced structural
knowledge possibly was not available to successful use in the control task (although,
in fact, the tutorial was administered prior to controlling the system in an application-
oriented context). Schoppek’s interpretation yet is partly different: Instead of “blaming”
the experimental group for inert structural knowledge he points at knowledge acquisi-
tion among subjects of the control group. Due to early experiences with the dynamic
system control subjects might have gained and later applied strategic knowledge which
compensates for the enhanced structural knowledge of experimental subjects. Since
even in a transfer task control subjects without a tutorial on strategic knowledge per-
formed equally well as the experimental group9impacts of specific knowledge about
input-output relations cannot explain the equal levels of control performance neither
can structural knowledge.
Instructed strategies, analogies, and the impact of goal specifity. Studies on
the effects of structural diagrams and other tutorials have shown that in CPS methods
exist which promote the acquisition of knowledge, structural knowledge in particular,
and which sometimes positively affect control performance as well. On the other hand
we have seen that structural knowledge might not be the only factor responsible for
successful problem solving (e. g Schoppek, 2002). Hence when looking for supporting
9This finding obviously contradicts Putz-Osterloh’s earlier results, claiming that on a transfer task
promoted structural knowledge leads to increased control performance.
factors in the process of CPS further possibilities need to be taken into account: In this
section, the impact of instructed strategies and analogies will be discussed as well as the
impact of goal specifity, which influences problem solvers’ spontaneous application of
strategies. Not in all cases it can be figured out whether one of these methods primarily
supports knowledge acquisition, control performance or both, leading us back to the
disputed relation between the two task demands.
Strategies. Before experimentally inducing and manipulating problem solvers’ use of
strategic approaches researchers have been analysing strategies subjects employ spon-
taneously and have assessed these strategies with respect to their effectiveness and sys-
tematicity. Implied in the concept of the Dynamis approach, to gain knowledge in a
Dynamis situation it is required to discriminate the effects of single input variables.
This can be accomplished by varying the relevant input variable and keeping all other
variables at a constant value of zero, a strategy which has been referred to as VOTAT
(Vary One Thing At a Time). VOTAT is often considered as the most promising strat-
egy to knowledge acquisition in a dynamic situation (see, e. g, Putz-Osterloh, 1993b,
Vollmeyer & Rheinberg, 1998, Rollett, 2002). Accordingly, manipulating more than two
inputs at a time is associated with unsystematic exploration. A more detailed discus-
sion of the effectiveness of VOTAT and other strategies is provided in part three when
strategies are considered from the perspective of problem solvers’ characteristics (see
the experimental analysis by Vollmeyer, Burns, & Holyoak, 1996). Eigendynamics can
be best detected by doing without any intervention at all, leaving all input variables
at the value of zero. Another possibility, yet less frequently employed, is to choose
interventions that are equal in absolute value, but opposite in signs on two successive
trials. In the case of no eigendynamics, after the second trial the initial state should be
re-attained. Concerning the detection of side effects applying particularly large input
values can be helpful to discriminate a basic input effect from a usually smaller side
effect (Schulz, 2003).
Rollett (2002) experimentally analysed and compared a number of such spontaneously
employed strategies on the base of data from 109 subjects in a first and 207 subjects
in a second study. Applying the dynamic system Biology Lab (Vollmeyer, Burns,
& Holyoak, 1996) different types of strategies were recorded and assessed as to their
effectiveness in gaining structural knowledge. Highest – and highly significant – posi-
tive correlations between knowledge acquisition and strategy use were found with keep-
ing constant one or two input variables out of three input variables in Biology Lab
(r= 0.51). Zero interventions, too, yielded positive correlations with the amount of
knowledge gained (r= 0.35). Negative correlations in contrast were associated with
respect to employing many different input values (r=0.50). Obviously, when trying
to understand a dynamic system by strategic exploration less is more.
Not surprisingly, due to its prominence and effectiveness the strategy of VOTAT has
become a central element in experimental research on strategy instruction. Vollmeyer,
Burns, and Holyoak (1996) systematically instructed the application of VOTAT in a
CPS task on the base of the Dynamis system Biology Lab. The employed version
of Biology Lab comprised eight variables, four exogenous variables, four endogenous
variables which are affected by five primary input-output relations and one instance
of eigendynamic. As implied in the cover story participants have to imagine they en-
ter a biological laboratory in order to find out how certain water qualities in a water
tank (temperature, salt, oxygen, current) influence the population of four species of sea
animals (prawns, sea bass, lobster, crabs). Out of a total number of 60 participants
(undergraduate students at the University of California, Los Angeles) 30 subjects were
provided with explanations and examples how to use VOTAT in subsequent phases of
exploration (three rounds), of control (one round) and transfer tasks (one round). The
other 30 subjects received no instruction on systematic use of strategies.
Given free exploration without specific target values in the three initial rounds to be
reached the author’s strategy instruction obviously “worked”: Throughout all three
rounds of exploration about eighty percent of subjects’ interventions could be classified
as VOTAT strategies rather than the unsystematic strategy of changing all variables or
other, less systematic strategies. In contrast, most participants of the non-instructed
control group did not adopt VOTAT spontaneously.10 The instructed group revealed
higher levels of knowledge acquisition, slightly higher levels of performance in the first
control task and especially increased control performance in the transfer control task.
The results from this part of the experiment confirm that instructing systematic strate-
gies such as VOTAT indeed has beneficial effects on dealing with complex dynamic
systems – provided that task demands are suitable to follow the VOTAT instruction.
What, however, makes task demands suitable to applying systematic strategies?
Goal specifity. An attempt to answer this last question is already implied in the
same study by Vollmeyer et. al. Besides the impact of strategy instruction the authors
examined another feature of the exploration setting: the matter whether subjects should
be informed about target values of the control task from the very beginning (goal speci-
fity) or whether during the first phases of free exploration they should simply be asked
to find out about the system as much as they can before they get to know the target
values to be reached and maintained in controlling the system (unspecific goal condi-
tion). One possible conception is that specific goals, encouraging efficient means-end
analysis of how to reduce the difference between an actual state and a goal state, might
be a suitable condition for acquiring knowledge (Anderson, 1987). On the other hand,
however, one might argue that specific goals and associated means-end analyses should
rather disturb an essential precondition of knowledge acquisition (Sweller, 1988): The
strategy of means-end analysis might interfere with problem solvers’ systematic induc-
tion of rules which would otherwise, in the absence of defined goals, have taken place.
This later view has been adopted by Vollmeyer, Burns, and Holyoak.
In order to test the hypothesis that goal-oriented strategies, i. e. means-ends analysis,
interfere with other systematic strategies adapt to gaining structural knowledge (e.g,
VOTAT) the authors had implemented goal speficity as an additional treatment factor
10This finding on the effectiveness of strategy instruction is hardly self-evident. Vollmeyer and Rhein-
berg (1998), e. g, attempted to manipulate VOTAT strategy instruction in the context of motivational
influences and found that differential strategy use among an instructed and a non-instructed group did
not occur but for the very beginning of a CPS task. In the later process of the task instructional effects
on strategy use disappeared, consequently not affecting knowledge acquisition and control performance
within the experimental design. Within each of the two experimental conditions of
strategy instruction half of the subjects were given no target values until the control
task, the other half was informed about the target values during the three learning
rounds already although it was made clear that their task was “learning as much as
possible” rather than merely aiming at the goal values.
For analysing participants’ use of strategies in the goal specifity condition the classifi-
cation distinguishing VOTAT from changing all inputs and other strategies was supple-
mented by the category of a difference reduction strategy. Difference reduction, a goal
oriented strategy, was diagnosed when observed systems’ states incrementally approxi-
mated the goal values and when subjects successfully reached the goals in the exploration
phases already. According to Vollmeyer and colleagues difference reduction implies a
strategic approach fundamentally different from VOTAT.
Prior to statistical testing the authors expected that in the context of goal specifity
participants would apply VOTAT only if strategies were instructed explicitly. In case of
no systematic instruction combined with specific goals subjects should make predomi-
nant use of the goal-oriented strategy of difference reduction. The results went beyond
this expectation as to the impact of goal specifity. Even when instructed on VOTAT
participants showed a strong linear trend away from this strategy, starting with VOTAT
application in 80 % of the cases in the first round of exploration and finishing with a
frequency of about 20 % use in the third round. Use of the strategy of difference re-
duction markedly increased during the three exploration phases for both instructed and
uninstructed subjects provided with specific goals.
Critically examining practical requirements in this experiment Schoppek (2002) has ar-
gued that, in fact, subjects aiming to reach the goal values had had no other choice than
to omit the VOTAT strategy since in the employed version of Biology Lab it would
be virtually impossible to reach the goals by manipulating one single input variable. In
a replication study he partly changed task demands and system characteristics to dis-
solve the confound between strategy instruction and goal specifity finding that subjects
made use of VOTAT in spite of goal specifity just as Vollmeyer et al. had expected first.
Although it thus remains disputable whether specific goals impair the use of instructed
strategies as well at least in the context of no explicit strategy instruction spontaneous
application of strategies seems to be affected by specific goals. What does this mean to
knowledge acquisition and control performance?
Vollmeyer, Burns, and Holyoak showed that, as expected, participants gained more
accurate knowledge about the dynamic system when no specific target values had been
announced in the initial phases of learning and exploration. With respect to control
performance in the learning task goal specifity did not give rise to significant increments
or decreases. One might suppose that input-output knowledge related to means-end
analysis in case of goal specifity has comparable effects on controlling as the enhanced
structural knowledge when no specific goal is given. In the transfer task, however,
subjects who had freely and systematically explored Biology Lab in the unspecific
goal condition proved superior in controlling the system. This effect has been interpreted
in terms of the subjects’ comprising structural knowledge. Referring to Sweller (1988),
Vollmeyer et al. conclude that “general problem solving methods applied to a specific
goal foster acquisition of knowledge about an isolated solution path but do not provide
an effective way of learning the overall structure of a problem space” (p. 75).
Partly intended as a replication of these findings, Vollmeyer and Burns (1996) attempted
a closer examination of mechanisms involved in the process of knowledge acquisition un-
der the condition of both specific and unspecific goals. Following their hypothesis they
experimentally compared the impact of goal specifity to another situational influence
in CPS which is assumed to have similar effects: instructing subjects with explicit
hypotheses to be tested vs. not instructing subjects. The theoretical notion behind
this procedure goes back to Klahr’s and Dunbar’s dual space theory (1988) according
to which learning can be guided by either one of two distinct processes: by explicitly
forming hypotheses about possible system characteristics and then “testing” these hy-
potheses in terms of experiments (predominant search in a hypothesis space) or by first
conducting experiments about system characteristics in order to induct rules or hypothe-
ses afterwards (predominant search in an experiment space). Searching in the space of
hypothesis is associated with greater systematicity and hence with greater knowledge
acquisition than searching in the space of experiments that may sometimes appear quite
Vollmeyer and Burns assumed that in practice hypothesis search and resulting knowl-
edge acquisition should be encouraged, of course, by providing subjects with an explicit
hypothesis to be tested when dealing with a dynamic system, but that not providing
specific goals in the dynamic system should have comparable effects. These two factors,
the instruction of an explicit hypothesis vs. no hypothesis instruction, and goal specifity
vs. no goal specifity, made up an experimental design including 15 subjects per group,
a total number of 60 subjects. Participants, students of the University of California,
Los Angeles, dealt with Biology Lab in an experimental procedure equivalent to the
authors’ previous study. Instruction of an explicit hypothesis was realised by naming
one hypothesis concerning a defined instance of eigendynamic as well as one hypothesis
concerning a numerically defined input-output relation; these hypotheses were assigned
to a (fictive) scientist.
With respect to goal specifity the effects found by Vollmeyer et al. (1996) could vir-
tually be replicated: Knowledge acquisition measured by structural causal knowledge
was reduced in the condition of a specific goal11, suggesting again that specific target
values prevent problem solvers from systematic searching in an hypothesis space but
make them apply goal-oriented strategies directly aiming at the target values. As ex-
pected, the instruction of an explicit hypothesis, too, tended to induce better knowledge
acquisition and higher achievement in a prediction task than if no hypothesis had been
presented. Deviating from the effect of goal specifity, as to system control subjects of
the hypothesis condition also outperformed subjects who had not been given an explicit
hypothesis. In general, however, it appears that unspecific goals and the presentation
of hypotheses to be tested affect problem solving in a similar way, both encouraging
subjects to systematically induct and scrutinise rules.
Closer analyses of the data showed that enhanced control performance with subjects
given a hypothesis was not confined to the variables whose effects were stated in the
hypothesis. Yet to some extent better prediction scores and better structural knowledge
11Only as to the measure quality of prediction which depends on more concrete input-output knowl-
edge no impairing effects of goal specifity were found.
gained by this group could be assigned to the information that was implied in the hy-
pothesis. In order to limit the scope of this alternative explanation a second experiment
was conducted, involving 236 participants. Three treatment conditions were realised:
An experimental group instructed with the correct hypothesis, another experimental
group instructed with a false hypothesis and a control group which received comparable
information about a particular relation within the dynamic system, yet not formulated
as a fictive scientist’s hypothesis.
Statistical analysis showed that on all relevant dimensions subjects instructed by an
explicit hypothesis performed better than subjects of the control group, even if the
hypothesis-instructed subjects had been dealing with false information. Subjects given
a false assumption did not generally perform worse than those given the correct infor-
mation. According to Vollmeyer and Burns, especially this last finding emphasises that
rather the mere instruction to test an assumption than the actual information implied
in this assumption makes problem solvers induct rules and test their own hypotheses
beyond the one given in the instruction.
Analogies. A final note concerning supporting factors in acquiring knowledge about
dynamic systems refers to the instruction of analogies. Effects of analogies have rarely
been studied in the context of CPS, hence the work done by Schulz (2003) in an unpub-
lished diploma thesis might represent a pilot study in this domain. Analogous reasoning,
a prominent concept realised in intelligence diagnostics, requires a person to transfer
knowledge structures from a known, familiar domain to another, hitherto unknown do-
main of comparable structure, but different contents. As with the instruction of explicit
hypotheses the cognitive demands involved in processing analogies, too, can be associ-
ated with increased search in a hypothesis space. Schulz thus assumed that instructing
analogies in a CPS task would facilitate the acquisition of structural knowledge. For
experimental testing an abstract Dynamis scenario named Wits, consisting of two en-
dogenous and two exogenous variables was employed in an internet experiment. Three
between subjects conditions of analogy instruction were manipulated: working on the
system without any analogy at all, dealing with one analogy and dealing with three
different analogies. All analogies were derived from familiar every day contexts such as
the context of an ecosystem.
Results based on the data of 53 participants revealed a marked, yet not significant effect
towards superior knowledge acquisition of subjects instructed with analogies. Somewhat
unclear and different from the author’s expectation the group provided with only one
analogy even performed better than the group provided with three different analogies.
On the whole, however, Schulz concludes that analogies obviously have beneficial effects
on acquiring structural knowledge about a complex dynamic system.
3.3 Person characteristics
In the first two parts we have dealt with the role of system and task characteristics
in CPS. Differing levels in performance even though subjects work on exactly identical
problems under identical conditions make clear that it is not the dynamic system as
such nor the task demand taken on its own which determines how well we do on a
computer-simulated CPS task. To a considerable extent our achievement rather depends
on ourselves: the strategies we employ deliberately or by intuition without explicit prior
instruction, our current motivation as well as generalised motivational tendencies and
our relevant cognitive abilities.
3.3.1 Strategies and systematic learning
The section about supporting factors which may enhance performance in CPS has al-
ready revealed the benefits of systematic, strategic operations in this context. Not
surprisingly, just as experimental inducement of strategies leads to increased control
performance and system identification, the same occurs if problem solvers apply appro-
priate strategies spontaneously without preceding request.
Branke (1991) conducted an experiment which involved 42 university students working
on a Dynamis simulation named Korallenriff (coral reef); in addition to stan-
dard measures of problem solving abilities she recorded and classified the application of
strategies as a personality factor and came to the conclusion that employing strategies
is significantly related to successful problem solving. The more strategic the approach,
the more knowledge will be gained and the better control performance will be.
Similarly, Putz-Osterloh (1993b) found that high increases in performance were associ-
ated with strategies classified as effective – especially strategies of selection – whereas
low increases in performance were associated with less effective strategies, i. e. changing
all inputs in the same trial. Not quite as expected, in Putz-Osterloh’s experiment this
effect could only be revealed when subjects had been promoted in gaining structural
knowledge by means of transparency. A control group without additional information
about the system’s structure seemed to increase performance both in the case of effective
as well as ineffective strategies. Satisfying explanations cannot be provided.
The particular relevance of VOTAT, the strategy of manipulating but one single input
variable on a trial, has been highlighted in an above discussed study by Vollmeyer, Burns,
and Holyoak (1996). VOTAT is compared and contrasted to the strategy of “chang-
ing all variables in a haphazard way” and a heterogenous collection of other strategies.
Problem solving records of 36 students (studying at the university of California, Los
Angeles) working on Biology Lab showed that a score on strategy systematicity was
highly positively related to the accuracy of knowledge acquisition (r= 0.76) and control
performance. While subjects employing VOTAT clearly performed best though not per-
fect, subjects characterised by haphazard strategies seemed to be impaired in structural
knowledge, control performance as well as achievement in a prediction task.
Rollett, too, corroborates the view that individual differences in applying strategies
help to explain the frequently observed large variances in problem solving performance.
Rollett’s experimental studies on the effectiveness of specific strategies (see the section
about instructed strategies) are supplemented by analyses on the benefits of information
utilisation. The term information utilisation refers to the question which and how
much information participants – when dealing with a complex problem – actually take
into consideration as compared to the type and amount of information participants
might make use of in case of expertise. Practically speaking, the “experts’” utilisable
information is rated for each subject and each episode of intervention separately by
analysing specific interventions and the resulting input and outcome states.
Correlation analyses based on samples of a 109 subjects in the first study and 207 sub-
jects in a second study showed significantly positive correlations between the assumed
experts’ information use and participants’ information use (correlation coefficients rang-
ing from 0.56 to 0.29). Since the highest coefficients were found for information utilisa-
tion in the first of several learning rounds one may conclude that problem solvers at the
beginning of their task make use of available information most efficiently whereas in the
later process of problem solving efficient use of information partly decays, possibly due
to further cognitive demands.
Combining the two concepts of strategic operating and information utilisation a problem
solvers’ typology has been Rollett’s major contribution to the research on individual
differences in CPS. Subjects are classified on two dimensions: on the dimension of
strategy use as either insufficient, sufficient or efficient users and on the dimension of
information use as either belonging to the lower, the medium or the upper third among
other performers. Frequency analyses on the resulting (nine) combinations revealed
that a majority of subjects characterised by efficient strategy use made highest use of
available information; only a minority of them made lowest use of information. In turn,
insufficient strategies were found primarily with subjects whose information use was
assessed to be in the lower third. Obviously, efficient use of strategies and information
coincide to some degree.
In her above mentioned study, Branke (1991) assessed that only half of the sample’s
participants preferred a strategic approach to the problem situation. In Putz-Osterloh’s
experiment (1993b) even as many as two thirds of the subjects could be characterised
by employing ineffective strategies. The analyses by Vollmeyer and colleagues (1996)
showed that in a first round of dealing with the dynamic system Biology Lab again
two thirds of the subjects started exploring the system by least systematic haphazard
strategies, but after three further rounds the share of unsystematic strategies declined
to about 20 %. More than half of the problem solvers (58 %) rather made use of VOTAT
in the later stages of exploration. As it seems, the use of systematic strategies will be
promoted by growing experience with the dynamic system.
Why, however, do problem solvers still dismiss or simply not apply strategic operations in
a dynamic computer-simulated task? To anticipate the subsequent sections, we will keep
in mind that close relations exist between intelligence and exploration skills (Kr¨oner,
2001) and that motivation, too, plays a major role in applying strategies (Vollmeyer &
Rheinberg, 1998; Rheinberg, Vollmeyer, & Rollett, 2001). Yet an earlier study by Fritz
and Funke (Fritz & Funke, 1988) suggests that the use of strategies in CPS might as well
depend on some specific learning abilities not covered by overall levels of intelligence.
The authors examined problem solving performance and the use of appropriate strate-
gies with a clinical sample of adolescents having “minimal cerebral dysfunctions” (MCD)
which are strongly associated with learning difficulties at school. Out of 53 adolescents,
14 displayed a stable MCD symptomatic, 17 – the group of potentially affected adoles-
cents – had once been given a diagnosis, but did not confirm it, another 16 adolescents
had no MCD diagnosis at all, thus serving as a control group. All participants worked on
a simplified version of Funke’s ¨
Okosystem (Funke, 1985), a dynamic system including
three exogenous and three endogenous variables, one instance of eigendynamic and no
side effects or delayed feedback.
It was hypothesised that in spite of their overall high intelligence subjects having MCD
would turn out to be inferior in strategic operating. Statistical analysis failed to reach
significance, yet the results showed a tendency towards the expectations. Various strate-
gies of CPS occurred in both the MCD groups and the control group, however, not
equally distributed. Whereas subjects of the control group frequently and successfully
employed zero intervention strategies and isolated manipulations of one single exogenous
variable, strategies most adapt to gain structural knowledge, subjects of the MCD group
more often tended to manipulate all three exogenous variables at a time, which hardly
enables the detection of isolated effects. From this characteristic difference the authors
inferred insufficient discriminatory abilities of participants having MCD.12 Lowered in-
tegrational abilities, too, were assumed since, compared to subjects of the control group,
those having MCD or being potentially affected required more time and experience to
acquire knowledge, i. e. to form an integrated structure of knowledge.
Interestingly, despite differing strategic approaches performance levels were about equal
among the three groups. MCD tends to be related to impaired strategic operations and
strategic operations definitely affect the process of problem solving as we have seen, but
there must still be other factors relevant to individual differences in this domain.
3.3.2 Motivation
It has widely been accepted that (at least some) motivation is necessarily required
for achievement in cognitive tasks. In a few experimental studies (e. g, Funke, 1992b)
measures of initial or current motivation have been implemented in order to ensure that
on the average participants are equally well motivated among different experimental
conditions.13 Yet the majority of investigations appears to do without motivational
measures of control, a fact which is not only due to economical considerations. The
particular difficulty of motivational records concerns the fluctuating nature of current
motivational states: Supposed an experimenter asks his or her subjects about some
defined aspects of motivation, say, interest in the task and expectations of successful
outcomes, how will the experimenter know whether the subjects’ self-assessments remain
valid over the course of the task’s duration, over at least one hour and often more than
that? Some facets of motivation, e. g, interest, which may appear dominant in the
beginning, may decrease or disappear, while other facets come into play which initially
have not been recorded at all, e. g, finishing one’s task quickly not to miss a later
appointment (compare Vollmeyer & Rheinberg, 1998). As Vollmeyer and Rheinberg
did in their study discussed below, researchers can, of course, employ brief intermittent
measures of present motivational states, yet being aware that the records might disturb
the process of learning and CPS.
Another perspective in examining motivational impacts in CPS is to consider aspects of
12Concerning motivation adolescents of the MCD group revealed significantly higher failure motiva-
tion, i. e. fear of potential failure. As we will see below, anticipation of failure as well should impair
the application of systematic strategies (see Vollmeyer & Rheinberg, 1998).
13The demand of comparable motivational levels will not necessarily be accomplished as Funke’s
findings show: Initial motivation already was lower for subjects who dealt with a system of merely
numerical presentation than for subjects who worked on an equivalent system with graphical display.
personality which are related to current motivational states, but are assumed to be more
stable over time and hence less affected by the hazards associated with current moti-
vation and intermitting records. The term motivational orientation or goal orientation
has been introduced to define relatively enduring motivational tendencies, an individ-
ual’s “tendency to favour specific types of goals, outcomes or consequences over some
others” (Niemivierta, 2002). Unfortunately, according to G¨orn et al. (2001), experts
have not yet fully agreed as to the question to what extent motivational orientations
display stable traits of personality or to what extent they are still related to fluctuating
motivational states.
In the present paper we will try to consider the impact of both motivational states and
trait dispositions on CPS. The four studies discussed in this section especially focus on
the question which motivational factors interact in which way with relevant cognitive
variables in problem solving. As Vollmeyer and Rheinberg (1998) state, motivation per
se does not affect cognition but is mediated by the frequency and duration, by the quality
of learning activities and the learner’s functional state such as his or her concentration.
With regard to motivational states in CPS Vollmeyer and Rheinberg (1998) have exam-
ined the impact of two specific factors: interest in the task and confidence of success vs.
fear of failure. In preliminary examinations these two facets of motivation had turned
out to be crucial specifically to the context of computer-simulated dynamic tasks. Hence,
for their experiment, Vollmeyer and Rheinberg recorded the two factors both as part
of a questionnaire intended to assess initial motivation and by means of brief question-
naires assessing motivation parallel to a dynamic problem solving task. The intermittent
questionnaires contained only one item each for task interest and confidence of success:
“I am still enjoying the task” and “I am sure I will find the correct solution”. In or-
der to elucidate how assumed motivational effects are mediated the authors added two
measures characterising learning activities: participants’ concentration on the task, es-
pecially “effortless concentration” which is associated with productive learning, secondly
the systematicity of applied strategies as can be inferred from the protocol of input and
output states. Effortless concentration was assessed in a one-item-scale questionnaire.
48 students of the University of California, Los Angeles, took part in Vollmeyer’s and
Rheinberg’s study. Subjects worked on a relatively complex version of the Dynamis
system Biology Lab whose cover story refers to a virtual biological laboratory; within
the “biology lab” participants have to find out how three chemical factors (salt, carbon,
calcium), the exogenous variables, influence the quality of water in a tank with respect
to three indicators (oxygenation, chlorine concentration, temperature), the exogenous
variables. Three rounds of mere exploration, each consisting of eight intervention trials,
were conducted, one subsequent round was assigned to system control and accomplishing
defined goal values.
Regarding the two motivational factors, unfortunately, only the factor confidence of
success vs. failure motivation proved stable enough to more detailed analysis. Subjects’
self-evaluations of initial interest in the task seemed to coincide too little with later
ratings of the task’s attractiveness, so that this factor was dismissed.
Path-analytical analysis of the remaining cognitive and motivational factors revealed
two indirect impacts (paths) and one direct impact subjects’ confidence of success has
on the quality of CPS. Indirect relations between confidence of success were firstly found
to be mediated by the systematicity of strategies applied: The more confident subjects
feel that they will succeed the more systematic strategies they will employ, an effect
which yet only became noticeable as late as the third round of exploration. Systematic
strategies in turn are positively related to the acquisition of knowledge and this again
has positive effects on the application of knowledge, leading to enhanced control perfor-
mance. Secondly, concentration appeared to be a mediator: The more confident about
successful outcomes, the higher participants scored on effortless concentration. Interest-
ingly, in Vollmeyer’s and Rheinberg’s model concentration revealed an enhancing effect
on control performance without enhancing knowledge acquisition as well. From this fact
the authors conclude that effortless concentration due to good motivation displays its
beneficial effect only when knowledge actually is applied; no explicit knowledge struc-
tures need to exist prior to application (see Vollmeyer & Rheinberg, p. 20). The third
path-analytical finding concerns a direct positive of impact confidence of success vs.
fear of failure on knowledge acquisition. Obviously, high confidence of success is associ-
ated with increased knowledge neither as a specific result of concentration or systematic
strategies, but possibly fostered by some other mediating factors we do not know yet.
Further specifications as to the interaction between current motivational states and cog-
nition in CPS have been provided in a subsequent study by Rheinberg and colleagues
(Rheinberg, Vollmeyer & Rollett, 2002). Motivational factors of research interest were
probability of success, anxiety, which corresponds about to fear of failure, interest in
the task and challenge, i. e. whether problem solvers feel that a task is a challenge to
them. These four factors formed a global score on initial motivation. Besides motiva-
tion, in this study relevant cognitive ability was examined as measured by the series
completion subtest from the Intelligenz-Struktur-Test (Intelligence-Structure-Test, I-S-
T, Amthauer, 1970)14. For cognitive measures characterising the process of CPS and
its quality the following were chosen: quality of goal achievement, quality of system
identification and a score on strategy systematicity.
107 students from the University of Potsdam and from local high schools were selected
to participate. The general procedure of the experiment was comparable to the above
study; the Dynamis scenario applied was an almost identical version of Biology Lab.
The main difference of Rheinberg’s, Vollmeyer’s and Rollett’s study as compared to
the previous concerns its methodological approach. Instead of applying path analysis
the authors ran cluster analyses, i. e. they tried to gain an insight into the relations
between initial motivation, ability and goal achievement by searching for different types
of learners who reveal characteristic patterns of motivation, ability and achievement.
Five such different types (groups or clusters) could be distinguished: Firstly, there were
subjects who – in line with theoretical expectations – scored high in goal achievement,
in relevant initial ability and motivation. These were termed the optimal learner type.
The mirror image of the optimal learner type scoring low in all three variables, too, was
found, and named the poor learner type. A third class of subjects proved to be relatively
low in performance despite high initial ability; as subjects’ motivation, however, was low,
performance deficits seem to have a profound explanation. Concerning these learners,
Rheinberg and colleagues speak of the underachiever type.
14In the series completion subtest participants try to detect regularities inherent in a series of numbers
and complete the series according to these regularities. The test focuses on logical numerical abilities.
Less straightforward patterns of motivational and cognitive variables occurred for sub-
jects of the two remaining clusters: Both of these groups showed about average ability
and initial motivation, yet one group performed far worse than the other. What might
explain this fact? As an analysis on strategic operation suggested, the better performers
were those who acted more strategically and systematically. Hence the authors added
a cluster of systematic learners vs. a cluster of unsystematic learners to their typology.
They conclude that obviously both motivational aspects as well as cognitive ability,
strategy systematicity and still additional factors need to be combined when examining
the variances of individual performance in a CPS task. Rheinberg, Vollmeyer and Rollett
also note that clustering subjects according to characteristic patterns provides a useful
description of motivational influences, but not an explanation in terms of cause-and-
In contrast to relatively fluctuating motivational states goal orientation as defined above
deals with more enduring traits of personality that determine which goals or expected
outcomes make us put effort, time and concentration into learning activities such as a
CPS task. Two “classical” learning goals, sometimes considered as exclusive goals, have
become prominent in psychological research: The goal of enhancing one’s competence
and the goal of demonstrating one’s competence. The former motivational orientation
has been named ego orientation or performance orientation, the latter has been named
task orientation or competence orientation (see Nicholls, 1984; Dweck, 1986, 1989).
More differentiated goal orientations have been suggested by Niemivierta (2002) who
adds achievement goals, i. e. the goal to succeed, performance-avoidance goals, i. e. the
goal to avoid situations of potential experiences of failure and avoidance goals, i. e. the
goal to get off with as little effort as possible to the distinction of performance goals and
learning goals.
Certainly, any of these goals can affect a learner’s performance in the domain of CPS.
Which, however, will provide most suitable conditions for successfully coping with a
task? Dweck (1986) as well as Nicholls (1984) suggest that goal orientation consid-
ered on its own has no marked impact on problem solving performance. Task oriented
learners who are particularly interested in the task itself should do generally well when
dealing with a complex problem. Those, however, whose ego is – due to performance
orientation – more involved should rather depend on the level of their abilities as they
themselves perceive it. An ego-oriented learner who is confident in his or her abilities
will perform just as well as a task-oriented learner of comparable abilities; yet the ego-
oriented learner will prove inferior in performance if belief in his or her capabilities is
poor (see G¨orn et al., 2001).
According to G¨orn, Vollmeyer and Rheinberg (2001), this interaction between motiva-
tional orientation and self-evaluation of abilities had been demonstrated in prior exper-
iments applying differential instructions to manipulate either learning or performance
goals as experimental factors in between subjects designs. orn and her colleagues
advocated another approach: Since they conceived motivational orientation rather as
a trait of personality than as a transient state to be experimentally induced they ex-
amined the same interaction hypothesis by measuring goal orientations in motivational
113 school students from upper grades, mostly eleventh graders, took part in the ex-
periment. For testing in classrooms the CPS task was administered as a paper and
pencil version of Biology Lab. Pupils had to imagine they were medical researchers
in a laboratory and had to find out how three types of medication named A, B, and
C, the exogenous variables, affected each of three body substances, the output variables
named serotonin, thyroxin, and hystamine. Corresponding to six learning phases in the
computer-based scenario six different states of the system were presented subsequently
on six sheets of paper. These had to be analysed in order to gain structural knowledge
as measured through causal diagrams. In a final equivalent of the application phase sub-
jects calculated how exogenous variables had to be manipulated so that defined outcome
states of the endogenous variables would result. Success in problem solving was analysed
with regard to knowledge application and a score comparable to control performance
in computer-simulated versions. Perceived subjective failure or success was recorded by
brief self-ratings after each of the six sessions. The above hypothesis of an interaction
between motivational orientation and self-evaluation of failure or success could not be
confirmed. The authors assume that this effect is less prominent when goal orientation
is examined as a personality factor than when it is experimentally induced.
A second hypothesis concerned the relation between performance and learning goals:
Does it seem appropriate to assume performance and learning goals as non-overlapping,
exclusive characteristics of learners? Thinking of goal orientation rather as a two-
dimensional construct of personality, G¨orn, Vollmeyer and Rheinberg classified subjects
according to four categories: task-oriented learners scoring high only on task orienta-
tion, ego-oriented learners scoring high only on ego orientation, high indifferent learners
scoring high on both orientations and low indifferent learners scoring high on neither
orientation. It was predicted that high indifferent learners would be most successful in a
CPS task because two goals, relative to only one, might enhance the beneficial impact of
motivation. Following the same logic, low indifferent learners were expected to perform
worst, i. e. even worse than ego-oriented subjects who care at least for demonstrating
competence as their dominant goal. While high indifferent learners indeed turned out
to perform best, unexpectedly low indifferent learners, too, did well on their task, a
finding which remains unclear. In spite of this, according to G¨orn and colleagues we
can at least suppose that the bipolar distinction of learners as either task-oriented or
ego-oriented appears too general.
Niemivierta (2002) continued the research on goal orientation in CPS by means of
two further approaches. With respect to subjects’ evaluations during problem solv-
ing Niemivierta analysed not merely self-related evaluations of success or failure and
test anxiety, but also subjects’ situational appraisals. Secondly, he combined the trait-
oriented notion of goal orientations (see G¨orn et al., 2001) with earlier experiments’
practice to induce either task-involving or ego-involving learning conditions. The main
question in Niemivierta’s work is how habitual motivational orientation interacts with
different instructional learning conditions with respect to situational appraisals and re-
sulting performance in a CPS task. Examples of situational appraisals are self-efficacy,
i. e. confidence to do well in a task, and claimed self-handicapping, i. e. the use of
anticipatory excuses (like being ill or in a bad mood) when success might be threatened.
Niemivierta examined situational appraisals, goal orientation and, of course, task perfor-
mance (knowledge acquisition and knowledge application) in a sample of one 143 school
students15, ninth graders from junior high schools in southern Finland. The students
dealt with the computer-simulated dynamic system Med Lab, which is nearly iden-
tical to Biology Lab, each under one of two different instructional conditions. Half
of the students, the task-involving condition, received task-focused instructions which
requested subjects to work on the problem as hard as they could, so that the current ver-
sion of Med Lab could be evaluated and revised. It was highlighted that the students
would not be tested with regard to individual successful performance. The other half
of the students, the ego-involving condition, received performance-focused instructions;
these obviously focused on individual performance as students were told that a few days
later, their teacher would announce the results.
It was hypothesised that subjects in the latter condition as compared to those receiv-
ing task-involving instructions would reveal higher levels of test anxiety, lower levels
of self-efficacy and more claimed self-handicapping, especially if the students had been
classified as habitually performance-oriented learners. Furthermore, performance in the
Med Lab task was expected to be lower with participants of the ego-involving condition.
Disconfirming the last expectation, levels of task performance did not differ significantly
among the two instructional conditions. Neither did test anxiety seem to be affected by
ego- vs. task-involving instructions; as one would guess, test anxiety was generally high-
est for participants reporting performance-avoidance goals, the goal to avoid situations
of potential failure. Yet with regard to situational appraisals Niemivierta’s assumptions
were supported: Ego-involving instructions obviously led to lower reported self-efficacy
or lower expected success, which may be interpreted as a self-protecting function in
the ego-involving and possibly ego-threatening instructional context. While subjects
with different patterns of goal orientations seemed to be comparable as to their overall
situational appraisals, in the ego-involving condition more self-handicapping and less
self-efficacy was reported when learners’ scored high on ego orientation as a personality
trait relative to habitually task-oriented students. Once again, the impact of different
goal orientations and multiple effects in the context of CPS are illustrated.
3.3.3 Intelligence
Last but not least, our final section concerns the role of intelligence in CPS tasks. In
research literature this issue has been discussed in rather controversial disputations. As
Funke (2003) states, in early research, test intelligence has been regarded and examined
as the major influencing factor on problem solving. Yet initial findings seemed to con-
tradict both common sense and scientific hypotheses. An investigation by Putz-Osterloh
(1981) applying tailor shop, a popular, non-Dynamis-based computer simulation, even
suggested that no systematic relations at all existed between test intelligence and control
performance in tailor shop. In fact, a tendency towards a negative relation occurred, i. e.
participants scoring high in a traditional intelligence test did worse in controlling the
complex scenario. Less striking, a following experiment (Putz-Osterloh & L¨uer, 1981)
revealed zero correlations between control performance and intelligence only if no addi-
tional information had been given to the problem solvers. In case of transparency, i. e.
if the system’s internal relations had been disclosed to subjects by means of a structural
15To later evaluation only the data of 100 subjects proved to be valid.
diagram, the authors found significantly positive correlations between the intelligence
score and the quality of system control.
Hence the more intelligent the problem solver the more successful he or she will interact
with a dynamic system, but this effect is mediated by transparency conditions. Why
should this make sense? Following the author’s interpretation working on a transparent
dynamic system requires systematic analysis of given information prior to its application,
a task demand which is relatively similar to the demands of traditional intelligence diag-
nostics. In case of intransparency – a typical feature of complex problems – participants
actively search for task-relevant information, a demand not covered in intelligence tests.
As self-guided search for information and analysis of given information require different
abilities, success in a standard problem solving paradigm and conventional intelligence
tasks may be unrelated (for summaries of the findings see also Funke, 2003).
In consequence, differential task demands and required abilities concerning these two
domains have increasingly been the focus of attention. Presenting his concept of “oper-
ative intelligence” D¨orner (1986) has highlighted the dynamic orientation as the main
task feature specific to CPS. Unlike in static intelligence tests which cover participants’
speed and accuracy abilities, working on complex dynamic systems additionally involves
skills such as anticipation and consideration of temporal processes, setting up appropri-
ate goals and subgoals (and occasionally modifying them in the course of the process),
using successful and adaptive strategies to search for relevant information. Ecological
validity, i. e. the proximity to “real” problems in the outside world is said to be much
higher for complex problem computer simulations as compared to traditional intelligence
On the other hand, at least some connections between problem solving and test intel-
ligence cannot be denied. Starting with an experiment by Funke (1983), the view that
scores in intelligence tests can predict the level of control performance has become re-
established. According to Beckmann and Guthke (1995) positive correlations between
intelligence and achievement in problem solving could be revealed consistently (p. 178).
Yet correlations appear too low to infer a direct impact of intelligence on the process of
problem solving. Following D¨orner’s approach, Beckmann and Guthke assume a multi-
tude of influential factors that mediate the effects of test intelligence. Transparency is
one such factor (see above), and the same applies, e.g, to task difficulty or goal defini-
tion. Highest correlations between intelligence scores and control performance in CPS
can be expected with problems of average difficulty (Beckmann & Guthke, p. 184).
Concerning goal definition, well-defined goals in a problem solving situation make the
demands more similar to demands of traditional intelligence tests than ill-defined goals
(e. g, Strohschneider, 1991). As further examples adding to this list, potentially mediat-
ing effects of semantic embedding and learning ability were analysed in an experiment
done by Beckmann and Guthke.
The researchers tested 92 school students successively on a CPS task (first session), two
conventional intelligence tests (second session) and two comparable short-term learning
tests (third and fourth session). Experimental manipulation as to semantic embedding
was realised by assigning half of the pupils to a semantically embedded Dynamis system
named Kirschbaum while the other half worked on Maschine, a Dynamis system
identical in structure and parameters, but lacking semantic embedding. The measures
of academic intelligence comprised a subtest on analogies taken from the Amthauer
Intelligenz-Struktur-Test (IST, Amthauer, 1973) and a reasoning task (“figure series”),
the pretest of the Lerntest “Schlussfolgerndes Denken” (LTS 3, Guthke et al., 1983).
The results confirmed the authors’ expectation that semantic embedding mediates the
influence of test intelligence on control performance: Significant correlations between
these two measures were found only if the complex problem situation was abstract. In
case of semantic embedding intelligence scores could not predict control performance
(for an equivalent finding see also M¨uller, 1993). This, however, obviously did not result
from semantic embedding per se, but from generally impaired knowledge acquisition in
the semantic context. As noted in the section about semantic embedding, semantics
can lead to “sham confidences” which prevent participants from gaining real and useful
knowledge. In turn, if hardly any knowledge is acquired by both intelligent and less
intelligent subjects, except for ad hoc control there is no reason why anyone should be
superior in knowledge application, i. e. in control performance.
The authors’ second question aimed at learning ability and the comparability of com-
plex problem situations and learning tests. The idea behind learning tests is to “record
not only a subject’s momentary performance in a one-time administration of a test pro-
cedure, as is done in the usual static intelligence test situation, but also the subject’s
responses to repeated, standardised questions that are built into the test” (Beckmann
& Guthke, 1995, p. 186). Hence, the concept of a learning test appears to share
some dynamic components with CPS tasks, although in learning tests more detailed
and instantaneous feedback is given to participants. The two learning tests chosen for
Beckmann’s and Guthke’s experiment were said to cover learning abilities that corre-
sponded to the intellectual abilities measured by the two conventional intelligence tests.
It was hypothesised that learning tests, sharing the dynamic feature with CPS, are bet-
ter instruments of predicting knowledge acquisition in a complex situation than static
intelligence tests would be. No effect was expected with regard to control performance.
In general, statistical analyses confirmed the expectations. Only in case of seman-
tic embedding learning tests and static intelligence tests were equally good or equally
bad predictors of the amount of knowledge gained since this amount of knowledge was
generally poor (see above). Beckmann and Guthke conclude that systematic relations
between academic intelligence and performance in complex problem situations definitely
exist. Zero correlations as reported in early research might have been methodological
artefacts, e. g, due to impaired and omitted knowledge acquisition, the authors suggest.
Existing correlations yet become more prominent if mediating factors such as semantic
embedding or learning ability are revealed.
More recently, Kr¨oner has studied the correlations between problem solving performance
and test intelligence from a practical point of view. His aim has been to develop and
validate a Dynamis system suitable to serve as a tool of intelligence diagnostics. Origi-
nal and revised versions of MultiFlux, the graphically embedded computer simulation
of a fictive machine, provided the base of three experimental studies. Control perfor-
mance and knowledge acquisition in MultiFlux was compared to achievement in the
Advanced Progressive Matrices (APM; Raven, 1958), a conventional intelligence test
which demands logical reasoning in completing graphical patterns.
The first study, a pilot study involving a sample of 28 students, on the whole supported
Kr¨oner’s major hypotheses: Intelligent subjects performed better on the MultiFlux
control task. They also tended to be superior in skilful exploration although, in part,
(indefinite) factors other than test intelligence seemed to contribute to exploration skills.
To reduce the influence of such unknown sources of variation and to rise the impact of
intelligence on exploration in dynamic system the second experiment was conducted. 96
high school students worked on a revised version of MultiFlux. Indeed correlations
between good exploration of MultiFlux and APM scores were found to be higher
. The third study employing a further revised scenario finally was designed to show
that the enhanced control performance of intelligent subjects is not merely the result
of better knowledge acquisition (due to more skilful exploration); instead there should
be a an additional direct and positive effect of test intelligence on control performance.
The data of a 101 pupils (grades ten to twelve) supported this claim. Kr¨oner resumes
that close relations exist between achievement in a conventional intelligence test and
the quality of dealing with a complex system, the correlations being high, though not
maximal. The indicators of test reliability are considered satisfactory so that, according
to Kr¨oner, MultiFlux might provide a simulation-based tool of intelligence diagnostics
in the future.
Amthauer, R. (1970). I-S-T 70. Intelligenz-Struktur-Test [I-S-T 70. Intelligence-
Structure-Test]. G¨ottingen: Hogrefe.
Amthauer, R. (1973). IST-70. Intelligenz-Struktur-Test [Test of the structure of intelli-
gence]. G¨ottingen: Hogrefe.
Anderson, J. R. (1987). Skill acquisition: Compilation of weak-method problem solu-
tions. Psychological Review, 94, 192-210.
Ashby, W. R. (1958). Requisite variety and its implications for control of complex
systems. Cybernetica, 1, 83-99.
Beckmann, J. F. (1994). Lernen und komplexes Probleml¨osen. Ein Beitrag zur Kon-
struktvalidierung von Lerntests [Learning and complex problem solving. A contribution
to the validation of learning tests]. Bonn: Holos.
Beckmann, J. F. & Guthke, J. (1995). Complex problem solving, intelligence and learn-
ing ability. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European
perspective (pp. 177-200). Hillsdale, NJ: Lawrence Erlbaum Associates.
Berry, D. C. & Broadbent, D. E. (1984). On the relationship between task performance
and associated verbalizable knowledge. Quarterly Journal of Experimental Psychology,
36A, 209-231.
Branke, S. (1991). Die Interaktion von Personen- und Aufgabenmerkmalen beim kom-
plexen Probleml¨osen [The interaction of person and task characteristics in complex prob-
lem solving]. Report, Psychologisches Institut der Universit¨at Mainz, Mainz.
Brehmer, B., & D¨orner, D. (1993). Experiments with computer-simulated microworlds:
Escaping both the narrow straits of the laboratory and the deep blue sea of the field
study. Computers in Human Behavior, 9, 171-184.
Dienes, Z. & Fahey, R. (1995). Role of specific instances in controlling a dynamic system.
Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 848-862.
orner, D. (1986). Diagnostik der operativen Intelligenz [Diagnosis of operative intelli-
gence]. Diagnostica, 32, 290-308.
orner, D. (1987). On the difficulties people have in dealing with complexity. In J.
Rasmussen, K. Duncan, & J. Leplat (Eds.), New technology and human error (pp. 97-
109). Chichester: Wiley.
Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist,
41, 1040-1048.
Dweck, C. S. (1989). Motivation. In A. Lesgold & R. Glaser (Eds.), Foundations for a
psychology of education (pp. 85-136). Hillsdale, NJ: Erlbaum.
Frensch, P. A. & Funke, J. (Eds.). (1995). Complex problem solving: The European
perspective. Hillsdale, NJ: Erlbaum.
Funke, J. (1983). Einige Bemerkungen zu Problemen der Probleml¨oseforschung oder: Ist
Testintelligenz doch ein Pr¨adiktor [Some comments on the problems in problem-solving
research or is test intelligence still a predictor]? Diagnostica, 29, 283-302.
Funke, J. (1985). Steuerung dynamischer Systeme durch Aufbau und Anwendung sub-
jektiver Kausalmodelle [Control of dynamic systems by building up and using subjective
causal models]. Zeitschrift f¨ur Psychologie, 193, 435-457.
Funke, J. (1992a). Dealing with dynamic systems: Research strategy, diagnostic ap-
proach and experimental results. German Journal of Psychology, 16(1), 24-43.
Funke, J. (1992b). Wissen ¨uber dynamische Systeme: Erwerb, Repr¨asentation und
Anwendung [Knowledge about systems: Acquistion, representation, and application].
Berlin: Springer
Funke, J. (1993). Microworlds based on linear equation systems: A new approach
to complex problem solving and experimental results. In G. Strube & K.-F. Wender
(Eds.), The cognitive psychology of knowledge (pp. 313-330). Amsterdam: Elsevier
Science Publishers.
Funke, J. (2003). Probleml¨osendes Denken [Problem-solving thinking]. Stuttgart: Kohl-
Funke, J. & M¨uller, H. (1988). Eingreifen und Prognostizieren als Determinanten von
Systemidentifikation und Systemsteuerung [Intervention and prediction as determinants
of system identification and system control]. Sprache & Kognition, 7, 176-186.
Gardner, P. H., & Berry, D. C. (1995). The effect of different forms of advice on the
control of a simulated complex system. Applied Cognitive Psychology, 9 (Special Issue),
orn, A., Vollmeyer, R., & Rheinberg, F. (2001). Effects of motivational orientation
and perceived ability on performance. Psychologische Beitr¨age, 43, 153-171.
Goshen-Gottstein, Y. & Moscovitch, M. (1995). Repetition priming for newly formed
and preexisting associations: Perceptual and conceptual influences. Journal of Experi-
mental Psychology: Learning, Memory, and Cognition, 21, 1229-1248.
Guthke, J., J¨ager, C., & Schmidt, I. (1983). Lerntest “Schlussfolgerndes Denken” (LTS)
[Learning test “Reasoning”]. Berlin: Psychodiagnostisches Zentrum der Humboldt-
ubner, R. (1987). Eine naheliegende Fehleinscatzung des Zielabstandes bei der zeitop-
timalen Regelung dynamischer Systeme [A common misperception of goal distance when
controlling dynamic systems in optimal time]. Zeitschrift f¨ur Experimentelle und Ange-
wandte Psychologie, 34, 38-53.
ubner, R. (1988). Die kognitive Regelung dynamischer Systeme und der Einfluss
analoger versus digitaler Informationsdarbietung [Cognitive regulation of dynamic sys-
tems and its effects on analog versus digital information presentation]. Zeitschrift f¨ur
Psychologie, 196, 161-170.
Klahr, D. & Dunbar, K. (1988). Dual space search during scientific reasoning. Cognitive
Science, 12, 1-55.
Kluge, A. (2003). Probleml¨oseleistungen und Szenarioschwierigkeit: ¨
Uber Wissenser-
werb f¨ur das Steuern technischer Systeme. Unpublished Habilitationsschrift, RWTH,
Kluwe, R. H. (1988). Methoden der Psychologie zur Gewinnung von Daten ¨uber men-
schliches Wissen [Psychological methods for collecting data on human knowledge]. In
H. Mandl & H. Spada (Eds.), Wissenspsychologie (pp. 359-385). M¨unchen: Psychologie
Verlags Union.
Kr¨oner, S. (2001). Intelligenzdiagnostik per Computersimulation [Measuring intelligence
by means of computer simulations]. M¨unster: Waxmann.
Moray, N., Lootsteen, P., & Pajak, J. (1986). Acquisition of process control skills. IEEE
Transactions on Systems, Man, and Cybernetics, SMC-16, 497-504.
uller, H. (1993). Komplexes Probleml¨osen: Reliabilit¨at und Wissen [Complex problem
solving: Reliability and knowledge]. Bonn: Holos.
Nicholls, J. G. (1984). Achievement Motivation: Concepts in ability, subjective experi-
ence, task choice, and performance. Psychological Review, 91, 328-346.
Niemivirta, M. (2002). Motivation and performance in context: The influence of goal
orientation and instructional setting on situational appraisals and task performance.
Psychologia, 45, 250-270.
Preußler, W. (1996). Zur Rolle expliziten und impliziten Wissens bei der Steuerung
dynamischer Systeme [On the role of explicit and implicit knowledge in controlling
dynamic systems]. Zeitschrift f¨ur Experimentelle Psychologie, 43, 399-434.
Preußler, W. (1997). Effekte des Kontexts auf den Wissenserwerb bei der Steuerung
eines dynamischen Systems [Effects of context on the acquisition of knowledge in con-
trolling a dynamic system]. Sprache & Kognition, 16, 48-59.
Preußler, W. (1998). Strukturwissen als Voraussetzung f¨ur die Steuerung komplexer
dynamischer Systeme [Structural knowledge as a precondition of controlling complex
dynamic systems]. Zeitschrift f¨ur Experimentelle Psychologie, 45, 218-240.
Preußler, W. (2001). Probleml¨oseerfolg als Bedingung f¨ur den Erwerb von Eingriffswis-
sen bei der Systemsteuerung [Successful problem solving as a precondition for acquisition
of control knowledge]. Zeitschrift f¨ur Experimentelle Psychologie, 48, 3, 214-225
Putz-Osterloh, W. (1981). ¨
Uber die Beziehung zwischen Testintelligenz und Probleml¨o-
seerfolg [The relation between test intelligence and problem solving success]. Zeitschrift
ur Psychologie, 189, 79-100.
Putz-Osterloh, W. (1993a). Strategies for knowledge acquisition and transfer of knowl-
edge in dynamic tasks. In G. Strube & K.-F. Wender (Eds.), The cognitive psychology
of knowledge (pp. 331-350). Amsterdam: Elsevier Science Publishers.
Putz-Osterloh, W. (1993b). Unterschiede im Erwerb und in der Reichweite des Wissens
bei der Steuerung eines dynamischen Systems [Differences in the acquisition and extent
of knowledge during system control]. Zeitschrift f¨ur Experimentelle und angewandte
Psychologie, 40, 386-410.
Putz-Osterloh, W. & L¨uer, G. (1981). ¨
Uber die Vorhersagbarkeit komplexer Problem-
oseleistungen durch Ergebnisse in einem Intelligenztest [On the predictability of the
performance in COMPLEX PROBLEM SOLVING by results from an intelligence test].
Zeitschrift f¨ur Experimentelle und Angewandte Psychologie, 28, 309-334.
Raven, J. C. (1958). Advanced progressive matrices (2nd Ed.). London: Lewis.
Reither, F. (1981). About thinking and acting of experts in complex situations. Simu-
lation & Games, 12, 125-140.
Rigas, G. & Brehmer, B. (1999). Mental processes in intelligence tests and dynamic
decision making tasks. In P. Juslin & H. Montgomery (Eds.), Judgement and decision
making: Neo-Brunswikean and process-tracing approaches (pp. 45- 65). Hillsdale, NJ:
Rollett, W. (2003). Strategieeinsatz, Informationsnutzung und Motivation beim Explo-
rieren und Steuern komplexer Systeme. Unpublished Dissertation, Humanwissenschaft-
liche Fakult¨at der Universit¨at Potsdam, Potsdam.
Schoppek, W. (2002). Examples, rules, and strategies in the control of dynamic systems.
Cognitive Science Quarterly, 2, 63-92.
Schulz, C. A. (2003). Strategien zum komplexen Probleml¨osen. Eine experimentelle
Analyse der Rolle von Analogien beim Wissenserwerb in komplexen, dynamischen Sys-
temen [Strategies for complex problem solving : an experimental analysis of the role
of analogies for knowledge acquisition in complex, dynamic systems]. Unpublished
Diplomarbeit, Psychologisches Institut der Universit¨at, Heidelberg. [online available
Sternberg, R. J. & Frensch, P. A. (Eds.)(1991). Complex problem solving: Principles
and mechanisms. Hillsdale, NJ: Erlbaum.
Stevens, J. (1992). Applied multivariate statistics for the social sciences. Second edition.
Hillsdale, NJ: Erlbaum.
Steyer, R., Schmitt, M., & Eid, M. (1999). Latent state-trait theory and research in
personality and individual differences. European Journal of Personality, 13, 389-408.
Strauß, B. (1993). Konfundierungen beim komplexen Probleml¨osen [Confundations in
complex problem-solving: The influence of the ratio of correct solutions on problem-
solving in complex situations]. Bonn: Holos.
Strohschneider, S. (1991). Probleml¨osen und Intelligenz: ¨
Uber die Effekte der Konkre-
tisierung komplexer Probleme [Problem solving and intelligence: The effects of problem
concreteness]. Diagnostica, 37, 353-371.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive
science, 12, 257-285.
Vollmeyer, R. & Burns, B. D. (1996). Hypotheseninstruktion und Zielspezifit¨at: Be-
dingungen, die das Erlernen und Kontrollieren eines komplexen Systems beeinflussen
[Hypothesis instruction and goal specificity: Factors influencing the learning and con-
trol of complex systems]. Zeitschrift f¨ur Experimentelle Psychologie, 43, 657-683.
Vollmeyer, R., Burns, B. D., & Holyoak, K. J. (1996). The impact of goal specificity on
strategy use and the acquisition of problem structure. Cognitive Science, 20, 75-100.
Vollmeyer, R. & Funke, J. (1999). Personen- und Aufgabenmerkmale beim komplexen
Probleml¨osen [Person and task effects within complex problem solving]. Psychologische
Rundschau, 50, 213-219.
Vollmeyer, R. & Rheinberg, F. (1998). Motivationale Ein߬usse auf Erwerb und Anwen-
dung von Wissen in einem computersimulierten System [Motivational influences on the
acquisition and application of knowledge in a computer-simulated system]. Zeitschrift
ur P¨adagogische Psychologie, 12, 11-23.
... The goal was to set up a (face-) valid, psychometric sound microworld to assess complex problem solving (CPS) that can immediately be applied in the school context. To this end, the development drew on the rich body of empirical knowledge that was derived from previous studies on microworlds (for an overview see for example Blech & Funke, 2005;Funke & Frensch, 2007). To enable educators to make full use of the GL, it can be administered within 50 minutes (i.e. the length of a typical school lesson), and in three different languages (English, French, and German In the GL (shown in Fig. 1), students explore how genes of fictitious creatures influence their characteristics. ...
... Although the method of knowledge assessment by causal diagrams was often successfully used in samples of university students (Blech & Funke, 2005;Funke, 2001), critique was raised that due to its high cognitive demand, it might be problematic in samples of lower cognitive capacity. ...
... Both skills are key characteristics of CPS (Funke, 2003). Most previous attempts to score control performance emphasized the (aggregated) deviation between the achieved values and the target values (Blech & Funke, 2005). This approach, however, was critized for making the scoring of a step dependent from the previous one if the scenario does not allow to reach the target value within one step. ...
Full-text available
Complex problem solving (CPS) is now established as a key aspect of today’s educational curricula and a central competence for international assessment frameworks. It has become clear that the educational context – particularly among general school-age students – places particular demands on the instruments used to assess CPS, such as computer-based microworlds. This chapter shows how these challenges can successfully be addressed by reviewing recent advancements in the field of complex problem solving. Using the example of the Genetics Lab, a newly developed and psychometrically sound microworld which emphasises usability and acceptance amongst students, this chapter discusses the challenges and opportunities of assessing complex problem solving in the classroom.
... Complex systems are characterized by uncertainty and nonlinear interactions (Blech and Funke, 2005;Diehl and Sterman, 1995;Forrester, 1993) making it difficult to understand relations between elements. Furthermore, consequences of actions are often delayed in time and diluted by natural dynamic changes (Karakul and Qudrat-Ullah, 2008), while feedback may also be distorted, subject to misinterpretation, or imperceptible (Sterman, 2006). ...
... Furthermore, consequences of actions are often delayed in time and diluted by natural dynamic changes (Karakul and Qudrat-Ullah, 2008), while feedback may also be distorted, subject to misinterpretation, or imperceptible (Sterman, 2006). The dynamics of such systems are determined by their underlying structure, so knowledge about the causal relations between the elements comprising such systems e referred to as structural knowledge (Davis et al., 2003) e is critical for performing effective decision-making in this context (e.g., Blech and Funke, 2005;Gagnon et al., 2012). However, decision-making is not solely dependent on the quality of structural knowledge, as heuristics e simple but effective strategies that do not require a profound knowledge of a situation e may also be used and can often lead to good performance in certain task ecologies that favour the simplicity principle (Gigerenzer and Gaissmaier, 2011;Gigerenzer and Selten, 2001;Shah and Oppenheimer, 2008). ...
While simple heuristics can be ecologically rational and effective in naturalistic decision making contexts, complex situations require analytical decision making strategies, hypothesis-testing and learning. Sub-optimal decision strategies – using simplified as opposed to analytic decision rules – have been reported in domains such as healthcare, military operational planning, and government policy making. We investigate the potential of a computational toolkit called “IMAGE” to improve decision-making by developing structural knowledge and increasing understanding of complex situations. IMAGE is tested within the context of a complex military convoy management task through (a) interactive simulations, and (b) visualization and knowledge representation capabilities. We assess the usefulness of two versions of IMAGE (desktop and immersive) compared to a baseline. Results suggest that the prosthesis helped analysts in making better decisions, but failed to increase their structural knowledge about the situation once the cognitive prosthesis is removed.
... Berry & Broadbent, 1984;Dörner, 1980;Funke, 1992; for a review see Osman, 2010). The study presented in this article was underpinned by the DYNAMIS or complex problem solving (CPS) approach, introduced by Funke (1992;; see Blech & Funke, 2005 for a review). CPS tasks consist of a number of inputs (variables that the problem solver intervenes on) and outputs (outcomes that are generated by the system) that are governed by a set of linear equations (this is referred to as the underlying structure of the system). ...
Full-text available
Many situations require people to acquire knowledge about, and learn how to control, complex dynamic systems of inter-connected variables. Numerous studies have found that most problem solvers are unable to acquire complete knowledge of the underlying structure of a system through an unguided exploration of the system variables; additional instruction or guidance is required. This paper examines whether providing structural information following an unguided exploration also improves control performance, and the extent to which any improvements are moderated by problem solvers’ fluid intelligence. A sample of 98 individuals attempted to discover the underlying structure of a computer-simulated complex dynamic system. After initially controlling the system with their limited knowledge, half of the sample received guidance concerning the actual underlying structure of the system. All participants then controlled the system again. In contrast to the results of previous studies, guidance did result in immediate improvements in control performance. Fluid intelligence moderated the extent to which problem solvers benefited from structural information. These results indicate that guidance in the form of structural information is critical in promoting knowledge and control over complex and dynamic systems.
... Several cognitive processes are involved in the solving of complex problems. The OECD (2013) established a framework of problem solving that is based upon the work of several cognitive psychologists (Baxter & Glaser, 1997;Blech & Funke, 2005, 2010Bransford, Brown, & Cockling, 1999;Funke & Frensch, 2007;Greiff & Funke, 2008;Klieme, 2004;Mayer & Wittrock, 2006;Osman, 2010;Reeff, Zabal, & Blech. C., 2006; Vosniadou & Ortony, 1989;Wirth & Klieme, 2003). ...
Full-text available
Complex problem solving (CPS) is a highly transversal competence needed in educational and vocational settings as well as everyday life. The assessment of CPS is often computer-based, and therefore provides data regarding not only the outcome but also the process of CPS. However, research addressing this issue is scarce. In this article we investigated planning activities in the process of complex problem solving. We operationalized planning through three behavioral measures indicating the duration of the longest planning interval, the delay of the longest planning interval and the variance of intervals between each two successive interactions. We found a significant negative average effect for our delay indicator, indicating that early planning in CPS is more beneficial. However, we also found effects depending on task and interaction effects for all three indicators, suggesting that the effects of different planning behaviors on CPS are highly intertwined.
... This argument is well supported by research in social cognition which establishes that the body movements might provide an immediate understanding of mental states and intentions of students (Fuchs & De Jaegher, 2009;Gallese, 2007). This research stream presents evidence that bodily states including postures, gaze, movement etc. are core to social cognition and can be used to interpret mental states (Barsalou, Niedenthal, Barbey, & Ruppert, 2003). Here, the concepts of synchrony, individual accountability, equality, and intra-individual variability in are investigated and shown to be identified from students' nonverbal behaviours in practice-based learning environments. ...
... However, students' intra-individual variability to our knowledge has not been investigated in the contexts of students' CPS in practice-based learning. As emphasised by various other researchers CPS tends to be inherently interactive, interdependent, and dynamic [21,22]. CPS can only occur if the students attempt to create a common ground about the problem/task they are dealing with [23]. ...
Conference Paper
Full-text available
Collaborative problem solving (CPS), as a teaching and learning approach, is considered to have the potential to improve some of the most important skills to prepare students for their future. CPS often differs in its nature, practice, and learning outcomes from other kinds of peer learning approaches, including peer tutoring and cooperation; and it is important to establish what identifies collaboration in problem-solving situations. The identification of indicators of collaboration is a challenging task. However, students physical interactivity can hold clues of such indicators. In this paper, we investigate two non-verbal indexes of student physical interactivity to interpret collaboration in practice-based learning environments: equality and intra-individual variability. Our data was generated from twelve groups of three Engineering students working on open-ended tasks using a learning analytics system. The results show that high collaboration groups have member students who present high and equal amounts of physical interactivity and low and equal amounts of intra-individual variability.
... A DIPF által vezetett TBA (Technology Bases Assessment) projekt másik lényeges eleme a komplex problémamegoldás számítógép-alapú mérése (Wirth és Klieme, 2003, Wirth és Funke, 2005). A MicroDYN megközelítés fényében teljes mértékben kihasználják a technológia adta lehetőségeket, és egy dinamikusan változó környezettel szembesítik a tesztelt személyeket (Blech és Funke, 2005; Greiff és Funke, 2008). A TAO-rendszer szerepel az EU ajánlásaiban, ezért a fejlesztő munkák koordinálására az Unió az Isprában működő Joint Research Center keretében elindított egy nemzetközi együttműködési programot. ...
... The process to be controlled typically consists of a large number of interrelated and cross-coupled variables (Moray 1997;Vicente 2007;, meaning that various aspects of a situation are not independent and therefore cannot be independently influenced, a characteristic called interconnectivity . Interconnectivity also stresses the importance of recognising unfamiliar and unintended feedback loops (Perrow 1984), control parameters with potential interactions and undesired and desired "parallel effects" (Blech and Funke 2005). Parallel effects are caused by ramified cause-and-effect chains, initiated by altering only one single input variable at the beginning of the chain . ...
This chapter introduces the terms acquisition, retention and transfer of knowledge and skill relevant for control room operators’ tasks. The chapter defines the differences between work experience and practice and between novices and experts, which is later relevant for the staged training design. The theories of instance-based learning, bottom-up and top-down learning are presented as well as learning mechanisms that support instance-based learning for novice learners in order to reduce complexity in early stages of learning to operate the plant. The chapter subsequently outlines the process of ongoing learning processes for becoming an expert through deliberate practice. Theories concerning instance-based learning for learning teamwork skills follow, as well as theories addressing skill decay due to longer periods of non-use based on the New Theory of Disuse. The chapter ends with a list of all 30 propositions regarding how control room operators’ learning takes place and which learning principles support these learning processes.
... The process to be controlled typically consists of a large number of interrelated and cross-coupled variables (Moray 1997;Vicente 2007;, meaning that various aspects of a situation are not independent and therefore cannot be independently influenced, a characteristic called interconnectivity . Interconnectivity also stresses the importance of recognising unfamiliar and unintended feedback loops (Perrow 1984), control parameters with potential interactions and undesired and desired "parallel effects" (Blech and Funke 2005). Parallel effects are caused by ramified cause-and-effect chains, initiated by altering only one single input variable at the beginning of the chain . ...
This chapter addresses the control room operator’s task and the issue of ‘what control room operators control’. A definition of the term complex technical system is provided, followed by an introduction and definition of the constituents of a complex system such as interconnectivity, dynamic effects, non-transparency and crew coordination affordances. In order to understand crew coordination complexity, plant operation team roles are explained. A list of operators’ tasks is provided, as well as differences between process industries batch/mix and process/flow industries relevant for understanding the operator’s control tasks. The constituent characteristics of a complex task are introduced. Finally, a distinction is made between routine and non-routine tasks as well as between normal and abnormal situations as relevant work conditions and context factors to be considered in training design.
Full-text available
There is a growing demand for assessment instruments which can be used in higher education, which cover a broader area of competencies than the traditional tests for disciplinary knowledge and domain-specific skills, and which measure students' most important general cognitive capabilities. Around the age of the transition from secondary to tertiary education, such assessments may serve several functions, including selecting the best-prepared candidates for certain fields of study. Dynamic problem-solving (DPS) is a good candidate for such a role, as tasks that assess it involve knowledge acquisition and knowledge utilization as well. The purpose of this study is to validate an online DPS test and to explore its potential for assessing students' DPS skills at the beginning of their higher education studies. Participants in the study were first-year students at a major Hungarian university (n = 1468). They took five tests that measured knowledge from their previous studies: Hungarian language and literature, mathematics, history, science and English as a Foreign Language (EFL). A further, sixth test based on the MicroDYN approach, assessed students' DPS skills. A brief questionnaire explored learning strategies and collected data on students' background. The testing took place at the beginning of the first semester in three 2-h sessions. Problem-solving showed relatively strong correlations with mathematics (r = 0.492) and science (r = 0.401), and moderate correlations with EFL (r = 0.227), history (r = 0.192), and Hungarian (r = 0.125). Weak but still significant correlations were found with certain learning strategies, positive correlations with elaboration strategies, and a negative correlation with memorization strategies. Significant differences were observed between male and female students; men performed significantly better in DPS than women. Results indicated the dominant role of the first phase of solving dynamic problems, as knowledge acquisition correlated more strongly with any other variable than knowledge utilization.
Full-text available
Forty-eight students participated in an experiment in which they had to learn the structure of a complex, computer-simulated system. Subjects manipulated the system and they could analyze the result. By doing so they could increase their knowledge about the system, that is, they could learn the relations underlying the system. In the subsequent application phase they could use this knowledge to reach specified goal values for the system. Performance was measured both in terms of the number of correct relations ("structure score", a measure of knowledge acquisition in the learning phase) and how close subjects got to the goal values ("solution error", a measure of knowledge application). We examined the impact on two motivational factors ("interest in the task" and "confidence of success vs. fear of failure") on both performance measures and by what processes these influences may be mediated. A path-analysis showed a path in which a more systematic strategy leads to more knowledge in the learning phase, which in turn leads to better performance in the application phase. However, the motivational factor "confidence of success vs. fear of failure" had a direct impact and two indirect influences on knowledge application. The two indirect impacts on performance were mediated by strategy systematicity, and "effortless concentration" respectively.
Contents: Part I:Reading, Writing, and Arithmetic. K.E. Stanovich, A.E. Cunningham, Reading as Constrained Reasoning. M. Bryson, C. Bereiter, M. Scardamalia, E. Joram, Going Beyond the Problem as Given: Problem Solving in Expert and Novice Writers. S.M. Sokol, M. McCloskey, Cognitive Mechanisms in Calculation. Part II:Social Sciences. J.F. Voss, C.R. Wolfe, J.A. Lawrence, R.A. Engle, From Representation to Decision: An Analysis of Problem Solving in International Relations. R.K. Wagner, Managerial Problem Solving. J. Funke, Solving Complex Problems: Exploration and Control of Complex Social Systems. E. Amsel, R. Langer, L. Loutzenhiser, Do Lawyers Reason Differently from Psychologists? A Comparative Design for Studying Expertise. Part III:Natural Sciences. M. Hegarty, Knowledge and Processes in Mechanical Problem Solving. A. Lesgold, S. Lajoie, Complex Problem Solving in Electronics. D.S. Kay, Computer Interaction: Debugging the Problems. Part IV:Games. P.A. Frensch, R.J. Sternberg, Skill-Related Differences in Game Playing. Part V:Conclusions. E. Hunt, Some Comments on the Study of Complexity.