Dynamis review: An overview about applications of
the Dynamis approach in cognitive psychology
August 24, 2005
The Dynamis review has been designed to provide an overview about applica-
tions of the Dynamis approach in cognitive psychology. Since the development of
computer-based scenarios Dynamis, based on linear structural equation systems,
has become increasingly popular as a tool for analysing decision making and com-
plex problem solving. Beginning with the role of system size, connectivity, and
types of relations in a system (e. g, eigendynamics and side eﬀects) it has been
discussed how formal system characteristics may inﬂuence the process of complex
problem solving. In a second part, essential task demands, i. e. knowledge ac-
quisition and knowledge application, have been speciﬁed further with particular
respect to their interdependency. Supporting inﬂuences as to eﬀective presentation
of information and task demands have been suggested in this context: graphical
presentation, semantic embedding, structural diagrams and other tutorials, in-
structed strategies, speciﬁc goals, and hypotheses. Research on the impact of
problem solvers’ habitual intellectual and strategic abilities as well as motivation
completed the presentation of the extensive contexts in which Dynamis systems
have been applied and possibly will be applied to in future research.
Correspondence concerning this report should be addressed to Joachim Funke, Department of
Psychology, Universit¨at Heidelberg, Hauptstraße 47-51, D-69117 Heidelberg, Germany.
E-mail: Joachim.Funke@psychologie.uni-heidelberg.de (http://funke.uni-hd.de)
1 General introduction 3
2 The Dynamis approach 3
3 Psychological research based on Dynamis 5
3.1 Systemcharacteristics ............................ 6
3.1.1 The number of variables . . . . . . . . . . . . . . . . . . . . . . . 6
3.1.2 The number of relations . . . . . . . . . . . . . . . . . . . . . . . 9
3.1.3 The quality of relations . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 Taskcharacteristics.............................. 15
3.2.1 Format of presentation . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2.2 Taskdemands............................. 20
3.3 Personcharacteristics............................. 36
3.3.1 Strategies and systematic learning . . . . . . . . . . . . . . . . . . 37
3.3.2 Motivation............................... 39
3.3.3 Intelligence .............................. 44
1 General introduction
The use of computer-simulated scenarios in problem solving research has become in-
creasingly popular during the last 25 years (for a representative collection of papers see,
e. g., the two editions from Sternberg & Frensch, 1991, and Frensch & Funke, 1995).
This new approach to problem solving seems attractive for several reasons. In con-
trast to static problems, computer-simulated scenarios provide the unique opportunity
to study human problem solving and decision making behaviour when the task envi-
ronment changes concurrently to subjects’ actions. Subjects can manipulate a speciﬁc
scenario via a number of input variables (typically ranging from 2 to 20, in some ex-
ceptional instances even up to 2000), and they observe the system’s state changes in a
number of output variables. In exploring and/or controlling a system, subjects have to
continuously acquire and use knowledge about the internal structure of the system.
Research on dynamic systems was motivated partly because traditional IQ tests turned
out to be weak predictors in non-academic environments (see Rigas & Brehmer, 1999, p.
45). According to their proponents, computer-simulated “microworlds” seem to possess
what is called “ecological validity”. Simulations of (simpliﬁed) industrial production
(e. g., Moray, Lootsteen, & Pajak, 1986), medical systems (e. g., Gardner & Berry, 1995),
or political processes (e. g., D¨orner, 1987) have the appeal of bringing “real world tasks”
to the laboratory. Brehmer and D¨orner (1993) argue that these scenarios escape both
the narrow straits of the laboratory and the deep blue sea of the ﬁeld study because
scenarios would allow for a high degree of ﬁdelity with respect to reality and at the same
time allow for systematic control of inﬂuential factors.
2 The Dynamis approach
In everyday life, a number of activities require the regulation and control of processes
which consist of quantitative variables (e.g., driving a car, controlling a CAD-machine).
Not only technical but also economic and ecological situations require that we ﬁrst have
to understand the system before goal-oriented action is possible. In many sciences,
systems with quantitative variables are represented successfully by means of the general
linear model (cf. Stevens, 1992). The use of linear structural equation systems as a
tool for problem solving research has been introduced by Funke (1985) under the name
of Dynamis (which was the name of the ﬁrst software shell for realising this type of
How can such a linear Dynamis model be used as a tool for analysing decision making
and problem solving? The subject is instructed that she or he has to deal with a system
that consists of some exogenous and endogenous variables. The exogenous ones can be
directly manipulated by the subject and, thus, can inﬂuence the endogenous variables
which can not be manipulated directly. The general task is (a) to ﬁnd out how the
exo- and endogenous variables are related to each other, and (b) to control the variables
in the system so that they reach certain goal values. Normally, these two subtasks of
system identiﬁcation and system control are separated experimentally as two steps of
the whole task (see Funke, 1993).
The basic structure of a simple linear Dynamis system, for example, consisting of four
Figure 1: Structure of the simple linear Dynamis system with two input variables A and B as
well as two output variables Y and Z. Variables are represented as boxes, connections between
them are marked by weighted arrows.
variables is shown in Figure 1 (adopted from Vollmeyer & Funke, 1999). Instead of
labelling the variables semantically, abstract letters are used. A system which contains
semantics from biology can be found in Vollmeyer, Burns, and Holyoak (1996).
In the example system from Figure 1, the variables A and B represent the exogenous
variables which have an eﬀect on the endogenous variables Y and Z. The numbers on
the arrows represent the weight with which the respective exogenous variables aﬀect
the endogenous ones. The system is described formally by two equations (one for each
Yt+1 = 2 ∗At(1)
Zt+1 = 3 ∗At−2∗Bt+ 0.5∗Yt+ 0.9∗Zt(2)
In these equations, the indices tand t+ 1 represent the actual state of the system which
itself goes on in discrete steps (= periods) on the time axis. From equation (1) it turns
out that the value of Y at period t+ 1 can be calculated from the value of A at period
t, times two. Similar, in equation (2) the value of Z at t+ 1 can be calculated from the
exogenous variables A and B at period t(with weight 3 and −2), from the value of Y
at that time (weight 0.5), and from its own value at period ttimes 0.9. Normally, such
a system is presented on a screen where all the variables are shown together with the
system’s history (for a certain period of time). What is not shown to the subjects is the
structure of the system because it has to be discovered by them during the exploration
In some systems, the endogenous variables have eﬀects on other endogenous ones (in
Figure 1, the eﬀect from Y to Z), an eﬀect which one might label as “indirect eﬀect”which
shows up only in case of manipulating the exogenous variable A. This variable A has
itself two eﬀects, one being larger (“main eﬀect” on Z), one being smaller (“side eﬀect”
on Y). Also, endogenous variables can inﬂuence themselves (in the example shown with
variable Z), thus representing an eﬀect one might call “eigendynamic” because of the
constant increase or decrease of this variable independent of other inﬂuences. As the
reader might imagine, there are many possibilities to construct linear systems with a
full range of eﬀects of the kind described above and, thus, making identiﬁcation and
control of such Dynamis systems to a hard problem. The two main task demands are
knowledge acquisition and knowledge application.
Task Demand 1: Knowledge Acquisition. The term “knowledge acquisition” (sys-
tem identiﬁcation) describes a complex learning situation during which the subject has
to ﬁnd out details about the connectivity of the variables and their dynamics. The
structural aspects of the system (= connectivity) cannot easily be separated from the
dynamic aspects because the system itself can only be analysed interactively over the
In the Dynamis situation, this identiﬁcation problem requires an identiﬁcation strategy,
that is, a certain way of manipulating the exogenous variables so that you can derive from
the consequences (in terms of values of the endogenous variables) the causal structure
of the system or at least to come to hypotheses about this structure which could be
tested subsequently. Identiﬁcation of system relation can occur in diﬀerent levels: (a)
as identiﬁcation of the existence or non-existence of a relation, (b) as speciﬁcation of a
direction, (c) as speciﬁcation of qualitative aspects of this (either positive or negative)
relation, and (d) ﬁnally as the exact quantitative speciﬁcation of the weight of this
Task Demand 2: Knowledge Application The term “knowledge application” (sys-
tem control) describes the situation of applying previously acquired knowledge in order
to reach a certain goal state within the system. The goal speciﬁcations are normally
given by the experimenter.
In the Dynamis situation, knowledge application requires two subgoals: ﬁrst, to trans-
form a given state of the endogenous variables by means of an input vector into the vector
of goal values, and second, to keep this goal state on a stable level because in a dynamic
system the goal state – once reached – may disappear quickly due to “eigendynamics”.
3 Psychological research based on Dynamis
The main aim of the present report is to provide a summary of various applications
and research contexts to which computer scenarios of the Dynamis family have already
been applied or possibly could be of use in the future. Central results and interesting
methodology especially of recent Dynamis studies are presented to give an introduction
to the most relevant ﬁndings in the extensive domain of Dynamis research.
The overview consists of three parts with subchapters: Part one deals with basic terms
and formal characteristics of Dynamis systems, i. e. the Dynamis algorithm. Experi-
mental studies examining the eﬀects these characteristics have on the process of complex
problem solving (CPS) will be discussed. Part two presents ﬁndings and theories con-
cerning task characteristics some of which may be implemented in Dynamis, other of
which refer to the general experimental setting. The third and ﬁnal part focuses on
the person interacting with the Dynamis task. Findings on some selected inﬂuences of
personality such as motivation and intelligence will be discussed in the context of CPS.
3.1 System characteristics
As noted above, system characteristics in Dynamis are integrated in the formal algo-
rithm underlying a problem solving scenario and can therefore be termed the “core” of
any Dynamis program. A very close connection exists between these formal charac-
teristics and the complexity of a system. Asked to imagine a complex system, in the
ﬁrst place one would probably think of a large number of diﬀerent variables, an even
larger number of relations between these variables, the interconnections being hardly
detectable and not at all transparent. The more possibilities to interact with a system,
the more diﬃcult it will be handled, one might expect. Yet experimental ﬁndings about
such features are a bit more intricate than just conﬁrming the basic notion that high
complexity means low performance.
Three aspects of system complexity will be considered in the following sections: the
number of variables in a system, the number of relations and the quality of relations
including special types of relation such as parallel or side eﬀects, eigendynamics and
3.1.1 The number of variables
The most obvious feature of a CPS scenario possibly is the number of the system’s
variables. This will also be referred to as the system’s size. Numbers of variables range
from as few as two variables in the smallest systems to more than 2000 variables in the
largest scenarios such as the well-known Lohhausen scenario developed by D¨orner et al.
(for a brief survey of CPS scenarios in psychological research see Funke, 1992b, p. 8-9).
CPS scenarios intended to be more “naturalistic”, i. e. closer to problems in real life,
usually employ larger numbers of variables whereas in Dynamis research, less variables
are common. Most of the studies discussed in the present paper applied Dynamis
systems containing six, seven or eight variables.
Systematic attempts to ﬁnd out how problem solving is aﬀected by diﬀerent sizes of
Dynamis scenarios have been undertaken by Funke (1992a, 1992b). In a study based
on the data of 40 participants (students at the University of Bonn) Funke compared
problem solving under two conditions: a “low task diﬃculty” scenario with only two
goal variables to be controlled versus a “high task diﬃculty” scenario with four goal
variables to be controlled. The system presented to subjects of the two experimental
groups was, in fact, the same, but the task was diﬀerent: In the “low complexity task”
two out of four goal variables simply had not to be considered as goal variables.
The complete scenario consisted of eight variables, four exogenous, four endogenous
variables. The employed system called Alt¨
ol (used oil) semantically refers to pollution
by used oil including exogenous variables such as “control procedures” or “price of crude
oil” and endogenous variables such as “maritime pollution” and “air pollution” (Funke,
1992b, S. 123).
Further experimental variation complemented the design: It was tested for eﬀects of
prior knowledge, eﬀects of diﬀerent degrees of controllability and eﬀects of presentation
format, i. e. whether input and output information were displayed as either numbers
(numerical type of presentation) or in charts (graphical type of presentation).1
With respect to dependent variables two types of measures appear relevant in this
context, claiming a brief explanation: the concept of control performance and the quality
Control performance provides a measure of how well problem solvers are able to reach
and maintain the goal states they aim at. At least since the 1990s, achievements in
control performance are usually computed as the mean deviations of the output values
actually observed from their corresponding goal values.
Quality of prediction is a less frequently employed diagnostic tool which, in Dynamis
research papers, has ﬁrst been mentioned by Funke and M¨uller (1988). It is a measure of
how well participants are able to predict future states of a CPS system, once the subjects
know the current state and following interventions. Quality of prediction is computed
in a similar manner as control performance: by the mean deviation of predicted values
from the values that would – according to the system’s underlying algorithm – actually
have followed the current status if the intervention had taken place.
Exactly as expected, Funke’s results showed a signiﬁcantly lower control performance
for the group dealing with the “high task diﬃculty” scenario. Obviously, it was easier
to control only two rather than four variables. However, this ﬁnding was conﬁned
to problem solving under the condition of a numerical type of presentation. When
input and output values were displayed graphically, the number of goal variables had no
eﬀect on control performance. The author only brieﬂy points at a possible explanation:
Perhaps only for numerical presentation, participants had the chance to analyse changes
in states of input and output variables thoroughly enough so that diﬀerent levels of
control performance could occur.
Secondly and not expected, but in line with the ﬁrst result, it was found that subjects
predicted future system states signiﬁcantly better when task diﬃculty was low (two goal
variables) than when it was high (four goal variables). On the whole, the experiment
supports the assumption that on several dimensions smaller systems, i. e. systems con-
taining fewer variables, are easier to deal with than larger systems with more variables.
This general interpretation has yet given rise to more speciﬁc research. An experiment
conducted by Preußler (1997) suggests that in addition to a system’s mere number
of variables, the variables’ quality may be just as important or even more important.
Preußler’s experiment showed that under certain conditions subjects controlling a dy-
namic system in fact perform better if the system contains more variables.
60 students at the University of Bayreuth took part in the study. Their task was to
control a Dynamis system named Linas (linear additive system). As in a typical
CPS experimental paradigm relations and outcomes of interventions were unknown a
priori. Since Linas variables have fancy names (e. g, Bulmin, Ordal, Trimol) no prior
semantic knowledge can be expected to play a role in this task. Two diﬀerent versions
of Linas were presented to the subjects: Half of the students dealt with a simpliﬁed
version which involved a set of four input variables and three output variables (control
group). The other half (experimental group) learned to control a full version of Linas.
1These further experimental factors, however, will be discussed below in more detail.
Four additional variables were included, but – though participants were not informed
about this fact – the extra variables were irrelevant to system control. Subjects in both
conditions controlled Linas for eight rounds with eight trials each. After that, they
accomplished a decision task which was intended to measure the structural knowledge
subjects had found out about Linas. Pairs of variable names were presented, and
participants had to decide whether there had been a relation between these two variables
The major results concentrate on both control performance and the amount of knowledge
acquired under the two conditions: Control performance, at least at the end of round
eight, was signiﬁcantly better in the experimental group as compared to the control
group. Yet with respect to the acquisition of knowledge, the pattern was reversed, i. e.
subjects controlling the simpliﬁed version outperformed those controlling the extended
system; they had found out more about the relations within Linas.
What might explain these ﬁndings? To Preußler, the results are exactly as predicted.
Larger systems with irrelevant additional variables provide redundant information which
requires cognitive capacity and hence means extra eﬀort to subjects. That is why, ac-
cording to Preußler, the experimental group did not show any superior performance in
the initial rounds. At the end of the task, however, the beneﬁts of redundancy outweigh
its advantages. The larger system provides a “multiple learning context”, learning con-
ditions which could have allowed participants to try diﬀerent approaches to the same
problem. Just as with transfer tasks in traditional learning psychology, multiple contexts
facilitate the control of the dynamic system. Preußler concludes that “learning contexts
requiring additional cognitive resources for problem solving can have diﬀerential eﬀects
on the acquisition of structural knowledge and the acquisition of control knowledge” (p.
48). Thus, cognitive load due to redundant information obviously can enhance control
performance and at the same time impair the acquisition of structural knowledge.
Another complementing contribution as to the impact of the number of variables in
Dynamis has been proposed by Strauß (1993) in experimental studies and theoretical
reﬂections. Strauß argues that in problem solving scenarios the number of variables
per se is less decisive as to controlling and gaining knowledge than the question of how
easily performance goals could be accomplished even if subjects were acting totally at
random. In formal terms: Compared to all possible solutions, which is the amount of
interventions leading to a certain goal state within a deﬁned period of time? Strauß has
termed this ratio the “share of correct solutions” (Strauß, 1993, p. 60). Using vector
algebra the share of correct solutions can either be computed exactly from the full set
of possible solutions or it can be estimated from samples.
In experimental studies Strauß has shown that performance in CPS scenarios can indeed
be aﬀected by this formal system characteristic. He compared subjects dealing with
problem solving systems that were identical but for the share of correct solutions. In
line with Strauß’s assumptions, higher shares of correct solutions were associated with
higher control performance whereas lower shares resulted in lower performance.
A second experiment revealed that even if two treatment conditions diﬀered in the
number of variables included, the share of correct solutions kept constant, regarding
control performance no marked diﬀerences between the groups were found. Accordingly,
for successful controlling of a dynamic system the system’s size hardly matters as long
as it is easy enough to reach deﬁned goal states when merely guessing.
Strauß pleads for the share of correct solutions to be treated as a system’s formal feature
which supplements the number of variables. Neither the number of variables nor the
share of correct solution considered on its own is convincing; instead, both concepts seem
confounded with one another. For full report, it should yet be mentioned that Strauß’s
ﬁndings proved to be valid for control performance, but that to CPS processes there
are more determining factors than just this. With respect to the amount of knowledge
acquired the share of correct solutions had no such eﬀects in Strauß’s experiments.
3.1.2 The number of relations
A second factor which determines the formal complexity of a system is the number of
relations between variables. Even large systems with many variables may appear rather
simple if there are only few interconnections between these variables. On the other hand,
comparatively small systems can be constructed, involving, say, as few as six variables
and yet being most intricate to solve since almost all variables aﬀect one another in an
unknown way. In research literature, this concept has been named the connectivity (“
Vernetztheit”) of a complex system.
Eﬀects of connectivity as an experimental treatment factor have been investigated as
early as 1985 by Funke (1985). The study involved a Dynamis CPS system named
Okosystem (ecosystem) whose cover story refers to gardening in an ecological context
(the variables being labelled as “poison”, “varmints”, “fertiliser”, “beetles”, “water
pollution”, “number of leaves”). In ¨
Okosystem there are three exogenous and three
endogenous variables, i.e. a total number of six variables. For experimental study, the
number of relations was manipulated to provide three diﬀerent conditions: four relations
in the condition of low connectivity, six relations for medium connectivity, eight relations
for high connectivity. A further experimental condition concerning the presentation
format resulted in a design with six groups, ten subjects per group. Participants task
was to control goal parameters in a ﬁctive ecosystem for ﬁve rounds with seven trials
In addition to control performance Funke measured the amount of knowledge subjects
gained while exploring ¨
Okosystem. Unlike in Preußler’s study employing the “pair-
task”, Funke used the method of causal diagrams. In this paradigm, problem solvers
are asked to formally express their ﬁndings about causal relationships with the help of
arrows and symbols in diagrams. They draw arrows from one rectangle displaying a vari-
able’s name to another once they know that any relation exists between these variables
(knowledge of relation). If they have found out that increments in one variable increase
the outcome of another variable or that by increments in one variable another variable
will decrease they add a plus or minus sign, respectively (knowledge of direction). In
case they even know the exact numerical factor underlying a relation they put down a
number (knowledge of strength). Hence, knowledge can be analysed on three increasing
levels of accuracy. For each type of knowledge the quality of system identiﬁcation is
calculated from a formula involving the number of correctly identiﬁed items relative to
the maximum detectable number of items and the number of mistakes.
The experiment supports the author’s expectation that higher connectivity yields both
lower control performance and less causal knowledge than low connectivity conditions.
Signiﬁcant diﬀerences between the connectivity conditions were found for all three grad-
uations of knowledge with knowledge of relation being most extensive, knowledge of
direction having an intermediate state and knowledge of strength being relatively sparse
in all three experimental groups.2More recently, Kluge (2003) reported comparable
eﬀects as to the impact of connectivity on CPS.
The interpretation that connectivity in a problem solving scenario can serve as a measure
of task complexity appears to be obvious. It was Strauß (1993) again who pointed
at a further explanation. Re-analysing Funke’s experiment Strauß demonstrated that
connectivity, the number of relations in ¨
Okosystem, has been confounded with the
share of correct solutions as deﬁned above: the less relations, the larger the share of
correct solutions, the more relations, the smaller the share of correct solutions. As Strauß
illustrates, controlling a system with only four relations between variables would be more
successful than controlling an eight-relations-system even for random interactions with
the system. These limitations may be taken into account even though one does not wish
to fully abandon the ﬁrst straightforward interpretation.
3.1.3 The quality of relations
The concept of connectivity can be speciﬁed further according to the types of relations
between variables. For example, it may occur that input variables aﬀect more than but
one output variable each. In turn, output or endogenous variables can depend on a
set of several input or exogenous variables. There may be parallel eﬀects. Just as well
it is possible that endogenous variables show inﬂuence on other endogenous variables,
indirect eﬀects which have been referred to as side eﬀects. A third possibility concerning
the quality of relations is the eﬀect of an endogenous variable on itself. Challenged by
problems in real life where systems’ states often change without active interventions,
without noticeable inﬂuences from known outside causes, in complex problem research,
too, eigendynamics have become an established concept and experimental paradigm. A
formal deﬁnition of eigendynamic (“ Eigendynamik”) is provided by Funke: “Eigendy-
namik means that an endogenous variable at time thas an eﬀect on its own state at
time t+ 1 independent of exogenous inﬂuences which might add to the eﬀect” (Funke,
1993, p. 322).
Parallel eﬀects. Parallel eﬀects have been studied with regard to two questions:
Given the fact that in a dynamic system one input variable aﬀects more than one output
variable, what is the impact of diﬀerent numerical weights shaping the relations between
input and output variables? Will the strongest relation, i. e. the relation formed by the
largest numerical weight, be detected and controlled more easily than other relations?
Secondly one may ask under which conditions a system will be identiﬁed and controlled
more successfully – if there is one input variable with parallel eﬀects on several output
2The ﬁnding reported here applies to experimental conditions in which interactions with the system
had immediate eﬀects on the outcome states of the next trials (immediate feedback). In systems with
so-called delayed feedback eﬀects of connectivity on knowledge and performance were not quite as
distinct. The inﬂuence of feedback delays will be described in a separate section at the end of part one.
variables or if there are several input variables that add to the outcome state of a single
The impact of parallel eﬀects of diﬀerent strength has been studied by Funke (1992b).
As part of a larger series of studies examining side eﬀects and eigendynamics, on the base
of causal diagrams drawn by forty-eight subjects Funke analysed how well participants
detected relations in a dynamic system called Sinus. The two relevant relations to be
compared were an input-output-relation with a large numerical factor of 3 (dominant
eﬀect) and an input-output-relation with a much smaller numerical factor of only 0.5
(subdominant eﬀect). The input variable was the same to both relations, the two output
variables were diﬀerent.
Frequency analyses were conducted in order to know in which category most of cases
would be found: subjects who noticed the dominant eﬀect at ﬁrst and then the sub-
dominant eﬀect, subjects who noticed the subdominant eﬀect earlier than the dominant
eﬀect, subjects who noticed both eﬀects at the same time or subjects who noticed neither
the dominant nor the subdominant relation. Since the majority of participants identiﬁed
the dominant relation earlier than the subdominant relation, it is obviously easier to
detect relations with larger impact on other variables although we do not know to what
extent identiﬁcation of a system is facilitated by increasing dominance. Neither there
have been ﬁndings relating dominance or subdominance to successful or less successful
control performance so far.
Another experiment described in the same publication (Funke, 1992b) dealt with the
second question risen above, the impact diﬀerent patterns of parallel eﬀects have on
solving a complex problem. In this context a formal concept from cybernetics (Ashby,
1958) has been introduced into the domain of CPS: the degree of controllability. Con-
trollability in technical terms means the ratio of two numbers of variable classes: the
number of variables that inﬂuence or control other variables divided by the number of
variables that are inﬂuenced or controlled. In Dynamis scenarios a system’s controlla-
bility is thus determined by the numbers of exogenous and endogenous variables. The
more exogenous variables as compared to endogenous variables, the more easily the sys-
tem should be controlled. Funke’s idea was to ﬁgure out whether this theoretical and
rather technical notion ﬁts with control performance shown by human problem solvers.
The design of the study has already been described in a previous section when dealing
with system complexity due to the number of variables. Besides the number of vari-
ables and two further experimental variations concerning the Dynamis scenario Alt¨
Funke’s experiment comprised three diﬀerent degrees of controllability as a within sub-
ject factor. Within the whole of eight variables three independent subsystems could be
separated: One exogenous variable aﬀecting two endogenous variables (1:2 relation or
low controllability), one exogenous variable aﬀecting one endogenous variable (1:1 rela-
tion or medium controllability) and two exogenous variables aﬀecting one endogenous
variable (2:1 relation or high controllability). For statistical analysis data of 40 students
The results showed eﬀects contrary to the expectations. In fact, signiﬁcant diﬀerences as
to control performance for the three subsets occurred, but performance was best when
the ratio of exogenous and endogenous variables was 1:2, i. e. for the subsystem which
had been named the low controllability condition. In turn, it was lowest for the reverse
relation, which should have facilitated control performance according to the author’s
hypothesis. Medium eﬀects were found for the 1:1 relation. In this condition high
performance resulted if the variables’ states had been presented in numbers whereas for
graphical display performance was just as low as in the so-called high controllability
Reﬂecting on these ﬁndings Funke has proposed methodological problems as alternative
explanations, i. e. lacking independence of the subsystems from semantic features. So it
remains unclear whether the psychological notion of controllability corresponds to the
technical concept or not.
Side eﬀects. Side eﬀects in the context of CPS, too, have been studied by Funke
(1992a, 1992b, 1993). For the study on side eﬀects a Dynamis system named Sinus
was chosen. Sinus comprises a total number of six variables, three endogenous variables,
three exogenous variables, these being linked by four basic relations. In a standard ver-
sion, two additional relations are implemented, one an instance of eigendynamics, the
other a side eﬀect of one endogenous variable on another. When dealing with Sinus par-
ticipants are asked to control population numbers of six diﬀerent tribes on a ﬁctive planet
in a solar system diﬀerent from ours. The tribes are named “Gaseln”, “Schmorken” and
“Sisen”, the endogenous variables or population numbers that have to be controlled,
and “Olschen”, “Mukern”, “Raskeln”, the exogenous variables or population numbers
that can be manipulated by direct interventions. Relations between the six tribes on
the ﬁctive planet have to be explored and controlled for ﬁve rounds consisting of seven
trials or simulated weeks. According to the standard procedure as employed in Funke’s
experiment rounds one to four are intended to serve knowledge acquisition due to the
exploration of Sinus. Speciﬁc goal values to be aimed at are only given as late as round
ﬁve when the main task demand is applying knowledge gained in earlier rounds.
In order to examine the impact of side eﬀects on knowledge and performance Funke mod-
iﬁed the system structure so that three increasing degrees of side eﬀects were realised:
One experimental condition without any side eﬀect at all, a second condition imple-
menting one small side eﬀect with a numerical magnitude of 0.2 and a third condition
implementing two side eﬀects of 0.2 and 0.5.
Statistical analysis was based on scores of control performance and causal diagrams from
24 students, eight subjects per group. It was found that increasing side eﬀects led to a
linear decrease in the amount and quality of knowledge just as assumed prior to testing.
The same eﬀect occurred as to control performance which also decreased for systems with
growing numbers of side eﬀects. This fact has been interpreted as a logical consequence
of limited knowledge acquisition associated with the side eﬀects. Interestingly, as more
detailed analysis revealed, in identifying the system’s formal structure, subjects tended
to compensate for mistakes in a rather systematic way: In case side eﬀects had not been
detected participants typically assumed additional relations between variables which, in
fact, had not been manipulated at all.
Eigendynamics. Concerning the eﬀects of eigendynamics an experiment analogous to
the study on side eﬀects was conducted (Funke 1992a, Funke 1992b, Funke 1993). Again
24 students worked on the dynamic system Sinus, following the procedure described
above. This time diﬀerent degrees of eigendynamics were implemented in Sinus (while
the number of side eﬀects was kept constant): One version without any eigendynamics
was provided, another version including one instance of eigendynamic with a magnitude
of 0.9 and a third version including two instances of eigendynamic, the magnitudes being
0.9 and 1.1.
The magnitude of eigendynamics represents a numerical factor by which a variable’s
outcome value is multiplied from trial to trial. In case no eigendynamic contributes
to the outcome state, this factor is set to the value of 1 (which is the default value in
Dynamis). Accordingly, eigendynamics deﬁned by the factor 0.9 lead to a ten percent
decrease of the former state in each following trial. Speaking in terms of the Sinus cover
story: Sisen, the Sinus tribe associated with eigendynamics of 0.9, will lose ten percent
of its population each week and gradually die out unless appropriate interactions are
undertaken. The reverse eﬀect on Gasen, the Sinus tribe associated with eigendynamics
of 1.1 will lead to a ten percent increase in population number if no counteracting
measures take place.
Just like for growing degrees of side eﬀects the author assumed that increasing levels
of eigendynamic, too, would deteriorate both the quality of control performance and
the quality of system identiﬁcation, i. e. the amount and type of knowledge. The
expectation was conﬁrmed as to the quality of control performance since the three
groups signiﬁcantly diﬀered in the predicted manner. The condition containing two
instances of eigendynamic turned out to be especially hard to control. There was,
however, no corresponding eﬀect on the quality of identiﬁcation. According to Funke,
“causal dependencies were detected equally well under all three conditions” (Funke,
1993, p. 323).3He concludes that acquisition of knowledge and its application in the
control task are diﬀerent tasks requiring diﬀerent abilities. Dissociations may therefore
Perhaps still more interesting than considering side eﬀects and eigendynamics each on
their own it is to compare and combine the ﬁndings. The above results suggest that
both side eﬀects and eigendynamics, adding complexity to a problem solving task, make
scenarios such as Sinus more diﬃcult to handle. While successful identiﬁcation of the
system’s structure obviously remains unaﬀected by increasing levels of eigendynamics,
side eﬀects impair the gaining of knowledge as well.
In order to corroborate the idea that in CPS systems side eﬀects are harder to detect than
eigendynamics, Funke applied combined frequency analyses on comparable data from
three experiments of equal structure (the two experiments described above plus a third
analogous study). Causal diagrams were selected from 48 participants who had worked
on a Sinus version including one instance of eigendynamics and one side eﬀect. The
subjects were classiﬁed according to four exclusive categories: subjects who detected
eigendynamics prior to the side eﬀect, subjects who detected the side eﬀect prior to
eigendynamics, subjects who detected both eigendynamics and the side eﬀect in the
same round and subjects who detected neither of these relations. The ﬁrst category was
considered as in line with the expectations, the second category directly opposed it, the
3In an earlier publication Funke (1992b) notes that older measures of control performance and
knowledge yielded eﬀects for quality of identiﬁcation as well. Obviously, however, the more recent
measures should be preferred.
other two categories were deemed neutral or irrelevant to the expectations.
From a statistically signiﬁcant ratio of 20 cases supporting the hypothesis and only ten
cases opposing it, it was concluded that in Sinus eigendynamics indeed were easier
to identify than side eﬀects. A closer look at typical faults in the identiﬁcation of the
system revealed that often side eﬀects which had remained unnoticed had been mistaken
for additional eigendynamics leading to about the same outcome values.
Why could eigendynamics in Sinus be detected with relatively little eﬀort? As Funke
(1993, p. 323) points out, there is a rather simple strategy to notice eigendynamics in
a CPS scenario: leaving the system to itself without any active interventions for one
or more trials. This strategy can be applied in a very appropriate manner especially
during the ﬁrst four rounds of the experiment, i. e. when no speciﬁc goal values are pro-
vided. Under these conditions free exploration is both possible and desired. Unfortunate
outcomes of interventions or omitted interventions do not matter as long as relations
within the system are well identiﬁed. Certainly, in real life problems many people would
rather refrain from strategies named “just wait and see”. That is why from our studies
we cannot necessarily conclude that eigendynamics are less relevant to problem solving
than, e. g, side eﬀects or parallel eﬀects.
Feedback delays. Another feature ﬁnally appears to be typical of complex problems
in the outside world beyond science laboratories. In reality, interventions rarely yield
eﬀects that can be noticed immediately after an operation has been carried out. Some
eﬀects may occur at once, others only occur after some time if at all.
In Dynamis eﬀects in the long term have been modelled by means of feedback delays.
In the Dynamis systems presented so far changes in a variable at a deﬁned time thad
an impact on outcome states of the trial following immediately, i. e. time t+ 1. This case
shall be termed immediate feedback. Yet changes at tdo not necessarily have to aﬀect
outcomes in t+ 1; instead they can “skip” one trial and aﬀect, say, outcomes in t+2 or
in times even further in the future. In this case the system’s reaction to an intervention
The already mentioned study by Funke (1985) employing ¨
Okosystem was conducted
to examine the role of feedback delays in CPS besides the impact of connectivity (see
above). Including three diﬀerent levels of complexity due to connectivity two diﬀerent
types of systems were constructed, one which contained immediate feedback exclusively
and another in which part of immediate feedback relations were replaced by correspond-
ing relations with delayed feedback. Half of the 60 participants were assigned to work
on each system.
It was expected that feedback delays should be more diﬃcult to identify and to control
than immediate feedback. Indeed, the results showed more and better knowledge gained
by participants who had been given immediate feedback instead of delayed feedback.
Signiﬁcant eﬀects appeared for three distinct types of knowledge: knowledge of rela-
tion, knowledge of direction, and knowledge of strength. With regard to interactions
between feedback delay and the second experimental manipulation, the complexity due
to connectivity, the feedback manipulation seemed to give rise to diﬀerential eﬀects of
connectivity. While for subjects solely confronted with immediate feedback increasing
connectivity consistently resulted in decreasing accuracy of knowledge, in the condition
of delayed feedback it was only the medium connectivity group who identiﬁed less rela-
tions than the low connectivity group; participants in the condition of high connectivity
identiﬁed the system almost as well as the low connectivity group. Funke has attributed
this unexpected ﬁnding to enhanced eﬀort and extra time spent on the task by subjects
dealing with both delayed feedback and high connectivity. Eﬀects of feedback delays
on control performance, unfortunately, turned out to be unsuitable to statistical test-
ing. Since in recent Dynamis studies feedback delays have hardly been examined we
can only guess that the immediate feedback should rather facilitate control performance
than systems including feedback delays.
3.2 Task characteristics
Occasionally recurring to the basics of part one, in the following sections we will consider
broader applications of CPS scenarios. Leaving the mere formal attributes of Dynamis
systems we will shift the focus to the interaction between the person, i. e. the problem
solver, and the system.
How should information be presented in order to facilitate working on a complex system?
What are relevant and appropriate task demands providing an insight into the process
of CPS, and how are diﬀerential demands – such as the acquisition and application of
knowledge – interrelated? Finally, which methods can be employed in terms of training
or task instructions to support successful interactions with the system? These may be
the leading questions to the section about task characteristics.
3.2.1 Format of presentation
The ﬁrst question risen above was: How should information be presented in order to
facilitate working on a complex system? Although it seems nearly impossible to ﬁnd a
deﬁnite answer applying to all instances of CPS we will have a closer look at the impact
two prominent task characteristics have on diﬀerent facets and demands in problem
solving: Numerical vs. graphical format of presentation and semantic embedding.
Numerical versus graphical format of presentation. In the sections on system
characteristics it has been brieﬂy pointed at the fact that in Dynamis scenarios infor-
mation as to the state of input and output variables can be displayed either in numbers
or in diagrams, which implies two diﬀerent modes of the user’s interface. All of the
studies discussed above – unless it has been noted explicitly – dealt with systems of the
numerical format. According to Beckmann (1994, p. 65) this mode of presentation is
far more common than graphical interfaces. Recent scenarios of the numerical mode
show variables’ states as numerical values in tables. For interventions numbers have to
be entered by using the keybord.
In his own studies Beckmann employed two Dynamis systems of the graphical type,
Kirschbaum (cherry tree) an Maschine (machine).4In these scenarios exogenous
variables can be manipulated by means of three diﬀerent mouse buttons on a control
4In fact, the systems are identical in their formal structure, but diﬀerent in semantic embedding.
This will be considered more closely in the following section.
panel: one button for zero interventions, another button for increasing the value of
a chosen exogenous variable, the third button for decreasing the value of a chosen
exogenous variable. The relative size of the values can be inferred from the length
of a vertical bar: the longer the bar, the higher the corresponding numerical value.
Positive or negative values are symbolised by the bars position above or below a “zero
intervention line”, respectively. Line diagrams, on the other hand, present the course of
previous states and actual states relative to top and bottom limitations and relative to
goal values marked by a straight horizontal line.
Another descriptive example of graphical features in Dynamis is given by the dynamic
system MultiFlux (Kr¨oner, 2001). MultiFlux has been developed as the simulation
of a ﬁctive machine. The machine consists of four regulators or control devices, i. e.
the exogenous variables, and four instruments depending on the regulators, i. e. the
endogenous variables. Variables are labelled control devices 1 to 4 and instruments
1 to 4. To problem solvers the task is to adjust regulators so that the instruments
will reach deﬁned goal adjustments. In doing so, they are confronted with a graphical
display optically similar to the control buttons in real technical instruments such as hi-ﬁ
While in Beckmann’s and Kr¨oner’s studies it has not been the primary aim to examine
impacts of graphical presentation on CPS, Funke (1992a, 1992b) conducted experimen-
tal comparisons of the two diﬀerent modes of presentation. In studies based on the
data of 80 students he analysed problem solving in a graphical vs. a numerical ver-
sion of the already mentioned eight variable system Alt¨
ol. Almost as in Beckmann’s
Kirschbaum and Maschine the graphical interface displayed the course of outcome
values in line diagrams. An additional horizontal line indicated the level of the corre-
sponding goal value. Type of presentation was one experimental factor among others
such as task diﬃculty, degree of controllability and a manipulation referring to the role
of prior knowledge.
Former studies dealing with scenarios other than Dynamis systems had suggested that
control performance would be facilitated if information were displayed in diagrams as
compared to numerical tables (see H¨ubner, 1987, 1988). Opposing that view, Funke
(1992a) hypothesised that subjects dealing with numerical interfaces would outperform
the graphical presentation group in control performance, but that the identiﬁcation
of the system would be better if graphical information were provided. Yet neither of
the these expectations could be supported (nor could be the opposite). Only when
additional factors such as task diﬃculty were taken into account signiﬁcant eﬀects of
interaction appeared: Provided that participants controlled a system with only two goal
variables (low task diﬃculty) better identiﬁcation of the system’s relations was found
for the numerical than for the graphical condition. In turn, if four goal variables had
to be controlled (high task diﬃculty), participants in the graphical condition identiﬁed
the system’s structure more easily. Funke concludes that “presentation format per se is
not a critical factor. However, it is obvious that depending on the nature of the task,
diﬀerential eﬀects occur: In order to cope with the more complex task the graphical
presentation which is less precise in presenting system information yields better results
(Funke, 1992a, p. 37)”.
The same notion is supported in another analysis by Funke (1992b). In this paper
the author reports a study which revealed no signiﬁcant diﬀerences as to the quality
of system control and the quality of system identiﬁcation. Interestingly, the format of
presentation seemed to aﬀect some motivational factors which had been recorded by
means of questionnaires for the purpose of experimental control. At the beginning of
the task subjects confronted with the numerical presentation reported higher “failure
motivation”, i. e. higher anxiety to fail on the task, and lower readiness to make an
eﬀort. Fortunately, however, at least in this context, impaired motivation had no strik-
ing detrimental eﬀects on the overall performance and identiﬁcation. Only for a third
diagnostic measure, the quality of prediction, numerical presentation as compared to the
graphical mode seemed to facilitate the task demands. However, since in both conditions
subjects had to predict future outcomes in numbers (instead of graphical predictions)
diﬀerential experience with the numerical mode might be regarded convincing enough
to oﬀer an alternative explanation.
Semantic embedding and the role of prior knowledge. Thinking of abilities
and skills relevant to CPS, personality factors such as intelligence and a “feeling” for
successful strategic approaches (compare D¨orner’s concept of operative intelligence, e. g,
D¨orner, 1986) probably come to our mind in the ﬁrst place. Yet just as well we might
think of practice and the knowledge a problem solver has gained prior to the task in con-
texts associated with the task demands. This, too, should aﬀect a person’s interaction
with a given dynamic system.
With regard to the role of prior knowledge the Dynamis scenarios discussed in the
above sections can be assigned to two classes. Scenarios such as Sinus (the ﬁctive
planet), Linas (fancy names) and Kr¨oner’s ﬁctive machine MultiFlux have been
constructed to be relatively neutral to prior knowledge. Although problem solvers may
be familiar with the concept of a machine, when confronted with MultiFlux, they will
not know whether and how control device number 1 is likely to aﬀect instrument A, for
instance. Systems of these properties will be referred to as systems without semantic
embedding. In contrast, semantically embedded Dynamis scenarios like Alt¨
Okosystem refer to rather familiar contexts. Dealing with ¨
Okosystem anyone with a
basic idea of gardening and ecology can make up his or her own expectations once he or
she has but realised the variables’ names: “poison”, “varmints”, “fertiliser”, “beetles”,
“water pollution”, “number of leaves”. Will poison lead to increasing water pollution?
Will fertiliser, brought into play, induce growth and increase the number of leaves?
Hypotheses such as these may be formed immediately unless there is no prior knowledge
In order to elucidate eﬀects of prior knowledge on CPS one common approach is to ex-
amine how “experts”, i. e. problem solvers with suﬃcient relevant prior knowledge, cope
with dynamic systems, as compared to “novices” without relevant prior knowledge (see,
e. g, Reither, 1981). This, however, does not concern Dynamis systems in particular
and will not be the subject of the present paper. A second approach which is more
closely related to Dynamis is found by comparing problem solving in semantically em-
bedded systems to problem solving in systems without semantic embedding. For valid
comparisons, of course, systems have to be identical in structure and user’s interface
but for the diﬀerent names of exogenous and endogenous variables.
This is exactly the essence of Beckmann’s (1994) research on semantically embedded sys-
tems. For experimental studies Beckmann employed the semantically embedded system
Kirschbaum (cherry tree) which contains three exogenous variables labelled “light”,
“water supply”, “warmth” and three exogenous variables named “cherries”, “leaves”,
“beetles”, hence appealing to general knowledge. According to the cover story, partic-
ipants explore and inﬂuence growth and growth conditions of a hardly known type of
cherry tree. In Maschine, the equivalent version without semantic embedding, exoge-
nous variables are represented as control devices, endogenous variables are represented
as instrumental displays. Preliminary examinations based on a sample of pupils made
Beckmann assume that semantic embedding as a system’s property had an eﬀect on
an identiﬁcation task, but seemingly not on control performance. In order to replicate
the quasi-experimental ﬁndings he realised a ﬁrst experiment which involved 40 partici-
pants, students at the University of Bonn. Half of the students explored the semantically
embedded system Kirschbaum for two rounds with seven intervention trials and after-
wards controlled the system for one round in order to accomplish deﬁned goal values.
The other half did the same task on Maschine.
As expected, subjects in the condition without semantic embedding identiﬁed the sys-
tem’s structure better than those dealing with the semantically embedded system. A
likely interpretation to this ﬁnding is that the semantic context in Kirschbaum triggers
participants’ prior knowledge and their assumptions about relations, which, in fact, are
incompatible with the relations speciﬁed in the Dynamis linear equations (hypothesis
of knowledge incompatibility). However, more astonishing, though also expected from
the preliminary analysis, control performance was about equal in the two groups; in
spite of less knowledge gained the subjects in the semantically embedded condition did
not control any worse.
If knowledge incompatibility is indeed a crucial determinant to diﬀerential achievements
in problem solving, why should control performance remain unaﬀected? Beckmann ar-
gues that even with regard to knowledge acquisition “explicit knowledge incompatibility”
oﬀers an at least incomplete explanation (see p. 172). Relations between variables in the
Kirschbaum scenario implied no serious incompatibilities to subject’s prior knowledge.
More noteworthy, a closer look at identiﬁcation patterns revealed that the relations sub-
jects detected well or not so well in the semantically embedded system coincided with
the equivalent relations in the scenario free of semantics.
Instead of knowledge incompatibility, Beckmann pleads for two processes of knowledge
acquisition which are diﬀerent in quality, not but in the quantity of resulting gained
knowledge. Concerning systems without semantic embedding a constructive process of
knowledge acquisition is proposed. Since, due to lacking semantics, no prior knowledge
is “activated” and no expectations are available, the default assumption about possible
relations between any two variables is that there is no relation. Only after interactions
with the system, an internal model about structural relations will be built up, succes-
sively extended and reﬁned. The reverse process should be the case for people dealing
with semantically embedded systems. According to Beckmann, semantic contexts as
in Kirschbaum give rise to the default assumption that each variable depends on all
other variables. The structural model of knowledge is formed and reﬁned in a reductive
process, gradually decomposing the model of general interconnection. As the reduc-
tive process takes more cognitive eﬀort less knowledge should be gained in semantically
embedded scenarios. Beckmann’s data obviously ﬁt well with the assumption of two
diﬀerent models of processes.
But what about control performance? What can explain the fact that control perfor-
mance did not seem impaired in a semantically embedded system? Beckmann’s idea
is that there are at least two diﬀerent ways of controlling dynamic systems, though
probably not equally useful. The ﬁrst process is control performance based on gained
knowledge. This is, of course, assumed to be the primary process guiding participants
who cope with Maschine and do well on both the identiﬁcation task and the control
task. But even for participants dealing with Kirschbaum episodes of good identiﬁca-
tion preceded or coincided with episodes of good control performance. Only if control
cannot – because of insuﬃcient knowledge – be based on knowledge a second process
named “ad hoc” control comes into play: basic rules or heuristics which do not apply to
a certain dynamic system in particular (“situationally unspeciﬁc heuristics”). By means
of “ad hoc” control subjects who controlled the semantically embedded version might
have compensated for limited structural knowledge.
To validate this notion, Beckmann designed a second experiment involving control con-
ditions which should only enable “ad hoc” control. Knowledge acquisition was said to
be prevented by leaving out the former two exploration phases and replacing them by
control phases with speciﬁc goal values to be accomplished. It was expected that as to
control performance no marked eﬀects of semantics should occur then. With regard to
scenarios and general procedure, the second experiment equalled the ﬁrst one. 51 school
students from Leipzig area participated.
Not fully consistent with the expectations, in experiment two results indicated some in-
ﬂuence of semantic embedding on control performance although not all eﬀects turned out
to be statistically signiﬁcant. For both experimental conditions, performance increased
in the course of the three rounds, yet for subjects in the Kirschbaum condition the
eﬀect was more prominent: The pupils working on Kirschbaum started with a lower
performance in the ﬁrst round as compared to the those working on Maschine, in the
second round the performance of the two groups was equal and in the ﬁnal third round
subjects of the semantically embedded condition even tended to control better.
Beckmann hence answers the second question as follows: Although control performance
with semantic contexts appears to be the same level as with contexts free of semantics
performance is indeed impaired. Consequences of the deteriorating eﬀect of semantic
embedding, however, are only obvious and determinant at the beginning of the process
of CPS. Early phases of exploration or additional rounds of control, fostering successful
“ad hoc” control will make the eﬀect vanish. As Beckmann illustrates impaired control
performance due to semantics, “it is obviously easier to understand and control a system
if ‘you know that you know nothing’ (Maschine) than if ‘you think you know something’
(Kirschbaum)” (Beckmann, 1994, p. 199). Further research on semantic embedding
and the role of intelligence in CPS is provided by Beckmann and Guthke (1995, see
Besides comparing experts vs. novices and experiments on semantic embedding there is
a third approach to study impacts of prior knowledge on problem solving. While in the
experimental designs described above systems with exactly equal structural relations,
but a diﬀerent “surface” have been employed, it is just as possible to compare two sys-
tems, both semantically embedded, with the same number and names of variables; these
will look the same in superﬁcial characteristics, but the underlying structure may be dif-
ferent. In research done by Funke (1992a, 1992b) two such systems, diﬀerent versions of
ol scenario were constructed. They were even identical in the general structure
but for the signs of parameters in linear equation models. One version, the matching
condition, was assumed to coincide with participants’ prior knowledge while the other
version, the mismatching condition which contained inverted signs of parameters, dis-
agreed with expected knowledge. It was statistically tested for eﬀects of knowledge
compatibility on the three measures quality of system control, quality of system identi-
ﬁcation and quality of prediction. Each of these variables turned out to be signiﬁcantly
impaired for the mismatching condition, the Alt¨
ol scenario containing counterintuitive
relations. The results, reﬂecting straightforward expectations, once again conﬁrm that
prior knowledge deﬁnitely requires consideration in cognitive research, even though in
most cases little is known about knowledge speciﬁc persons have about speciﬁc tasks
(compare Funke, 1992b, p. 139).
3.2.2 Task demands
As the introduction and preceding chapters have underlined already, there are at least
two essential demands to a person dealing with a complex dynamic system. Firstly,
he or she has to ﬁnd out how the system works, secondly, he or she is asked to reach
and maintain a certain goal state of the system. If psychological research wishes to
get a functional insight into processes of CPS, it needs to focus on both: on subjects’
performance in identifying the system, i. e. his or her acquiring knowledge, as well as on
subjects’ control performance, often termed as knowledge application. In this section we
will try to elucidate how acquisition and application of knowledge depend on each other.
Prior to that, however, let us brieﬂy recapitulate that in experimental practice several
indicators exist to record the type and amount of gained knowledge and the quality of
Measuring acquisition of knowledge. Regarding acquisition of knowledge, the far
most common experimental indicator is the structural score gained from the method of
causal diagrams in the paper and pencil format (see, e. g, Funke, 1985). Computer-based
modiﬁcations of this method exist in some rather exceptional cases. In studies, e. g, by
Beckmann (1994) or Schulz (2003) the analogous format of arrows in diagrams is ei-
ther complemented or replaced by verbal and numerical elements. Beckmann recorded
participants’ structural knowledge by means of sequential verbal questions after each
experimental trial. Subjects were confronted with very general questions ﬁrst (such as
the question whether they had found out anything new about any relation in the dy-
namic system) and later – in case subjects had answered accordingly – more speciﬁc
questions followed (such as the question concerning the strength of the relation between
two deﬁned variables). Hence, the level of speciﬁty was adapted to participants’ indi-
vidual knowledge. Verbal answers of the question sequence then were translated into
an analogous format, i. e. into arrows and symbols between exogenous and endogenous
variables implemented in the system’s graphical user’s interface (see Beckmann, 1994, p.
72). In Schulz’s diploma thesis (2003), the method of causal diagrams was completely
transformed into a tabular display leaving room for the same information as can be
speciﬁed in a graphical causal diagram.5
Although the method of causal diagrams has been criticised due to its potential reactiv-
ity, i. e. the fact that the way of recording knowledge draws subjects’ attention to causal
relations to an unnatural, exaggerated extent (see Kluwe, 1988, p. 370), there are two
major advantages of this method. Firstly, causal diagrams allow diﬀerentiated analyses
of three levels of system identiﬁcation: the identiﬁcation of the existence or non-existence
of a relation, the identiﬁcation of a relation’s direction and the identiﬁcation of a quan-
titative weight indicating the strength and direction of a relation. Secondly, measures
developed on the base of causal diagrams have proved highly reliable, especially the
indicator quality of knowledge acquisition, which has been validated by M¨uller (1993)
in extensive studies. Quality of knowledge acquisition deﬁnes a weighted diﬀerence of
ratios of correct and false answers relative to the maximum numbers of correct and
false items (see also Funke, 1992b, p. 81 ﬀ.). Another approach to measure structural
knowledge is found with Preußler’s “pair-task” (Preußler, 1996, 1997, 1998). In the
“pair-task” pairs of two variables’ names are presented to subjects who decide whether
a relation between these variables exists or not. According to Preußler, the pair-task
is considered less reactive than the method of causal diagrams, yet information about
gained knowledge is conﬁned to the mere knowledge of existent or non-existent relations.
While causal diagrams and the “pair-task” aim at recording abstract structural knowl-
edge about causal relations in a system, in terms of acquired knowledge it is also possible
to examine knowledge on a more concrete level, closer to the application of knowl-
edge. Schoppek (2002), e. g, emphasises the distinction between structural knowledge
and input-output knowledge which “represents speciﬁc input values together with the
corresponding output values”. Accordingly, in order to consider this type of speciﬁc
knowledge as well, participants may be asked which will be the resulting outcome states
provided certain input states and interventions. The measure calculated from such pre-
dictions and the actually resulting outcome states is named the quality of prediction
(see Funke, 1992b, Funke & M¨uller, 1988). Another way to infer input-output knowl-
edge is to present speciﬁc system’s states which either correspond to dynamic situations
actually occurred (target situations) or to additionally constructed distractor situations
which resemble target situations but have never been encountered by subjects (Preußler,
2001). Subjects are requested to assess whether a presented situation is “old”, i. e. a
target, or whether it is completely “new” to them, i. e. a distractor. On the base of
correct and false recognition answers knowledge scores can be calculated.
Even though the latter measures can possibly be regarded as less reactive than causal
diagrams, problem solvers confronted with these demands still know that they are asked
about explicit structural knowledge they have gained. It was Preußler (1996) who sug-
gested and conducted an attempt of recording rather implicit knowledge. In Preußler’s
lexical decision task participants are interrupted in the actual task of CPS just as with
causal diagrams or the “pair-task”, but they do not know that the intermittent task
5The reason for this practice were not general objections against graphical causal diagrams, but a
speciﬁc experimental manipulation which might have interfered with graphical display of the system’s
intends to gain an insight into their knowledge. Subjects in the experimental setting are
simply asked to decide as quickly as they can whether a variable’s name presented in a
temporal sequence on the screen refers to a “real” variable or to a similar, non-existent
variable’s name. In diagnosing knowledge the decisive measure is provided by records
of reaction time rather than by the mere distinction of correct or false reactions. As
common in research on associative priming (compare Goshen-Gottstein & Moscovitch,
1995) it is assumed that problem solvers will react to a correct name more quickly if the
preceding item represents the name of a variable which is – within the examined com-
plex system – indeed related to the other variable. The approach seems both appealing
for its comparable non-reactivity and prone to criticism: Certainly it is a daring busi-
ness to infer associations from reaction times and structural knowledge from superﬁcial
associations (compare Funke, 2003, p.166).
Measuring control performance. Control performance on the other hand is mea-
sured almost consistently throughout recent experimental studies by means of the quality
of system control. This measure calculates mean logarithm deviations of systems’ ob-
served states from deﬁned goal states (see, e. g, M¨uller, 1993, p. 61). Shown by M¨uller’s
validating analyses, quality of system control as an experimental tool is just as reliable
as the quality of system identiﬁcation.
A measure found in older studies is given by mean absolute deviations from goal values
(e. g, Funke & M¨uller, 1988). In the exceptional case of a special paper and pencil
Dynamis adaptation (G¨orn, Vollmeyer & Rheinberg, 2001) merely counting the number
of accomplished goals can be an indicator of control performance as well.
Knowledge acquisition: active exploration vs. passive observation. So far
we have considered processes of knowledge acquisition as a result of subject’s speciﬁc
interventions and experiences with dynamic systems. Phases of system control and
active exploration have been interwoven with the formation of structural knowledge. In
reality, however, this is not necessarily the case: In some situations we better learn from
pure observation. Maybe this is true for some aspects of CPS as well?
This question was risen and investigated in an study by Funke and M¨uller (1988; see also
Funke 1992a, 1992b, 1993) based on the data of 32 college students. Using Sinus, the
authors implemented a new experimental design which is suitable to comparing eﬀects of
active intervention vs. passive observation on problem solving. Active intervention was
realised as in traditional experiments whereas participants in the condition of passive ob-
servation received the intervention data from an “experimental twin” of the intervention
group (yoked control-design). During four initial exploration phases they but observed
input states, the twin’s intervention values and corresponding outcome states. Like the
intervention group they ﬁlled out causal diagrams after each round. These blocks of
either active or passive exploration were followed by one round requiring active control
of the dynamic system from all participants.
Prior to statistical testing Funke and M¨uller expected a general superiority of the inter-
vention group with regard to both the amount of knowledge and the quality of system
control. Their assumption was only partly supported. As path-analytical expectation re-
vealed, subjects who actively explored Sinus showed indeed better control performance
than the passive observers. The observers, however, had gained signiﬁcantly more and
better knowledge about the system, especially with regard to exact numerical param-
eters. An explanation proposed by the authors points at diﬀerential perception of the
experimental demands: While subjects asked to actively explore and control the system
may consider the ﬁnal control task as their primary task demand, to their experimental
twins in the observers’ condition, knowledge acquisition and producing valid causal dia-
grams, being their only active contribution during the exploration blocks, should appear
far more important.
Partially diﬀering results and interpretations are found in Beckmann’s (1994) research
which is dedicated to learning in traditional learning tests as compared to CPS in the
ﬁrst place, but which additionally deals with eﬀects of active vs. passive exploration on
problem solving. Beckmann’s terms active and passive exploration correspond to Funke’s
and M¨ullers concepts of active intervention vs. passive observation, respectively. Only
unlike in the above experiment, passive exploration in Beckmann’s practical realisation
implies observing system’s states which have been prepared by the experimenters in
order to represent information resulting from possible optimal exploration strategies.
The yoked-control design is replaced by displaying informative data identical to all
observers. With regard to the scenario employed (Maschine, a six variable system
free of semantic embedding), with regard to general procedure, and the number of
participants (40 students) Beckmann’s experiment seems comparable to Funke’s and
The results, however, are diﬀerent. Beckmann could not replicate the ﬁnding that sub-
jects in the observers’ condition gained signiﬁcantly more structural knowledge than
subjects who actively intervened. The observers even tended to identify less precise al-
though this eﬀect failed statistical signiﬁcance. Concerning control performance, Beck-
mann’s participants, too, performed better if they had actively explored the dynamic
system. This ﬁnding has been interpreted in terms of diﬀerent procedural skills that
arise from diﬀerent task demands. Beckmann assumes that when actively controlling a
system former observers can only rely on the declarative knowledge they have acquired
during observation while active explorers on the other hand gain and make use of both
declarative and procedural, often implicit knowledge about the system’s reactions.
For the subsequent sections on knowledge acquisition and knowledge application we can
at least keep in mind that knowledge can be gained both from participants’ active in-
tervention as well as from passive observation. As we will see with studies providing
additional structural knowledge to subjects besides the knowledge from active explo-
ration both active and passive ways of knowledge acquisition can be combined.
A closer look at the relation between knowledge acquisition and control per-
formance. It is commonly assumed that acquiring knowledge represents a necessary
precondition for controlling a dynamic system. As in the above text, too, most scien-
tists interpret system control in terms of applying gained knowledge.6That is why in
6Interestingly, the reverse path, i. e. whether and how control performance in turn aﬀects knowledge
acquisition has been examined far less frequently. Assumptions as to the question whether successful
control performance encourages increased knowledge acquisition appear to be controversial. Preußler’s
(2001) research on the base of a Dynamis scenario suggests that at least gaining speciﬁc input-output
experimental practice, phases of free exploration, aiming at knowledge acquisition, usu-
ally precede phases of controlling a dynamic system with respect to deﬁned goal values,
aiming at knowledge acquisition.
Although in general there is empirical evidence for this theoretical notion, we will see
that ﬁndings on the relation between knowledge acquisition and control performance are
not quite as distinct for several reasons: Knowledge relevant to control demands can be
explicit and abstract or rather implicit and concrete to diﬀering degrees; furthermore,
mediating factors exist that make it more or less probable that problem solvers in control
tasks actually make use of acquired knowledge.
Correlation analyses. A straightforward approach to investigate the relation be-
tween knowledge acquisition and control performance is to analyse statistical (a posteri-
ori) correlations between these two indicators in CPS. Funke and M¨uller (1988) already
found positive statistical correlations (r= 0.41 or – for revised diagnostic measures in
Funke, 1992b – r= 0.54) hence indicating that the degree of knowledge acquisition will
determine the level of control performance. The possibly most prominent demonstra-
tion of this relation is found with M¨uller’s (1993) analyses based on the theory of latent
state-trait models (e. g Steyer, Schmitt, & Eid, 1999). LISREL analyses, the practical
implementations of this model, enable separating manifest variables, i.e. variables that
are empirically observed, from latent variables, i. e. variables assumed to form an un-
derlying, non-observable psychological construct. With respect to CPS it was M¨uller’s
idea to distinguish manifest knowledge and control performance in speciﬁc Dynamis
situations from corresponding latent traits of system identiﬁcation and system control
as problem solvers would reveal irrespective of situational inﬂuences.
For empirical validation 143 school students worked on Sab, the Dynamis computer
simulation of an abstract machine (a precursor of Kr¨oner’s MultiFlux system). For
LISREL requirements two sessions with two equivalent Sab versions were conducted,
unfortunately reducing the number of complete data to 78 participants. On the base
of the LISREL procedure M¨uller showed that for the latent variables corresponding to
manifest system identiﬁcation and system control correlations were signiﬁcantly posi-
tive (r= 0.83 for the ﬁrst session, r= 0.86 for the second session; for a brief summary
of M¨uller’s procedure and results see also Funke, 2003, p. 163-165). M¨uller’s analy-
sis furthermore suggests that the trait of system identiﬁcation can be regarded as a
one-dimensional construct, dismissing any implicit or procedural knowledge speciﬁc to
control performance. According to M¨uller, control performance can fully be predicted
on the base of subjects’ abstract knowledge about causal relations in the system (p.208).
Another interesting aspect of M¨uller’s work provides one possible explanation as to em-
pirical ﬁndings inconsistent with M¨uller’s positive correlations between identiﬁcation
and control of a complex system. In a small experimental sample of 20 students M¨uller
showed that the amount of knowledge gained obviously depends on some facet of pre-
senting information about the system’s state. Students who were – during a problem
solving task – confronted with a simultaneous presentation of actual and past system’s
knowledge is not systematically related to control achievements. Yet in this section the discussion of
relevant Dynamis studies will focus on the more obvious relation: how control performance depends
on acquired knowledge.
states gained more and better knowledge than those who viewed but one system’s state
at a time, the actual system’s state which was replaced by the subsequent state in the
following trial (sequential presentation). Given simultaneous presentation participants
seemed to beneﬁt from extra time spent on the task whereas in the condition of sequen-
tial presentation this was not observed. The author concludes that simultaneous display
provides the possibility of analysing the course of the system’s development, a necessary
condition for acquiring knowledge. In turn, since only with simultaneous display sub-
jects acquired suﬃcient knowledge, only in this condition positive correlations between
system identiﬁcation and system control may occur whereas for sequential presentation
correlations are – or rather seem – insigniﬁcant.
The notion that acquired structural knowledge can predict goal attainment is also sup-
ported within more comprising path-analytical models by Vollmeyer and Rheinberg
(1998), Beckmann and Guthke (1995) as well as Kr¨oner (2001). Correlation coeﬃcients
range from 0.51 with Beckmann and Guthke to 0.65 in Kr¨oner’s experiment or 0.59 when
it was statistically controlled for the impact of test intelligence. Similarly, Vollmeyer,
Burns, and Holyoak (1996) found that for both an initial control task as well as for
a transfer control task structural knowledge measured by a structural score revealed
signiﬁcantly negative correlation coeﬃcients when correlated with the solution error,
the absolute diﬀerence between a target value and the observed value (r=−0.57 for
the initial control task, r=−0.65 for a transfer control task). All of the correlations
reached signiﬁcance or even high signiﬁcance making a general dissociation between
system identiﬁcation and system control appear highly improbable.
Yet it should be mentioned that the obvious relation between gaining and applying
knowledge may be mediated by various inﬂuences. One such inﬂuence has already
been discussed with Beckmann’s (1994) study on the impact of semantic embedding in
complex dynamic systems. Positive correlations between achievement on the two task
demands rather occurred when the system was abstract than semantically embedded. In
Kluge’s (2003) work task diﬃculty has proved to be another mediating factor. In a large-
scale experiment involving 496 subjects working on the Dynamis system ColorSim
(see below), among other factors formal task complexity was manipulated by three levels
of increasing connectivity. Correlation coeﬃcients as to acquired knowledge and control
performance were found to be high for a low or medium degree of connectivity (r= 0.70
and r= 0.68, respectively), but it was considerably lower for subjects who had dealt
with the most complex version (r= 0.44). Possibly in the latter case additional factors,
e. g, the amount of time spent on the task became more inﬂuential, hence lowering the
Manipulating structural knowledge: the impact of transparency. The a pos-
teriori correlations described in the preceding section obviously provide an appropriate
tool to detect that there is, in fact, a positive relation between the two task demands in
CPS and that this relation is relatively powerful. Yet the results remain conﬁned to a
descriptive level because no causal relation is implied by correlation analyses. In order to
ﬁnd out more about the mechanisms which determine how knowledge is translated into
control performance experimental manipulations in this context are necessary. If the
researcher is interested in the eﬀects of structural knowledge on control performance, he
or she can try to systematically promote or impair subjects’ structural knowledge and
observe whether and how control performance will be aﬀected. The ﬁrst methodolog-
ical approach has been realised in a number of experimental studies. Since structural
knowledge is fostered by providing diagrams which present the system’s structure, mak-
ing the system “transparent” to a problem solver, the term transparency will be applied
to systems whose structure is disclosed to subjects.
In a sample of 50 university students Putz-Osterloh (1993a, 1993b) conducted an ex-
periment on possible eﬀects of transparency on problem solving in order to examine the
relation between knowledge acquisition and system control. Participants dealt with a
version of Linas which involves four input variables (named A, B, C, D), seven output
variables (having fancy names) and 15 relations, some of these characterised by delayed
eﬀects. In a ﬁrst part of the experiment subjects’ exploring and controlling the sys-
tem was required. The task was identical for all subjects but for a manipulation on
structural knowledge: Subjects of the experimental group (25 students) were given a
structural diagram displaying the relations within Linas; the diagram was explained
by the experimenter, and by standardised questions it was ensured that subjects had
understood relevant information. Subjects of the control group (25 students) received
no structural information.
Putz-Osterloh assumed that students of the experimental group would beneﬁt from the
extra knowledge due to transparency and would perform more successfully in controlling
Linas. Yet the expectation could not be conﬁrmed. As to control performance no
systematic diﬀerences between the two transparency conditions were found although
subjects of the experimental group indeed seemed to have more structural knowledge
than those of the control group.7A ﬁrst conclusion therefore contradicts the above
suggested coincidence of knowledge acquisition and control performance. On the base
of the experiment’s ﬁrst part one might rather suppose that these are two independent
processes in CPS. According to Putz-Osterloh and similar to Beckmann’s (1994) notion
of ad hoc control, it might be possible to successfully control a system despite little
knowledge, acting almost in terms of trial and error.
When, however, can structural knowledge still support performance in a complex dy-
namic system? A second part of the experiment suggests that structural knowledge
provided by diagrams can, although not applicable immediately after its acquisition,
enhance control performance in a transfer task. Half a year after the ﬁrst session, 38
students of both the experimental and control group from Putz-Osterloh’s original sam-
ple worked on a modiﬁed or “ﬂawed” version of Linas in which the eﬀect of one input
variable had been omitted. Participants were informed about this fact, yet they did
not know which variable had been changed. They were requested to name the deﬁcient
variable after controlling the dynamic system.
As expected, this time subjects formerly supported by structural knowledge outper-
formed subjects of the initial control group, again applying more eﬃcient strategies.
They also proved to be more successful in diagnosing the modiﬁcation. Interestingly,
for participants of the former experimental group a positive correlation was found con-
cerning the amount of reproduced structural knowledge (as could be inferred from the
7Especially the ﬁnding that participants supported by transparency adopted more eﬃcient strategies
made the author assume that these subjects indeed made use of their enhanced structural knowledge.
diagnosis task) and control performance. Subjects of the former control group whose
knowledge acquisition had not been systematically promoted on the other hand revealed
no such systematic correlation. Instead, Putz-Osterloh states a dissociation: Problem
solvers who easily detected the system’s modiﬁcation tended to do relatively badly on the
control task, not adopting their strategies to the modiﬁed system’s structure, whereas
other subjects seemed to cope successfully with the changed version although they did
not detect which variable had been manipulated.
Possibly, Putz-Osterloh argues, in the initial task subjects confronted with abstract
knowledge indeed acquired helpful, comprising structural knowledge, but the demands
of analysing and interpreting the graphical structural diagram occupied their cognitive
capacities almost completely so that they could not apply their comprising knowledge
right from the beginning. Supporting eﬀects of structural knowledge on control perfor-
mance only became obvious in the transfer task after the process of knowledge acquisition
had been completed. With respect to the relation of acquiring structural knowledge and
accomplishing the control task Putz-Osterloh, partly disagreeing, e. g., with M¨uller’s
(1993) interpretation, assumes that both demands can still be conceived as two separate
goals. As shown by the dissociation found in Putz-Osterloh’s control group the goals
are not necessarily coordinated, but they well may be if problem solvers’ structural
knowledge reaches a suﬃcient extent.
Preußler (1996) as well examined the role of structural knowledge and control perfor-
mance in Dynamis situations and adapted the view that there might not necessarily be a
direct translation of structural knowledge into applied knowledge in control performance.
Two successive experiments emphasise that in actually applying gained knowledge at
least two conditions need to be accomplished: Firstly, structural knowledge has to be
gained in the context of application instead of abstract “inert” knowledge, secondly,
knowledge acquisition should take place prior to application.
With respect to structural knowledge, Preußler did not but analyse explicit verbalised
knowledge recorded by the “pair-task”, but also implicit associative knowledge about
structural relations which was inferred from a lexical decision task. Both qualities
of knowledge were to be considered to examine whether implicit rather than explicit
knowledge was associated with successful control performance as Berry and Broadbent
(1984) had suggested.
Similar to Putz-Osterloh, yet closer to knowledge application, structural knowledge
in the transparency condition was promoted by speciﬁc examples and explanations.
Within a total sample of 50 students (studying at the University of Bayreuth) half
of the students, the experimental group, passed through a practice phase before the
control task: The experimenter told them – according to standardised examples – how to
manipulate certain input variables and explained subsequent eﬀects on output variables
to them. The remaining subjects forming the control group explored the dynamic system
without requests and explanations before they worked on the same system for eight
rounds with eight trials each. The Dynamis system chosen for this task was a version
Three experimental hypotheses were to be tested for statistically. Firstly, the experimen-
tal group systematically supported in gaining structural knowledge should reveal more
knowledge as to both the explicit and implicit measure. Secondly, as a consequence of
enhanced knowledge acquisition the experimental group should outperform the control
group in controlling Linas. Thirdly, the amount of gained knowledge, both implicit
and explicit, should be a predictor of control performance. The ﬁrst experiment, how-
ever, supported neither of these assumptions. Only marginal support was found with
regard to enhanced implicit knowledge in the experimental group (ﬁrst hypothesis) and
a positive relation between knowledge and system control (third hypothesis).
It was the second experiment involving another 48 university students for participation
which conﬁrmed that subjects systematically promoted in knowledge acquisition dis-
played indeed higher levels of implicit knowledge as compared to those of the control
group. Control performance, too, tended to be higher within the experimental group.
What had been changed in the second experiment relative to the ﬁrst experiment? Ac-
cording to Preußler, the procedure of instructing knowledge had been extended to more
systematic analysis in the practice phase. This fact emphasises the importance of in-
structing knowledge appropriately, ready to application in controlling the system. As
expected, in the second experiment, too, when acquiring knowledge had been success-
fully promoted control performance turned out to be positively related to the amount
of implicit knowledge (at least for Linas relations which could be detected quite eas-
ily). Since comparable positive correlations also occurred with knowledge acquisition
and control performance in the early practice phase, Preußler corroborates the inter-
pretation from correlation analyses (e. g, M¨uller, 1993, Funke, 1992b) that knowledge
precedes and determines subjects’ coping with the control task. She yet emphasises the
importance of eﬀective knowledge instruction. To a considerable extent eﬀectiveness
appears to depend on the time of instruction. Obviously, it is suitable to promote struc-
tural knowledge prior to its application in a control task so that the two processes of
analysing and controlling will less interfere with one another.
A following study by Preußler (1998) attempted more detailed speciﬁcations of the
suggested conditions under which induced structural knowledge will be predictive of
success in a control task. Unlike in the previous experiment, Preußler was especially
interested in the role of explicit structural knowledge; explicit knowledge was deemed
more determinant than implicit knowledge because it enables subjects’ making up and
testing explicit hypothesis which might be helpful in control tasks. In accordance with
the earlier ﬁndings emphasis was put ﬁrstly on early instruction of explicit knowledge
(before the control task) and on even more application-oriented instructions.
Application-oriented promotion of structural knowledge in an experimental group meant
a practice phase involving a graphical structural diagram for transparency combined
with practice tasks: Goal values had to be attained by particular manipulations; the
task lasted until participants themselves had found out correct solutions. In a control
group by contrast structural diagrams were omitted and the same practice tasks could be
ﬁnished without ﬁnding the correct solutions. The experimental procedure comprised
two subsequent control tasks, one learning phase after the practice phase, then after
some delay a second control task, the transfer phase. In total, 56 Bayreuth University
students took part in a ﬁrst experiment and another 54 students in a second experiment.
All of them worked on diﬀerent Linas implementations.
Results from the ﬁrst experiment suggested supporting eﬀects of the application-oriented
treatment on structural knowledge (measured by the pair task) and performance: Struc-
tural knowledge of the experimental group, after the ﬁrst control task already, was found
to be signiﬁcantly higher than with subjects of the control group. Interpreted as a conse-
quence of enhanced and successfully applied structural knowledge, in both the learning
phase and the transfer phase the experimental group outperformed the control group.
Unlike the control group, the experimental group displayed “positive transfer”, i. e. in
the transfer control task right from the beginning they performed equally well as at
the end of the ﬁrst learning control task. Concerning the relation between structural
knowledge and control performance, as expected, positive correlations were revealed, yet
only for problem solving in the transfer task. The partly deviation from expectations
as to the learning task cannot fully be interpreted. However, unlike in Putz-Osterloh’s
experiment (1993a, 1993b; see above) there is at least evidence that subjects’ control
performance will beneﬁt from induced structural knowledge in the ﬁrst control task
already instead of deferred beneﬁts which are revealed but on a transfer task.
Preußler’s second experiment dealt with the question whether the enhanced control
performance of the experimental group results from the mere fact that (some kind
of) knowledge is instructed or whether successful control performance is promoted by
the particular structural knowledge implied in the transparency condition. For this
purpose the experimental group (receiving an almost identical treatment as in the ﬁrst
experiment) was compared to a control group receiving a training on speciﬁc, more
concrete knowledge about isolated eﬀects of intervention in the practice phase.
Obviously, conﬁrming the author’s expectations, structural knowledge proved to be
superior in promoting control performance. Although speciﬁc interventional knowledge
might, of course, not be regarded as futile to control performance, evidence occurred
that the practice on reaching isolated goals in the practice phase might have interfered
with the successful coordination of these goals as required to system control. Additional
analyses showed that less time was required to pass through the training on abstract
knowledge (experimental group) as to the training of speciﬁc interventional knowledge
(control group), suggesting that the application of structural knowledge is not merely
more successful, but also more economic.
To summarise, the later ﬁndings of Preußler (1998) partly reconcile experimental in-
vestigation with the ﬁndings of correlation analyses claiming positive relations between
knowledge application and knowledge acquisition. Provided that knowledge is acquired
prior to system control in an application-oriented context, the equivalent positive cor-
relations as in a posteriori analyses will appear.
The quality of knowledge and knowledge instruction. Studies on the impact
of transparency made obvious that in general appropriate knowledge and appropriate
ways of instructing knowledge will enhance the application of this knowledge in con-
trol tasks. Some questions, however, remain: With respect to the type of knowledge
Preußler (1998) suggested that promoting structural knowledge is most helpful in prob-
lem solving whereas speciﬁc interventional knowledge has less beneﬁcial eﬀects. Is this
necessarily the case? Additionally, one might consider the way in which the acquisition
of (structural) knowledge is supported. Early application-oriented learning seems to
facilitate later control performance (Preußler, 1998) – but maybe there is more to an
eﬀective instruction or tutorial?
For systematically comparing diﬀerent modes of knowledge instruction Kluge (2003)
designed an experiment involving the Dynamis system ColorSim.ColorSim, a sys-
tem of the graphical presentation format, is made up of three abstract input variables
(X, Y, Z) and three output or goal variables representing the names of colours (“yel-
low”, “black”, “green”). In the most simple realisation output variables are aﬀected
by four basic relations and one instance of eigendynamic; the most complex realisation
includes two instances of eigendynamic as well as two side eﬀects in addition to the
four basic relations. In Kluge’s experiment three degrees of connectivity – manipulated
between subjects – gave rise to three experimental conditions: low, medium and high
Knowledge instruction as well was realised by means of three treatment conditions:
explanatory screen mask, guided exploration and a combination of the two former in-
structional types. The main diﬀerence between the explanatory screen and guided ex-
ploration is the extent to which subjects become actively involved in the process of
acquiring structural knowledge. While guided exploration requires active exploration,
testing self-constructed hypotheses and independent reasoning,8the explanatory screen
mask but presents and explains information to be received and analysed, yet very com-
prising information; besides graphical displays of structural diagrams verbal descriptions
and corresponding numerical equations are provided so that the system’s algorithm can
be explained in successive steps.
Based on the statistical results from a sample of as many as 496 subjects Kluge concluded
that none of the three instructional conditions generally proved superior to another. Ef-
fectiveness rather seemed to depend on the level of task diﬃculty. When the scenario
had been classiﬁed as simple subjects of all three instructional conditions controlled
ColorSim equally well; only knowledge acquisition was reduced for participants con-
fronted with guided exploration as compared to control subjects. Regarding medium
task diﬃculty the combination of guided exploration and the explanatory screen turned
out to promote both knowledge acquisition and control performance in a more eﬀective
way than guided exploration without additional explanatory elements. As the author
assumes, when dealing with simple problems self-employed learning might be helpful
and successful, but with more complex problems the advantages of self-instruction de-
cline relative to the beneﬁts of objective explanations. For high task diﬃculty each of
the three instructional modes revealed deﬁcits that might, according to Kluge, only be
compensated for by further support such as leaving more time to cope with the complex
The question whether structural knowledge in CPS is most important relative to other
kinds of knowledge has been studied by Schoppek (2002). Schoppek promotes the theory
that in general both abstract causal knowledge, i. e. structural knowledge, as well as
speciﬁc interventional knowledge named input-output knowledge may be relevant to
CPS. Yet the size of the complex system should determine which type of knowledge
plays a dominant role. Analysing previous studies (involving both scenarios of the
Dynamis approach and other scenarios) Schoppek found that in scenarios implying but
a small number of possible solutions problem solvers seemed to acquire input-output
8Guided exploration is about equivalent to the instruction of testing hypothesis in studies on the
eﬀects of systematic rule-induction (see Vollmeyer, Burns, & Holyoak, 1996).
knowledge spontaneously. This kind of interventional knowledge obviously suﬃced the
demand of controlling the system. In contrast, for large systems making it impossible
to store isolated combinations of input and output values to a suﬃcient degree subjects
tended to develop structural causal knowledge even if they had not been instructed to
Schoppek’s argumentation, however, goes further and – as the conclusion of his experi-
mental investigation – states that beyond structural knowledge and speciﬁc knowledge
of input-output relations a third type of knowledge might come into play, operative or
strategic knowledge. This kind of knowledge is deﬁned as “knowledge about how to
proceed in order to accomplish a task”. Strategic knowledge can imply aspects of proce-
dural input-output knowledge, but it may as well comprise quite abstract representations
about a sequence of operator steps.
Which experimental evidence supports Schoppek’s concept? With respect to hypothe-
ses and general procedure the study Schoppek conducted on the impact of structural
knowledge resembles previous studies on transparency (Putz-Osterloh, 1993a, 1993b)
and enriched tutorials in promoting structural knowledge (Preußler, 1996, 1998). In a
between subjects design 80 students studying at the University of Bayreuth controlled
Linas, preceded by a learning phase of either one of two conditions: Subjects of the
experimental group passed through a computer tutorial which guided them to analyse
eﬀects of input variables; subjects of the control group explored the system and tried to
reach goal values.
In accordance with Putz-Osterloh (1993a, 1993b) it was found that on the average par-
ticipants of the two conditions controlled the system equally well although subjects of
the experimental group had gained more structural knowledge. Following Preußler’s ear-
lier interpretation (1996, 1998; see above) one should assume that enhanced structural
knowledge possibly was not available to successful use in the control task (although,
in fact, the tutorial was administered prior to controlling the system in an application-
oriented context). Schoppek’s interpretation yet is partly diﬀerent: Instead of “blaming”
the experimental group for inert structural knowledge he points at knowledge acquisi-
tion among subjects of the control group. Due to early experiences with the dynamic
system control subjects might have gained and later applied strategic knowledge which
compensates for the enhanced structural knowledge of experimental subjects. Since
even in a transfer task control subjects without a tutorial on strategic knowledge per-
formed equally well as the experimental group9impacts of speciﬁc knowledge about
input-output relations cannot explain the equal levels of control performance neither
can structural knowledge.
Instructed strategies, analogies, and the impact of goal speciﬁty. Studies on
the eﬀects of structural diagrams and other tutorials have shown that in CPS methods
exist which promote the acquisition of knowledge, structural knowledge in particular,
and which sometimes positively aﬀect control performance as well. On the other hand
we have seen that structural knowledge might not be the only factor responsible for
successful problem solving (e. g Schoppek, 2002). Hence when looking for supporting
9This ﬁnding obviously contradicts Putz-Osterloh’s earlier results, claiming that on a transfer task
promoted structural knowledge leads to increased control performance.
factors in the process of CPS further possibilities need to be taken into account: In this
section, the impact of instructed strategies and analogies will be discussed as well as the
impact of goal speciﬁty, which inﬂuences problem solvers’ spontaneous application of
strategies. Not in all cases it can be ﬁgured out whether one of these methods primarily
supports knowledge acquisition, control performance or both, leading us back to the
disputed relation between the two task demands.
Strategies. Before experimentally inducing and manipulating problem solvers’ use of
strategic approaches researchers have been analysing strategies subjects employ spon-
taneously and have assessed these strategies with respect to their eﬀectiveness and sys-
tematicity. Implied in the concept of the Dynamis approach, to gain knowledge in a
Dynamis situation it is required to discriminate the eﬀects of single input variables.
This can be accomplished by varying the relevant input variable and keeping all other
variables at a constant value of zero, a strategy which has been referred to as VOTAT
(Vary One Thing At a Time). VOTAT is often considered as the most promising strat-
egy to knowledge acquisition in a dynamic situation (see, e. g, Putz-Osterloh, 1993b,
Vollmeyer & Rheinberg, 1998, Rollett, 2002). Accordingly, manipulating more than two
inputs at a time is associated with unsystematic exploration. A more detailed discus-
sion of the eﬀectiveness of VOTAT and other strategies is provided in part three when
strategies are considered from the perspective of problem solvers’ characteristics (see
the experimental analysis by Vollmeyer, Burns, & Holyoak, 1996). Eigendynamics can
be best detected by doing without any intervention at all, leaving all input variables
at the value of zero. Another possibility, yet less frequently employed, is to choose
interventions that are equal in absolute value, but opposite in signs on two successive
trials. In the case of no eigendynamics, after the second trial the initial state should be
re-attained. Concerning the detection of side eﬀects applying particularly large input
values can be helpful to discriminate a basic input eﬀect from a usually smaller side
eﬀect (Schulz, 2003).
Rollett (2002) experimentally analysed and compared a number of such spontaneously
employed strategies on the base of data from 109 subjects in a ﬁrst and 207 subjects
in a second study. Applying the dynamic system Biology Lab (Vollmeyer, Burns,
& Holyoak, 1996) diﬀerent types of strategies were recorded and assessed as to their
eﬀectiveness in gaining structural knowledge. Highest – and highly signiﬁcant – posi-
tive correlations between knowledge acquisition and strategy use were found with keep-
ing constant one or two input variables out of three input variables in Biology Lab
(r= 0.51). Zero interventions, too, yielded positive correlations with the amount of
knowledge gained (r= 0.35). Negative correlations in contrast were associated with
respect to employing many diﬀerent input values (r=−0.50). Obviously, when trying
to understand a dynamic system by strategic exploration less is more.
Not surprisingly, due to its prominence and eﬀectiveness the strategy of VOTAT has
become a central element in experimental research on strategy instruction. Vollmeyer,
Burns, and Holyoak (1996) systematically instructed the application of VOTAT in a
CPS task on the base of the Dynamis system Biology Lab. The employed version
of Biology Lab comprised eight variables, four exogenous variables, four endogenous
variables which are aﬀected by ﬁve primary input-output relations and one instance
of eigendynamic. As implied in the cover story participants have to imagine they en-
ter a biological laboratory in order to ﬁnd out how certain water qualities in a water
tank (temperature, salt, oxygen, current) inﬂuence the population of four species of sea
animals (prawns, sea bass, lobster, crabs). Out of a total number of 60 participants
(undergraduate students at the University of California, Los Angeles) 30 subjects were
provided with explanations and examples how to use VOTAT in subsequent phases of
exploration (three rounds), of control (one round) and transfer tasks (one round). The
other 30 subjects received no instruction on systematic use of strategies.
Given free exploration without speciﬁc target values in the three initial rounds to be
reached the author’s strategy instruction obviously “worked”: Throughout all three
rounds of exploration about eighty percent of subjects’ interventions could be classiﬁed
as VOTAT strategies rather than the unsystematic strategy of changing all variables or
other, less systematic strategies. In contrast, most participants of the non-instructed
control group did not adopt VOTAT spontaneously.10 The instructed group revealed
higher levels of knowledge acquisition, slightly higher levels of performance in the ﬁrst
control task and especially increased control performance in the transfer control task.
The results from this part of the experiment conﬁrm that instructing systematic strate-
gies such as VOTAT indeed has beneﬁcial eﬀects on dealing with complex dynamic
systems – provided that task demands are suitable to follow the VOTAT instruction.
What, however, makes task demands suitable to applying systematic strategies?
Goal speciﬁty. An attempt to answer this last question is already implied in the
same study by Vollmeyer et. al. Besides the impact of strategy instruction the authors
examined another feature of the exploration setting: the matter whether subjects should
be informed about target values of the control task from the very beginning (goal speci-
ﬁty) or whether during the ﬁrst phases of free exploration they should simply be asked
to ﬁnd out about the system as much as they can before they get to know the target
values to be reached and maintained in controlling the system (unspeciﬁc goal condi-
tion). One possible conception is that speciﬁc goals, encouraging eﬃcient means-end
analysis of how to reduce the diﬀerence between an actual state and a goal state, might
be a suitable condition for acquiring knowledge (Anderson, 1987). On the other hand,
however, one might argue that speciﬁc goals and associated means-end analyses should
rather disturb an essential precondition of knowledge acquisition (Sweller, 1988): The
strategy of means-end analysis might interfere with problem solvers’ systematic induc-
tion of rules which would otherwise, in the absence of deﬁned goals, have taken place.
This later view has been adopted by Vollmeyer, Burns, and Holyoak.
In order to test the hypothesis that goal-oriented strategies, i. e. means-ends analysis,
interfere with other systematic strategies adapt to gaining structural knowledge (e.g,
VOTAT) the authors had implemented goal speﬁcity as an additional treatment factor
10This ﬁnding on the eﬀectiveness of strategy instruction is hardly self-evident. Vollmeyer and Rhein-
berg (1998), e. g, attempted to manipulate VOTAT strategy instruction in the context of motivational
inﬂuences and found that diﬀerential strategy use among an instructed and a non-instructed group did
not occur but for the very beginning of a CPS task. In the later process of the task instructional eﬀects
on strategy use disappeared, consequently not aﬀecting knowledge acquisition and control performance
within the experimental design. Within each of the two experimental conditions of
strategy instruction half of the subjects were given no target values until the control
task, the other half was informed about the target values during the three learning
rounds already although it was made clear that their task was “learning as much as
possible” rather than merely aiming at the goal values.
For analysing participants’ use of strategies in the goal speciﬁty condition the classiﬁ-
cation distinguishing VOTAT from changing all inputs and other strategies was supple-
mented by the category of a diﬀerence reduction strategy. Diﬀerence reduction, a goal
oriented strategy, was diagnosed when observed systems’ states incrementally approxi-
mated the goal values and when subjects successfully reached the goals in the exploration
phases already. According to Vollmeyer and colleagues diﬀerence reduction implies a
strategic approach fundamentally diﬀerent from VOTAT.
Prior to statistical testing the authors expected that in the context of goal speciﬁty
participants would apply VOTAT only if strategies were instructed explicitly. In case of
no systematic instruction combined with speciﬁc goals subjects should make predomi-
nant use of the goal-oriented strategy of diﬀerence reduction. The results went beyond
this expectation as to the impact of goal speciﬁty. Even when instructed on VOTAT
participants showed a strong linear trend away from this strategy, starting with VOTAT
application in 80 % of the cases in the ﬁrst round of exploration and ﬁnishing with a
frequency of about 20 % use in the third round. Use of the strategy of diﬀerence re-
duction markedly increased during the three exploration phases for both instructed and
uninstructed subjects provided with speciﬁc goals.
Critically examining practical requirements in this experiment Schoppek (2002) has ar-
gued that, in fact, subjects aiming to reach the goal values had had no other choice than
to omit the VOTAT strategy since in the employed version of Biology Lab it would
be virtually impossible to reach the goals by manipulating one single input variable. In
a replication study he partly changed task demands and system characteristics to dis-
solve the confound between strategy instruction and goal speciﬁty ﬁnding that subjects
made use of VOTAT in spite of goal speciﬁty just as Vollmeyer et al. had expected ﬁrst.
Although it thus remains disputable whether speciﬁc goals impair the use of instructed
strategies as well at least in the context of no explicit strategy instruction spontaneous
application of strategies seems to be aﬀected by speciﬁc goals. What does this mean to
knowledge acquisition and control performance?
Vollmeyer, Burns, and Holyoak showed that, as expected, participants gained more
accurate knowledge about the dynamic system when no speciﬁc target values had been
announced in the initial phases of learning and exploration. With respect to control
performance in the learning task goal speciﬁty did not give rise to signiﬁcant increments
or decreases. One might suppose that input-output knowledge related to means-end
analysis in case of goal speciﬁty has comparable eﬀects on controlling as the enhanced
structural knowledge when no speciﬁc goal is given. In the transfer task, however,
subjects who had freely and systematically explored Biology Lab in the unspeciﬁc
goal condition proved superior in controlling the system. This eﬀect has been interpreted
in terms of the subjects’ comprising structural knowledge. Referring to Sweller (1988),
Vollmeyer et al. conclude that “general problem solving methods applied to a speciﬁc
goal foster acquisition of knowledge about an isolated solution path but do not provide
an eﬀective way of learning the overall structure of a problem space” (p. 75).
Partly intended as a replication of these ﬁndings, Vollmeyer and Burns (1996) attempted
a closer examination of mechanisms involved in the process of knowledge acquisition un-
der the condition of both speciﬁc and unspeciﬁc goals. Following their hypothesis they
experimentally compared the impact of goal speciﬁty to another situational inﬂuence
in CPS which is assumed to have similar eﬀects: instructing subjects with explicit
hypotheses to be tested vs. not instructing subjects. The theoretical notion behind
this procedure goes back to Klahr’s and Dunbar’s dual space theory (1988) according
to which learning can be guided by either one of two distinct processes: by explicitly
forming hypotheses about possible system characteristics and then “testing” these hy-
potheses in terms of experiments (predominant search in a hypothesis space) or by ﬁrst
conducting experiments about system characteristics in order to induct rules or hypothe-
ses afterwards (predominant search in an experiment space). Searching in the space of
hypothesis is associated with greater systematicity and hence with greater knowledge
acquisition than searching in the space of experiments that may sometimes appear quite
Vollmeyer and Burns assumed that in practice hypothesis search and resulting knowl-
edge acquisition should be encouraged, of course, by providing subjects with an explicit
hypothesis to be tested when dealing with a dynamic system, but that not providing
speciﬁc goals in the dynamic system should have comparable eﬀects. These two factors,
the instruction of an explicit hypothesis vs. no hypothesis instruction, and goal speciﬁty
vs. no goal speciﬁty, made up an experimental design including 15 subjects per group,
a total number of 60 subjects. Participants, students of the University of California,
Los Angeles, dealt with Biology Lab in an experimental procedure equivalent to the
authors’ previous study. Instruction of an explicit hypothesis was realised by naming
one hypothesis concerning a deﬁned instance of eigendynamic as well as one hypothesis
concerning a numerically deﬁned input-output relation; these hypotheses were assigned
to a (ﬁctive) scientist.
With respect to goal speciﬁty the eﬀects found by Vollmeyer et al. (1996) could vir-
tually be replicated: Knowledge acquisition measured by structural causal knowledge
was reduced in the condition of a speciﬁc goal11, suggesting again that speciﬁc target
values prevent problem solvers from systematic searching in an hypothesis space but
make them apply goal-oriented strategies directly aiming at the target values. As ex-
pected, the instruction of an explicit hypothesis, too, tended to induce better knowledge
acquisition and higher achievement in a prediction task than if no hypothesis had been
presented. Deviating from the eﬀect of goal speciﬁty, as to system control subjects of
the hypothesis condition also outperformed subjects who had not been given an explicit
hypothesis. In general, however, it appears that unspeciﬁc goals and the presentation
of hypotheses to be tested aﬀect problem solving in a similar way, both encouraging
subjects to systematically induct and scrutinise rules.
Closer analyses of the data showed that enhanced control performance with subjects
given a hypothesis was not conﬁned to the variables whose eﬀects were stated in the
hypothesis. Yet to some extent better prediction scores and better structural knowledge
11Only as to the measure quality of prediction which depends on more concrete input-output knowl-
edge no impairing eﬀects of goal speciﬁty were found.
gained by this group could be assigned to the information that was implied in the hy-
pothesis. In order to limit the scope of this alternative explanation a second experiment
was conducted, involving 236 participants. Three treatment conditions were realised:
An experimental group instructed with the correct hypothesis, another experimental
group instructed with a false hypothesis and a control group which received comparable
information about a particular relation within the dynamic system, yet not formulated
as a ﬁctive scientist’s hypothesis.
Statistical analysis showed that on all relevant dimensions subjects instructed by an
explicit hypothesis performed better than subjects of the control group, even if the
hypothesis-instructed subjects had been dealing with false information. Subjects given
a false assumption did not generally perform worse than those given the correct infor-
mation. According to Vollmeyer and Burns, especially this last ﬁnding emphasises that
rather the mere instruction to test an assumption than the actual information implied
in this assumption makes problem solvers induct rules and test their own hypotheses
beyond the one given in the instruction.
Analogies. A ﬁnal note concerning supporting factors in acquiring knowledge about
dynamic systems refers to the instruction of analogies. Eﬀects of analogies have rarely
been studied in the context of CPS, hence the work done by Schulz (2003) in an unpub-
lished diploma thesis might represent a pilot study in this domain. Analogous reasoning,
a prominent concept realised in intelligence diagnostics, requires a person to transfer
knowledge structures from a known, familiar domain to another, hitherto unknown do-
main of comparable structure, but diﬀerent contents. As with the instruction of explicit
hypotheses the cognitive demands involved in processing analogies, too, can be associ-
ated with increased search in a hypothesis space. Schulz thus assumed that instructing
analogies in a CPS task would facilitate the acquisition of structural knowledge. For
experimental testing an abstract Dynamis scenario named Wits, consisting of two en-
dogenous and two exogenous variables was employed in an internet experiment. Three
between subjects conditions of analogy instruction were manipulated: working on the
system without any analogy at all, dealing with one analogy and dealing with three
diﬀerent analogies. All analogies were derived from familiar every day contexts such as
the context of an ecosystem.
Results based on the data of 53 participants revealed a marked, yet not signiﬁcant eﬀect
towards superior knowledge acquisition of subjects instructed with analogies. Somewhat
unclear and diﬀerent from the author’s expectation the group provided with only one
analogy even performed better than the group provided with three diﬀerent analogies.
On the whole, however, Schulz concludes that analogies obviously have beneﬁcial eﬀects
on acquiring structural knowledge about a complex dynamic system.
3.3 Person characteristics
In the ﬁrst two parts we have dealt with the role of system and task characteristics
in CPS. Diﬀering levels in performance even though subjects work on exactly identical
problems under identical conditions make clear that it is not the dynamic system as
such nor the task demand taken on its own which determines how well we do on a
computer-simulated CPS task. To a considerable extent our achievement rather depends
on ourselves: the strategies we employ deliberately or by intuition without explicit prior
instruction, our current motivation as well as generalised motivational tendencies and
our relevant cognitive abilities.
3.3.1 Strategies and systematic learning
The section about supporting factors which may enhance performance in CPS has al-
ready revealed the beneﬁts of systematic, strategic operations in this context. Not
surprisingly, just as experimental inducement of strategies leads to increased control
performance and system identiﬁcation, the same occurs if problem solvers apply appro-
priate strategies spontaneously without preceding request.
Branke (1991) conducted an experiment which involved 42 university students working
on a Dynamis simulation named Korallenriff (coral reef); in addition to stan-
dard measures of problem solving abilities she recorded and classiﬁed the application of
strategies as a personality factor and came to the conclusion that employing strategies
is signiﬁcantly related to successful problem solving. The more strategic the approach,
the more knowledge will be gained and the better control performance will be.
Similarly, Putz-Osterloh (1993b) found that high increases in performance were associ-
ated with strategies classiﬁed as eﬀective – especially strategies of selection – whereas
low increases in performance were associated with less eﬀective strategies, i. e. changing
all inputs in the same trial. Not quite as expected, in Putz-Osterloh’s experiment this
eﬀect could only be revealed when subjects had been promoted in gaining structural
knowledge by means of transparency. A control group without additional information
about the system’s structure seemed to increase performance both in the case of eﬀective
as well as ineﬀective strategies. Satisfying explanations cannot be provided.
The particular relevance of VOTAT, the strategy of manipulating but one single input
variable on a trial, has been highlighted in an above discussed study by Vollmeyer, Burns,
and Holyoak (1996). VOTAT is compared and contrasted to the strategy of “chang-
ing all variables in a haphazard way” and a heterogenous collection of other strategies.
Problem solving records of 36 students (studying at the university of California, Los
Angeles) working on Biology Lab showed that a score on strategy systematicity was
highly positively related to the accuracy of knowledge acquisition (r= 0.76) and control
performance. While subjects employing VOTAT clearly performed best though not per-
fect, subjects characterised by haphazard strategies seemed to be impaired in structural
knowledge, control performance as well as achievement in a prediction task.
Rollett, too, corroborates the view that individual diﬀerences in applying strategies
help to explain the frequently observed large variances in problem solving performance.
Rollett’s experimental studies on the eﬀectiveness of speciﬁc strategies (see the section
about instructed strategies) are supplemented by analyses on the beneﬁts of information
utilisation. The term information utilisation refers to the question which and how
much information participants – when dealing with a complex problem – actually take
into consideration as compared to the type and amount of information participants
might make use of in case of expertise. Practically speaking, the “experts’” utilisable
information is rated for each subject and each episode of intervention separately by
analysing speciﬁc interventions and the resulting input and outcome states.
Correlation analyses based on samples of a 109 subjects in the ﬁrst study and 207 sub-
jects in a second study showed signiﬁcantly positive correlations between the assumed
experts’ information use and participants’ information use (correlation coeﬃcients rang-
ing from 0.56 to 0.29). Since the highest coeﬃcients were found for information utilisa-
tion in the ﬁrst of several learning rounds one may conclude that problem solvers at the
beginning of their task make use of available information most eﬃciently whereas in the
later process of problem solving eﬃcient use of information partly decays, possibly due
to further cognitive demands.
Combining the two concepts of strategic operating and information utilisation a problem
solvers’ typology has been Rollett’s major contribution to the research on individual
diﬀerences in CPS. Subjects are classiﬁed on two dimensions: on the dimension of
strategy use as either insuﬃcient, suﬃcient or eﬃcient users and on the dimension of
information use as either belonging to the lower, the medium or the upper third among
other performers. Frequency analyses on the resulting (nine) combinations revealed
that a majority of subjects characterised by eﬃcient strategy use made highest use of
available information; only a minority of them made lowest use of information. In turn,
insuﬃcient strategies were found primarily with subjects whose information use was
assessed to be in the lower third. Obviously, eﬃcient use of strategies and information
coincide to some degree.
In her above mentioned study, Branke (1991) assessed that only half of the sample’s
participants preferred a strategic approach to the problem situation. In Putz-Osterloh’s
experiment (1993b) even as many as two thirds of the subjects could be characterised
by employing ineﬀective strategies. The analyses by Vollmeyer and colleagues (1996)
showed that in a ﬁrst round of dealing with the dynamic system Biology Lab again
two thirds of the subjects started exploring the system by least systematic haphazard
strategies, but after three further rounds the share of unsystematic strategies declined
to about 20 %. More than half of the problem solvers (58 %) rather made use of VOTAT
in the later stages of exploration. As it seems, the use of systematic strategies will be
promoted by growing experience with the dynamic system.
Why, however, do problem solvers still dismiss or simply not apply strategic operations in
a dynamic computer-simulated task? To anticipate the subsequent sections, we will keep
in mind that close relations exist between intelligence and exploration skills (Kr¨oner,
2001) and that motivation, too, plays a major role in applying strategies (Vollmeyer &
Rheinberg, 1998; Rheinberg, Vollmeyer, & Rollett, 2001). Yet an earlier study by Fritz
and Funke (Fritz & Funke, 1988) suggests that the use of strategies in CPS might as well
depend on some speciﬁc learning abilities not covered by overall levels of intelligence.
The authors examined problem solving performance and the use of appropriate strate-
gies with a clinical sample of adolescents having “minimal cerebral dysfunctions” (MCD)
which are strongly associated with learning diﬃculties at school. Out of 53 adolescents,
14 displayed a stable MCD symptomatic, 17 – the group of potentially aﬀected adoles-
cents – had once been given a diagnosis, but did not conﬁrm it, another 16 adolescents
had no MCD diagnosis at all, thus serving as a control group. All participants worked on
a simpliﬁed version of Funke’s ¨
Okosystem (Funke, 1985), a dynamic system including
three exogenous and three endogenous variables, one instance of eigendynamic and no
side eﬀects or delayed feedback.
It was hypothesised that in spite of their overall high intelligence subjects having MCD
would turn out to be inferior in strategic operating. Statistical analysis failed to reach
signiﬁcance, yet the results showed a tendency towards the expectations. Various strate-
gies of CPS occurred in both the MCD groups and the control group, however, not
equally distributed. Whereas subjects of the control group frequently and successfully
employed zero intervention strategies and isolated manipulations of one single exogenous
variable, strategies most adapt to gain structural knowledge, subjects of the MCD group
more often tended to manipulate all three exogenous variables at a time, which hardly
enables the detection of isolated eﬀects. From this characteristic diﬀerence the authors
inferred insuﬃcient discriminatory abilities of participants having MCD.12 Lowered in-
tegrational abilities, too, were assumed since, compared to subjects of the control group,
those having MCD or being potentially aﬀected required more time and experience to
acquire knowledge, i. e. to form an integrated structure of knowledge.
Interestingly, despite diﬀering strategic approaches performance levels were about equal
among the three groups. MCD tends to be related to impaired strategic operations and
strategic operations deﬁnitely aﬀect the process of problem solving as we have seen, but
there must still be other factors relevant to individual diﬀerences in this domain.
It has widely been accepted that (at least some) motivation is necessarily required
for achievement in cognitive tasks. In a few experimental studies (e. g, Funke, 1992b)
measures of initial or current motivation have been implemented in order to ensure that
on the average participants are equally well motivated among diﬀerent experimental
conditions.13 Yet the majority of investigations appears to do without motivational
measures of control, a fact which is not only due to economical considerations. The
particular diﬃculty of motivational records concerns the ﬂuctuating nature of current
motivational states: Supposed an experimenter asks his or her subjects about some
deﬁned aspects of motivation, say, interest in the task and expectations of successful
outcomes, how will the experimenter know whether the subjects’ self-assessments remain
valid over the course of the task’s duration, over at least one hour and often more than
that? Some facets of motivation, e. g, interest, which may appear dominant in the
beginning, may decrease or disappear, while other facets come into play which initially
have not been recorded at all, e. g, ﬁnishing one’s task quickly not to miss a later
appointment (compare Vollmeyer & Rheinberg, 1998). As Vollmeyer and Rheinberg
did in their study discussed below, researchers can, of course, employ brief intermittent
measures of present motivational states, yet being aware that the records might disturb
the process of learning and CPS.
Another perspective in examining motivational impacts in CPS is to consider aspects of
12Concerning motivation adolescents of the MCD group revealed signiﬁcantly higher failure motiva-
tion, i. e. fear of potential failure. As we will see below, anticipation of failure as well should impair
the application of systematic strategies (see Vollmeyer & Rheinberg, 1998).
13The demand of comparable motivational levels will not necessarily be accomplished as Funke’s
ﬁndings show: Initial motivation already was lower for subjects who dealt with a system of merely
numerical presentation than for subjects who worked on an equivalent system with graphical display.
personality which are related to current motivational states, but are assumed to be more
stable over time and hence less aﬀected by the hazards associated with current moti-
vation and intermitting records. The term motivational orientation or goal orientation
has been introduced to deﬁne relatively enduring motivational tendencies, an individ-
ual’s “tendency to favour speciﬁc types of goals, outcomes or consequences over some
others” (Niemivierta, 2002). Unfortunately, according to G¨orn et al. (2001), experts
have not yet fully agreed as to the question to what extent motivational orientations
display stable traits of personality or to what extent they are still related to ﬂuctuating
In the present paper we will try to consider the impact of both motivational states and
trait dispositions on CPS. The four studies discussed in this section especially focus on
the question which motivational factors interact in which way with relevant cognitive
variables in problem solving. As Vollmeyer and Rheinberg (1998) state, motivation per
se does not aﬀect cognition but is mediated by the frequency and duration, by the quality
of learning activities and the learner’s functional state such as his or her concentration.
With regard to motivational states in CPS Vollmeyer and Rheinberg (1998) have exam-
ined the impact of two speciﬁc factors: interest in the task and conﬁdence of success vs.
fear of failure. In preliminary examinations these two facets of motivation had turned
out to be crucial speciﬁcally to the context of computer-simulated dynamic tasks. Hence,
for their experiment, Vollmeyer and Rheinberg recorded the two factors both as part
of a questionnaire intended to assess initial motivation and by means of brief question-
naires assessing motivation parallel to a dynamic problem solving task. The intermittent
questionnaires contained only one item each for task interest and conﬁdence of success:
“I am still enjoying the task” and “I am sure I will ﬁnd the correct solution”. In or-
der to elucidate how assumed motivational eﬀects are mediated the authors added two
measures characterising learning activities: participants’ concentration on the task, es-
pecially “eﬀortless concentration” which is associated with productive learning, secondly
the systematicity of applied strategies as can be inferred from the protocol of input and
output states. Eﬀortless concentration was assessed in a one-item-scale questionnaire.
48 students of the University of California, Los Angeles, took part in Vollmeyer’s and
Rheinberg’s study. Subjects worked on a relatively complex version of the Dynamis
system Biology Lab whose cover story refers to a virtual biological laboratory; within
the “biology lab” participants have to ﬁnd out how three chemical factors (salt, carbon,
calcium), the exogenous variables, inﬂuence the quality of water in a tank with respect
to three indicators (oxygenation, chlorine concentration, temperature), the exogenous
variables. Three rounds of mere exploration, each consisting of eight intervention trials,
were conducted, one subsequent round was assigned to system control and accomplishing
deﬁned goal values.
Regarding the two motivational factors, unfortunately, only the factor conﬁdence of
success vs. failure motivation proved stable enough to more detailed analysis. Subjects’
self-evaluations of initial interest in the task seemed to coincide too little with later
ratings of the task’s attractiveness, so that this factor was dismissed.
Path-analytical analysis of the remaining cognitive and motivational factors revealed
two indirect impacts (paths) and one direct impact subjects’ conﬁdence of success has
on the quality of CPS. Indirect relations between conﬁdence of success were ﬁrstly found
to be mediated by the systematicity of strategies applied: The more conﬁdent subjects
feel that they will succeed the more systematic strategies they will employ, an eﬀect
which yet only became noticeable as late as the third round of exploration. Systematic
strategies in turn are positively related to the acquisition of knowledge and this again
has positive eﬀects on the application of knowledge, leading to enhanced control perfor-
mance. Secondly, concentration appeared to be a mediator: The more conﬁdent about
successful outcomes, the higher participants scored on eﬀortless concentration. Interest-
ingly, in Vollmeyer’s and Rheinberg’s model concentration revealed an enhancing eﬀect
on control performance without enhancing knowledge acquisition as well. From this fact
the authors conclude that eﬀortless concentration due to good motivation displays its
beneﬁcial eﬀect only when knowledge actually is applied; no explicit knowledge struc-
tures need to exist prior to application (see Vollmeyer & Rheinberg, p. 20). The third
path-analytical ﬁnding concerns a direct positive of impact conﬁdence of success vs.
fear of failure on knowledge acquisition. Obviously, high conﬁdence of success is associ-
ated with increased knowledge neither as a speciﬁc result of concentration or systematic
strategies, but possibly fostered by some other mediating factors we do not know yet.
Further speciﬁcations as to the interaction between current motivational states and cog-
nition in CPS have been provided in a subsequent study by Rheinberg and colleagues
(Rheinberg, Vollmeyer & Rollett, 2002). Motivational factors of research interest were
probability of success, anxiety, which corresponds about to fear of failure, interest in
the task and challenge, i. e. whether problem solvers feel that a task is a challenge to
them. These four factors formed a global score on initial motivation. Besides motiva-
tion, in this study relevant cognitive ability was examined as measured by the series
completion subtest from the Intelligenz-Struktur-Test (Intelligence-Structure-Test, I-S-
T, Amthauer, 1970)14. For cognitive measures characterising the process of CPS and
its quality the following were chosen: quality of goal achievement, quality of system
identiﬁcation and a score on strategy systematicity.
107 students from the University of Potsdam and from local high schools were selected
to participate. The general procedure of the experiment was comparable to the above
study; the Dynamis scenario applied was an almost identical version of Biology Lab.
The main diﬀerence of Rheinberg’s, Vollmeyer’s and Rollett’s study as compared to
the previous concerns its methodological approach. Instead of applying path analysis
the authors ran cluster analyses, i. e. they tried to gain an insight into the relations
between initial motivation, ability and goal achievement by searching for diﬀerent types
of learners who reveal characteristic patterns of motivation, ability and achievement.
Five such diﬀerent types (groups or clusters) could be distinguished: Firstly, there were
subjects who – in line with theoretical expectations – scored high in goal achievement,
in relevant initial ability and motivation. These were termed the optimal learner type.
The mirror image of the optimal learner type scoring low in all three variables, too, was
found, and named the poor learner type. A third class of subjects proved to be relatively
low in performance despite high initial ability; as subjects’ motivation, however, was low,
performance deﬁcits seem to have a profound explanation. Concerning these learners,
Rheinberg and colleagues speak of the underachiever type.
14In the series completion subtest participants try to detect regularities inherent in a series of numbers
and complete the series according to these regularities. The test focuses on logical numerical abilities.
Less straightforward patterns of motivational and cognitive variables occurred for sub-
jects of the two remaining clusters: Both of these groups showed about average ability
and initial motivation, yet one group performed far worse than the other. What might
explain this fact? As an analysis on strategic operation suggested, the better performers
were those who acted more strategically and systematically. Hence the authors added
a cluster of systematic learners vs. a cluster of unsystematic learners to their typology.
They conclude that obviously both motivational aspects as well as cognitive ability,
strategy systematicity and still additional factors need to be combined when examining
the variances of individual performance in a CPS task. Rheinberg, Vollmeyer and Rollett
also note that clustering subjects according to characteristic patterns provides a useful
description of motivational inﬂuences, but not an explanation in terms of cause-and-
In contrast to relatively ﬂuctuating motivational states goal orientation as deﬁned above
deals with more enduring traits of personality that determine which goals or expected
outcomes make us put eﬀort, time and concentration into learning activities such as a
CPS task. Two “classical” learning goals, sometimes considered as exclusive goals, have
become prominent in psychological research: The goal of enhancing one’s competence
and the goal of demonstrating one’s competence. The former motivational orientation
has been named ego orientation or performance orientation, the latter has been named
task orientation or competence orientation (see Nicholls, 1984; Dweck, 1986, 1989).
More diﬀerentiated goal orientations have been suggested by Niemivierta (2002) who
adds achievement goals, i. e. the goal to succeed, performance-avoidance goals, i. e. the
goal to avoid situations of potential experiences of failure and avoidance goals, i. e. the
goal to get oﬀ with as little eﬀort as possible to the distinction of performance goals and
Certainly, any of these goals can aﬀect a learner’s performance in the domain of CPS.
Which, however, will provide most suitable conditions for successfully coping with a
task? Dweck (1986) as well as Nicholls (1984) suggest that goal orientation consid-
ered on its own has no marked impact on problem solving performance. Task oriented
learners who are particularly interested in the task itself should do generally well when
dealing with a complex problem. Those, however, whose ego is – due to performance
orientation – more involved should rather depend on the level of their abilities as they
themselves perceive it. An ego-oriented learner who is conﬁdent in his or her abilities
will perform just as well as a task-oriented learner of comparable abilities; yet the ego-
oriented learner will prove inferior in performance if belief in his or her capabilities is
poor (see G¨orn et al., 2001).
According to G¨orn, Vollmeyer and Rheinberg (2001), this interaction between motiva-
tional orientation and self-evaluation of abilities had been demonstrated in prior exper-
iments applying diﬀerential instructions to manipulate either learning or performance
goals as experimental factors in between subjects designs. G¨orn and her colleagues
advocated another approach: Since they conceived motivational orientation rather as
a trait of personality than as a transient state to be experimentally induced they ex-
amined the same interaction hypothesis by measuring goal orientations in motivational
113 school students from upper grades, mostly eleventh graders, took part in the ex-
periment. For testing in classrooms the CPS task was administered as a paper and
pencil version of Biology Lab. Pupils had to imagine they were medical researchers
in a laboratory and had to ﬁnd out how three types of medication named A, B, and
C, the exogenous variables, aﬀected each of three body substances, the output variables
named serotonin, thyroxin, and hystamine. Corresponding to six learning phases in the
computer-based scenario six diﬀerent states of the system were presented subsequently
on six sheets of paper. These had to be analysed in order to gain structural knowledge
as measured through causal diagrams. In a ﬁnal equivalent of the application phase sub-
jects calculated how exogenous variables had to be manipulated so that deﬁned outcome
states of the endogenous variables would result. Success in problem solving was analysed
with regard to knowledge application and a score comparable to control performance
in computer-simulated versions. Perceived subjective failure or success was recorded by
brief self-ratings after each of the six sessions. The above hypothesis of an interaction
between motivational orientation and self-evaluation of failure or success could not be
conﬁrmed. The authors assume that this eﬀect is less prominent when goal orientation
is examined as a personality factor than when it is experimentally induced.
A second hypothesis concerned the relation between performance and learning goals:
Does it seem appropriate to assume performance and learning goals as non-overlapping,
exclusive characteristics of learners? Thinking of goal orientation rather as a two-
dimensional construct of personality, G¨orn, Vollmeyer and Rheinberg classiﬁed subjects
according to four categories: task-oriented learners scoring high only on task orienta-
tion, ego-oriented learners scoring high only on ego orientation, high indiﬀerent learners
scoring high on both orientations and low indiﬀerent learners scoring high on neither
orientation. It was predicted that high indiﬀerent learners would be most successful in a
CPS task because two goals, relative to only one, might enhance the beneﬁcial impact of
motivation. Following the same logic, low indiﬀerent learners were expected to perform
worst, i. e. even worse than ego-oriented subjects who care at least for demonstrating
competence as their dominant goal. While high indiﬀerent learners indeed turned out
to perform best, unexpectedly low indiﬀerent learners, too, did well on their task, a
ﬁnding which remains unclear. In spite of this, according to G¨orn and colleagues we
can at least suppose that the bipolar distinction of learners as either task-oriented or
ego-oriented appears too general.
Niemivierta (2002) continued the research on goal orientation in CPS by means of
two further approaches. With respect to subjects’ evaluations during problem solv-
ing Niemivierta analysed not merely self-related evaluations of success or failure and
test anxiety, but also subjects’ situational appraisals. Secondly, he combined the trait-
oriented notion of goal orientations (see G¨orn et al., 2001) with earlier experiments’
practice to induce either task-involving or ego-involving learning conditions. The main
question in Niemivierta’s work is how habitual motivational orientation interacts with
diﬀerent instructional learning conditions with respect to situational appraisals and re-
sulting performance in a CPS task. Examples of situational appraisals are self-eﬃcacy,
i. e. conﬁdence to do well in a task, and claimed self-handicapping, i. e. the use of
anticipatory excuses (like being ill or in a bad mood) when success might be threatened.
Niemivierta examined situational appraisals, goal orientation and, of course, task perfor-
mance (knowledge acquisition and knowledge application) in a sample of one 143 school
students15, ninth graders from junior high schools in southern Finland. The students
dealt with the computer-simulated dynamic system Med Lab, which is nearly iden-
tical to Biology Lab, each under one of two diﬀerent instructional conditions. Half
of the students, the task-involving condition, received task-focused instructions which
requested subjects to work on the problem as hard as they could, so that the current ver-
sion of Med Lab could be evaluated and revised. It was highlighted that the students
would not be tested with regard to individual successful performance. The other half
of the students, the ego-involving condition, received performance-focused instructions;
these obviously focused on individual performance as students were told that a few days
later, their teacher would announce the results.
It was hypothesised that subjects in the latter condition as compared to those receiv-
ing task-involving instructions would reveal higher levels of test anxiety, lower levels
of self-eﬃcacy and more claimed self-handicapping, especially if the students had been
classiﬁed as habitually performance-oriented learners. Furthermore, performance in the
Med Lab task was expected to be lower with participants of the ego-involving condition.
Disconﬁrming the last expectation, levels of task performance did not diﬀer signiﬁcantly
among the two instructional conditions. Neither did test anxiety seem to be aﬀected by
ego- vs. task-involving instructions; as one would guess, test anxiety was generally high-
est for participants reporting performance-avoidance goals, the goal to avoid situations
of potential failure. Yet with regard to situational appraisals Niemivierta’s assumptions
were supported: Ego-involving instructions obviously led to lower reported self-eﬃcacy
or lower expected success, which may be interpreted as a self-protecting function in
the ego-involving and possibly ego-threatening instructional context. While subjects
with diﬀerent patterns of goal orientations seemed to be comparable as to their overall
situational appraisals, in the ego-involving condition more self-handicapping and less
self-eﬃcacy was reported when learners’ scored high on ego orientation as a personality
trait relative to habitually task-oriented students. Once again, the impact of diﬀerent
goal orientations and multiple eﬀects in the context of CPS are illustrated.
Last but not least, our ﬁnal section concerns the role of intelligence in CPS tasks. In
research literature this issue has been discussed in rather controversial disputations. As
Funke (2003) states, in early research, test intelligence has been regarded and examined
as the major inﬂuencing factor on problem solving. Yet initial ﬁndings seemed to con-
tradict both common sense and scientiﬁc hypotheses. An investigation by Putz-Osterloh
(1981) applying tailor shop, a popular, non-Dynamis-based computer simulation, even
suggested that no systematic relations at all existed between test intelligence and control
performance in tailor shop. In fact, a tendency towards a negative relation occurred, i. e.
participants scoring high in a traditional intelligence test did worse in controlling the
complex scenario. Less striking, a following experiment (Putz-Osterloh & L¨uer, 1981)
revealed zero correlations between control performance and intelligence only if no addi-
tional information had been given to the problem solvers. In case of transparency, i. e.
if the system’s internal relations had been disclosed to subjects by means of a structural
15To later evaluation only the data of 100 subjects proved to be valid.
diagram, the authors found signiﬁcantly positive correlations between the intelligence
score and the quality of system control.
Hence the more intelligent the problem solver the more successful he or she will interact
with a dynamic system, but this eﬀect is mediated by transparency conditions. Why
should this make sense? Following the author’s interpretation working on a transparent
dynamic system requires systematic analysis of given information prior to its application,
a task demand which is relatively similar to the demands of traditional intelligence diag-
nostics. In case of intransparency – a typical feature of complex problems – participants
actively search for task-relevant information, a demand not covered in intelligence tests.
As self-guided search for information and analysis of given information require diﬀerent
abilities, success in a standard problem solving paradigm and conventional intelligence
tasks may be unrelated (for summaries of the ﬁndings see also Funke, 2003).
In consequence, diﬀerential task demands and required abilities concerning these two
domains have increasingly been the focus of attention. Presenting his concept of “oper-
ative intelligence” D¨orner (1986) has highlighted the dynamic orientation as the main
task feature speciﬁc to CPS. Unlike in static intelligence tests which cover participants’
speed and accuracy abilities, working on complex dynamic systems additionally involves
skills such as anticipation and consideration of temporal processes, setting up appropri-
ate goals and subgoals (and occasionally modifying them in the course of the process),
using successful and adaptive strategies to search for relevant information. Ecological
validity, i. e. the proximity to “real” problems in the outside world is said to be much
higher for complex problem computer simulations as compared to traditional intelligence
On the other hand, at least some connections between problem solving and test intel-
ligence cannot be denied. Starting with an experiment by Funke (1983), the view that
scores in intelligence tests can predict the level of control performance has become re-
established. According to Beckmann and Guthke (1995) positive correlations between
intelligence and achievement in problem solving could be revealed consistently (p. 178).
Yet correlations appear too low to infer a direct impact of intelligence on the process of
problem solving. Following D¨orner’s approach, Beckmann and Guthke assume a multi-
tude of inﬂuential factors that mediate the eﬀects of test intelligence. Transparency is
one such factor (see above), and the same applies, e.g, to task diﬃculty or goal deﬁni-
tion. Highest correlations between intelligence scores and control performance in CPS
can be expected with problems of average diﬃculty (Beckmann & Guthke, p. 184).
Concerning goal deﬁnition, well-deﬁned goals in a problem solving situation make the
demands more similar to demands of traditional intelligence tests than ill-deﬁned goals
(e. g, Strohschneider, 1991). As further examples adding to this list, potentially mediat-
ing eﬀects of semantic embedding and learning ability were analysed in an experiment
done by Beckmann and Guthke.
The researchers tested 92 school students successively on a CPS task (ﬁrst session), two
conventional intelligence tests (second session) and two comparable short-term learning
tests (third and fourth session). Experimental manipulation as to semantic embedding
was realised by assigning half of the pupils to a semantically embedded Dynamis system
named Kirschbaum while the other half worked on Maschine, a Dynamis system
identical in structure and parameters, but lacking semantic embedding. The measures
of academic intelligence comprised a subtest on analogies taken from the Amthauer
Intelligenz-Struktur-Test (IST, Amthauer, 1973) and a reasoning task (“ﬁgure series”),
the pretest of the Lerntest “Schlussfolgerndes Denken” (LTS 3, Guthke et al., 1983).
The results conﬁrmed the authors’ expectation that semantic embedding mediates the
inﬂuence of test intelligence on control performance: Signiﬁcant correlations between
these two measures were found only if the complex problem situation was abstract. In
case of semantic embedding intelligence scores could not predict control performance
(for an equivalent ﬁnding see also M¨uller, 1993). This, however, obviously did not result
from semantic embedding per se, but from generally impaired knowledge acquisition in
the semantic context. As noted in the section about semantic embedding, semantics
can lead to “sham conﬁdences” which prevent participants from gaining real and useful
knowledge. In turn, if hardly any knowledge is acquired by both intelligent and less
intelligent subjects, except for ad hoc control there is no reason why anyone should be
superior in knowledge application, i. e. in control performance.
The authors’ second question aimed at learning ability and the comparability of com-
plex problem situations and learning tests. The idea behind learning tests is to “record
not only a subject’s momentary performance in a one-time administration of a test pro-
cedure, as is done in the usual static intelligence test situation, but also the subject’s
responses to repeated, standardised questions that are built into the test” (Beckmann
& Guthke, 1995, p. 186). Hence, the concept of a learning test appears to share
some dynamic components with CPS tasks, although in learning tests more detailed
and instantaneous feedback is given to participants. The two learning tests chosen for
Beckmann’s and Guthke’s experiment were said to cover learning abilities that corre-
sponded to the intellectual abilities measured by the two conventional intelligence tests.
It was hypothesised that learning tests, sharing the dynamic feature with CPS, are bet-
ter instruments of predicting knowledge acquisition in a complex situation than static
intelligence tests would be. No eﬀect was expected with regard to control performance.
In general, statistical analyses conﬁrmed the expectations. Only in case of seman-
tic embedding learning tests and static intelligence tests were equally good or equally
bad predictors of the amount of knowledge gained since this amount of knowledge was
generally poor (see above). Beckmann and Guthke conclude that systematic relations
between academic intelligence and performance in complex problem situations deﬁnitely
exist. Zero correlations as reported in early research might have been methodological
artefacts, e. g, due to impaired and omitted knowledge acquisition, the authors suggest.
Existing correlations yet become more prominent if mediating factors such as semantic
embedding or learning ability are revealed.
More recently, Kr¨oner has studied the correlations between problem solving performance
and test intelligence from a practical point of view. His aim has been to develop and
validate a Dynamis system suitable to serve as a tool of intelligence diagnostics. Origi-
nal and revised versions of MultiFlux, the graphically embedded computer simulation
of a ﬁctive machine, provided the base of three experimental studies. Control perfor-
mance and knowledge acquisition in MultiFlux was compared to achievement in the
Advanced Progressive Matrices (APM; Raven, 1958), a conventional intelligence test
which demands logical reasoning in completing graphical patterns.
The ﬁrst study, a pilot study involving a sample of 28 students, on the whole supported
Kr¨oner’s major hypotheses: Intelligent subjects performed better on the MultiFlux
control task. They also tended to be superior in skilful exploration although, in part,
(indeﬁnite) factors other than test intelligence seemed to contribute to exploration skills.
To reduce the inﬂuence of such unknown sources of variation and to rise the impact of
intelligence on exploration in dynamic system the second experiment was conducted. 96
high school students worked on a revised version of MultiFlux. Indeed correlations
between good exploration of MultiFlux and APM scores were found to be higher
. The third study employing a further revised scenario ﬁnally was designed to show
that the enhanced control performance of intelligent subjects is not merely the result
of better knowledge acquisition (due to more skilful exploration); instead there should
be a an additional direct and positive eﬀect of test intelligence on control performance.
The data of a 101 pupils (grades ten to twelve) supported this claim. Kr¨oner resumes
that close relations exist between achievement in a conventional intelligence test and
the quality of dealing with a complex system, the correlations being high, though not
maximal. The indicators of test reliability are considered satisfactory so that, according
to Kr¨oner, MultiFlux might provide a simulation-based tool of intelligence diagnostics
in the future.
Amthauer, R. (1970). I-S-T 70. Intelligenz-Struktur-Test [I-S-T 70. Intelligence-
Structure-Test]. G¨ottingen: Hogrefe.
Amthauer, R. (1973). IST-70. Intelligenz-Struktur-Test [Test of the structure of intelli-
gence]. G¨ottingen: Hogrefe.
Anderson, J. R. (1987). Skill acquisition: Compilation of weak-method problem solu-
tions. Psychological Review, 94, 192-210.
Ashby, W. R. (1958). Requisite variety and its implications for control of complex
systems. Cybernetica, 1, 83-99.
Beckmann, J. F. (1994). Lernen und komplexes Probleml¨osen. Ein Beitrag zur Kon-
struktvalidierung von Lerntests [Learning and complex problem solving. A contribution
to the validation of learning tests]. Bonn: Holos.
Beckmann, J. F. & Guthke, J. (1995). Complex problem solving, intelligence and learn-
ing ability. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European
perspective (pp. 177-200). Hillsdale, NJ: Lawrence Erlbaum Associates.
Berry, D. C. & Broadbent, D. E. (1984). On the relationship between task performance
and associated verbalizable knowledge. Quarterly Journal of Experimental Psychology,
Branke, S. (1991). Die Interaktion von Personen- und Aufgabenmerkmalen beim kom-
plexen Probleml¨osen [The interaction of person and task characteristics in complex prob-
lem solving]. Report, Psychologisches Institut der Universit¨at Mainz, Mainz.
Brehmer, B., & D¨orner, D. (1993). Experiments with computer-simulated microworlds:
Escaping both the narrow straits of the laboratory and the deep blue sea of the ﬁeld
study. Computers in Human Behavior, 9, 171-184.
Dienes, Z. & Fahey, R. (1995). Role of speciﬁc instances in controlling a dynamic system.
Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 848-862.
D¨orner, D. (1986). Diagnostik der operativen Intelligenz [Diagnosis of operative intelli-
gence]. Diagnostica, 32, 290-308.
D¨orner, D. (1987). On the diﬃculties people have in dealing with complexity. In J.
Rasmussen, K. Duncan, & J. Leplat (Eds.), New technology and human error (pp. 97-
109). Chichester: Wiley.
Dweck, C. S. (1986). Motivational processes aﬀecting learning. American Psychologist,
Dweck, C. S. (1989). Motivation. In A. Lesgold & R. Glaser (Eds.), Foundations for a
psychology of education (pp. 85-136). Hillsdale, NJ: Erlbaum.
Frensch, P. A. & Funke, J. (Eds.). (1995). Complex problem solving: The European
perspective. Hillsdale, NJ: Erlbaum.
Funke, J. (1983). Einige Bemerkungen zu Problemen der Probleml¨oseforschung oder: Ist
Testintelligenz doch ein Pr¨adiktor [Some comments on the problems in problem-solving
research or is test intelligence still a predictor]? Diagnostica, 29, 283-302.
Funke, J. (1985). Steuerung dynamischer Systeme durch Aufbau und Anwendung sub-
jektiver Kausalmodelle [Control of dynamic systems by building up and using subjective
causal models]. Zeitschrift f¨ur Psychologie, 193, 435-457.
Funke, J. (1992a). Dealing with dynamic systems: Research strategy, diagnostic ap-
proach and experimental results. German Journal of Psychology, 16(1), 24-43.
Funke, J. (1992b). Wissen ¨uber dynamische Systeme: Erwerb, Repr¨asentation und
Anwendung [Knowledge about systems: Acquistion, representation, and application].
Funke, J. (1993). Microworlds based on linear equation systems: A new approach
to complex problem solving and experimental results. In G. Strube & K.-F. Wender
(Eds.), The cognitive psychology of knowledge (pp. 313-330). Amsterdam: Elsevier
Funke, J. (2003). Probleml¨osendes Denken [Problem-solving thinking]. Stuttgart: Kohl-
Funke, J. & M¨uller, H. (1988). Eingreifen und Prognostizieren als Determinanten von
Systemidentiﬁkation und Systemsteuerung [Intervention and prediction as determinants
of system identiﬁcation and system control]. Sprache & Kognition, 7, 176-186.
Gardner, P. H., & Berry, D. C. (1995). The eﬀect of diﬀerent forms of advice on the
control of a simulated complex system. Applied Cognitive Psychology, 9 (Special Issue),
G¨orn, A., Vollmeyer, R., & Rheinberg, F. (2001). Eﬀects of motivational orientation
and perceived ability on performance. Psychologische Beitr¨age, 43, 153-171.
Goshen-Gottstein, Y. & Moscovitch, M. (1995). Repetition priming for newly formed
and preexisting associations: Perceptual and conceptual inﬂuences. Journal of Experi-
mental Psychology: Learning, Memory, and Cognition, 21, 1229-1248.
Guthke, J., J¨ager, C., & Schmidt, I. (1983). Lerntest “Schlussfolgerndes Denken” (LTS)
[Learning test “Reasoning”]. Berlin: Psychodiagnostisches Zentrum der Humboldt-
H¨ubner, R. (1987). Eine naheliegende Fehleinsch¨atzung des Zielabstandes bei der zeitop-
timalen Regelung dynamischer Systeme [A common misperception of goal distance when
controlling dynamic systems in optimal time]. Zeitschrift f¨ur Experimentelle und Ange-
wandte Psychologie, 34, 38-53.
H¨ubner, R. (1988). Die kognitive Regelung dynamischer Systeme und der Einﬂuss
analoger versus digitaler Informationsdarbietung [Cognitive regulation of dynamic sys-
tems and its eﬀects on analog versus digital information presentation]. Zeitschrift f¨ur
Psychologie, 196, 161-170.
Klahr, D. & Dunbar, K. (1988). Dual space search during scientiﬁc reasoning. Cognitive
Science, 12, 1-55.
Kluge, A. (2003). Probleml¨oseleistungen und Szenarioschwierigkeit: ¨
werb f¨ur das Steuern technischer Systeme. Unpublished Habilitationsschrift, RWTH,
Kluwe, R. H. (1988). Methoden der Psychologie zur Gewinnung von Daten ¨uber men-
schliches Wissen [Psychological methods for collecting data on human knowledge]. In
H. Mandl & H. Spada (Eds.), Wissenspsychologie (pp. 359-385). M¨unchen: Psychologie
Kr¨oner, S. (2001). Intelligenzdiagnostik per Computersimulation [Measuring intelligence
by means of computer simulations]. M¨unster: Waxmann.
Moray, N., Lootsteen, P., & Pajak, J. (1986). Acquisition of process control skills. IEEE
Transactions on Systems, Man, and Cybernetics, SMC-16, 497-504.
M¨uller, H. (1993). Komplexes Probleml¨osen: Reliabilit¨at und Wissen [Complex problem
solving: Reliability and knowledge]. Bonn: Holos.
Nicholls, J. G. (1984). Achievement Motivation: Concepts in ability, subjective experi-
ence, task choice, and performance. Psychological Review, 91, 328-346.
Niemivirta, M. (2002). Motivation and performance in context: The inﬂuence of goal
orientation and instructional setting on situational appraisals and task performance.
Psychologia, 45, 250-270.
Preußler, W. (1996). Zur Rolle expliziten und impliziten Wissens bei der Steuerung
dynamischer Systeme [On the role of explicit and implicit knowledge in controlling
dynamic systems]. Zeitschrift f¨ur Experimentelle Psychologie, 43, 399-434.
Preußler, W. (1997). Eﬀekte des Kontexts auf den Wissenserwerb bei der Steuerung
eines dynamischen Systems [Eﬀects of context on the acquisition of knowledge in con-
trolling a dynamic system]. Sprache & Kognition, 16, 48-59.
Preußler, W. (1998). Strukturwissen als Voraussetzung f¨ur die Steuerung komplexer
dynamischer Systeme [Structural knowledge as a precondition of controlling complex
dynamic systems]. Zeitschrift f¨ur Experimentelle Psychologie, 45, 218-240.
Preußler, W. (2001). Probleml¨oseerfolg als Bedingung f¨ur den Erwerb von Eingriﬀswis-
sen bei der Systemsteuerung [Successful problem solving as a precondition for acquisition
of control knowledge]. Zeitschrift f¨ur Experimentelle Psychologie, 48, 3, 214-225
Putz-Osterloh, W. (1981). ¨
Uber die Beziehung zwischen Testintelligenz und Probleml¨o-
seerfolg [The relation between test intelligence and problem solving success]. Zeitschrift
f¨ur Psychologie, 189, 79-100.
Putz-Osterloh, W. (1993a). Strategies for knowledge acquisition and transfer of knowl-
edge in dynamic tasks. In G. Strube & K.-F. Wender (Eds.), The cognitive psychology
of knowledge (pp. 331-350). Amsterdam: Elsevier Science Publishers.
Putz-Osterloh, W. (1993b). Unterschiede im Erwerb und in der Reichweite des Wissens
bei der Steuerung eines dynamischen Systems [Diﬀerences in the acquisition and extent
of knowledge during system control]. Zeitschrift f¨ur Experimentelle und angewandte
Psychologie, 40, 386-410.
Putz-Osterloh, W. & L¨uer, G. (1981). ¨
Uber die Vorhersagbarkeit komplexer Problem-
l¨oseleistungen durch Ergebnisse in einem Intelligenztest [On the predictability of the
performance in COMPLEX PROBLEM SOLVING by results from an intelligence test].
Zeitschrift f¨ur Experimentelle und Angewandte Psychologie, 28, 309-334.
Raven, J. C. (1958). Advanced progressive matrices (2nd Ed.). London: Lewis.
Reither, F. (1981). About thinking and acting of experts in complex situations. Simu-
lation & Games, 12, 125-140.
Rigas, G. & Brehmer, B. (1999). Mental processes in intelligence tests and dynamic
decision making tasks. In P. Juslin & H. Montgomery (Eds.), Judgement and decision
making: Neo-Brunswikean and process-tracing approaches (pp. 45- 65). Hillsdale, NJ:
Rollett, W. (2003). Strategieeinsatz, Informationsnutzung und Motivation beim Explo-
rieren und Steuern komplexer Systeme. Unpublished Dissertation, Humanwissenschaft-
liche Fakult¨at der Universit¨at Potsdam, Potsdam.
Schoppek, W. (2002). Examples, rules, and strategies in the control of dynamic systems.
Cognitive Science Quarterly, 2, 63-92.
Schulz, C. A. (2003). Strategien zum komplexen Probleml¨osen. Eine experimentelle
Analyse der Rolle von Analogien beim Wissenserwerb in komplexen, dynamischen Sys-
temen [Strategies for complex problem solving : an experimental analysis of the role
of analogies for knowledge acquisition in complex, dynamic systems]. Unpublished
Diplomarbeit, Psychologisches Institut der Universit¨at, Heidelberg. [online available
Sternberg, R. J. & Frensch, P. A. (Eds.)(1991). Complex problem solving: Principles
and mechanisms. Hillsdale, NJ: Erlbaum.
Stevens, J. (1992). Applied multivariate statistics for the social sciences. Second edition.
Hillsdale, NJ: Erlbaum.
Steyer, R., Schmitt, M., & Eid, M. (1999). Latent state-trait theory and research in
personality and individual diﬀerences. European Journal of Personality, 13, 389-408.
Strauß, B. (1993). Konfundierungen beim komplexen Probleml¨osen [Confundations in
complex problem-solving: The inﬂuence of the ratio of correct solutions on problem-
solving in complex situations]. Bonn: Holos.
Strohschneider, S. (1991). Probleml¨osen und Intelligenz: ¨
Uber die Eﬀekte der Konkre-
tisierung komplexer Probleme [Problem solving and intelligence: The eﬀects of problem
concreteness]. Diagnostica, 37, 353-371.
Sweller, J. (1988). Cognitive load during problem solving: Eﬀects on learning. Cognitive
science, 12, 257-285.
Vollmeyer, R. & Burns, B. D. (1996). Hypotheseninstruktion und Zielspeziﬁt¨at: Be-
dingungen, die das Erlernen und Kontrollieren eines komplexen Systems beeinﬂussen
[Hypothesis instruction and goal speciﬁcity: Factors inﬂuencing the learning and con-
trol of complex systems]. Zeitschrift f¨ur Experimentelle Psychologie, 43, 657-683.
Vollmeyer, R., Burns, B. D., & Holyoak, K. J. (1996). The impact of goal speciﬁcity on
strategy use and the acquisition of problem structure. Cognitive Science, 20, 75-100.
Vollmeyer, R. & Funke, J. (1999). Personen- und Aufgabenmerkmale beim komplexen
Probleml¨osen [Person and task eﬀects within complex problem solving]. Psychologische
Rundschau, 50, 213-219.
Vollmeyer, R. & Rheinberg, F. (1998). Motivationale Einﬂ¨usse auf Erwerb und Anwen-
dung von Wissen in einem computersimulierten System [Motivational inﬂuences on the
acquisition and application of knowledge in a computer-simulated system]. Zeitschrift
f¨ur P¨adagogische Psychologie, 12, 11-23.