Int. J. Cont. Engineering Education and Lifelong Learning, Vol. 16, Nos. 3/4, 2006
Copyright © 2006 Inderscience Enterprises Ltd.
WWW-intensive concept mapping for metacognition
in solving ill-structured problems
Open University of the Netherlands, OTEC
P.O. Box 2960, 6401 DL Heerlen, NL
University of Twente
Faculty of Behavioural Science
P.O. Box 217, 7500 AE Enschede, NL
Abstract: Concept mapping is one of the most intimate and most dynamic
learning support activities that needs still a drastic further evolution of methods
and tools. WWW-based concept mapping gains momentum quite fast now and
needs a solid reviewing of the various approaches and the empirical effects.
This article bridges the technological advance of WWW-based concept
mapping tools and its more recent effects on learning by problem solving. The
results show that it added value manifests in the phases of idea generation and
selection. The mapping approach caused a broader perception and a greater
diversity of ideas. The conclusion is that further investments are needed to
make WWW-based mapping more accessible and integrated in WWW-based
learning management systems.
Keywords: WWW-based learning tools; concept mapping; problem solving;
Reference to this paper should be made as follows: Stoyanov, S. and
Kommers, P. (2006) ‘WWW-intensive concept mapping for metacognition in
solving ill-structured problems’, Int. J. Cont. Engineering Education and
Lifelong Learning, Vol. 16, Nos. 3/4, pp.297–316.
Biographical notes: Slavi Stoyanov is with Educational Technology Expertise
Centre at Open University of The Netherlands. He has a PhD degree on
Instructional Technology from the University of Twente, The Netherlands. The
professional interests of Stoyanov include domains such as cognitive mapping,
learning to solve ill-structured problems, individual differences in learning, and
Piet Kommers is Associate Professor at Twente University and Lector at
Fontys Professional University in The Netherlands. His interest and prior
research is conceptual learning tools, virtual reality for training and recently
mobile learning. His dominant paradigm is that media provoke alternative
learning modes and it is now the task for schools to learn themselves.
S. Stoyanov and P. Kommers
Concept mapping is widely recognised as strategic method for articulating prior
knowledge and stimulate metacognition in the learner. A growing number of conceptual
representation tools become available via the WWW lately and it stimulates
educationalists to see its pragmatic benefits for collaborative distance learning.
1 In the first place there is the family of mapping tools that derive configurational
schemes from the hyperlinks as if the user already crawls through the many pages.
Instead of the browsing via local hotspots, it is the mapping device that presents a
wider landscape so that the learner can take his decisions at a more global meta
The first example of resource-based concept mapping is TouchGraph1 that
allows learners to trace the more dominant links (in terms of frequency) that
have been created in a certain domain.
A second example of it is the WWW-based tool KartOO.2 It is a meta search
engine with visual display interfaces. When you click on OK, KartOO launches
the query to a set of search engines, gathers the results, compiles them and
represents them in a series of interactive maps through a proprietary algorithm.
Given ‘Concept Map’ as root, it will bring the key arena of sites that elaborate
further on Concept Mapping.
2 In the second place there are the WWW-based concept mapping tools that allow
users to actually build momentary conceptual fields in order to share common or
adjacent conceptual frameworks with co-learners at a distance. Examples of these are
‘Brain’: Dynamic Mind Mapping with Personal Brain.3 It allows the learner to
express the momentary interest and map it unto the vast constellation of existing
relations defined by other experts.
The operational support to learning for solving ill-structured problems is considered as
one of the most demanding tasks of the contemporary instructional design paradigm
(Jonassen, 2004). It means introduction to specific techniques that facilitate learners to
construct appropriate solutions in situations which are characterised with insufficient and
sometimes vague information, existence of alternative and often conflicting approaches,
lack of clear-cut problem-solving procedures and no agreement upon what can be
accepted as an appropriate solution (Jonassen, 2004; Schön, 1996). Some of the
techniques that have been discussed are concept map, causal modelling, influence
diagrams, expert system shell, modelling dynamic systems (Jonassen, 2004), cognitive
flexibility hypertext (Spiro and Jehng, 1990), and questions prompts (Ge and Land, 2004;
King, 1991). Concept map is recognised potentially as a powerful problem-solving tool
but the discussion on the role of the technique in solving ill-structured problems still has
to address a number of substantial questions (Jonassen, 2004; Stoyanov, 2001). Some of
WWW-intensive concept mapping for metacognition 299
What are the characteristics of concept map that make this technique an effective and
efficient problem-solving tool?
What are the differences and similarities of concept map compared to other forms of
cognitive mapping techniques?
Are there any individual differences that can affect the concept mapping outputs in a
situation of problem-solving?
What is the role of the instruction in constructing a concept map as problem-solving
An attempt of providing some insights in relation to these questions is given in
the following sections. Figure 1 presents a concept map on the content that is a subject of
Figure 1 Concept map on concept mapping instruction
S. Stoyanov and P. Kommers
This study presents the empirical validation of the role of concept mapping instruction for
learning to solve ill-structured problems. The results of the study were operationalised as
design solutions implemented in a web-based application, called SMILE Maker
(Stoyanov, 2001). The tool stands for solution, mapping and intelligent learning
environment. The software guides the learner through the process of problem solving as
facilitating her/him in constructing different types of concept maps: map information
collection, map idea generation, map idea selection and map idea implementation. For
each of these types of maps, specific problem solving heuristics and techniques are
proposed, which support the cognitive processes of knowledge elicitation, knowledge
representation, knowledge reflection, and knowledge creation. The research presented
here focuses on the experimental verification of the concept mapping method, applied in
SMILE, but the investigation of learning and problem solving effectiveness of the
software application falls behind the scope of this paper.
2 Concept mapping as a problem-solving tool
Concept map is defined as a graphical technique to represent the conceptual organisation
of a particular subject domain and to grasp the perception of learners on this knowledge
structure (Huai and Kommers, 2004; Jonassen et al., 1998; Kommers and Lanzing, 1998;
Novak, 1998). The terms of concept map and concept mapping have been used
interchangeably. In this study, concept map refers to the result of the concept mapping
process. Concept mapping technique has been used mostly as a graphical advanced
organiser (Novak and Gowin, 1984; Novak, 1998) and assessment technique
(Constantinou, 2004; Fernández et al., 2004; Novak, 1998; Weber, 2004; Willerman and
Harg, 1991) from teachers, and as a learning aid for students to organise their declarative
and structural knowledge (Gulmans, 2004; Jonassen et al., 1993; Lumer and Hesse, 2004;
Lumer and Ohly, 2004; Novak, 1998). There are relatively few reports on the role of
concept mapping as a tool that supports learning for solving ill-structured problems
(Jonassen, 2004; Stoyanov, 2001). Jonassen (2004) defines concept map as a tool for
representing the semantic organisation of problems by learners. He finds important for
instructors to encourage students to make semantic concept maps before learners begin to
Most of the definitions of concept mapping describe the techniques as a knowledge
representation tool (Gulmans, 2004; Huai and Kommers, 2004; Jonassen et al., 1998;
Kenedy and McNaught, 1998; Kommers and Lanzing, 1998; Reimann, 1999; Sherry and
Trigg, 1996). As a knowledge representation tool, concept mapping has some features
that could make it a powerful problem-solving tool (Stoyanov, 2001). The technique is an
adequate, flexible and intuitive way of externalising the mental model of problem solver.
Concept map shows the pattern of knowledge items arranged in the problem space as
applying a simple graphical format: nodes represent thoughts and labelled links express
their interrelationships. Concept map is a concise, compact and parsimonious technique,
which is at the same time rich in information, because of the integration of verbal and
visual coding. The technique capitalises on the advantages of graphical representations,
without losing the flexibility and richness of the natural language system. Concept map
can expresses a variety of problem-solving representations (facts, analogies, feelings) and
a variety of relationships between them (descriptive, structural, causal, metaphorical).
WWW-intensive concept mapping for metacognition 301
The definition of concept map as a knowledge representation tool reflects only one
aspect of psychological conditions involved in problem-solving. Some of the issues
attributed to these conditions are restricted processing and high cognitive load due to
the limited capacity of working memory, difficulties with searching and retrieval in
long-term memory structure, and changing the dominant thinking patterns. These
limitations of cognitive system provoke some negative problem-solving effects such as
functional fixedness (Duncker, 1945, cited in Eysenck and Keane, 2000; Wertheimer,
1987), problem set (Luchins and Luchins, 1991, cited in Eysenck and Keane, 2000;
Wertheimer, 1987) and analysis paralysis (Kaufmann, 2001; von Wodtke, 1993). What
makes concept mapping a powerful problem-solving technique is that it is not only
knowledge representation tool, but it has also a potential to be knowledge elicitation,
knowledge reflection and knowledge changing tool if appropriate instruction is provided
Concept mapping as knowledge elicitation tool supports the access to and the search
in the long-term memory structures. It allows a quick recognition and retrieval of the
available knowledge because of the isomorphic correspondence between map and
cognitive structures (Eysenck and Keane, 2000; Wandersee, 1990). Concepts and labels
can act as cues for guiding the search through problem-solving space. In addition the
visual presentation of mental patterns makes easier pattern recognition. Recognition is a
faster cognitive process than retrieval, but also contributes to a more effective retrieval
(Eysenck and Keane, 2000). Ill-structured situations not always require development of a
completely new solution. The most appropriate solution for a given problem-solving
situation may already exist. The question however is to find it as performing broad and
deep searching and checking many alternatives.
As knowledge reflection tool concept mapping supports the cognitive processes
related to effective functioning of working memory. Concept map is a cognitive
artifact that allows problem solver to look at his/her mental model and to reflect on the
outcomes of thinking. Apart from this support to reflection-on individual cognitive
reality, as a result, concept mapping supports reflection-in-action (Schön, 1996) of
cognitive processes as well. The nature of ill-structured problem-solving situations
make difficult for people to look inside and control their thinking processes. Two parallel
processes are always running – thinking about the problem itself and thinking about
thinking on the problem. Concept mapping can be beneficial in such sort of situations as
externalising the internal problem-solving patterns and stimulating the self-appraisal
metacognitive functions. In this way it enhances the internal locus of control on
Concept map has a potential to reduce the cognitive load, which is a recognised threat
for a successful learning (Sweller et al., 1998). As a cognitive artifact, concept map is an
external extension of working memory. It makes possible for problem solver to grasp
complex interactions among thoughts that could otherwise exceed problem solver
cognitive capacity. The externalisation of mental problem-solving representations
involves effectively perception, which amplifies the performance of memory and
thinking. It frees up cognitive resources necessary for memory and thinking.
As knowledge changing tool, concept map has a potential to change the dominant
thinking pattern and to create a new one (if and when needed) that reflects better the
problem-solving situation. The technique allows manipulation of problem-solving
representations. As a cultural artifact, concept map mediates the interaction of a problem
S. Stoyanov and P. Kommers
solver with the objective problem-solving situation (Vygotsky, 1978). It is a sort of
‘transitional object’ (Eden and Ackerman, 2002; Lane, 1997) representing mental
models, which problem solver can play with. The position of knowledge items and spatial
configuration can be changed purposely in order to see different perspectives and to
explore new possibilities. This leads to generation of new ideas. Working upon a concept
map we are building upon our cognitive structures. While improving, modifying or
changing completely the external model of a thinking pattern, we are improving,
modifying or changing this pattern effectively. The potential of concept mapping to
change the problem-solving patterns in ill-structured situations is a distinguishing
characteristic of this technique.
2.1 Concept map as a part of the cognitive mapping paradigm
Concept mapping is a member of the cognitive mapping family, which includes among
other causal mapping (Eden and Ackerman, 2002), dynamic mapping (Vennix, 1997),
mind mapping (Buzan and Buzan, 1996), and hexagon mapping (Hodgson, 1999). The
discussion on the role of concept mapping in problem solving does not provide sufficient
information about the specific characteristics of concept mapping that make it a more
appropriate problem-solving tool than the other cognitive mapping formats (Jonassen,
2004). A comparative analysis of the theoretical background, procedures and software of
different mapping approaches is given elsewhere (Stoyanov, 2001). For the purposes of
this study we provide just a short description of these characteristics of concept mapping
as a problem-solving tool.
Although referring to different theories, all of the referred cognitive mapping
approaches use a non-linear spatial format as an explicit metaphor to represent the way
knowledge items are interconnected. The theories behind the different mapping strategies
provide empirical evidence that human mind stores and organise information in a map
format. There are however some substantial differences in the way the mapping
approaches represent cognitive constructs involved in problem-solving. Concept map is
the only mapping technique that allows different formats of spatial organisation of ideas.
It can be either hierarchy or network, as network itself opens many possibilities. Causal
mapping directly suggests a hierarchical structure, while mind mapping imposes a
hierarchy without an explicit reference to this type of organisation. Causal mapping and
dynamic mapping apply mostly unlabelled causal links. The graphical organisation of
mind mapping suggests structural links. The relationships can be other than structural, but
they are not explicit and the reader has to make inferences about their type. Hexagon
mapping does not use links at all. Concept map is the only technique that provides
opportunity for applying any sort of labelled idiosyncratic links. The graphical
conventions of concept mapping make the technique the most flexible and expressive
mapping technique (Alpert, 2004; Heeren and Kommers, 1992).
2.2 Mapping software
Most of the mapping approaches are implemented in specific software applications:
Inspiration® (2003) – concept mapping, Mind Manager® (2004) – mind mapping,
Decision Explorer® (2003) – causal mapping, STELLA 7.0, (2000) – dynamic mapping,
and Idons-For-Thinking 2.0 (1999) – hexagon mapping). They provide explicit support
for only the graphical conventions of a particular mapping approach but not for how a
particular cognitive mapping method can be used for an effective problem solving. It is
assumed that the function of cognitive mapping as a problem solving tool is self-evident,
it is given by affordance and it is embedded within the graphical functions. Mapping
software support mostly knowledge representation features of mapping approaches, but
do not support knowledge elicitation, knowledge reflection, and knowledge creation
functions. The mapping software applications serve mostly as graphical tools but not as
cognitive tools for problem solving.
WWW-intensive concept mapping for metacognition 303
2.3 Concept mapping and individual differences
The cognitive structures and processes modelled by a concept map have individual
dimensions. It should be expected that a concept map reflects the characteristics of the
individual constructs involved in problem-solving (Huai and Kommers, 2004; Jonassen,
2004; Oughton and Reed, 2000; Stoyanov, 2001). We should be able to identify these
individual constructs’ features in the specific graphical organisation of the concept map
elements. The taxonomy of individual differences consists of constructs such as level of
knowledge, cognitive styles, learning styles, personality traits to list but a few. Important
issues related to the individual constructs are their large number and diversity,
multi-layers structure, and instability over time, space and task. The reported issues can
be managed for research and design purposes through a selection of the stylistic
preferences as a representative for individual differences. We use the term ‘stylistic
preferences’ referring to either learning style or cognitive style. Stylistic preferences are
an integrative construct including abilities and personality dimensions. Stylistic
preferences play an intermediate role between abilities and behaviour.
The research on relationships between concept mapping and styles returns some
inconsistent data. Ayersman and von Minden (1995) reported no significant difference
among Kolb’s learning styles of diverger, assimilator, converger and accommodator
(Kolb, 1998) in relation to hypermedia knowledge. In contrast, Oughton and Reed (2000)
found an interaction effect between Kolb’s learning styles and level of prior hypermedia
knowledge on several features of concept mapping production such as number of
concepts, number of links, level of depths, preserved concepts, omitted concepts, and
added concepts. Assimilators and divergers were the most productive on their maps and
had the deepest level of processing on their maps. Huai and Kommers (2004) measured
the effect of cognitive styles, knowledge dimensions and concept mapping approaches on
learning achievements. The cognitive styles are holist, serialist, un-known styles and
versatiles (Pask, 1988). Knowledge dimensions were defined as declarative and
procedural knowledge. Two concept mapping approaches were developed: globalisic and
specialistic. The gobalistic approach was designed to match the holistic dimension of
cognitive style. The specialistic concept mapping approach was designed to match the
serialistic dimension of cognitive style. The findings of the study do not suggest an effect
of concept mapping at cognitive style level. In addition, it was not found whether style’s
accommodation or compensation affects the learning achievements. In earlier research
Huai (2000) assumed relationships between cognitive styles (holist/serialist), type of
memory (short-term/long-term), and concept mapping method (serialistic/globalistic) on
learning outcomes. She reported a relationship between cognitive style and type of
memory. Holists try to compensate their weak short-term memory with globalistic
concept mapping approach. Serialists having a good short-term memory adopt a
serialistic concept mapping approach.
S. Stoyanov and P. Kommers
2.4 Instruction on concept mapping
The consideration of concept mapping as knowledge representation tool bring about the
assumption that the instruction on concept mapping applying graphical conventions a
sufficient condition that automatically makes the technique an effective problem-solving
tool (Jonassen, 2004). Our experience of using concept mapping to support learning to
design software applications for educational and training purposes showed that it was not
be the case. The instruction based on graphical conventions of concept mapping is a
necessary but not a sufficient condition for making concept mapping an effective
problem-solving tool. There should be another type of instruction that takes into
consideration the characteristics of problem-solving process. It introduces a set of
heuristics and more concrete techniques that support the cognitive processes of
knowledge elicitation, knowledge representation, knowledge reflection, and knowledge
creation in each of the problem-solving phases, namely, analysis of problem situation,
idea generation, idea selection and solution implementation. It is this type of concept
mapping instruction that transforms the possibility of being an effective problem
solving tool into reality. Some of the other cognitive mapping approaches apply a
problem-solving instruction in addition to the graphical instruction. Dynamic mapping
(Vennix, 1997) uses Delphi method and Nominal group technique. Hexagon mapping
(Hodgson, 1999) proceeds with some of the principles and techniques of lateral thinking
(De Bono, 1990). The problem related to the type of instruction that makes concept
mapping an effective problem solving tool in ill-structured situations brings two
hypothetical ideas. One assumes that the instruction on graphical conventions is sufficient
condition for making concept map an effective problem-solving tool. Another assumes
that the instruction on graphical conventions should be complemented with some
problem-solving heuristics and techniques. The two instructions together constitute
the necessary and the sufficient conditions for concept mapping to be an effective
problem-solving tool. The graphical instruction on concept mapping applies the classical
procedure introduced by Novak and Gowin (1984). The problem-solving instruction,
developed for the purposes of this study, took into consideration the problem-solving
process. For each of the stages of problem-solving process, a number of heuristics was
suggested (for details see Stoyanov, 2001, pp.175–177). The set of heuristics was based
on the strengths of the rational approaches to problem-solving such as explicitness,
generality and soundness (Wagner, 1992), but also took into account the intuitive, non
linear and thinking-while-doing way people approach problems (Mintzberg, 1992; Schön,
1996; Wagner, 1992). The problem solving guidelines reflected brainstorming principles
(Osburn, 1963, cited in Van Gundy, 1997), rational problem-solving approach (Kepner
and Tregoe, 1981, cited by Van Gundy, 1997), synectics method (Gordon, 1961, cited in
Van Gundy, 1997), and lateral thinking techniques (De Bono, 1990). The heuristics
were aimed at supporting knowledge elicitation, knowledge representation, knowledge
reflection and knowledge changing. Some randomly taken examples of heuristics
that support knowledge elicitation during problem-solving phase of analysis of the
“Try to scan everything you know about the problem situation. Map everything
that comes spontaneously to your mind, as one items is built upon another. The
items might be existing solutions, facts, hunches, metaphors, feelings. Produce
as many information items as possible. Avoid any attempt to judge them during
the process of free association.”
Some examples for knowledge reflection heuristics are:
WWW-intensive concept mapping for metacognition 305
“Make an evaluation trip on the map. Remove or change (if it is necessary)
some of the nodes and some of the links. Try to improve your map.”
Some examples for knowledge representation guidelines are:
“Try to make clusters. Draw and label links between items. The links can be
descriptive (is a), structural (part of, belongs to), causal (leads to, influenced
by), or metaphorical (like).”
An example for knowledge changing heuristic is ‘Change the places of the nodes in the
map, if necessary’.
Some guidelines that support knowledge elicitation during idea generation phase are
“Look at the map analysis of situation that just has been made. Start to
formulate solution by scratch, as many as possible. Write down everything that
pops-up to your mind without any judgment.”
An example for knowledge changing heuristic during the idea generation is the following:
“Take randomly one of the marginal concepts and put it at the very central
place of the map. Try to reconfigure the map from this new perspective. Use
the new vision as a stimulus for a free association in order to generate as many
ideas as possible.
Play with labels. Randomly select a pair of nodes and change the links’ label.
Use this as a provocation for producing as many solutions as you can.”
An example for knowledge representation heuristics during the idea generation is the
“Draw a resulting map containing all ideas generated. Link the nodes and label
the links as is a; part of, like; leads to, and etc.”
An example for knowledge reflection heuristic during the idea generation phase is
“Try to find a trend or pattern among the ideas you have generated. Is it
possible to make clusters? If you find repetition of some of the ideas, it should
attract your attention. Try to add some more ideas.”
In addition to investigating the effectiveness of the concept mapping instruction method,
we want to determine the role of individual differences in each of the two hypothetical
conditions. To test the validity of the assumptions we design and conduct an experiment.
The research questions that are going to be addressed are as follows:
What is the effect of type of concept mapping instruction on solving ill-structured
What is the effect of individual differences on the construction of concept maps
given an ill-structured problem-solving situation?
S. Stoyanov and P. Kommers
The experimental method applied a factorial experimental design (2 × 2) with a post-test
control group. This experimental design was selected because a random assignment to the
conditions was possible at a certain stage. The combination of random assignment and a
control group served to eliminate the majority of threats to both the external and internal
validity of the study. Although the proportion of dropouts was reported as a potential
threat to internal validity, not controlled for this type of design, it did not prove to be a
problem in the current study. The research was conducted in a one-day session and the
size of groups remained constant throughout the duration of the study.
The experimental design included two independent variables: type of instruction on
concept mapping and learning style. The type of instruction had two levels: the classical
concept mapping instruction method and the new concept mapping instruction method.
The new concept mapping instruction method introduced a set of problem-solving
heuristics in addition to the concept mapping graphical conventions.
The experimental design defined learning style as a second independent variable. It
had also two levels: doers and thinkers. Learning style should be controlled because of
the possibility of being a source of alternative explanation of the effect of instruction on
mapping production, if found. This variable was included in the experimental design
schema as a second independent variable because it was expected that it can be a good
predictor for a possible differential effect on mapping production and a possible
interaction effect with the type of concept mapping instruction.
The dependent variable in this study was concept mapping production. The
operationalisation of the variable is based on the approach of Novak and Gowin (1984) in
scoring concept maps and the criteria for creative thinking developed by Guilford (1967),
both adapted for measuring the effectiveness of concept mapping instruction.
The operationalisation of the dependent variable included two criteria, each having
several indicators. These criteria were broad perception and divergence. Broad perception
defined the extent to which problem solver represents comprehensively the problem
situation. The indicators that described this criterion were as follows:
a number of nodes
b number of links
a variety of nodes – facts, data, metaphors/analogies, personal experience,
opinions, hypotheses, feelings
b variety of labels – descriptive, structural, causal, interrogative, and remote
c variety of links – one-directional, bi-directional, and cross-links.
Divergence was defined as the extent to which problem solver produced alternative
solutions. The indicators that described this criterion were:
Fluency – number of ideas
Flexibility – variety of ideas: ready-made solutions, elaboration, and unconventional
The study tested two hypotheses. The first hypothesis states that the experimental group
using the new method for concept mapping instruction will score significantly higher on
mapping production than the control group, which applies the classical concept mapping
instruction method. According to the second hypothesis individual differences in learning
styles will predict the differences in mapping production and will generate an interactive
effect with the type of instruction.
WWW-intensive concept mapping for metacognition 307
3.1 Subjects and instrument
Fifty-two fourth-year undergraduate students were tested for their learning style.
Thirty-two of them were randomly selected and then were equally assigned to the
experimental and the control group, according to their learning styles. As a reinforcement
to increase the motivation of the students to participate in the experiment, several demo
versions of mapping software tools were installed for free to be used after the experiment.
The Learning Style Questionnaire (LSQ) of Honey and Mumford (1992) was used to
measure learning styles of students. It consists of 80 items to identify four learning styles:
activist, reflector, theorist, and pragmatist. The instrument is a psychometrical validation
of Kolb’s (1998) experiential theory but provides better internal consistence of items than
the original Kolb’s Learning Style Inventory (see for more details de Ciantis and Kirton,
1996). The test-retest reliability of the LSQ is reported to be high – 0.89. In order to
ensure better representation of learning styles for the purposes of this experiment, the
four scales were merged into two – Thinkers (Theorists and Reflectors) and Doers
(Activists and Pragmatists). Honey and Mumford (1992) recommended reducing the four
styles to two for research purposes.
The reliability of mapping production coding was checked as well. Firstly, two
evaluators independently coded six maps each (three from the experimental group and
three from the control group) and compared the results of their scoring. The intercoder
reliability was a relation between the number of agreements and the total number of
agreements and disagreements (Miles and Huberman, 1994). The intercoder reliability
initially got the value of 80%. Because this was assumed as not a very high reliability, the
two evaluators discussed the value of each of the indicators in order to make closer their
judgments. The discussion increases the value of the intercoder reliability up to 95%.
The learning style questionnaire was distributed among the subjects to be filled in.
Based on the results, the students were proportionally assigned to the control and
the experimental groups in order for both learning styles (thinkers and doers) to be
The students in the control group were introduced to the classical concept mapping
method. The experimental group had to apply the new concept mapping instruction
A case to be solved was presented to the students in both the control and the
experimental group and they were asked to use the procedures they had been introduced
to solve the case. The case, called the ‘George’s Career Dilemma’ represents a situation
in which a last year university student is confronted with a problem to take decision about
S. Stoyanov and P. Kommers
Two-way analysis of variance was chosen as an appropriate statistical procedure for the
factorial experimental design applied in this study. The fixed factors were concept
mapping instruction and learning style. The dependant variable was mapping production.
An alpha level of 0.05 was used for all data analysis.
The experimental group scored significantly higher than the control group on the
indicator fluency of nodes of the broad perception criteria – F (1, 28) = 6.297, p = 0.018.
(See Table 1 for the direction of concept mapping instruction effect).
Table 1 Mean figures of broad perception – nodes
Number of nodes
Variety of nodes
Metaphors and analogies
Note: N = 32 (Classical concept mapping instruction – 16; New concept mapping
instruction – 16)
The subjects in the experimental group produced considerably more information
items than the subjects in the control group. The experimental group also demonstrated
significantly higher results on the flexibility of nodes – F (1, 28) = 55.446, p = 0.0001.
The distribution of the different types of nodes shows that the students in
the experimental group included relatively much more statistical data and figures
– F (1,28) = 12.802, p = 0.000, personal experience – F (1, 28) = 11.510, p = 0.002,
hypotheses – F (1,28) = 13.810, p = 0.001, feelings – F (1, 28) = 62.837, p = 0.000, and
metaphors and analogies – F (1, 28) = 8.269, p = 0.008, than the students in the control
group. No one of the maps in the control group contained the following types of nodes:
statistical data and figures, personal experience, and hypotheses. The perception of the
problem space in the control group was dominated mostly by facts – F (1, 28) = 50.948,
p = 0.000 and opinions – F (1, 28) = 17.372, p = 0.000. The data show that the new
concept mapping instruction method stimulates students in the experimental group
better to express the complexity of their problem-solving representations than the
students applying the classical concept mapping instruction. Students in the experimental
group use not only facts but also feelings, metaphors and analogies, and assumptions
types of nodes.
There was not a significant difference between the experimental and the control group
on the indicator fluency of links (Table 2 presents mean values of the Broad Perception
– links indicators).
WWW-intensive concept mapping for metacognition 309
Means figures of broad perception – links
Number of links
Variety of links
Note: N = 32 (Classical concept mapping instruction – 16; New concept mapping
instruction – 16)
As the students in the experimental group produced more nodes, the students in
the control group use relatively more links per node. The subjects working with
the classical concept mapping instruction scored significantly higher than their fellows
in the experimental group on the relative number of bi-directional – F (1, 28) = 9.965,
p = 0.004, and cross-links – F (1, 28) = 5.029, p = 0.033. The students in the experimental
group use mostly one-directional links – F (1, 28) = 16.490, p = 0.000. A possible
explanation might be that subjects using the classical method were forced to use the
whole repertoire of possible links because they had to represent everything on one
sheet of paper. The students in the experimental group had more room to place their
problem-solving representations because of the instruction to make at least two maps
– one for analysis of problem situation and one for idea generation. This particular
feature of the new concept mapping instruction gave the subjects in the experimental
group more memory space, mapped into different sections – analysis of problem situation
and idea generation. While the traditional method put all the problem-solving activities
in one picture, the new concept mapping instruction created a picture of the whole
problem-solving process, sharing the cognitive load between the problem-solving stages.
While the simplicity of the types of links freed up the memory processes, the complexity
of the labels’ structure provided a deeper perception of the problem-solving space.
The variety of link labels – F (1, 28) = 5.645, p = 0.025, was greater in the
experimental conditions (Table 3 shows the mean figures of links’ labels).
Table 3 Means figures of broad perception – labels
Variety of labels
Note: N = 32 (Classical concept mapping instruction – 16; New concept mapping
instruction – 16)
S. Stoyanov and P. Kommers
The students in the classical concept mapping instruction group used predominantly
descriptive types of links’ labels – F (1, 28) = 12.948, p = 0.001. They did not use remote
association labels at all. The experimental group used more structural – F (1, 28) = 8.483,
p = 0.007, causal – F (1, 28) = 6.192, p = 0.019, interrogative – F (1, 28) = 5.358,
p = 0.028, and remote associative – F (1, 28) = 13.064, p = 0.000, links. The new method
used a more complex verbal code combined with a simpler link structure. It provided a
deeper perception of the problem space while reducing the cognitive overload.
The experimental group was superior to the control group in regard to the criteria of
divergence. The scores on number of ideas, F (1, 28) = 20.171, p = 0.000, and variety of
ideas, F (1, 28) = 9.031, p = 0.006, were significantly higher than the same indicators of
the control group. (See Table 4 for some descriptive statistics data).
Table 4 Means for indicators of divergence
Number of ideas
Variety of ideas
Note: N = 32 (Classical concept mapping instruction – 16; New concept mapping
instruction – 16)
Certainly the results were expected as far as the control group did not get an explicit
instruction of using all these types of nodes, links and types of labels. However, the data
revealed at least three important facts:
1 the new concept mapping instruction worked and supported problem solving
2 with the classical concept mapping method, people tend to use particular types of
nodes, links, and labels
3 the expressiveness of knowledge representation can be improved.
The analysis of the learning style variable showed that Thinkers tended to use
significantly more structural types of links than the Doers – F (1,28) = 4.419, p = 0.045.
They also formulated substantially more assumptions items than doers. The result is close
to being a significant at the 0.5 level of probability – F (1,28) = 3.851, p = 0.060.
Thinkers naturally tend to classify information and to present it into clusters. They tend
also to generate more hypotheses. A good prerequisite for this is a well-established
structure. Doers expressed more feelings in the perception of the problem-solving space
– F (1,28) = 4.047, p = 0.054. This is probably because they are more extravert-oriented
people. With the new instruction Thinkers reduced considerably the number of
cross-links – F (3,26) = 5.722, p = 0.024. Thinkers applying the classical concept
mapping approach needed more cross-links to express the structural complexity of the
problem-solving space. The new instruction gave them opportunities to distribute the
structural complexity among several maps.
The data showed no interaction effect between the two independent variables
Instruction and Style on the dependent variable of Map Production. The new concept
mapping instruction proved to have a main effect across all learning styles.
WWW-intensive concept mapping for metacognition 311
The experimental results support the hypothesis that the new concept mapping instruction
method is significantly better than the traditional concept mapping instruction in a
problem solving situation. The question however is how and why it is a better instruction
method for solving ill-structured problems. The new instruction method proves to be
more effective in the analysis of problem situation and the idea generation. It enables a
broaden perception with more and diverse information items and more complex labels on
the links. The new problem solving instruction promotes a broader and more complex
cognitive structure with a dominance of the structural, interrogative, causal and remote
associative types of links. The classical concept mapping method used more simple,
descriptive types of links.
The new concept mapping instruction gives more space for scanning not only
cognitive but also affective problem-solving representations. The psychological distance
between the types of information items on the scale of objectivity-subjectivity is larger in
the experimental group.
Data, for example, are very objective and feelings are very subjective. This increases
the possibility for breaking the fixedness of existing patterns and stimulates creative
combinations in the idea generation phase.
The students in the experimental group knew that they have to start with the map
analysis of problem situation and then they have to continue with constructing the map
idea generation. The externalisation of cognitive and affective structures by a sequence of
maps involves perception. Perception itself takes over some of the mental tasks during
problem-solving, thus contributing to reducing the memory overload. It makes the
reasoning processes easier and more flexible. While the traditional method draws
one picture trying to include all problem-solving activities, the new type of instruction
creates a picture of the whole problem-solving process distributing the cognitive load
between the problem-solving stages. The new method brings a perspective and a
direction to the activities. It is a cognitive aid for guiding and planning through the stages
of problem-solving. The problem-solving instruction supports not only reflection-on a
particular map production (analysis of situation, or idea generation). The students in the
experimental group produced several versions of the map analysis of problem situation
and the map idea generation as a result of reflection on their mapping production.
Although the data did not support the assumption for an interaction effect between
concept mapping instruction method and learning style, the experimental results
suggested that the new concept mapping instruction method brought a general beneficial
effect regardless of different learning styles. It tends to develop skills, which are not
prerogative to no one of the styles. Thus the method has the potential to develop a
comprehensive versatile style.
The new concept mapping instruction method produced better results than the
classical concept mapping instruction because the new method operationally
supported the cognitive conditions of knowledge elicitation, knowledge reflection,
knowledge representation and knowledge changing. In general, the number of nodes and
links of broad perception criterion and number of ideas of divergence criterion are
indicators of knowledge elicitation. The variety of nodes, links (broad perception),
and ideas (divergence) are operationalisations of the knowledge representation.
S. Stoyanov and P. Kommers
Knowledge reflection can be expressed by the extent to which clusters and patterns are
identified. Knowledge changing implies a number of original solutions that have been
5.1 Knowledge elicitation
The new concept mapping instruction method offers special techniques for a broad and
deep retrieval of cognitive and affective structures during the analysis of problem
situation. In the idea generation phase the new instruction method stimulates production
of many alternative solutions. The heuristics that it applies are combinations between
some problem-solving techniques and the specific characteristics of cognitive mapping.
5.2 Knowledge representation
The new concept mapping method promotes a variety of problem-solving types of
representations and a variety of links between them to build a meaningful network when
exploring a problem-solving space. It stimulates using not only objective (facts, statistics)
but also subjective (feelings, intuitions, assumptions) knowledge items. The method
manages the complexity of a problem-solving situation through a set of different types of
links: descriptive, structural, causal, and remote associative. The new concept mapping
method has a capacity to represent very rich picture of a problem situation as combining
verbal and visual coding within a simple graphical format. The externalisation of the
mental problem-solving representations frees up and extends the limited capacity of
working memory thus reducing the cognitive overload.
5.3 Knowledge reflection
The new instruction method on concept mapping makes the internal problem-solving
representations explicit. A problem solver is able to reflect-on the results and the process
of problem-solving. The new concept mapping instruction method offers some guidelines
and techniques for organising the problem-solving space in a particular way. Mostly it is
the case of some convergent activities within each of the phases of the method.
Knowledge reflection, for example, is supported by the suggestions for reorganising the
problem space, more specifically, clustering some of the items and eliminating others.
The visualisation of the problem space through cognitive maps helps the
manipulation of the knowledge items in a variety of ways. Because of the close
correspondence between internal mental structures and the external mode of their
representation one could change the way she or he looks at the problem.
5.4 Knowledge changing
The opportunity that the new concept mapping instruction provides for a manipulation of
nodes in the maps can change dominant thinking patterns and create new ones. The new
method proposes some easy to apply techniques that stimulate creating of original and
unconventional ideas. The new concept mapping instruction challenges the assumption
that it is not possible to modify old pattern in such extent that it can result in creating a
perfectly new one (De Bono, 1990).
WWW-intensive concept mapping for metacognition 313
There are not sufficient data to claim that learning style is a strong predictor for a
differential effect on mapping production. However, some of the figures in the analysis of
the mapping production suggest that learning style preferences should not be ignored.
The analysis did not confirm the hypothesis predicting an interaction effect between the
type of instruction and learning style. There might be two possible explanations for not
confirming this hypothesis. The first one reflected the fact that the experiment modelled a
situation, which was dealing primarily with problem-solving and in a less extent with
learning. The second reason was the selection of a right measuring instrument. Although
the Learning Style Questionnaire (Honey and Mumford, 1992) solved some of the
reliability problems of Learning Style Inventory (Kolb, 1998), it still returned
inconsistent data due to measuring three different independent constructs: style, level and
process (see for more details de Ciantis and Kirton, 1996).
The instruction on concept mapping that includes problem-solving heuristics proved a
better approach in ill-structured problem situations than the classical concept mapping
instruction. The new instruction on concept mapping creates conditions for effective
knowledge elicitation, knowledge representation, knowledge reflection and knowledge
changing. However, the experimental results should be carefully generalised as more
research is needed not only related to concept mapping instruction, but also to the role of
instruction on other problem-solving tools.
We reported data related to the role of concept mapping instruction only in the
analysis of problem situation and the idea generation phases. The problem-solving
process includes additionally at least the phases of idea selection and solution
implementation. We have some data about the role of problem-solving instruction in
these problem-solving phases. However, the graphical techniques used in the idea
selection and the solution implementation were not properly concept maps. Matrix for
example seemed more appropriate graphical technique for idea selection. A sort of PERT
diagram was used for the solution implementation phase. It would be useful to study the
effectiveness of different graphical techniques, including concept mapping, for selecting
ideas and implementing of the solutions. Individual differences can be tested also as a
predictor for the preferences of learners to particular graphical technique. Further
research on concept mapping instruction should optimise some of the experimental
conditions developed for the purposes of the current study. The coding of the types of
nodes needs an improvement. The indicators for knowledge reflection should be
explicitly included in the map production scoring schema. Having clusters and different
versions of maps’ structures could be indicators for knowledge reflection.
We gave students in this study a domain independent problem with an idea to include
experimental subjects from other faculties and universities. Follow up research should
challenge students with a domain specific problem. For our reference situation such a
problem could be designing a website for educational or training purpose.
The experiment for determining the role of instruction for solving ill-structured
problems reflected primarily problem-solving as learning being a secondary concern.
Further research should better model the situation of learning to solve ill-structured
problems. The individual differences should be better represented through a selection of
an appropriate instrument.
S. Stoyanov and P. Kommers
Most of the mapping approaches are supported by software (Inspiration®, 2004; Mind
Manager®, 2004; Decision Explorer®, 2004; STELLA 7.0, 2000; Idons-for-Thinking 2.0,
1999), which makes easier and more efficient the construction of a map and the
manipulation of nodes and links. Testing the effectiveness of the instruction embedded in
different mapping software applications could be the next research challenge.
Further research can include on a conceptual level the assumption of a possible
interaction between the indicators of mapping production. The statistical test should
check the Sums-of-Squares and Cross-Products (SSCP) matrices for determining the
effect of type of instruction on the means of various groupings of a joint distribution of
Alpert, S. (2004) ‘Flexibility of expressiveness: state of the practice. Conceptual representation for
in-depth learning’, in P. Kommers (Ed.) Cognitive Support for Learning, Amsterdam: IOS
Ayersman, D. and von Minden, A. (1995) ‘Individual differences, computers and instruction’,
Computers in Human Behaviour, Vol. 11, Nos. 3–4, pp.371–390.
Buzan, T. and Buzan, B. (1996) ‘The mind map’, Book, New York: Plume.
Constantinou, C. (2004) ‘Concept mapping for performance assessment in physics’, in P. Kommers
(Ed.) Cognitive Support for Learning, Amsterdam: IOS Press, pp.155–165.
De Bono, E. (1990) Lateral Thinking for Management, London: Penguin Books.
de Ciantis, S. and Kirton, M. (1996) ‘A psychometric reexamination of Kolb’s experiental learning
cycle construct: a separation of level, style, and process’, Educational and Psychological
Measurement, Vol. 56, No. 5, pp.809–820.
Decision Explorer® (Computer software) (2004) Banxia® Software: Kendal, Cumbria, UK.
Duncker, K. (1945) ‘On problem solving’, Psychological Monographs, Vol. 58, No. 270.
Eden, C. and Ackerman, F. (2002) Making Strategy. The Journey of Strategic Management,
London, UK: Sage Publications.
Eysenck, M. and Keane, M. (2000) Cognitive Psychology, London: LEA.
Fernández, H., Kommers, P. and Asensio, M. (2004) ‘Conceptual representation for in-depth
learning’, in P. Kommers (Ed.) Cognitive Support for Learning, Amsterdam: IOS Press,
Ge, X. and Land, S. (2004) ‘A conceptual framework for scaffolding ill-structured problem-solving
process using question prompts and peer interaction’, Educational Technology Research and
Development, Vol. 52, No. 2, pp.5–22.
Gordon, W. (1961) Synectics, New York: Harper and Row.
Guilford, J.P. (1967) The Nature of Human Intelligence, New York: McGraw-Hill.
Gulmans, J. (2004) ‘Mapping for the constructivistic acquisition of concepts?’, in P. Kommers
(Ed.) Cognitive Support for Learning, Amsterdam: IOS Press, pp.31–45.
Heeren, E. and Kommers, P. (1992) ‘Flexibility of expressiveness: critical factor in the design of
concept. Mapping tools for learning’, in P. Kommers et al. (Eds.) Cognitive Tools for
Learning, Berlin: Springer-Verlag, pp.86–101.
Hodgson, A. (1999) Hexagons for Systems Thinking, http://www.orglearn.nl/Archives/Hexagons/
hex1.rtft.html (retrieved 16 March 1999).
Honey, P. and Mumford, A. (1992) The Manual of Learning Styles, Maidenhead, Berkshire:
Published and Distributed by Peter Honey.
Huai, H. (2000) ‘Cognitive styles: its relations to memory capacity and concept mapping’, Doctoral
Dissertation, University of Twente, The Netherlands, Enschede: Febodruck.
Huai, H. and Kommers, P. (2004) ‘Cognitive styles and concept mapping’, in P. Kommers (Ed.)
Cognitive Support for Learning, Amsterdam: IOS Press, pp.209–228.
Idons-For-Thinking 2.0 (Computer software, demo version) (1999) Idon Software,
http://www.idonresources.com/demo.html (downloaded 16 March 1999).
Inspiration® (Computer software) (2003) Beaverton Hillsdale, Portland, Inspiration Software, Inc.
Jonassen, D.H. (2004) Learning to Solve Problems. An Instructional Design Guide, San Francisco,
Jonassen, D., Beissner, K. and Yacci, M. (1993) Structural Knowledge, New Jersey: LEA.
Jonassen, D., Reeves, T., Hong, H., Harvey, D. and Peters, K. (1998) ‘Concept mapping as
cognitive learning and assessment tools’, Journal of Interactive Learning Research, Vol. 8,
Nos. 3–4, pp.289–308.
Kaufmann, G. (2001) ‘Problem-solving and creativity’, in J. Henry (Ed.) Creative Management,
London: Sage, pp.47–78.
Kennedy, D. and McNaught (1998) ‘Use of concept mapping in the design of learning tools
for interactive multimedia’, Journal of Interactive Learning Research, Vol. 8, Nos. 3–4,
Kepner, C. and Tregoe, B. (1981) The New Rational Manager, London: John Martin Publishing.
King (1991) ‘Effect if training in strategic questioning on children’s problem-solving performance’,
Journal of Educational Psychology, Vol. 83, No. 3, pp.307–317.
Kolb, D. (1998) Experiential Learning, Englewood Cliffs, New York: Prentice-Hall.
Kommers, P. and Lanzing, J. (1998) ‘Students’ concept mapping for hypermedia design: navigation
through World Wide Web (WWW) space and self-assessment’, Journal of Interactive
Learning Research, Vol. 8, Nos. 3–4, pp.421–456.
Lane, D. (1997) ‘Modelling as learning: a consultancy methodology’, in J. Morecroft and
J. Sterman (Eds.) Modeling for Learning Organizations, Portland, OR: System Dynamics
Series, Productivity Press, pp.93–120.
Luchins, A. and Luchins, E. (1991) ‘Task complexity and order effects in computer presentation of
water jar problems’, Journal of General Psychology, Vol. 118, pp.45–72.
Lumer, J. and Ohly, K-P. (2004) Cognitive Support for Learning, in P. Kommers (Ed.) Amsterdam:
IOS Press, pp.81–89.
Lumer, J. and Hesse, M. (2004) ‘Concept mapping in the teaching of biology’, in P. Kommers
(Ed.) Cognitive Support for Learning, Amsterdam: IOS Press, pp.81–89.
Miles, M.B. and Huberman, A.M. (1994) Qualitative Data Analysis, 2nd ed., Thousand Oaks: Sage
Mind Manager® (Computer software) (2004) Mindjet™: Larkspur, CA.
Mintzberg, H. (1992) ‘The manger’s job: folklore and fact’, in H. Mintzberg and J.B. Quin (Eds.)
The Strategy Process Concepts and Contexts, Englewood Cliffs, NJ: Prentice Hall, pp.21–31.
Novak, J. (1998) Learning, Creating and Using Knowledge. Concept Maps as Facilitative Tools in
Schools and Corporation, Mahwah, NJ: LEA.
Novak, J. and Gowin, D. (1984) Learning How to Learn, Cambridge: Cambridge University Press.
Osburn, A. (1963) Applied Imagination, 3rd ed., New York: Scribner’s.
Oughton, J.M. and Reed, W.M. (2000) ‘The effect of hypermedia knowledge and learning styles on
student-centered concept maps about hypermedia’, Journal of Research on Computing in
Education, Vol. 32, No. 3, pp.366–383.
Pask, G. (1988) ‘Learning strategies, teaching strategies, and conceptual or learning style’, in
R. Schmeck (Ed.) Learning Strategies and Learning Styles, New York: Plenum Press,
WWW-intensive concept mapping for metacognition 315
316 Download full-text
S. Stoyanov and P. Kommers
Reimann, P. (1999) ‘The role of external representations in distributed problem-solving’, Learning
and Instruction, Vol. 9, pp.411–418.
Schön, D. (1996) The Reflective Practitioner. How Professionals Think in Action, London: Arena.
Sherry, L. and Trigg, M. (1996) ‘Epistemic forms and epistemic games’, Educational Technology,
Vol. 36, No. 3, pp.38–44.
Spiro, R. and Jehng, J-C. (1990) ‘Cognitive flexibility and hypertext: theory and technology for
the nonlinear and multidimensional traversal of complex subject matter’, in D. Nix and
R. Spiro (Eds.) Cognition, Education and Multimedia. Exploring Ideas in High Technology,
Hillsdale, New Jersey: LEA, pp.163–205.
STELLA 7.0 (Computer software, demo version) (2000) High Performance System, Inc. The
System Thinking Company, http://www.hps-inc.com/ (downloaded 19 November 2000).
Stoyanov, S. (2001) ‘Mapping in the educational and training design’, Doctoral Dissertation,
University of Twente, The Netherlands, Enschede: Print Partners Ipskamp.
Sweller, J., van Merrienboer, J. and Paas, F. (1998) ‘Cognitive architecture and instructional
design’, Educational Psychology Review, Vol. 10, pp.251–296.
Van Gundy, A. (1997) Techniques of Structured Problem-Solving, New York: Van Nostrand.
Vennix, J. (1997) Group Model Building. Facilitating Team Learning Using System Dynamics,
Chichester: John Wiley and Sons.
von Wodtke, M. (1993) Mind Over Media, New York: McGraw-Hill.
Vygotsky, L. (1978) Mind and Society, Cambridge, MA: Harvard University Press.
Wagner, R. (1992) ‘Managerial problems solving’, in R. Sternberg and P. Frensch (Eds.) Complex
Problem-Solving, Hillsdale, NJ: LEA, pp.159–184.
Wandersee, J. (1990) ‘Concept mapping and the cartography of cognition’, Journal of Research in
Science Teaching, Vol. 27, No. 10, pp.923–936.
Weber, S. (2004) ‘Evaluating structural knowledge with concept mapping’, in P. Kommers (Ed.)
Cognitive Support for Learning, Amsterdam: IOS Press, pp.123–136.
Wertheimer, M. (1987) Productive Thinking, New York: Harper and Brothers Publishers, (original
work published in 1943).
Willerman, M. and Harg, R.A. (1991) ‘The concept map as an advance organizer’, Journal of
Research in Science Teaching, Vol. 28, No. 8, pp.705–711.
KartOO is the WWW-based concept view on topics as learners develop their learning plans
Brain: ‘Dynamic Mind Mapping with Personal Brain’ http://www.thebrain.com/LPS/PBMM/.
for visualising dynamic patterns in WWW-based references