Cognitive Mapping on User Interface Design
Chui Yin Wong
Interface Design Program,
Faculty of Creative Multimedia,
63100 Cyberjaya, Malaysia.
Abstract—Cognitive mapping has been used to assess spatial
knowledge and decodes information about the relative
locations and attributes of phenomena in the everyday
spatial environment. This paper describes the application of
a cognitive mapping technique in relation to user interface
design. We refer to Object-Action Interface Model as an
underlying principle to draw the relation with cognitive
mapping. The most commonly used cognitive mapping
technique, namely concept map, has been applied on a bus
ticket booking mobile solution using a computer software
Keywords — Cognitive Mapping; Concept Map; User
Interface Design; Object-Action Interface Model; Human-
Today we are living in an era of convergence with rapid
technology evolution. We experience tremendous
technological advancement and foresee the boundaries
between each domain is diminishing and evolving in a
ubiquitous society. The impact of ubiquitous society will
revolutionize the pattern of one’s lifestyle. One needs to
know that we are no longer interacting with conventional
products, but surrounded with smart devices and
intelligent environments that will gradually ‘invisible’ in
our daily lives. It has become an exciting and also
challenge for researchers and designers, in particular,
interface designer to think of the next generation of user
interface design. By understanding this, one needs to
understand the history and evolution of user interface
There are many types of user interfaces that evolve over
the decades starting from 1980s of command-based
interface to WIMP (Windows, Icons, Menu and Pointing
Device) and GUI (graphical user interface), and 1990s of
Web, speech, appliance interfaces to the most recent
2000s of mobile, multi-modal, shareable, tangible,
augmented and mixed reality, wearable and robotic
interfaces . The evolution of user interface uncovers
aplenty opportunities for interface designers to think out-
of-the box of the next user interface design solution. To
conceptualize the interaction styles for an interface
solution, this can be an uphill task for interface designers
when it comes to a more complex and innovative user
The cognitive mapping method has been used to assess
spatial knowledge and examine how it is created , and
it has been applied in various disciplines such as
anthropology, psychology, disability studies, artificial
intelligence and cybernetics, computer science, cognitive
neuroscience, behavioral economics and finance,
ergonomics and user interface design, philosophy of mind,
linguistics and geography. In terms of geography, the
relation of cognitive mapping is related to pointing-
direction and orientation, inter-point distance examination,
path integration, landmark navigation and route learning.
In this paper, we are particularly interested to focus on
the use and application of cognitive mapping on
ergonomics and user interface design from a design
perspective. However, there are limited studies of
applying cognitive mapping into user interface design, in
particular on user task-action-object to draw interface
design solutions. Therefore, as interface design
researchers and practitioners, we are interested to apply
how cognitive mapping technique, in particular concept
map, can help interface designers to conceptualize the
interaction style for interface design solutions. The
motivation of this study is to apply a structural graphical
diagram using Cmap tool to conceptualize interface design
solution for interface designers.
COGNITIVE MAPPING AND OBJECT-ACTION
A. Cognitive Mapping
Cognitive mapping is defined as ‘a process of a series
of psychological transformations by which an individual
acquires, codes, stores, recalls, and decodes information
about the relative locations and attributes of phenomena
in their everyday spatial environment’ . Cognitive
mapping is a neuropsychological process, with both
conscious and unconscious aspects. Although cognitive
mapping can be generated with or without conscious
intent, they are not always self-intimating.
Cognitive mapping encompasses the terms of causal
mapping [4,5], semantic mapping  and concept
mapping , which usually also refers to mental models,
mental maps, cognitive maps, scripts, schemata, and
frames of reference. Cognitive mapping techniques are
usually used to identify subjective beliefs and to portray
these beliefs externally . The general approach is to
extract subjective statements from individuals, within a
particular problem domain, about meaningful concepts
and relations among these concepts, and then to describe
these concepts and relations in some kind of graphical
layout . The outcome of a cognitive mapping technique
is usually referred as a cognitive map.
B. Cognitive Map
As mentioned earlier, the representation of cognitive
mapping is in graphical structure or diagram form to
2010 International Conference on Computer Applications and Industrial Electronics (ICCAIE 2010), December 5-7, 2010, Kuala Lumpur, Malaysia
978-1-4244-9053-0/10/$26.00 ©2010 IEEE 287
present information, which is called cognitive map. A
cognitive map is defined as ‘an overall mental image or
representation of the space and layout of a setting’ ,
which means that the act of cognitive mapping is ‘the
mental structuring process leading to the creation of a
cognitive map’. Cognitive map is also a term users refer
to one’s internal representation of the experienced world.
Cognitive mapping includes the various processes used to
sense, encode, store, decode, and use this information.
Cognitive maps are invariably incomplete and partially
distorted, features that can be revealed in external
representations or in spatial behaviors.
The term of ‘cognitive map’ was developed by Tolman
 to describe an individual’s internal mental
representation of the concepts and relations among
concepts. In psychology, this internal mental
representation is used to understand the environment and
make decisions accordingly. Cognitive maps are regarded
as ‘internally represented schemas or mental models for
particular problem-solving domains that are learned and
encoded as a result of an individual’s interaction with
[th]eir environment’ . Therefore, cognitive maps
provide a presentation for what is known and believed,
and exhibit the reasoning behind purposeful actions .
To understand fully the nature and function of a
cognitive map, it is required to understand ‘what cognitive
maps are’, ‘how they are formed’, and ‘how they work’.
Therefore, there are three fundamental questions need to
be addressed in the process, which are: (i) what do people
need to know? (ii) what do people know? (iii) how do
people get their knowledge?
• (i) What do people need to know?
Given an individual with the limitations and spatial
environment complex characteristics, two basic types of
information one needs to possess for everyday’s spatial
behavior and survival skills are the location and attributes
of phenomena [3,11]. A cognitive map consists of both
attributes. Since location and attribution are properties of
objects as well as of phenomena, we first need to
understand what an ‘object’ is.
Locational information is a subjective geometry of
space, which consists of 2 major components – distance
and direction. For instance, we need to know the location
we are currently located i.e. Kuala Lumpur and the next
destination, Petaling Jaya city, in terms of the distance and
direction. To learn the attributes of phenomena, we need
to know the attributive information of what kinds of
phenomena exist, and is complementary to locational
information, which indicates what is at a particular
location and why anybody would want to go there.
The relationship between and attribute and an object
depends on the scale of analysis of the problem at hand
which defines what an object is and what an attribute and
location information about. An object is usually identified
and defined by a set of attributes and bits of locational
information. However, what an object is at one spatial
scale can become an attribute at another. By
understanding the meaning of an object and its attribute of
locational information, we denote the relationship to
Object-Action Interface (OAI) model  in interface
design. Section III will illustrate more on how cognitive
mapping being applied on an interface design application.
• (ii) What do people know?
Cognitive maps are complex, highly selective, abstract,
and generalized representations in various forms.
Cognitive maps are sometimes distorted so that the size
(scale) of represented phenomena indicates relative
connotative significance. It can be viewed as incomplete,
distorted, schematized, and augmented when we find the
differences between intergroup and individual exist. For
instance, the spatial environment contains many regular
and recurrent features; people share common information-
processing capabilities and strategies; and spatial behavior
patterns display similar origins, destinations and
frequencies. Therefore, we need to be careful in
interpreting the absence of phenomena from cognitive
maps as reflecting cognitive discontinuity space. The
distortion of cognitive maps indicates the cognitive
transformations of both distance and direction.
We conclude that we view the world in the way that
how we see it because it allows us to view it in that way.
Our view accords with our plans or ‘inner mind’ for use of
the environment. In other words, the differences between
the ‘real world’ and the cognitive maps serve us a useful
purpose in spatial behavior. People generally behave in
the world ‘as they view it’. Any incompleteness of
cognitive maps, these serve as the basis for spatial
• (iii) How do people get their knowledge?
People generally gain their knowledge through sensory
modalities, and through various interaction style and
vicarious sources of information. For instance, we gain
knowledge and learn information from different sensory
modalities such as sight, sound, touch, smell and taste,
which is integrated by our nervous system. Much of our
overall conception of the world is based upon information
extracted from visual and auditory scanning of the
environment. Recently, haptic sensory play an important
role to vision and audition to humans, especially in
multimedia environment. Such integration of multi-
sensory modalities results in unified perceptual
experiences that are coherent across sensory modalities
and interpreted by our mind that form perception,
information and interpretation of meaning. By
understanding these phenomena, we relate how the
underlying philosophy of cognitive mapping associate
with OAI model  and help to inform the interaction
style on user interface.
C. Concept Map as a Cognitive Mapping Technique
The most popular cognitive mapping technique is
concept mapping. Concept mapping is developed by ,
which is based on the theory by  who emphasized the
importance of prior knowledge in being able to learn
about new concepts.  concluded that ‘meaningful
learning involves the assimilation of new concepts and
propositions into existing cognitive structures.’ To
transform the existing cognitive structure into learning of
new concepts, concept map is developed to draw a
graphical representation to address such concern.
Basically, a concept map is a graphical representation
of the structure of knowledge. It is defined as ‘a
structured process, focused on a topic or construct of
interest, involving input from one or more participants,
that produces an interpretable pictorial view (concept
map) of their ideas and concepts and how these are
interrelated’ . There are several purposes that concept
mapping can achieve :
- To generate ideas (i.e. brainstorming);
- To design a complex structure (i.e. long texts,
hypermedia, large websites, etc.);
- To communicate complex ideas;
- To aid learning by explicitly integrating new and
- To assess understanding or diagnose
A concept map is a graphical representation where
nodes (points or vertices) represent concepts, and links
(arcs or lines) represent the relationship between concepts.
The links, with labels to represent the type of relationship
between concepts, can be one-way, two-way, or non-
directional. The concepts and the links may be
categorized, and the concept map may show temporal or
causal relationships between concepts.  defined
concept as ‘a perceived regularity in events or objects, or
records of events or objects, designated by a label’.
Concepts are connected with labeled arrows, in a
downward-branching hierarchical structure. The
relationship between concepts is articulated in linking
phrases such as ‘gives rise to’, ‘results in’, ‘is required
by’, or ‘contributes to’.
Propositions are statements about some object or event
in the universe, either naturally occurring or constructed.
Propositions usually contain two or more concepts
connected using linking words or phrases to form a
meaningful statement. It is sometimes called semantic
units, or units of meaning . Another important
characteristic of concept map is the inclusion of cross-
link. Cross-links are relationships or links between
concepts in different segments or domains of the concept
map. Figure 1 shows an example of a concept map and its
key features :
Figure 1: A concept mapping with its key features .
Concept mapping is useful in generating ideas,
designing a complex structure, communicating complex
ideas, aiding learning by explicitly integrating new and
old knowledge, as well as assessing understanding or
diagnosing misunderstanding [11,16]. As a consequence,
a concept map is a type of cognitive map which is in
In essence, concept map has been used in many
industries and disciplines such as healthcare decision-
making , information systems , mathematics ,
architecture , knowledge management  and so
forth. There are few studies related in relation of cognitive
map with ergonomics and user interface design such as
requirement capture of an electronic database ,
information architecture and the web , and tangible
user interface for wayfinding .
D. Object-Action Interface (OAI) Model
We refer to Object-Action Interface (OAI) Model as our
underlying theory, which is a descriptive and explanatory
model that focuses on task and interface objects and
actions . The OAI model reflects the higher level of
design where most designers dealing with the user
interface (UI) tools in creating any interface design
solution, and provide valuable guidance for designers of
interfaces, online help, and training processes.
Graphical User Interface (GUI) has long replaced the
command language syntax, and lays the foundation for the
current computer operating systems and mobile devices.
When interface designers designing an interface solution,
it is advisable to use real world ‘object-action’ as
metaphor in the user interface process. Thus, GUIs with
its direct manipulation attributes coupled with visual
appearance of icons and menu represent best solution of
real world ‘object-action’ in the interface metaphor. For
example, using dustbin as a recycle bin on the desktop of
a personal computer is easily recalled and associated by
users for throwing unwanted files into a bin. As a result,
the emphasis now is on the visual display of user-task
objects and actions.
It is important to understand user tasks in order to do an
object-action design. The task includes the universe of
real-world objects with which users work to accomplish
their intentions and the actions that they apply those
objects . Task actions usually start from high level
intentions that are decomposed into intermediate goals and
individual steps. Tasks include hierarchy of objects and
actions at high and low levels. Tasks may not be perfect at
the first place, but they are comprehensive and useful, and
interface designers are advised to improve the tasks-
actions from time to time in the design process. Users
generally accept a separation of their tasks into high-and
low-level objects and actions.
Like tasks, interfaces also include hierarchy of objects
and actions at high and low levels. For instance, an
interface ‘object’ like the concept of ‘storage’ is usually
referred as computers or a workstation that store
information at a high-level concept. To decompose it, the
stored information can then be refined into a set of
directory entities and each file is then an object that has a
lower-level structure consisting of lines, fields, characters,
fonts and so on . Figure 2 below shows the task and
interface concepts depicted in OAI model:
Figure 2: Task and interface concepts with hierarchies of objects and
Designers are encouraged to emulate the ‘real-world
objects’ such as books, pen, notes, and ‘actions’
performed in daily activities such as chatting with friends,
answering telephones. It is best to create the metaphoric
representations of the interface objects and actions based
on the common user tasks-activities and objects we are
familiar with in our environment. Once the task-objects
and actions and their decompositions are defined,
designers need to make the interface actions ‘visible’ to
the users, so that users can decompose their plans into a
series of intermediate actions such as clicking a file,
moving the file, and copying a document, and decompose
into more details user actions on that particular object.
APPLICATION OF COGNITIVE MAPPING TECHNIQUE
ON INTERFACE DESIGN
A. Cmap Tools for Interface Design
Concept map, as a graphical tool, is the most commonly
used of cognitive mapping technique for organizing and
representing knowledge. Concepts map are a powerful
tool for capturing, representing, and archiving knowledge
of individuals, but also a useful tool to create new
knowledge. Florida Institute for Human and Machine
Cognition (IHMC) had developed the Cmap Tools, a
computer software program for concept map . It
allows flexibility and collaboration between individuals
around the world to produce a concept map synchronously
In constructing a good concept map, one has to go
through several steps such as identify a focus question
, where the question clearly specifies the problem or
issues the concept map should help to resolve. Given a
selected domain and a defined question or problem in this
domain, the next step is to identify the key concepts that
apply to this domain, within 15-25 concepts.
These concepts could be listed down, and then from this
list a rank ordered list should be established from the most
general and most inclusive concepts. Then, the next step is
to choose the explicit linking words to relate to concepts
in order to form good proposition, and continue building
concept hierarchy. This can be done writing using the
conventional way of Post-its™ notes, or computer
generated IHMC Cmap Tools. As in constructing a
preliminary concept map, both allow for moving the
concepts around easily, but the latter allows for more
flexibility and quicker iteration in the process. Usually one
round of concept map construction is to form the
understanding ground. It requires several revisions to
make the final concept materialized. Once the preliminary
map is built, cross-links should be sought; those are links
between concepts in different segments or domains of
knowledge on the map that help to illustrate how these
domains are related to one another. It is important to
understand the relationship of cross-links between the sub-
domains in the concept map.
As interface design researchers and practitioners, we
are interested to apply the principle of concept map using
Cmap Tools with the underlying Object-Action Interface
(OAI) model for interface design solutions. Using the
same abovementioned approach, we first identify an
interface design project instead of a focus question.
The ‘concept’ usually refers to objects on interface such
as mobile screens, user interface (UI) elements i.e.
controls, buttons, menus, search boxes, icons, and so
forth. To visualize the overall concept map, an interface
designer usually works together with users to formulate a
Task Analysis [24, 25] by understanding the user tasks
before constructing a concept map. Hence, the proposition
here refers to action that will be triggered by the users
such as touch, click, press, select, and so forth. In many
situations, interface designers find bottlenecks to
conceptualize how many screen pages, and also the UI
elements appear in the interface. Hence, concept map
comes in handy as a design tool to assist them to visualize
an interaction style, in relation to input methods and
output display for an interface solution.
In this particular case, a bus ticket booking mobile
solution, namely TRA-K, is drawn as an example (see
Figure 3: A cognitive mapping developed using Cmap Tool for TRA-K, a bus ticket booking mobile solution.
(Courtesy from Interface Design Department).
In TRA-K bus ticketing mobile solution, for example,
the user will need to trigger an ‘action’ of using ‘touch’
(an interaction style) for a main menu display (‘concept’)
on a mobile device. This is followed by ‘touching’
(action) another mobile screen (object) entitles ‘promotion
discount’ (concept). To check out the ‘latest updates’
(concept), the user needs to ‘scroll’ down (action) the
scroll bar for ‘more information’ (concept).
The interface designers can use different color coding
to differentiate the high-level tasks, which are usually
associated with the interface hierarchies that reflect how
many categories of mobile screens will appear on an
interface solution. In TRA-K, there are five color codings
being used to differentiate the high-level interface
hierarchies of objects and actions. For instance, light blue
is used for ‘device settings’ (i.e. language preferences,
on/off button), pink for ‘transport information’, purple for
‘promotion discount’, green for ‘user sign-up’, yellow for
‘current activities’ and ‘latest updates’. Each high-level
interface hierarchy is cross-linked with an action, and
consists of sub-interface or sub-tasks (sub-domain).
For interface designers, a concept map assists them to
conceptualize interaction styles, in relation to input
methods and output display for an interface solution. As
the design project moves along, designers usually can
propose several alternatives of interaction style of how a
user will interact and trigger an action on one ‘concept’
(an UI element) to another ‘concept’. For instance, instead
of using ‘touch’ (action) from the ‘main menu’ screen
(concept) to ‘promotion discount’ screen (concept),
interface designers can provide an alternative interaction
style of triggering another action such as ‘sliding’ (action)
between both concepts.
In summary, cognitive mapping concerns who we think
about space, and how those thoughts are used and
reflected in human spatial behavior. In order to traverse
space, we make hundreds of complex spatial choices and
decisions, in most cases without any references to source
such as maps, instead relying on our knowledge of where
places are. Cognitive mapping provides a guide of how
people learn, process, and use spatial information that
relates to the environment that surrounds them. We apply
the concept map using Cmap Tools as one of the most
commonly use cognitive mapping techniques on user
interface design. In essence, Cmap Tools serves as an
effective design tool for interface designers in
conceptualizing interaction styles and designing user
tasks-objects-actions for interface solutions.
Despite the benefit as mentioned earlier, given the time
constraints during the development, interface designers
find challenges in juggling between their design skills of
creating creative interface solutions, task analysis and
cognitive mapping. Thus, we are keen in finding out how
useful for interface designers in translating task analysis
into cognitive mapping in the real scenario in the future.
Future studies will also compare both task analysis and
cognitive mapping as to which tool is more effective in
decomposing user task-object-action scenarios for
The author would like to thank Janet Choo, Khong
Chee Weng and Kimberly Chu from Interface Design as
part of the TRA-K design project. In addition, the author
also likes to acknowledge the use of CmapTools
developed by Florida Institute for Human and Machine
Cognition (IHMC) in this paper.
Disclaimer. The authors wish to emphasize that the
names and images shown in this paper are only for
training and educational purposes, and does not
intentionally infringe the rights of individuals or
organisations. Any company names, registered trademarks
or commercial names mentioned or shown in the text
solely belong to their respective owners, organisations and
 J. Preece, H. Sharp and Y. Rogers, Interaction Design: Beyond
Human-Computer Interaction (2
.) West Sussex, England: John
Wiley & Sons, Ltd., 2007.
 Intraspec.ca, “Cognitive Mapping: Definition, Examples and
Resources”, [Online] Available at: http://intraspec.ca/cogmap.php
[access on 21st May 2008]
 RM Downs, and D. Stea. “Cognitive Maps and Spatial Behavior:
Process and Products,” in Image and Environment, RM Downs
and D. Stea, Eds. Chicago: Aldine Publishing Company, 1973, pp.
 J. Swan, “Using cognitive mapping in management research:
Decisions about technical innovation”, British Journal of
Management, vol. 8, 2, pp. 183-198, 1997.
 GA. Kelly, The Psychology of Personal Constructs, New York:
Norton & Company Inc., 1955.
 T. Buzan, The Mind Map Book, London: BBC Books, 1993.
 JD. Novak, “How do we learn our lesson? Taking students
through the process”, The Science Teacher, vol. 60, 3, pp. 50-55,
 CM. Fiol and AS. Huff, “Maps for managers: where are we?
Where do we go from here?”, Journal of Management Studies,
vol. 29, 3, pp. 267-285, 1992.
 P. Arthur and R. Passini, Wayfinding: People, Signs and
Architecture. Toronto: McGraw-Hill Ryerson, 1992.
 E. Tolman, “Cognitive maps and rats and men”, Psychological
Review, vol. 55, 4, pp. 189-208, 1948.
 K. Siau and X. Tan, “Use of Cognitive Mapping Techniques in
Information Systems Development”, The Journal of Computer
Information Systems, July 2008. [online] Available at:
management/company-strategy/11565366-1.html [access on 20th
 B. Shneiderman and C. Plaisant, Designing the User Interface:
Strategies for Effective Human-Computer Interaction (4
Boston, USA: Pearson Education, Inc., 2005.
 D. P. Ausubel, Educational Psychology: A Cognitive view. New
York: Holt, Rinehart and Wisnton, 1968.
 W. M.K. Trochim, Concept Mapping, 2006. [online] Available at:
http://www.socialresearchmethods.net/kb/conmap.htm [access on
19 April 2008]
 JD Novak and AJ. Cañas. “The Theory Underlying Concept Maps
and How to Construct and Use Them”, Technical Report IHMC
CmapTools 2006-01, Rev 01-2008, Florida Institute for Human
and Machine Cognition, 2008.
 DH. Jonassen, K. Beissner and MA Yacci, Structural Knowledge:
Techniques for conveying, assessing, and acquiring structural
knowledge, Hillsdale, NJ: Lawrence Erlbaum Associates, 1993.
 M. Kane and P. Q. McMahon, “Using Concept Mapping to
Improve Health Care Decision Making”, Health care Biller, pp.4-
5, July 2002.
 K. Afamasaga-Fuata’i, Concept Mapping in Mathematics:
Research into Practice. New York: Springer, 2009.
 M. L. Lehman, Why a Mental Map is important for architects to
understand. [online] Available at:
important-for-architects-to-understand/ [access on 24
 J.W. Coffey, R.R. Hoffman, A.J. Caňas and K.M. Ford, “A
Concept Map-based knowledge Modeling Approach to Expert
Knowledge Sharing,” IKS 2002: The IASTED International
International Conference on Information and Knowledge Sharing,
 A. Pallant, P. Timmer and S. McRae, “Cognitive Mapping as a
Tool for Requirement Capture”, in Contemporary Ergonomics
1996, S.A. Robertson, Eds. London: Taylor and Francis, 1996, pp.
 L. Bollini and G. Palma, “Cognitive Maps: New Paradigms in
Information Architecture and Interface Design for the Web”, In
Concept Maps: Theory, Methodology, Technology, Proc. Of the
First Int. Conference on Concept Mapping, A.J. Caňas, J.D.
Novak, F.M. González, Eds, Pamplona, Spain, 2004.
 E. Sharlin, B. Watson, S. Sutphen, L. Liu, R. Lederer, J. Frazer,
“A tangible user interface for assessing cognitive mapping
ability”, International Journal of Human-Computer Studies, vol.
67, Elsevier, 2008, pp. 269-278.
 J. Hackos and J.C. Redish. User and Task Analysis for Interface
Design. US: Wiley, 1998.
 B. Kirwan and L. Ainsworth (Eds.) A Guide to Task Analysis.
London: Taylor and Francis.