Naïve Cartography: How Intuitions about Display Configuration Can Hurt Performance.

Full text

Page 1

Naı ¨ve Cartography: How Intuitions about Display
Configuration Can Hurt Performance
Mary Hegarty
Department of Psychology / University of California / Santa Barbara / CA / USA
Harvey S. Smallman
Pacific Science and Engineering Group / San Diego / CA / USA
Andrew T. Stull
Department of Psychology / University of California / Santa Barbara / CA / USA
Matt S. Canham
Department of Psychology / Central Oregon Community College / Bend / OR / USA
Abstract
Map-making has traditionally been the domain of professional cartographers, but with the advent of interactive display
systems, users now have the flexibility to create and configure their own digital maps and other visual displays.
This flexibility can be beneficial only if users have good intuitions about which display configurations are effective
or ineffective for different tasks. Here we examine people’s intuitions about display effectiveness and whether these
intuitions match the actual effectiveness of different displays. Surveys of undergraduate students and post-graduate
meteorology students reveal that they consistently prefer enhanced displays, especially those that add animation and
realism. These naı ¨ve intuitions contrast with the principles of cartography, which emphasize the importance of abstracting
from the real world to create simple displays that make task-relevant information salient. Both a review of objective
studies and a new study presented here support traditional principles of cartography and are inconsistent with naı ¨ve
intuitions. We interpret these studies in relation to new theoretical notions of users’ folk fallacies about how perception
works, and derive implications for the design of interactive display systems and education.
Keywords: cognition, naı ¨ve theories, intuitions, animation, 3D displays, realism, detail
Re ´sume ´
La conception de carte se faisait traditionnellement par des cartographes professionnels. Cependant, depuis l’arrive ´e des
syste `mes d’affichage interactifs, les utilisateurs peuvent maintenant cre ´er et configurer leurs propres cartes nume ´riques et
autres affichages visuels. Cette possibilite ´ s’ave `re avantageuse seulement si les utilisateurs savent d’instinct quelles
configurations sont efficaces ou inefficaces pour diffe ´rentes ta ˆches. Dans l’article, on examine ce que les gens conside `rent
efficaces en matie `re d’affichage et si leur instinct correspond a ` l’efficacite ´ re ´elle des diffe ´rents affichages. Un sondage
mene ´ aupre `s d’e ´tudiants en me ´te ´orologie, tant au premier cycle qu’aux cycles supe ´rieurs, a re ´ve ´le ´ qu’ils pre ´fe `rent syste ´m-
atiquement les affichages e ´volue ´s, particulie `rement les affichages anime ´s et re ´alistes. Cette impression naı ¨ve va a `
l’encontre des principes de la cartographie, qui misent sur l’abstraction du monde re ´el pour cre ´er des affichages simples
faisant ressortir des renseignements pertinents pour une ta ˆche donne ´e. Les e ´tudes objectives ante ´rieures a ` ce sujet et une
nouvelle e ´tude pre ´sente ´e ici appuient toutes les principes traditionnels de la cartographie et s’opposent aux impressions
cartographica (volume 44, issue 3), pp. 171–186doi:10.3138/carto.44.3.171
171

Page 2

naı ¨ves. On interpre `te ces e ´tudes en fonction des nouvelles notions the ´oriques sur les fausses croyances populaires des
utilisateurs en matie `re de perception, et on en de ´duit des re ´percussions possibles sur la conception de syste `mes d’affichage
interactifs et la formation.
Mots cle ´s : cognition, the ´ories naı ¨ves, intuitions, animation, affichages tridimensionnels, re ´alisme, de ´tail
1. Introduction
In recent years, there has been a dramatic increase in
the availability of interactive visualization technologies
(Card, Mackinlay, and Schneiderman 1999; Thomas and
Cook 2005). Computer visualizations and geographic
information systems (GIS) increasingly allow users to
select and customize their own displays of both georefer-
enced and abstract data. Further, using the World Wide
Web, people can now choose any place on the globe and
download or display a satellite image of that place, a map
that emphasizes routes through the place, or a map that
emphasizes terrain. Professionals such as meteorologists
have access to highly interactive systems in which they
can add and subtract variables at will and animate the
development of – in the case of meteorologists – weather
systems. Scientists can visualize and explore their data by
rotating them in three dimensions.
The design of maps and other visuo-spatial displays has
traditionally been the domain of professional cartogra-
phers and graphic designers. However, flexible interactive
digital displays put design in the hands of users. When
users are allowed to configure their own displays, effective
performance depends on their knowledge and intuitions
about which displays are most effective. Therefore, inter-
active displays can be beneficial only if users have good
intuitions about what makes a useful visual display and
which display configurations are effective or ineffective
for different tasks.
The goal of this paper is to examine people’s intuitions
about the effectiveness of different types of visual displays,
and how these intuitions match both the actual effective-
ness of displays and the wisdom of cartographers and
information designers. Specifically, the essence of cartog-
raphy is to abstract from the real world and create displays
that make task-relevant information salient while mini-
mizing extraneous clutter. As Jacques Bertin has put it,
‘‘Simplification is an obligation of the communication
process’’ (1983, 166). Similarly, in a cognitive analysis of
principles of graphics design, Stephen Kosslyn states as a
cardinal rule that ‘‘no more or less information should be
provided than is needed by the user’’ (1989, 211). E.R.
Tufte (1983) cautions against including extraneous
information in visual displays, calling this information
‘‘chartjunk.’’ According to these experts, good displays
simplify and abstract from the real world that they repre-
sent. In the studies documented here, our aim was
to discover whether people’s intuitions about display
effectiveness align with these expert principles.
Cognitive scientists refer to intuitions about scientific
phenomena as folk knowledge or naı ¨ve theories (Medin
and Atran 2004; McCloskey, Caramazza, and Green
1980; Clement 1982). In this context, the term ‘‘naı ¨ve’’
refers to the fact that these ‘‘theories’’ are constructed on
the basis of everyday experience rather than formal study.
For example, cognitive scientists use the term ‘‘naı ¨ve phy-
sics’’ to refer to intuitions about physical phenomena,
such as the commonly held but false belief that a ball
emerging from a curved tube will continue to follow a
curved path. Whether correct or erroneous, these intui-
tions have a powerful influence on behaviour and reason-
ing. In a related paper, Max Egenhofer and David Mark
(1995) introduce the term ‘‘naı ¨ve geography’’ to refer to
common-sense models of the geographical world and
propose that this is a necessary basis for the development
of GIS for use by non-experts. By analogy, naı ¨ve cartogra-
phy, as defined here, refers to common-sense intuitions
about how to best visualize georeferenced data and is an
important factor in the use of GIS that put powerful
visualization techniques in the hands of non-expert users.
Recently, there have been several claims that people prefer
displays that simulate the real world with greater fidelity
over simpler and more abstract displays. Researchers in
human–computer interaction have claimed that pervasive
beliefs about computer displays include the assumptions
that (1) dynamic displays are superior to static displays,
(2) three-dimensional (3D) displays are better than two-
dimensional (2D) displays, and (3) solid modelling is
better than wire-frame modelling (Scaife and Rogers
1996). Further, the few display principles that have been
codified by human-factors engineers also tend to empha-
size the benefits of realism (as documented by Smallman
and St. John 2005). For example, a preference for high-
fidelity displays is codified in human-factors principles
such as Stanley Roscoe’s complementary principles of
pictorial realism and the moving part (1968) – which
emphasize that displays should mimic the real-world
situations that they represent – and in the intuitions of
designers that realistic depictions ‘‘minimize interpretive
effort’’ (Dennehy, Nesbitt, and Sumey 1994, 111).
Similarly, cartographers have suggested that people may
have more confidence in data that are presented in realis-
tic displays (Fabrikant and Boughman 2006). These
claims suggest that a desire for high-fidelity displays
may be a general principle of naı ¨ve cartography.
Studies that directly measure people’s intuitions about
displays support the view that faith in realistic displays
is a basic principle of naı ¨ve cartography; they also show
Mary Hegarty, Harvey S. Smallman, Andrew T. Stull and Matt S. Canham
172
cartographica (volume 44, issue 3)

Page 3

that this faith is often misplaced. This is true both for the
content of displays and for the way in which displays
depict the 3D world (see Smallman and St. John 2005).
For example, US Navy users prefer realistic 3D-rendered
icons of ships and planes over less realistic, more abstract
symbols to populate their tactical displays. But different
ships are visually similar in the real world, so following a
design principle of maximizing realism here has the unan-
ticipated disadvantage of creating ship icons that are
harder to discriminate. Consequently, people perform
better with ‘‘symbicons’’ that pare down realism to max-
imize discriminability (Smallman and others 2001).
A similar trend can be seen in the depiction of space.
Some 40 years ago, cartographers first created perspective
views of scenes on flat screens to more naturally convey
terrain and depth relief. These displays, originally termed
‘‘3-D maps’’ (Jenks and Brown 1966), have since become
more widely known as, simply, ‘‘3D displays.’’ Although
they are undeniably compelling, there are several pro-
blems with these displays. Notably, depths compress
much faster than widths when a scene is rendered in
perspective; yet users behave as if oblivious to this distinc-
tion, assuming instead that everything shrinks with dis-
tance at the same rate and thus misperceiving relative
distances and angles. In a recent study, participants were
shown highly detailed 3D terrain maps and smoother,
more simplified maps and were asked to predict which
would be more effective for laying routes across terrain.
Participants predicted better performance with the most
realistic and detailed 3D maps, but in fact they performed
more accurately with the simplified maps that removed
task-irrelevant details (Smallman and others 2007). In
similar studies using more abstract displays, participants
often prefer 3D graphs over 2D graphs, even when the
third dimensionpresents
(Carswell, Frankenberger, and Bernhard 1991; Levy and
others 1996). Again, however, realistic but irrelevant
depth cues about the third dimension do not facilitate
performance and have negative, if small, effects on
graph interpretation (Zacks and others 1998).
no relevantinformation
In previous research on interpreting weather maps, we
have found that participants prefer complex maps that
display extraneous variables, especially when the maps
add realism. In a naturalistic observation of Navy meteor-
ologists performing a weather forecasting task, partici-
pants accessed weather maps via the web that were
more complex than they needed, displaying variables
that were extraneous to their task (Smallman and
Hegarty 2007). In follow-up laboratory tasks, about a
third of undergraduate participants preferred more com-
plex maps, especially those that added realism and famil-
iarity (terrain details and state boundaries), and this was
true regardless of whether they were asked to respond on
the basis of efficiency (‘‘with which map would you per-
form fastest’’) or preference (‘‘which map would you
prefer to use’’) (Hegarty, Smallman, and Stull 2008, 882;
also see Canham, Hegarty, and Smallman 2007). Despite
these preferences, adding realism to weather maps slowed
average response times by half a second (about 10%), and
as the number of displayed meteorological variables
increased, time to answer questions increased significantly
(Hegarty and others 2008; Canham and others 2007).
Additional realism masked the most task-relevant infor-
mation, thus impeding performance. The use of visual
displays seems to be an instance of the decorrelation
between users’ intuitions and performance that is widely
acknowledged in the human-factors literature (e.g., Andre
and Wickens 1995).
Users’ desire for complex and high-fidelity displays defies
basic principles of cartography and graphics design. As
described above, both cartographers and cognitive scien-
tists have proposed that good visual displays abstract from
the real world and create displays that make task-relevant
information salient while minimizing extraneous clutter
(Bertin 1983; Kosslyn 1989; Tufte 1983). Naı ¨ve cartogra-
phy therefore appears to contradict the expert knowledge
of cartographers, graphic design experts, and cognitive
psychologists.
This article provides more objective evidence that users
prefer enhanced displays and believe such displays to be
more effective. Whereas other studies have looked at spe-
cific display enhancements, ours is the first systematic
study to include questions about several display enhance-
ments (including animation, 3D, and detail) within the
same questionnaire, enabling us to examine whether
people discriminate between different types of display
enhancements or between different functions of graphics
in their preferences. We assessed whether users prefer
certain display enhancements (e.g., animation) over
others (e.g., 3D) and whether they think that more
enhanced displays are always better. For example, some
researchers have suggested that users always want more
enhanced displays (Scaife and Rogers 1996), whereas
others suggest that preferences are more nuanced, such
that users express different preferences in different situa-
tions (Levy and others 1996). We also draw a distinction
between those display enhancements that users prefer and
those that they consider more efficient/effective. For exam-
ple, people may prefer to work with more realistic displays
for aesthetic reasons despite being aware that these dis-
plays may impede their performance.
A second goal of the studies presented here was to begin
to study the effects of experience on users’ naı ¨ve cartog-
raphic intuitions. In one previous study, participants were
more likely to choose a simpler display to perform a
comprehension task after they had performed the task
with both simple and complex displays, which suggests
that they became aware of the added difficulty associated
with realistic displays and changed their preferences
accordingly (Hegarty and others 2008). In this article we
Naı ¨ve Cartography: How Intuitions about Display Configuration Can Hurt Performance
cartographica (volume 44, issue 3)
173

Page 4

examine the preferences of meteorologists, who create and
use visuo-spatial displays constantly in their profession,
and contrast these with the preferences of naı ¨ve individ-
uals. Finally, we examine how users’ preferences for dif-
ferent display enhancements compare to the actual
effectiveness of these enhancements. To this end, we pre-
sent data from a new experiment that compared the per-
formance of expert meteorologists and naı ¨ve individuals
using more and less enhanced weather maps, as well as
reviewing literature on the effectiveness of other display
enhancements.
2. Study 1
The goal of Study 1 was to gather basic data on users’
preferences for different display attributes (animation,
realism, 3D displays, and detail). We developed a ques-
tionnaire to evaluate intuitions about the effectiveness of
these display attributes and gave this questionnaire to a
large group of undergraduate students specializing in
socialsciences, naturalsciences,
Students were asked both about their preferences for dif-
ferent attributes of displays and to rate the effectiveness of
these different display attributes for a variety of everyday
tasks, which were chosen for their relevance to college
students (e.g., learning from textbooks, navigating with
the use of maps, and understanding weather reports).
andhumanities.
We also sought to examine whether people simply assume
that more enhanced displays are always better or whether
they discriminate between aesthetic preferences and
display efficiency, between different types of display
enhancements, or between different situations in their
preferences. To this end, we used factor analysis to
reduce the variability in a large number of measured vari-
ables (in this case, questionnaire items) to a smaller
number of latent variables (or factors) corresponding to
different dimensions of variability in the data.
2.1 METHOD
2.1.1 Participants and Procedure
The participants were 739 students (481 female, 256 male,
and 2 unreported) enrolled in psychology classes at the
University of California, Santa Barbara, with a mean age
of 18.55 years (range: 17–22). They completed the ques-
tionnaire as an online component of their classes.
2.1.2 Questionnaire
The questionnaire (see Table 1) consisted of 17 self-refer-
ential statements about the use of some type of visual
display (diagram, animation, map, etc.), for example,
‘‘I learn more effectively from diagrams that are pictorially
realistic.’’ Participants responded by indicating their
level of agreement with the statement on a scale from 1
(strongly disagree) to 7 (strongly agree); all items were
worded such that agreement with the statement indicated
a preference for or a belief in the effectiveness of the dis-
play characteristic described. Item 1 was the statement
‘‘Pictorially realistic diagrams require much less effort to
comprehend than less realistic ones.’’ Eight of the remain-
ing items expressed preferences for a characteristic of
visual display (e.g., ‘‘I prefer to use a map that shows as
many details as possible when I am trying to decide which
route to follow’’): two related to realism, two to 3D dis-
plays, two to animation, and two to detail. The other eight
items expressed the idea that one of these characteristics
makes displays more effective (e.g., ‘‘I learn more effec-
tively from diagrams that depict objects in three dimen-
sions’’). Again, two items addressed each characteristic.
Before completing the questionnaire, participants read
the instructions reproduced in Appendix A.
2.2 RESULTS
Table 1 shows the means and standard deviations for the
different scale items. The neutral response for each item
was 4 (‘‘neither agree nor disagree’’), so mean values
greater than 4 suggest that, on average, participants
favoured display enhancements; mean values less than 4
suggest that, on average, participants did not favour these
enhancements. The means for all but two items were sig-
nificantly greater than 4 (t(730)410, p50.001 in all
cases). Interestingly, the two exceptions (items 11 and
12) addressed the use of 3D maps to find routes.
The mean for item 11 was not significantly greater than
4 (t(730)¼1.72, p¼0.08), while the mean for item 12 was
significantly smaller (t(730)¼-2.39, p¼0.02). We also
computed a measure of effect size, equal to the mean
deviation from the neutral value for each scale item
divided by the standard deviation. Most of the effect
sizes reported in Table 1 are in the medium (0.4–0.7) or
large (?0.8) range (see Cohen 1988). Thus, except for
finding routes in 3D maps, participants strongly preferred
displays with more realism.
Responses to the questionnaire items were included in a
factor analysis. The initial extraction of the factors was
accomplished using the principal axis method, followed
by a varimax (orthogonal) rotation. Four factors had
eigenvalues greater than 1, suggesting that the items
could be reduced to four factors. These factors accounted
respectively for 35.3%, 7.6%, 6.9%, and 4.1% of the
variance among the 17 questionnaire items, for a total
of 53.83%.
Table
this table the columns represent the factors, or latent vari-
ables, and the rows indicate the ‘‘loading’’ of each ques-
tionnaire item on each factor, which can be interpreted
as the correlation between the individual questionnaire
item and the factor: items with high loadings on a
factor indicate what dimension of variance is captured
2presents therotated factorpattern.In
Mary Hegarty, Harvey S. Smallman, Andrew T. Stull and Matt S. Canham
174
cartographica (volume 44, issue 3)

Page 5

by that factor. As Table 2 shows, the items with highest
loadings on Factor 1 related to display details; on Factor 2,
to 3D displays; on Factor 3, to realism; and on Factor 4, to
animation. This suggests that people distinguish between
different display enhancements, such that those who have
a strong preference for animation do not necessarily have
as strong a preference for 3D, detail, and so on. There
is no evidence that separate factors reflect participants’
display preferences as opposed to the types of displays
they considered more effective, given that the ‘‘prefer’’
and ‘‘effective’’ items for each display attribute load on
the same factor. Furthermore, there is no evidence
for separate factors based on the real-world context in
which the display would be used (learning, route finding,
etc.). For example, Factor 1 is defined by items related to
the use of detailed displays for both navigating and learn-
ing, and each of the other factors includes items about
different real-world contexts.
We computed separate subscales for the items relating to
detail, 3D displays, realism, and animation, equal to the
mean score across the items for each of these display
characteristics. Participants had the strongest preference
for animation (M¼5.06, SD¼0.96), followed by realism
(M¼4.92, SD¼0.99), detail (M¼4.75, SD¼1.09), and
3D displays (M¼4.23, SD¼1.05). These differences were
highly significant (t(738)43.5, p50.001) for all pairs of
comparisons. The mean scores for the preference items
(M¼4.76, SD¼0.82) were not significantly different
from those for items on display effectiveness (M¼4.77,
SD¼0.79). We classified students’ declared major as
psychology (n¼174), other social science (n¼175),
other natural science (n¼129), humanities (n¼80), or
undecided (n¼181) and found no significant differences
in display preferences as a function of choice of college
major (F(3,554)53, p40.14 in all cases). There were no
sex differences in preference for any of the display
enhancements (t(735)51.8, p40.05 in all cases).
Study 1 thus indicates a strong preference for display
enhancements, especially animation and realism, on the
part of student participants. These preferences were
shared by science majors and non-science majors and by
male and female participants. The factor analysis indicates
that students distinguished between different display
enhancements, which might be thought of as superficial
features of a display, but shows no evidence that they
were sensitive to the context in which the display would
Table 1. Mean, standard deviation, and effect size for the different items in the Visual Display Questionnaire used in Study 1
Questionnaire ItemMean (/7)SD Effect Size
1Pictorially realistic diagrams require much less effort to comprehend than less realistic
ones.
I prefer to learn from diagrams that show as many details as possible.
I learn more effectively from diagrams that depict objects in three dimensions.
I am more effective in finding the best route to follow when looking at a realistically
coloured and detailed map.
I prefer to use a map that shows as many details as possible when I am trying to decide
which route to follow.
I prefer to learn from diagrams that are pictorially realistic.
When learning about a dynamic process (such as how a pump or other moving object
works) I prefer to view an animated diagram, so that I can watch the moving parts.
I learn more effectively from diagrams that display as many details as possible.
I learn more effectively from diagrams that are pictorially realistic.
If I wanted to learn how the weather will change in the next few days, I would be most
effective when viewing an animated weather map.
Using a three-dimensional map allows me to navigate more effectively when deciding
which route to follow.
I prefer to use a three-dimensional map when I am trying to decide which route to follow.
When looking at a weather map on TV or the Internet, I prefer to have the movement of
weather systems animated, so I can watch how the systems are moving across the
country.
I prefer to learn from diagrams that show objects in three dimensions.
When studying a dynamic process (such as how a pump works) I learn most effectively
from an animated diagram of the process.
I navigate most effectively when using maps that display as many details as possible
when deciding on a route to follow.
I prefer to use a realistically coloured map when I am trying to decide on a route to
follow.
4.781.240.63
2
3
4
4.72
4.54
5.00
1.32
1.20
1.33
0.55
0.45
0.75
54.801.380.58
6
7
5.03
5.51
1.10
1.15
0.94
1.31
8
9
4.81
5.05
4.71
1.28
1.12
1.34
0.63
0.94
0.5310
114.10 1.340.07
12
13
3.88
4.77
1.35
1.34
?0.09
0.57
14
15
4.41
5.24
1.20
1.14
0.34
1.09
164.67 1.330.50
174.99 1.220.81
Naı ¨ve Cartography: How Intuitions about Display Configuration Can Hurt Performance
cartographica (volume 44, issue 3)
175