Content uploaded by Edzer Pebesma
Author content
All content in this area was uploaded by Edzer Pebesma on Sep 19, 2014
Content may be subject to copyright.
Usability of interactive and non-interactive
visualisation of uncertain geospatial information
Lydia E. Gerharz1 & Edzer J. Pebesma1
1Institute for Geoinformatics, University of Münster
Lydia.Gerharz@uni-muenster.de
Abstract. Showing uncertainties of geospatial data in maps in a useful and
comprehensible way for skilled and unskilled users is a problem that is still
not solved ultimately. To evaluate the usability of some commonly used
visualisation techniques with a special focus on interactivity, an explorative
study has been conducted with ten interviewees with a geosciences
background. Each participant was asked to solve tasks and give personal
opinions on three methods applied to the same data set. As an outcome,
uncertainty was considered as being helpful for decision making in general.
The results also show a clear preference for the simplest method of
displaying value and uncertainty in adjacent maps, whereas the more
sophisticated Aguila visualisation system was judged as helpful for expert
users. Interactivity for the methods was preferred by the majority of the
users.
1 INTRODUCTION
Geospatial data is collected and processed to represent and describe real world
characteristics. This representation is naturally restricted by computational
power, model approximation, and limits to measurement availability and
accuracy. Input data and each modelling step are sources of errors that are
propagated through the processing chain to the final results. As we base our
decisions for planning or assessment tasks on those results, it is inevitable to
include also the reliability of the data to allow meaningful reasoning. Showing
not only what we know, but also the degree of information we do not know, can
be realized for instance by giving interval estimates instead of single point
estimates for the results of an analysis. We limit ourselves here to the uncertainty
of attributes, and will not address positional uncertainty.
1.1 Visualising spatial uncertainties
For geospatial information the data reliability can be included into map
representations. This yields the problem of mapping two dimensions, the value
and its uncertainty, in one spatial representation, under avoidance of visual and
cognitive overload of the user. To face this challenge, different types of
presentation techniques have emerged (MacEachren 1992), such as:
• Adjacent map pairs, displaying results and their uncertainty (e.g. standard
error) separately
• Sequential presentation of results and uncertainty
• Bi-variate map, merging results and uncertainty in a single map
Additionally, different visualisation modes can be distinguished, namely:
• Static, as one or more static maps, e.g. printed on paper
• Dynamic, e.g. automatic animation of realisations on a computer screen
• Interactive, the user interacts with the uncertainty representation
From the combination of these presentation techniques and modes, numerous
different methods were designed for varying purposes and user groups.
Metaphors, like for instance blurring the uncertain regions as if they were out of
focus, are commonly applied for qualitative uncertainty information, making use
of the intuitive perception for uncertainty of the user. Transparent overlays and
mixing pixel methods can be used for static bi-variate maps and blinking pixels,
highlighting certain regions, are an example for dynamic representations.
Generally, it is assumed that static methods are easier comprehensible especially
for non-experts, whereas interactive methods offer the control over the amount of
information shown which can be useful for understanding the structure of the
data. It is also hypothesised that bi-variate maps are less useful as adjacent maps,
because bi-variate maps probably contain too much information for the user.
Although many different methods exist for representing uncertainty in geospatial
information, not all of them seem equally helpful. Which representation method
is most helpful also depends on the user’s knowledge. Tversky and Kahneman
(1974) warn that users with different levels of experience use different criteria
for their decisions. For instance, novice users base their decision often on
heuristics rather than on statistical uncertainties.
1.2 Usability testing
However, only few investigations have been conducted to test the performance
and acceptance of different uncertainty mapping techniques for a use case
(MacEachren et al. 2005). Furthermore, the findings of these studies vary
significantly regarding which method is most useful, depending on the purpose
and design of the study. Interactive methods are usually covered by toggling
between the value and uncertainty map lying upon each other. Aerts et al. (2003)
found this to be a helpful method, compared to static representation of adjacent
maps. Evans (1997) in contrast found that toggling is less helpful than other
static and dynamic methods. However, most of the studies results are conform in
the fact that giving uncertainty information is helpful rather than confusing for
the user if it is presented in a useful way.
To verify the hypotheses mentioned above, we decided to perform a usability
study with a small set of test persons. The aim was to compare the usefulness for
decision support and user acceptance of three different uncertainty representation
methods with a focus on interactive vs. static methods. We hypothesise that
interactive uncertainty representation methods take more time to learn but are
more useful for quantification and decision making tasks. Influence of user
experience on the performance was neglected in this study, as the number of
participants was too low.
2 STUDY DESIGN & METHODOLOGY
For the study, ten participants with backgrounds in geography or geoinformatics,
all non-experts in statistics were interviewed for approximately 30 minutes. All
three visualisation methods were presented to them in different order on a
monitor while all answers and comments were recorded. The data set used for the
testing was a residual kriging analysis for annual PM10 concentration over
Europe in 2005, using the kriging standard deviation as uncertainty estimate. The
study had two main parts, one on task performance and one on user opinion.
2.1 Visualisation techniques
To cover interactive and non-interactive methods as well as bi-variate and
separate maps we chose three different methods. First, adjacent maps of the value
and the uncertainty were used as a simple and commonly used static visualisation
method. The legend for the value maps showed concentrations whereas the
legend for the uncertainty map only ranged from low to high. For the bi-variate
map type, a method called Whitening (Hengl & Toomanian 2006) was applied.
Whitening uses the Hue-Saturation-Intensity colour model to combine value and
uncertainty in one map. Uncertain values are displayed with reduced colour
saturation in the map, yielding paler regions that move out of the user’s focus.
This resulted in a two dimensional legend, ranging from high to low
concentrations and 40 to 100 % normalised error.
As a third and interactive method we used Aguila (Pebesma et al., 2007), an
interactive visualisation tool that stores value and uncertainty as cumulative
probability functions for each pixel in space and time. Fig. 1 shows Aguila with
the interactive linked windows of the map, the cursor & value window and the
cumulative probability function. In the map, the threshold values associated to a
certain probability, here 0.7, is shown. The user controls the threshold probability
of the values shown in the map by moving the line control in the probability
distribution function window. By moving the cursor in the map window, the
cumulative probability distribution function and concentration values of each
pixel could be visualised. Alternatively, Aguila can display the cumulative
probability of exceeding (POE) a certain value in the map. The control in the
cumulative probability function window changes to a vertical line and the user is
able to control the threshold value for the POE map.
After introducing the data set, each method was shown and explained to the
interviewee. In Aguila, the different options were demonstrated with the data set
and the user got the possibility to explore the functions himself as long as desired
before answering the questions.
Figure 1: Map, values and cumulative distribution function window of Aguila.
2.2 Tasks
The first two tasks had to be solved by the participant once for each method
consecutively, whereas the third task was only asked once and could be solved
by using any of the methods. Task performance was not measured by time or
errors, but comments of the participants were demanded and recorded. The
participant had to assess how easy it was to solve with the current method the
following tasks:
1. Identify the approximate concentration and its uncertainty for Norway and
North Italy.
2. Imagine being a European policy maker: Where is the annual threshold of
50 µg/m³ exceeded? Can you say how likely this will happen?
3. If you could decide where new measurement stations should be built,
which criteria would you use to identify possible locations? Which
methods do you use for this task?
The aim of the second part was to identify the preferred methods of the
participants and their opinion on uncertainty visualisation in general. For the
presented methods, each participant had to judge if he/she understood all three
methods, which method was easiest to comprehend, and which method was
preferred. To get an overview of the usability of uncertainty mapping in general,
each participant was asked to give his/her opinion on three questions:
• Do you prefer generally static or interactive methods?
• Does the uncertainty visualisation make the map too complex?
• Do you think visualising uncertainty is beneficial for making decisions?
The study was conducted as an interview, so these questions were aimed as
starting point for further comments of the participants. A statistical analysis of
the task performance and answer was not planned for this explorative study, due
to the limited number of interviewees.
3 RESULTS
The results for the first task in tab. 1 shows that the best performance is obtained
by adjacent maps. The concentration for both regions was identified correctly by
all participants with all three methods. Not all participants were able to identify
the uncertainty correctly, especially with the Whitening method. The opinion of
the participants reflects the difficulties that occurred during the task, leading to
the poorer performance of Whitening and Aguila.
Table 1: Results for task 1.
Concentration
correct Uncertainty
correct Easy to anwer
Aguila 10/10 5/10 4/10
Whitening 10/10 3/10 5/10
Adjacent maps 10/10 9/10 10/10
The second task showed generally bad results. Only one person identified the
fully correct answer by using Aguila and none answered it correctly using the
other methods. People tended to identify too many regions with the Whitening
and adjacent maps for threshold exceedance. Some users were able to identify the
correct regions with Aguila, but most could not identify the probability of
exceedance for the threshold with any of the methods, although some suspected
it should be possible with Aguila. Again, the participants judged adjacent maps
as the easiest one to answer this task.
For the third task all participants preferred to use the adjacent maps, yielding
overall good results. The majority of participants used the concentration as well
as the uncertainty information to identify potential locations.
During the user opinion part, all participants stated they understood Whitening
and adjacent maps, whereas only six of them thought they understood Aguila
(tab. 2). None of the interviewees preferred Whitening, whereas half of them
judged adjacent maps as their preferred method and some preferred a
combination of adjacent maps and Aguila. Adjacent maps were rated to be useful
for getting a brief overview, whereas Aguila could be useful for a more
thoroughly analysis to assess the uncertainty quantitatively. This corresponds
also to the results that all ten participants rated adjacent maps as most easily
comprehensible method. Several people found Aguila helpful but too
complicated to understand. Some of them suspected it to be helpful as an expert
tool that would take more time to learn than was given in this study. It was also
clear during the interviews that the time was too short to learn and remember all
the functions of Aguila. The Whitening method was mentioned by some
immediately as useless. They stated that it was too difficult to distinguish
between color hue and saturation although the principle was easy to comprehend.
Two participants explained adjacent maps to be so easy comprehensible and
usable through the separation of value and uncertainty into two maps.
Table 2: Results for opinion questions.
Understood Preferred Most easily to understand
Aguila 6/10 1/10 (4 with
adjacent maps) 0/10
Whitening 10/10 0/10 0/10
Adjacent maps 10/10 5/10 (4 with
Aguila) 10/10
For the general opinion questions, seven of ten participants stated to prefer
interactive methods over static. Some suggested making the adjacent maps
method interactive, either by overlay and toggling or by using interactive linked
windows, allowing scrolling a cursor through both maps simultaneously. All ten
participants found uncertainty helpful for decision making. Half of the
participants thought uncertainty makes maps more complex but not too complex,
whereas some stated for the case of Aguila and Whitening the representations are
too complex.
During the whole study it became clear that the term “uncertainty” was not
equally clear to every participant and got mixed up with “error”, “probability”
and “certainty”. A couple of participants criticised that uncertainty should be
shown quantitatively, which was only the case for Aguila. Others suggested to
translate the uncertainty for non-statisticians as Van de Kassteele & Velders
(2006) did by applying the IPCC terminology on probabilities of exceedances.
4 DISCUSSION
Due to the small number of participants, the results could not be tested on
statistical significance and should be treated cautiously rather as indicators.
Nevertheless, the outcomes of this explorative study clearly support the
hypothesis that visualising uncertainties in geospatial information supports
decision making processes, consistent to the findings of previous studies (Evans
1997, Aerts et al. 2003)
The results for the first task part indicate that adjacent maps are easier to use
for the participants but perform not always best in solving the tasks where
quantification is required. People had problems using Aguila, but suspected it
being helpful if they had more time to learn the program. This could mean that
the users understood the principle of Aguila but could not use it adequately with
the limited knowledge given by the short introduction. Whitening seemed to be
not useful or preferred and showed the poorest results for the three tasks in
general. It seems that the principle of Whitening is immediately easy to
comprehend but not useful for getting the information necessary for making the
decisions. Also, many interviewees did not feel comfortable in using Whitening
as also some felt in using Aguila, leading to the preference of adjacent maps.
Especially while using Aguila, the users seemed to be uncertain if they
understood and correctly applied the program.
The hypothesis that bi-variate maps are overloaded by information and too
complex for the user is supported by the results of this study. The two
dimensional legend of Whitening overstrained most of the users, whereas the
adjacent maps could be applied by the participants much easier. Although Aguila
also separates the statistical dimension from the map, this method seemed to be
much more complicated for the participants to understand and use. Also the
interactivity makes the learning process more complicated but is necessary to
make use of the full advantages in comparison to static methods.
5 CONCLUSION
This work was intended and conducted as an explorative study to investigate
the usability of interactive and non-interactive visualisation in decision making
processes. As an outcome we found the simplest method of showing value and
uncertainty map next to each other the most efficient and preferred one. On the
other hand, interactivity is suspected to support the perception of uncertainty
better than static representations with an adequate learning period.
For decision support this could mean that different methods should be used
and offered, depending on the user’s background, experience and task. A
combination of simple adjacent representations for a first overview and a more
complex system like Aguila for thorough and detailed analyses could possibly be
a solution in future decision support systems.
6 ACKNOWLEDGEMENTS
This study was realised in the HEIMTSA project funded by the European Union
Sixth Framework programme contract no. 036913. We also thank Bruce Denby
from NILU, Norway for the preparation of the residual kriging results, and the
interviewees for their help.
7 REFERENCES
Aerts, J. C. J. H., K. C. Clarke and A. D. Keuper (2003) Testing Popular
Visualization Techniques for Representing Model Uncertainty. Cartography
and Geographic Information Science 30(3): 237-248.
Evans, B. J. (1997). Dynamic display of spatial data reliability: does it benefit the
map user? Computers and Geosciences 23: 409–422.
Hengl, T. and N. Toomanian (2006) Maps are not what they seem: representing
uncertainty in soil-property maps.) Proceedings of the 7th International
Symposium on Spatial Accuracy Assessment in Natural Resources and
Environmental Sciences (Accuracy 2006). M. Caetano and M. Painho. pp.
805-813.
MacEachren, A. M. (1992). Visualising Uncertain Information. Cartographic
Perspective 13: 10-19.
MacEachren, A. M., A. Robinson, S. Hopper, S. Gardner, R. Murray, M.
Gahegan and E. Hetzler (2005). Visualising Geospatial Information
Uncertainty: What We Know and What We Need to Know. Cartography and
Geographic Information Science 32(3): 139-160.
Pebesma, E. J., K. de Jong and D. J. Briggs (2007). Interactive visualisation of
uncertain spatial and spatio-temporal data under different scenarios: an air
quality example. International Journal of Geographical Information Science
21(5): 515–527.
Tversky, A, and D. Kahneman (1974). Judgement under uncertainty: Heuristics
and biases. Science 185: 1124-1131.
van de Kassteele, J. and G. J. M. Velders (2006). Uncertainty assessment of local
NO2 concentrations derived from error-in-variable external drift kriging and
its relationship to the 2010 air quality standard. Atmospheric Environment
40(14): 2583-2595.