Conference PaperPDF Available

Beyond a Visuocentric Way of a Visual Web Search Clustering Engine: The Sonification of WhatsOnWeb.

Conference Paper

Beyond a Visuocentric Way of a Visual Web Search Clustering Engine: The Sonification of WhatsOnWeb.

Abstract

It is widely accepted that spatial representation is processed by an amodal system. Recent studies show that blind subjects have a better motion ability than sighted people in performing spatial exploration guided only by auditory cues. The sonification method offers an effective tool able to transmit graphic information, overcoming the digital divide risen by a visuocentric modality in which contents are conveyed. We present a usability evaluation aiming at investigate the interaction differences between both blind and sighted users while surfing WhatsOnWeb, a search engine that displays the information by using graph-drawing methods on semantically clustered data. We compare the visual presentation of three different layouts with the sonificated ones, demonstrating both qualitatively and quantitatively that blind and sighted users perform with no significant differences the interaction. These results remark that the digital divide could be decreased by going beyond the visuocentric way of the commonly adopted visual content representation.
K. Miesenberger et al. (Eds.): ICCHP 2010, Part I, LNCS 6179, pp. 351–357, 2010.
© Springer-Verlag Berlin Heidelberg 2010
Beyond a Visuocentric Way of a Visual Web Search
Clustering Engine: The Sonification of WhatsOnWeb
Maria Laura Mele1, Stefano Federici1,2, Simone Borsci1, and Giuseppe Liotta3
1 ECoNA - Interuniversity Centre for Research on Cognitive Processing in Natural
and Artificial Systems, 'Sapienza' University of Rome, IT
2 Department of Human and Education Sciences, University of Perugia, IT
3 Department of Electronic and Information Engineering, University of Perugia, IT
marialaura.mele@uniroma1.it, stefano.federici@unipg.it,
simone.borsci@uniroma1.it, giuseppe.liotta@diei.unipg.it
Abstract. It is widely accepted that spatial representation is processed by an
amodal system. Recent studies show that blind subjects have a better motion
ability than sighted people in performing spatial exploration guided only by
auditory cues. The sonification method offers an effective tool able to transmit
graphic information, overcoming the digital divide risen by a visuocentric mo-
dality in which contents are conveyed. We present a usability evaluation aiming
at investigate the interaction differences between both blind and sighted users
while surfing WhatsOnWeb, a search engine that displays the information by
using graph-drawing methods on semantically clustered data. We compare the
visual presentation of three different layouts with the sonificated ones, demon-
strating both qualitatively and quantitatively that blind and sighted users per-
form with no significant differences the interaction. These results remark that
the digital divide could be decreased by going beyond the visuocentric way of
the commonly adopted visual content representation.
Keywords: Sonification, Information visualization, Accessibility, Usability.
1 Introduction
Many studies agree that the spatial representation of information is independent from
the way in which the sensory inputs are displayed; in particular, some authors pointed
out that blind subjects have a better performance in processing spatial auditory inputs
than sighted people [1, 2]. Indeed, it has been highlighted that blind people show a
motion ability in performing spatial exploration tasks guided by only natural acoustic
cues, functionally equivalent to the visually guided way for sighted people [3].
Starting with these suggestions, an amodal system of spatial representation has
been proposed, by explaining the involvement of the auditory, haptic, and kinesthetic
information in the spatial mapping processing of blind people [4]. At the base of the
spatial information elaboration process some different strategies lie, related to both
nature of the information and different points of body references: allocentric vs ego-
centric. During the spatial orientation, totally or partially blind subjects show their
preferences for a body-centered strategy, based on corporal references points, rather
352 M.L. Mele et al.
than for an allocentric strategy, often adopted in mental rotation and scanning tasks
[5, 6]. Therefore, the nature of sound seems to be able to communicate the complexity
of static or dynamic data representation, by keeping their inner relations unchanged
[7].
In this work, we want to introduce a usability evaluation study of a sonificated ver-
sion of a search clustering engine called WhatsOnWeb (WoW), an application tool
based on new graph visualization algorithms, implemented at the Department of
Computer Engineering (DIEI) of the University of Perugia [8]. WoW conveys the
indexed dataset using graph-drawing methods on semantically clustered data [9]: the
visuo-spatial data representation provides the whole information by conveying it in
one single browseable page. By rebuilding the output in graphics mode, WoW’s lay-
outs provide what is conversely ordered in any Search Engines Report Page (SERP)
where the query outputs are in a top-down hierarchical sequence, starting from the
greatest ranking level website to the lowest in several results pages [10,11].
Unlike the operation of common search engines (e.g. Google and Yahoo), WoW
does not use ranking as a hierarchical organization criterion of information, but it
represents the information order even by a semantic association between data, making
all the contents easily and simultaneously available and learnable. Moreover, the
disappearance of browsing text - totally replaced by a gestaltic (synoptic) graphic one
- allows an easy and quick use of the information by reducing accessibility barriers.
Just as Ivory, Yu, and Gronemyer [12] claimed, blind users took twice as long as
sighted participants to explore search results, and three times as long to explore web
pages. Those results show the gap that exists on the interaction with traditional search
engines between blind people, who use screen readers to surf the Web, and sighted
people. These considerations seem to suggest that accessibility is actually not enough:
there is a strong necessity to implement search engines that are both accessible and
usable. In this way, as Federici et al. pointed out [13], WoW overcomes the efficiency
limitation of SERP search engines previously shown, since its structure overcomes
the limitations of a top-down flat representation by introducing different ways to
convey the spatial information.
2 What Is WhatsOnWeb
In order to redesign the WhatsOnWeb system accomplishing the accessibility and
usability principles, we followed the user centered design (UCD) in accordance with
the ISO 13407 “Human-centered design processes for interactive systems” [14]. First,
we proceeded by decoupling the WOW algorithm and subsequently making an auto-
matic accessibility analysis using the RAVEn Eclipse plugin provided by IBM [15],
then we implemented the software code by accomplishing the Java Foundation
Classes and the guidelines provided by Sun and IBM [16]. Specifically, a composite
architecture was produced by allowing the vocalization function, considering both the
use of screen readers and the need of an autonomous integrated synthesizer. Finally,
an appropriate support to the navigation peripherals has been arranged: the navigation
structure of WhatsOnWeb has been created as much independent as possible from the
peripheral, considering the future aim of extending the system also to the Brain Com-
puter Interfaces. Each graph has been organized by following different levels, using
Beyond a Visuocentric Way of a Visual Web Search Clustering Engine 353
cluster nodes - i.e. the semantic sets of results- with possibility of expansion and col-
lapse by users click - i.e. deep level of information - and leaf nodes - i.e. the sites
resulted from the research - as terminal constructs.
In a recent study conducted by Di Giacomo et al. (2008) [17] on a WhatsOnWeb
prototype (http://whatsonweb.diei.unipg.it:8080/wow3.2/), the effectiveness and effi-
ciency of the four different layouts implemented -TreeMap, Layered, Radial, and
Orthogonal - were compared through a navigation task and a satisfaction question-
naire. Findings showed that 56% of people judged the TreeMap as the best graphic
interface layout compared to the other ones. These results were confirmed by the
opinions expressed by the participants in the satisfaction questionnaires. Following
the experimental data on the TreeMap model, it has been developed the actual WoW
prototype by adding a new layout called Spiral TreeMap (spiral tree organization) as a
new visualization layout, where the most relevant node is set at the center of the
screen and the less significant data are progressively set around. This organization in
the process of being tested, together with sonification, should allow a significant im-
provement of the layout effectiveness and efficiency.
3 Sonification of WhatsOnWeb: Design and Implementation
Sonification is the “transformation of data relations into perceived relations in an
acoustic signal for the purposes of facilitating communication or interpretation” [18].
In most of the sonificated systems priority is usually given to the mapping of the
sound attribution to data, but not to the interactivity with the user: in order to over-
come this limit and to guarantee that the sonification represents both the interaction
design and information, Zhao, Plaisant, and Shneiderman (2008) [19] provided the
Action by Design Component (ADC) framework, a sonification model designed to
permit an active and dynamic navigation into the interaction environment. For this
reason, we chosen the ACD framework as a theoretical background for the sonifica-
tion of WhatsOnWeb, in which the indexed data are organized by semantic correla-
tions resulting in abstract information.
The sonification of WoW is combined with visual events describing both global
and particular browsing information. While the global information is visualized after
“search” action, the temporization technique provides to increase the intensity of each
cluster. From the first to the last ranking organization result is guaranteed to the user
through an overall overview of information which allows the first mental representa-
tion of the framework that users are going to browse. The complexity of the tone of
each node is related with the complexity of its paraverbal information: for example,
while browsing a cluster node, an harmonic chord will be executed suggesting the
semantic links with the other peaks. A Low-latency (less than 100 ms) of short sounds
have been used in order to grant a kind of active interaction in which sound informa-
tion processing and keeping does not implicate a short term memory overload [20].
Moreover, WoW browsing is granted by the auditory reiterable feedback which pro-
vides spatial information in order to facilitate user orientation. Indeed, WoW provides
to user a persistent signal which indicate his/her current position in the interface, as it
happens in visual navigation. Spatial cues are uttered by a stereo-audio overview
which simulates the position of selected nodes within a Cartesian coordinate plane;
354 M.L. Mele et al.
the information identification and memorization is strengthened by a verbal feedback
voiced by an integrated synthesizer. We implemented three different ways of sonifica-
tion, by using the sound’s volume, pitch, tone, blinking, and grid reference to transmit
visual features in a univocal way. We tested three different sonification models. In the
first one, VolumeSonification model, the Euclidean distance coding for a node com-
pared to a significant reference is rendered through the sound volume level, whereas
the panning is used to strengthen the node detection on the abscissas axis as absolute
information. The second, BlinkAndPitchSonification, conveys spatial relations
through an independent mapping of the two axes of the Cartesian plane (x, y) respec-
tively with the frequency of the sound blinking together with panning, and with the
note pitch. Finally, in order to optimize the graphic representation in terms of sound,
the PanAndPitchSonification has been created solely considering the panning for the
x axis and the pitch for the y axis.
4 Usability Evaluation of the Redesigned WoW
The experimental analysis of the reengineered and sonificated WoW software evalu-
ated the usability of the different layouts: TreeMap, Layered Radial, and Spiral
TreeMap.
Experimental Procedures. 1) The first phase investigates the usability of the sonifi-
cated WoW by an expert evaluation. Three experts, with more than five years of ex-
perience in the usability evaluation, assessed the software by the Nielsen’s heuristic
list [21]: a user scenario has been carried out in order to test each of the implemented
layouts. In particular, the experts’ tasks were to test the usability and the layout dif-
ferences between the three models of sonification: PanAndPitch, PitchAndVolume,
and BlinkAndPitch. The heuristic evaluation identified a small set of usability with a
medium and high level of severity, suggesting us that it is necessary the redesign of
the layout. Finally, all the evaluators suggested us to unify two of the sonification
models - PanAndPitch and BlinkAndPitch - proposing a new model called PanAnd-
PitchBlinking. The P&PB model conveys spatiality through the two axes of the Carte-
sian plane (x, y) by using the panning technique (x axis) and the note pitch (y axis)
and it employs the blink effect to represent the rank order of each vertex. 2) Follow-
ing the expert analysis we fix the errors of the application and, then, we performed a
usability test with two groups of participants: 4 totally blind users and 4 sighted users
(mean age 28, equally distributed by sex). This phase of evaluation aims at investigate
both the quality of users’ interaction with the visual and sonificated WoW and the
users’ satisfaction. In order to achieve these evaluation goals, we used the Partial
Concurrent Thinking Aloud (PCTA) [22] and the System Usability Scale (SUS) [23]
questionary. Each user tested the WoW after a clear and essential description of the
task and a preliminary exploration (lasting 3 min) of the layout. The experimental
task, provided by a scenario, consisted in an exhaustive search of the meaning of the
word “Armstrong” by using the WhatsOnWeb search engine. The keyboard naviga-
tion has been carried out by using either three typologies of layout -Radial, Layered e
Spiral TreeMap- or the PanAndPitch Blinking sonification. At the end of the evalua-
tion session, we interviewed all the subjects about their layout preferences and finally
they were asked to complete the SUS survey.
Beyond a Visuocentric Way of a Visual Web Search Clustering Engine 355
Experimental Results. Problems identification during the PCTA protocols were col-
lected and matched with the heuristic analysis of the first evaluation phase. All the
subjects found 19 problems, 9 out of them are related to the visual performance and
11 to the auditory performance. The one-way ANOVA analysis, carried out by SPSS
18 on task completion times for each layout, shows no significant differences (p>.05)
between the two groups and between the kind of layout - Layered layout (sighted
M=50,25'', blind M= 132,5), Spiral TreeMap layout (sighted M= 263,25'', blind M=
236'') - whereas significative difference was found on the Radial layout (F(1,6)=13,690;
p<0.05). The analysis of the SUS score shows no significant differences (p>.01) be-
tween the two participants’ groups. Therefore, since these results highlight similar
levels of efficacy, efficiency, and satisfaction between the two groups for both infor-
mation presentation modalities, the sonificated modality and the visual modality per-
formances seem to be homogeneous.
5 Conclusion
Thanks to a geometric spatial representation, WoW seems to make easier for a user
the information manipulation and findability. A global homogeneity on the evaluation
data shows that the use of WoW, or in general of a system that grants its accessibility,
reduce the digital divide. Many accessibility and usability studies pointed out that
subjects with visual disability are the main excluded from the ICTs because of the
missing information concerning “not only just to text, but also to graphics, tables, and
figures” that screen readers can not actually translate [24].
From the application point of view, WoW seems to be a versatile system that might
be used not only for searching in the World Wide Web, but also for retrieving docu-
ments in smaller environments. Moreover, according to Anderson’s theories which
states that the human knowledge is organized through semantic categorizations [25],
WoW, since clusterize information in semantic node, by emulating cognitive mental
information processing, makes easier to all users achieve and elaborate ICT informa-
tion. The indexed data representation way provided by WoW aims to optimize the
effectiveness and efficiency of the interaction. Moreover, the information visualiza-
tion techniques used to implement the architecture of WoW allow a remarkable re-
duction of the number of sensory controls necessary to perform and complete tasks.
At the same time, the WoW system provides an extensible and device-independent
architecture to lead the events through a two interaction states. The reduction of the
number of events necessary to perform a searching task allows the user navigation
through control systems and/or communication systems, such as Brain Computer
Interfaces, eye-trackers, tongue controllers and speech/sound interfaces.
References
1. De Vega., M., Cocude, M., Denis, M., Rodrigo, M.J., Zimmer, H.: The interface between
language and visuo-spatial representations. In: Denis, M., et al. (eds.) Imagery, Language
and Visuo-spatial Thinking, pp. 109–136 (2001)
356 M.L. Mele et al.
2. Avraamides, M., Loomis, J., Klatzky, R.L., Golledge, R.G.: Functional equivalence of spa-
tial representations derived from vision and language: Evidence from allocentric judg-
ments. Journal of Experimental Psychology: Human Learning, Memory & Cognition 30,
801–814 (2004)
3. Bryant, D.J.: A Spatial Representation System in Humans 3(16) (1992)
4. Millar, S.: Understanding and representing space: theory and evidence from studies with
blind and sighted children. Oxford University Press, Oxford (1994)
5. Olivetti Belardinelli, M., Federici, S., Delogu, F., Palmiero, M.: Sonification of Spatial In-
formation: Audio-tactile Exploration Strategies by Normal and Blind Subjects. In: Stepha-
nidis, C. (ed.) Universal Access in HCI, Part II, HCII 2009. LNCS, vol. 5615, pp. 557–
563. Springer, Heidelberg (2009)
6. Delogu, F., Palmiero, M., Federici, S., Zhao, H., Plaisant, C., Olivetti Belardinelli, M.:
Non-visual exploration of geographic maps: does sonification help? Disability and Reha-
bilitation: Assistive Technology (accept June 07, 2009)
7. Kramer, G.: Auditory display: Sonification, audification, and auditory interfaces. In: Pro-
ceedings of the First International Conference on Auditory Display, ICAD 1992. Addison-
Wesley, Reading (1994)
8. Di Giacomo, E., Didimo, W., Grilli, L., Liotta, G.: Graph Visualization Techniques for
Web Meta-search Clustering Engines. IEEE Transactions on Visualization and Computer
Graphics 13(2), 294–304 (2007)
9. Rugo, A., Mele, M.L., Liotta, G., Trotta, F., Di Giacomo, E., Borsci, S., Federici, S.: A
Visual Sonificated Web Search Clustering Engine. Cognitive Processing 10(suppl. 2),
286–289 (2009)
10. Borsci, S., Federici, S., Mele, M.L., Stamerra, G.: Global Rank: improving a qualitative
and inclusive level of web accessibility. In: 4th Biennial Disabilities study conference,
CeDR, Lancaster University (2008)
11. Borsci, S., Federici, S., Stamerra, G.: Web usability evaluation with screen reader users:
Implementation of the Partial Concurrent Thinking Aloud technique. Cognitive Processing
(accepted October 28, 2009)
12. Ivory, M.Y., Yu, S., Gronemyer, K.: Search result exploration: a preliminary study of
blind and sighted users’ decision making and performance. In: Extended abstracts of CHI
2004, pp. 1453–1456. ACM, New York (2004)
13. Federici, S., Borsci, S., Mele, M.L., Stamerra, G.: Web Popularity - An illusory perception
of a qualitative order in information. Universal Access in the Information Society (accept
June 07, 2009)
14. International Organization for Standardization. International Standard ISO 13407:1999:
Human-centred design processes for interactive systems (1999),
http://www.iso.org/iso/iso_catalogue/catalogue_tc/
catalogue_detail.htm?csnumber=21197
15. IBM Rule-based Accessibility Validation Environment,
http://www-03.ibm.com/able/resources/raven.html
16. Java Foundation Classes,
http://java.sun.com/javase/technologies/desktop/
17. Di Giacomo, E., Didimo, W., Grilli, L., Liotta, G., Palladino, P.: WhatsOnWeb+, an en-
hanced visual search clustering engine. In: Proceedings of the IEEE PacificVis., pp. 167–
174 (2008)
18. Kramer, G., Walker, B., Bonebright, T., Cook, P., Flowers, J., Miner, N., Neuhoff, J.:
Sonification Report: Status of the Field and Research Agenda, International Community
for Auditory Display (1997)
Beyond a Visuocentric Way of a Visual Web Search Clustering Engine 357
19. Zhao, H., Plaisant, C., Shneiderman, B., Lazar, J.: Data Sonification for Users with Visual
Impairment: A Case Study with Georeferenced Data. ACM Trans. Computer-Human In-
teraction 15(1), 1–28 (2008)
20. Atkinson, R.C., Shiffrin, R.M.: The control of short-term memory. Scientific American,
82–90 (August 1971)
21. Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: Proc. ACM CHI
1994 Conf., Boston, MA, April 24-28, pp. 152–158 (2004)
22. Borsci, S., Federici, S., Lauriola, M.: On the Dimensionality of the System Usability Scale
(SUS): A Test of Alternative Measurement Models. Cognitive Processing 10(3), 193–197
(2009)
23. Brooke, J.: SUS: a quick and dirty usability scale. In: Jordan, P.W., Thomas, B., Weerd-
meester, B.A., McClelland, A.L. (eds.) Usability Evaluation in Industry (1996)
24. Jay, C., Stevens, R., Glencross, M., Chalmers, A.: How people use presentation to search
for a link: expanding the understanding of accessibility on the web. In: W4A 2006 (2006)
25. Anderson, J.R.: The architecture of cognition. Harvard University Press, Cambridge
(1993)
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
A verbal protocol technique, adopted for a web usability evaluation, requires that the users are able to perform a double task: surfing and talking. Nevertheless, when blind users surf by using a screen reader and talk about the way they interact with the computer, the evaluation is influenced by a structural interference: users are forced to think aloud and listen to the screen reader at the same time. The aim of this study is to build up a verbal protocol technique for samples of visual impaired users in order to overcome the limits of concurrent and retrospective protocols. The technique we improved, called Partial Concurrent Thinking Aloud (PCTA), integrates a modified set of concurrent verbalization and retrospective analysis. One group of 6 blind users and another group of 6 sighted users evaluated the usability of a website using PCTA. By estimating the number of necessary users by the means of an asymptotic test, it was found out that the two groups had an equivalent ability of identifying usability problems, both over 80%. The result suggests that PCTA, while respecting the properties of classic verbal protocols, also allows to overcome the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In this way, PCTA reduces the efficiency difference of usability evaluation between blind and sighted users.
Article
Full-text available
Purpose: This study aims at evaluating the effectiveness of sonification as a mean to provide access to geo-referenced information to users with visual impairments. Method: Thiry-five participants (10 congenitally blind, 10 with acquired blindness and 15 blindfolded sighted) completed four tasks of progressive difficulty. During each task, participants first explored a sonified map by using either a tablet or a keyboard to move across regions and listened to sounds giving information about the current location. Then the participants were asked to identify, among four tactile maps, the one that crossmodally corresponds to the sonifed map they just explored. Finally, participants answered a self-report questionnaire of understanding and satisfaction. Results: Participants achieved high accuracy in all of the four tactile map discrimination tasks. No significant performance difference was found neither between subjects that used keyboard or tablet, nor between the three groups of blind and sighted participants. Differences between groups and interfaces were found in the usage strategies. High levels of satisfaction and understanding of the tools and tasks emerged from users' reports.
Article
Full-text available
Usability does not exist in any absolute sense; it can only be defined with reference to particular contexts. This, in turn, means that there are no absolute measures of usability, since, if the usability of an artefact is defined by the context in which that artefact is used, measures of usability must of necessity be defined by that context too. Despite this, there is a need for broad general measures which can be used to compare usability across a range of contexts. In addition, there is a need for "quick and dirty" methods to allow low cost assessments of usability in industrial systems evaluation. This chapter describes the System Usability Scale (SUS) a reliable, low-cost usability scale that can be used for global assessments of systems usability.
Article
Past research (e.g., J. M. Loomis, Y. Lippa, R. L. Klatzky, & R. G. Golledge, 2002) has indicated that spatial representations derived from spatial language can function equivalently to those derived from perception. The authors tested functional equivalence for reporting spatial relations that were not explicitly stated during learning. Participants learned a spatial layout by visual perception or spatial language and then made allocentric direction and distance judgments. Experiments 1 and 2 indicated allocentric relations could be accurately reported in all modalities, but visually perceived layouts, tested with or without vision, produced faster and less variable directional responses than language. In Experiment 3, when participants were forced to create a spatial image during learning (by spatially updating during a backward translation), functional equivalence of spatial language and visual perception was demonstrated by patterns of latency, systematic error, and variability.
Conference Paper
Several published sets of usability heuristics were compared with a database of existing usability problems drawn from a variety of projects in order to determine what heuristics best explain actual usability problems. Based on a factor analysis of the explanations as well as an analysis of the heuristics providing the broadest explanatory coverage of the problems, a new set of nine heuristics were derived: visibility of system status, match between system and the real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, and helping users recognize, diagnose, and recover from errors.
Article
Introduction: questions and terms 1. Modality and cognition in developmental theories and evidence 2. The modalities as convergent sources of spatial information 3. Neuropsychological evidence on convergence 4. Shape coding by vision and touch 5. Spatial coding: studies in small-scale space 6. Information and understanding large-scale space 7. Non-verbal representation: images, drawings, maps, and memory 8. Some practical implications 9. A theory of spatial understanding and development
Book
Excerpts available on Google Books (see link below). For more information, go to publisher's website : http://www.routledge.com/books/details/9780805822335/
Article
Posted 05/23/1992. Reviews evidence for the functional equivalence of spatial representations of observed environments and environments described in discourse. It is argued that people possess a spatial representation system that constructs mental spatial models on the basis of perceptual and linguistic information. Evidence for a distinct spatial system is reviewed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)