Mobile science learning for the blind

Abstract

Science learning for blind people is limited. For a variety of reasons there is a very low emphasis on science learning for such users, especially those from deprived communities. We have designed, implemented and evaluated the usability of AudioNature, an audio-based interface implemented for pocketPC devices to assist with science learning in users with visual impairments. The usability and the cognitive impact of the device were evaluated. Users accepted the interface, enjoyed the interaction with AudioNature, felt motivated, and learned science. Preliminary results provided evidence that points towards gains in problem solving skills and showed that game-based learning activities facilitate the user's interaction with the software.

Full text

Mobile Science Learning for the Blind
Abstract
Science learning for blind people is limited. For a
variety of reasons there is a very low emphasis on
science learning for such users, especially those from
deprived communities. We have designed, implemented
and evaluated the usability of AudioNature, an audio-
based interface implemented for pocketPC devices to
assist with science learning in users with visual
impairments. The usability and the cognitive impact of
the device were evaluated. Users accepted the
interface, enjoyed the interaction with AudioNature, felt
motivated, and learned science. Preliminary results
provided evidence that points towards gains in problem
solving skills and showed that game-based learning
activities facilitate the user’s interaction with the
software.
Keywords
Blind learners, virtual acoustic environments, usability,
science learning, mobile learning.
ACM Classification Keywords
K.3.1. [Computer And Education]: Computer Uses in
education - Computer-assisted instruction H.5.2.
[Information Interfaces and Presentation]: User
Interfaces - User-centered design
Copyright is held by the author/owner(s).
CHI 2008, April 5 – April 10, 2008, Florence, Italy
ACM 978-1-60558-012-8/08/04.
Jaime Sánchez
Department of Computer Science
University of Chile
Blanco Encalada 2120
Santiago, Chile
j
sanchez@dcc.uchile.cl
Héctor Flores
Department of Computer Science
University of Chile
Blanco Encalada 2120
Santiago, Chile
hflores@dcc.uchile.cl
Mauricio Sáenz
Department of Computer Science
University of Chile
Blanco Encalada 2120
Santiago, Chile
msaenz@dcc.uchile.cl
CHI 2008 Proceedings · Works In Progress April 5-10, 2008 · Florence, Italy
3201

Introduction
Blind learners hardly learn science by doing [3].
Activities such as conducting laboratory experiments,
where observation is one of the most important
processes, are almost impossible to cope with for blind
users. The same logic applies to on-site, out-of-
classroom work designed to achieve concrete
experiences in direct contact with the environment,
which most of the time is seem as too risky for blind
learners. How can we get science learning together for
students with visual impairments? A possible answer
arises from technology. If students can not interact
directly with the processes of nature for their
understanding, then we can provide them with virtual
simulators which, by means of proper feedbacks
tailored to their capacities, emulate what sighted
students could observe by using the scientific method.
The use of a mobile device in this context allows users
to interact playfully in non-static places for science
learning purposes [2,8]. Thus, on the one hand, the
user can learn science while traveling or waiting, and
on the other hand, the mobile device represents a
learning tool to be used in school activities outside the
classroom such as in a park or zoo. The use of mobile
technology and wireless communication has greatly
expanded in recent years. This situation implies a great
opportunity for learning and research, but also imposes
new challenges for resource management and
infrastructure [1].
In the last few years a diversity of software [4,6,7] and
methodologies [5] have been developed involving blind
users as the main participants, assisting them in their
learning with interactive technology. Very few studies
use mobile devices, focusing mainly on haptic
interfaces and desktop applications [6,7]. Some
software have been developed for blind children science
learning using desktop PC [6].
A common issue in these studies is that most
applications for mobile devices do not consider the blind
user’s mental model in a mobile context, using sound
peripherally and achieving a low level of interaction.
This study presents the design, implementation,
usability, evaluation and cognitive impact of using
AudioNature, an audio-based game interface
implemented for pocketPC devices to assist with
science learning for blind users.
AudioNature Design
AudioNature is an audio-based virtual simulator for
science learning implemented in a mobile device
(pocketPC) platform. To adjust the software to the
mental model of visually impaired users, a user-
centered design is used. The system presents an
ecosystem that has been altered and challenges
learners to return it to normality through interactive
tasks and problem solving.
We illustrate our proposal with an ecological problem,
which led us to come to know and understand concepts
and the consequences of actions that determine the
behavior of a balanced or unbalanced ecological
system. We challenged students to propose solutions
according to classic problem solving stages. Since the
problem is not simple to identify, learners need to
determine and apply strategies and then evaluate the
outcomes. If these results are unexpected or targeted
in the wrong direction, they reformulate the strategy
and return to interacting with the environment to verify
the results.
CHI 2008 Proceedings · Works In Progress April 5-10, 2008 · Florence, Italy
3202

The ecosystem consists of five natural variables,
classified as biological and climatic. The biological
variables correspond to three species: dogs, birds of
prey and rats. The climatic variables correspond to rain
and temperature. The user has to interact with these
variables, thus generating a biological impact on the
ecosystem. Together with these variables there are two
species, horses and cougars, which can be affected by
the decisions made when users interact with the
simulator.
Possible effects on the environment take place over
time and reduce or increase the number of biological
components in the ecosystem. AudioNature presents
random alterations to the ecosystem that learners then
have to create solutions for. The game considers an
internal timer, so the student can stop the game and
continue later, or even allocate actions and effects to
be performed in the time scheduled. The whole
interaction with AudioNature triggers auditory
feedback, allowing users to control the actions,
situations and their consequences.
Interfaces. The visual interfaces used in AudioNature
were designed with high color contrasts to model the
interaction of users with residual vision. The user’s
interaction with visual interfaces was based on a
maximum of five components distributed on the
pocketPC’s screen. This number was restricted because
of the reduced space available in mobile devices. Figure
1 shows the main interface of AudioNature, which
contains four visual elements: Time- zone indicating
the time elapsed in the game; Variables- active zone
for the effects the user inflicts upon the ecosystem;
Effect level- zone presenting the levels to modify each
variable; and Effect area- zone for the execution of
selected effect actions by dragging.
All interface elements in AudioNature are represented
through audio. Early end-user usability evaluations,
performed thorough inspections and attention to
practical user acceptation, permitted us to opportunely
define which sounds should be used and which
locutions were accepted by users. Feedback interaction
through audio was sufficient to generate a complete
user mental model of the virtual world without requiring
haptic feedback.
Interaction with AudioNature. Blind users’ interaction
with AudioNature is exercised through the touch screen
and the available buttons. The interaction triggers
audio feedback, which is used to receive instructions or
for decision making cues.
Usability Evaluation of AudioNature
Method. The methodology used consisted of applying
evaluative instruments and concrete materials in order
to evaluate the usability of AudioNature and to make a
preliminary assessment of the use of this application in
supporting the learning process for problem solving
skills and biological concepts such as biological
interaction and balance in an ecosystem. As we worked
with an intact and a reduced number of participants
with wide ranging diversity in their visual disabilities
and other associated deficits, control and experimental
groups were not feasible, and thus a case study method
was chosen.
Participants. The sample consisted of 10 learners aged
19 to 31 (5 men and 5 women), in which five of them
have low non-functional vision and five of them have
figure 1. Main Interface of AudioNature.
CHI 2008 Proceedings · Works In Progress April 5-10, 2008 · Florence, Italy
3203

acquired total blindness. All of them were legally blind,
therefore the terms blind, visually impaired, users with
residual vision and user with visual disabilities are used
indistinctly throughout the paper. Two special education
teachers, specialists in visually impaired users,
supported this evaluation as facilitators and observers
by filling in some items of the cognitive tests.
Usability and Cognitive Instruments. The instrument
used in the usability evaluation was an end-user
usability test. This instrument is divided into 24
statements in a likert scale (12 statements regarding
interaction with AudioNature and 12 statements related
to the hardware used). To evaluate the impact of
AudioNature on the learning of biological interaction
concepts and problem solving skills, preliminary pre-
tests and post-tests were administered. Cognitive
testing consisted of different questionnaires to evaluate
the learning of biology concepts, the behaviors and
skills of visually impaired users and their performance
with the Cognitive Tasks.
Procedure. Users evaluated AudioNature during 2
sessions of 1.5 hours each over a one-month period
(see figure 2). In each session all participants spent 30
minutes on the evaluation. Each user was provided with
a pocketPC and a pair of headsets for interaction.
Cognitive tasks (see figure 2) were carried out in 2
sessions of 2.5 hours each. In each session all
participants spent 30 minutes on the evaluation. For
every cognitive task, the procedure used consisted of
the following activities: 1. The user explores the
concrete material, 2. The user performs actions to
interact and progress in the activity, 3. The user
chooses a certain randomly determined action, 4. The
user solves tasks and problems posed in the activities,
5. The user starts using AudioNature, 6. The user
executes actions to interact and progress in the game,
7. The user selects the correct effect on the simulator,
solves the tasks and problems posed, and 8. The user
joins the rest of the test group once again and
continues participating in the game by using concrete
materials. At each of these stages, the student
experienced some type of learning. Through interaction
with AudioNature they learned concepts and then
applied them in the learning activities without
technology.
Results. The usability evaluation of AudioNature showed
that the interaction between users and the mobile
device through sound feedback support is a good
combination to aide in the learning of science for these
users. Results of the users’ evaluations of AudioNature
(see figure 3) indicate that the highest scores
correspond to users with residual vision, but still legally
blind.
Users with total blindness obtained the lowest scores.
This led us to redesign both the visual and audio
software interfaces, to better fit the needs of these
users (see figure 3). The statements that obtained the
highest difference between users with residual vision
and the totally blind were: 6. The general design of the
software is appropriate (low vision 5.0, totally blind
3.3), 7. The presentation of the information is
appropriate (low vision 4.3, totally blind 3.0), 8. The
software’s interface is pleasant (low vision 5.0, totally
blind 3.0). Five of the seven users who interacted with
AudioNature noticed that the reduction or increase in
the volume of animal sounds corresponded indeed to
the change of the number of animals in the simulation.
This is interesting, since it confirms that the sound-
figure 2. Cognitive Tasks.
figure 3. Software Evaluation.
CHI 2008 Proceedings · Works In Progress April 5-10, 2008 · Florence, Italy
3204

based mobile system was well accepted by users with
residual vision in relating the volume of these sounds to
the number notions.
Users with residual vision assigned, on average, a high
score (4.3) to the hardware used, unlike the totally
blind users who evaluated the hardware with a low
average score (3.0 score). The results obtained were: I.
Hardware was easy to use (4.7), II. Buttons are
correctly placed (4.0), III. Hardware is easy to
manipulate (4.2), IV. The design of the hardware is
adequate (3.9), V. The physical interface is pleasant
(3.8), VI. Hardware has all expected functionalities
(3.6), and VII. Hardware has all expected capacities
(3.8).
The hardware utilized was well accepted by users with
residual vision. The hardware was accepted only fairly
well by totally blind users. There was no mention of any
difficulties when using the mobile device (see figure 4).
This result is important because users were accustomed
to working with desktop computers by using only a
keyboard, which provides an appropriate tactile aid for
interaction. This means that they were able to work
with small tactile aids, such as touch screen borders
and the buttons available on the device with relative
ease compared to a desktop keyboard. This also implies
that the proposed software interface grants good use
support, since it does not require additional aids for
use.
Some comments about the interface made by users
were: "The sounds are clear and real, the voices are
distinguishable and understandable", "the buttons could
be bigger and less sensitive to pressure", "In general, I
liked everything because the subject of AudioNature is
very interesting", "The sounds and images of animals
are nice and I can image what I do", "I was
complicated with the option in the middle of the
screen", "I didn’t like the appearance of secondary
voices in the background, complementing the main
voice".
Figure 5 shows a preliminary pre-test/post-test
cognitive evaluation. Most users demonstrated gains in
their ability to solve cognitive tasks after having used
AudioNature. Case 7 obtained the lowest average score
(44.3%) whereas the lowest gain was attained by case
3. This means that there was an increase of 21.6 points
in the general average of the post-test in relation to the
pre-test. Case 8 obtained the highest pre-test average
score (84%) while case 2 obtained the highest post-
test score (100%). This indicates an increase of 16
points in the general average among the highest post-
test scores in relation to the pre-test scores.
Conclusions & Future Work
We have introduced AudioNature, a proof-of-concept
for the use of PDA simulation games by blind people to
learn biological concepts in mobile contexts. Usability
and preliminary cognitive evaluations for using
AudioNature, along with playful cognitive activities were
implemented with users in interactive processes for
science learning. The results obtained were highly
satisfactory and challenging. Learners learned biology
concepts and performed problem solving tasks
correctly.
According to the set of evaluations used together with
each process involved in the cognitive intervention,
such as entry activities, cognitive task sessions and
evaluation activities, we confirm the appropriateness of
figure 5. Cognitive Evaluation.
figure 4. Hardware Evaluation.
CHI 2008 Proceedings · Works In Progress April 5-10, 2008 · Florence, Italy
3205

designing game-based cognitive interventions for blind
user rehabilitation through interaction with mobile
devices. This form of involvement allowed users to
perform more securely in the development of their
cognitive processes. Playful, concrete interventions
facilitated a more confident mode of user-software
interaction, triggering the development and elaboration
of action theories to control the environments provided
by the software.
Thus, through this study we have shown that certain
interface elements must be designed carefully to make
them more usable and pertinent to blind people, for
example, enabling them to make a correct relationship
between quantity and volume of a sound, making
sensitive buttons not very large, and avoiding to
Overlay voices that cause confusion. Mobile devices are
designed for visual use. Studies like this help us to
understand the interaction between blind people and
mobile devices in order to design applications such as
AudioNature that go beyond technological devices such
as PCs that obligate blind learners to remain in a static
corporal position, diminishing their possibilities for
learning, interacting, and communicating while moving,
and thus better integrating themselves into society. We
will make a deeper cognitive evaluation in order to
prove the impact of this kind of software.
We propose for future work the task of identifying
whether the biology content embedded in the game
helps and assist student science learning. Finally,
testing AudioNature for learning purposes in different
places beyond the classroom such as a museum or a
zoo would add even more information about the
contexts and scenarios for which this tool is better
suited.
Acknowledgments
This report was funded by the Chilean National Fund of
Science and Technology, Fondecyt, Project 1060797
References
1. Anderson, P. & Blackwood, A. (2004). Mobile and
PDA Technologies and their future use in education.
JISC Technology & Standards Watch, Bristol, UK.
www.jisc.ac.uk/uploaded_documents/ACF11B0.pdf
2. Csete, J., Wong, Y. & Vogel, D. (2004). Mobile
devices in and out the classroom. In Cantoni, L. &
McLoughlin (Eds.). Proc. of ED-MEDIA 2004, June
21-26, Lugano, Switzerland, pp. 4729-4736.
3. Kumar, D., Ramasamy, R., Stefanich, G. (2001)
Science instruction for students with visual
impairments. ERIC 464805, pp. 1-2.
4. Sahyun, S., Gardner, S., & Gardner, C., Audio and
Haptic Access to Math and Science -Audio graphs,
Triangle, the MathPlus Toolbox, and the Tiger
printer. Proc. of the 15th IFIP WCC, Vienna,
September 1998, pp. 78-86.
5. Sánchez, J. & Flores, H. (2004). Memory
enhancement through audio. Proc. of ACM Assets
2004, Atlanta, Georgia, USA, October 18-20, pp. 24-
31.
6. Sánchez, J. (2007). A model to design interactive
learning environments for children with visual
disabilities. Journal Education and Information
Technologies, 12(3), pp. 149-163.
7. Sánchez, J., Sáenz, M. (2006). 3D sound interactive
environments for blind children problem solving
skills. Behaviour & Information Technology, 25(4),
pp. 367-378.
8. Williams, M., Jones, O., Fleuriot, C. & Wood, L.
(2005). Children and emerging wireless
technologies: Investigating the potential for spatial
practice. Proc. of ACM CHI, April 2-7 2005, Portland,
Oregon, USA, ACM Press, pp.819-828.
CHI 2008 Proceedings · Works In Progress April 5-10, 2008 · Florence, Italy
3206