ArticlePDF Available

Abstract and Figures

This study presents the design, implementation, and usability evaluation of AudioNature, an audio-based interface implemented for pocketPC devices to assist science learning in users with visual impairments. The usability evaluation sessions helped to redesign the interface and adjust it to the user's mental model while operating a pocketPC mobile device. Users accepted the interface, enjoyed the interaction with AudioNature, felt motivated, and learned science. Cognitive evaluations provided evidence that points toward gains in problem-solving skills and showed that game-based learning activities facilitate the user's interaction with the software. Users showed more self-confidence when designing and elaborating action theories to solve the problems posed.
Content may be subject to copyright.
C
YBER
P
SYCHOLOGY
& B
EHAVIOR
Volume 11, Number 3, 2008
© Mary Ann Liebert, Inc.
DOI: 10.1089/cpb.2007.0110
Virtual Mobile Science Learning for Blind People
JAIME SÁNCHEZ, Ph.D., and HÉCTOR FLORES
ABSTRACT
This study presents the design, implementation, and usability evaluation of AudioNature, an
audio-based interface implemented for pocketPC devices to assist science learning in users
with visual impairments. The usability evaluation sessions helped to redesign the interface
and adjust it to the user’s mental model while operating a pocketPC mobile device. Users ac-
cepted the interface, enjoyed the interaction with AudioNature, felt motivated, and learned
science. Cognitive evaluations provided evidence that points toward gains in problem-solv-
ing skills and showed that game-based learning activities facilitate the user’s interaction with
the software. Users showed more self-confidence when designing and elaborating action the-
ories to solve the problems posed.
356
INTRODUCTION
L
EARNERS WITH VISUAL IMPAIRMENTS
hardly learn
science by doing.
2,3
Activities such as conduct-
ing laboratory experiments, where observation is
one of the most important processes, are almost im-
possible to cope with for blind users. The same logic
applies to onsite, out-of-classroom work designed
to achieve concrete experiences in direct contact
with the environment, which most of the time is
seen as too risky for learners with visual impair-
ments.
The use of a mobile device in this context allows
users to interact playfully in nonstatic places for sci-
ence learning purposes.
1,6,8
Thus, on the one hand, the
user can learn science while traveling or in waiting
contexts, and on the other hand, the mobile device
provides a learning tool to be used in school activi-
ties outside the classroom, such as in a park or zoo.
6
In the last few years, a diversity of software and
methodologies
4,5,7
have been developed involving
blind users as the main participants, assisting them
in their learning with interactive technology. Very
few studies use mobile devices,
1,7
focusing mainly
on haptic interfaces
4
and desktop applications.
5,7
These authors have shown that audio-based virtual
environments are a powerful incentive for blind
learners to develop and train cognitive skills and
learn specific content.
DESIGN OF AUDIONATURE
AudioNature is an audio-based virtual simulator
for science learning implemented in a mobile device
(pocketPC) platform. To adjust the software to the
mental model of visually impaired users, a user-cen-
tered design was used. The system presents an
ecosystem that has been altered and challenges
learners to return it to normality through interac-
tive tasks and problem solving.
Department of Computer Science, University of Chile, Santiago, Chile.
Rapid Communication
AU1
Visual interfaces
The visual interfaces used in AudioNature were
designed with high-color contrasts to model the in-
teraction of users with residual vision.
Audio interfaces
All interface elements in AudioNature are repre-
sented through audio. Early usability evaluations
permitted us to opportunely define which sounds
had to be used and which locutions were accepted
by users. When learners begin an interaction with
AudioNature, they listen to a welcome message and
an introduction to the software. Then, an audio cue
is played together with information related to the
objects and elements provided by the software. An-
imals, for instance, are represented by real sounds;
instructions and variables are represented through
text-to-speech (TTS) with a synthesized voice.
Interaction with AudioNature
The interaction of blind users with AudioNature
is made through the touch screen and the available
buttons. While the user interacts with AudioNature,
an audio feedback is reproduced, thus allowing the
user to realize the effects of actions and the state of
the ecosystem.
USABILITY EVALUATION
OF AUDIONATURE
Method
The methodology used consisted of applying
evaluative instruments and concrete materials in or-
der to evaluate the usability of AudioNature and to
make a preliminary assessment of the use of this ap-
plication in supporting the learning process for
problem-solving skills and biological concepts such
as biological interaction and balance in an ecosys-
tem.
Participants
The sample consisted of 10 learners aged 19 to 31
(5 men and 5 women). Five of them have low, non-
functional vision, and five of them have acquired
total blindness. Two special education teachers sup-
ported this evaluation as facilitators and observers
by filling in some items of the cognitive tests.
Instruments
The instrument used in the usability evaluation
was an end-user’s usability test. To evaluate the im-
pact of AudioNature on the learning of biological
interaction concepts and problem-solving skills,
pretests and posttests were applied.
After that, users participated in hands-on activi-
ties concerning two cognitive tasks. These tasks
made use of concrete materials to allow users with
visual impairments to interact playfully with objects
and concepts integrated in AudioNature.
Procedure
Users evaluated AudioNature during two ses-
sions of 1.5 hours each over a month. Each user was
provided with a pocketPC and a pair of headsets
for interaction. In this evaluation, the user interacted
with AudioNature using all the functions available
with the help of the facilitators and then answered
the end-user’s usability test.
Cognitive tasks were carried out in two sessions
of 2.5 hours each. In these activities, the user inter-
acted with the concrete material and AudioNature
in the same session. The user first explored the con-
crete material, then played with AudioNature, and
then joined the rest of the testing group and con-
tinued with the concrete materials.
RESULTS
The usability evaluation of AudioNature showed
that the interaction between users and the mobile
device through sound feedback support is a good
combination to aid in the learning of science in these
users (see Fig. 1).
Users with total blindness obtained the lowest
scores. Statements 6, 7, and 8 regarding software ob-
tained the highest difference between users with re-
sidual vision and users who were totally blind. Five
of the seven users who interacted with AudioNa-
ture noticed that the reduction or increase in the vol-
ume of animal sounds corresponded indeed to the
change of the number of animals in the simulation.
This is interesting, because it confirms that a sound-
based mobile system was well accepted by users
with residual vision relating the volume of these
sounds to the number notions.
Users with residual vision assigned, on average, a
high score (4.3 out of 5.0 points) to the hardware used,
unlike totally blind users, who evaluated with a low
average score (3.0). Totally blind users fairly accepted
the hardware. There was no mention of any difficul-
ties when using the mobile device (see Fig. 1B). Fig-
ure 2 shows preliminary pretest/posttest cognitive
evaluation. Most users demonstrated gains in solving
cognitive tasks after using AudioNature.
VIRTUAL MOBILE SCIENCE LEARNIGN FOR BLIND PEOPLE 357
On average, pretest scores were 65.64% and
posttest scores reached 84.96%, increasing by 19.25
points. In only one case was there no difference be-
tween pretest/posttest scores. The other nine cases
increased their scores.
Content evaluation showed that, on average,
learners’ correct answers increased at posttest, and
the minimum and maximum correct answers also
increased. Cognitive tasks showed a high attain-
ment in all learners.
CONCLUSION
We have introduced AudioNature, a proof-of-
concept for the use of PDA simulation games by
blind people to learn biological concepts in mobile
contexts. Usability and preliminary cognitive eval-
uations in using AudioNature, along with playful
cognitive activities, were implemented involving
users in interactive processes for science learning.
The results obtained were highly satisfactory and
SÁNCHEZ AND FLORES
358
FIG. 1. Usability evaluation of AudioNature (A) and pocketPC (B).
FIG. 2. Pretest/posttest gains in cognitive impact.
A
B
challenging. Learners learned biology concepts and
performed problem-solving tasks correctly.
The usability evaluation during software imple-
mentation led us to define, modify, and improve the
design of the interfaces and the interaction pro-
cesses in order to adjust them to the mental model
of blind learners. Mobility allowed them more in-
teractions over distances, freedom to move, and the
extension of the space daily navigated. Users felt en-
joyment and were motivated when they interacted
with AudioNature.
According to the set of evaluations used together
with each process involved in the cognitive inter-
vention, such as entry activities, cognitive tasks ses-
sions, and evaluation activities, we confirm the ap-
propriateness of designing game-based cognitive
interventions for blind user rehabilitation through
interaction with mobile devices. The rationale be-
hind a mobile device design is easy access for ex-
tensive use over a long period of time. Then, com-
paring the increase in performance in the blind and
control groups may give some insight into the im-
pact of auditory feedback on the blind brain.
Mobile devices are designed for visual use. Studies
like this one can help us to understand the interaction
of blind people with mobile devices in order to design
applications such as AudioNature that go beyond
technological devices such as PCs, which obligate
blind learners to remain in a static corporal position.
Improved designs will increase their possibilities for
learning, interacting, and communicating while mov-
ing and therefore integrating themselves into society.
ACKNOWLEDGMENTS
This report was funded by the Chilean National
Fund of Science and Technology, Fondecyt, Project
1060797.
REFERENCES
1. Csete J, Wong Y, Vogel D. (2004) Mobile devices in and
out the classroom. In: Cantoni L, McLoughlin C, eds.
Proceedings of ED-MEDIA June 21–26, 2004. Lugano,
Switzerland, pp. 4729–36.
2. Kumar D, Ramasamy R, Stefanich G. Science instruc-
tion for students with visual impairments. ERIC Digest
2001; pp. 1–2.
3. Mastropieri M, Scruggs T. (1992) A practical guide for
teaching science to students with special needs in inclusive
settings. West Lafayette, IN: Purdue Research Center,
Purdue University.
4. Sahyun S, Gardner S, Gardner C. (1998) Audio and
haptic access to math and science-audio graphs. Pro-
ceedings of the 15th IFIP WCC. Vienna, pp. 78–86.
5. Sánchez J, Sáenz M. 3D sound interactive environ-
ments for blind children problem solving skills. Be-
haviour & Information Technology 2006; 25:367–78.
6. Sánchez J, Salinas A, Sáenz M. (2007) Mobile game-
based methodology for science learning. In: Jacko J, ed.
Human-computer interaction, Part IV. Series: Lecture
Notes in Computer Science (Vol. 4553), Springer,
Berlin: Heidelberg, pp. 322–31.
7. Westin T. (2004) Game accessibility case study: Ter-
raformers—a real-time 3D graphic game. Proceedings of
the 3rd ICDVRAT 2004. Oxford, UK, pp. 95–100.
8. Williams M, Jones O, Fleuriot C, Wood L. (2005) Chil-
dren and emerging wireless technologies: investigat-
ing the potential for spatial practice. Proceedings of
ACM CHI, April 2–7 2005. Portland, OR: ACM Press,
pp. 819–28.
Address reprint requests to:
Dr. Jaime Sánchez
Department of Computer Science
University of Chile
Blanco Encalada 2120
Santiago, Chile 2777
E-mail: jsanchez@dcc.uchile.cl
VIRTUAL MOBILE SCIENCE LEARNIGN FOR BLIND PEOPLE 359
... Different audio base virtual environments are also helpful for blind people to learn, develop and improve their skills. Audio Nature [15] is software, that worked on mobile devices for learning for the visually impaired. The interface models in software were represented through audio to an adjacent location because it's easy for users to interact with them. ...
Article
Full-text available
A patent is legal protection which gives an inventor the right to exclude others from performing a certain activity in the country of issuance. Innovations in form of patents are very beneficial for societies i.e., economic growth, reducing unemployment, increasing motivation and enhancing educational support. Visually impaired people are also part of society who faced several difficulties during access to educational material. Continual enrichment in technology has yielded a lot of innovative solutions for visually impaired people which have been patented. These solutions are in form of published articles, systems, applications and websites. Researchers can access journals, conferences, book chapters and technical reports easily but normally patents are unused sources of information in education. The main reason behind this is the complex structure of the patents which causes difficulty in reading. The proposed article aims to summarize the patent information in such a way that researchers and developers can easily access the information. Discuss some important patent offices from all over the world, along with their patent tools. A detailed analysis was conducted for patent documents that are of help to visually-impaired people learning mathematics.
... A significant increase in the mean score of the learning effectiveness test was observed both within the experimental group and the control group, and no significant difference was found between the two groups. Multisensory teaching material used in the lecture provided students with additional stimulations to supplement auditory stimulation, playing the role of cognitive scaffolding and helping them more effectively learn abstract knowledge such as science content (e.g., Fraser & Maguvhe, 2008;Sánchez & Flores, 2008;Verver, Vervloed, Yuill, & Steenbergen, 2020;Volpe & Gori, 2019). It may be that multisensory teaching materials were also helpful in promoting learning effectiveness in lecture-based classrooms. ...
Article
Visual challenges for learners who are blind may lead to decreased learning effectiveness, motivation, and engagement in learning science-related courses due to accessibility issues. Therefore, the purpose of the study was to integrate the multi-sensory scaffolding gamification teaching model using mobile technology and card games to develop an accessible biology learning activity for totally blind students. The mobile technology provides cognitive scaffolding through the auditory sense. Moreover, the card games emphasize learners' collaboration process, which leads to more in-depth peer interaction. With the integration of mobile technology and card games, the researchers systematically investigated students’ learning effectiveness, motivation, flow state, game acceptance, and behavioral patterns in the proposed gamification science course unit. Thirty-two middle school students with total blindness participated in this study. The findings indicated that the learning effectiveness of the learners within both groups (multi-sensory scaffolding teaching materials were applied) showed a significant increase. The results also showed that the design of the multi-sensory scaffolding met the learning needs of learners with total blindness in science education. These findings can contribute to the future development of designing and teaching science-related courses by integrating an accessible multi-sensory scaffolding approach that enriches and increases the understanding of abstract science concepts of students who are totally blind.
... The relevant papers are from 1999 to 2014, being 80% of the papers from 2008 on. The selected papers were: [13,51,2,35,3,4,8,15,12,17,18,41,42,24,25,30,28,29,32,33,31,37,39,38,40,36,34,47,48,49]. Among these, 25 papers described 21 distinct applications: 17 multimodal games and four multimodal navigation virtual environment. ...
Conference Paper
Full-text available
Multimodal serious video games are relevant tools to enhance the cognitive skills of people who are blind. For this purpose, it is necessary that designers and developers be able to create interfaces and interactions using the multimodal elements properly. Thus, there is a need to know the relevant elements for such applications, as well as to understand their roles and relationships. In this paper, we propose and discuss a 4-dimension classification: Interface, Interaction, Cognition, and Evaluation. Such classification was assembled from the features related to the design and evaluation of 21 multimodal video games and environments, identified via a bibliographic review based on the systematic review approach. Besides, we classify and discuss the 21 multimodal applications into the proposed classification.
... The relevant papers are from 1999 to 2014, being 80% of the papers from 2008 on. The selected papers were: [13,51,2,35,3,4,8,15,12,17,18,41,42,24,25,30,28,29,32,33,31,37,39,38,40,36,34,47,48,49]. Among these, 25 papers described 21 distinct applications: 17 multimodal games and four multimodal navigation virtual environment. ...
Conference Paper
Full-text available
Multimodal serious video games are relevant tools to enhance the cognitive skills of people who are blind. For this purpose, it is necessary that designers and developers be able to create user interfaces and interactions using the multimodal components properly. Thus, there is a need to know the key components to be considered for such applications, as well as to understand their roles and relationships. In this paper, we propose and discuss a 4-dimension classification: Interface, Interaction, Cognition, and Evaluation, to analyze the design of multimodal videogames for the cognition of people who are blind. Such classification was assembled from features related to the design and evaluation of a number of multimodal video games and virtual environments, identified via literature review based on systematic review methodology. We also classify and discuss multimodal applications within the proposed categorization.
Article
Blind and low-vision (BLV) youth have been excluded from technology-mediated learning in school for two decades due to the systemic inaccessibility of K-12 instructional technologies. Accessibility guidelines that address BLV youth’s needs would help schools address this systemic inequity. But such guidelines do not yet exist nor does the robust knowledgebase required to write them. Herein, I report three key findings from a mixed methods user study of BLV youth’s lived experiences learning with access technologies (AT) and instructional technologies in U.S. schools. First, these youth participants are using a broader range of AT than youth in previous studies. Second, the youth in this study frequently developed their AT literacy outside of school and most did not begin developing AT literacy in earnest until they were teens. Third, the youth defined accessible instructional technologies as those that are (1) easy to learn to use and (2) easy to use to learn and they used an inquiry process to evaluate accessibility. Building on the findings, I offer six preliminary design guidelines for accessible instructional technologies. Finally, I urge other scholars to study BLV youth’s digital access needs so that standards setting bodies eventually have sufficient data to write youth accessibility guidelines.
Article
Full-text available
Resumen Debido a las enormes distancias, el sentido de la visión es la principal herramienta con la que cuenta el observador para contemplar los fenómenos astronómicos, lo que representa una desventaja para las personas con limitaciones visuales. Dado lo anterior, este artículo presenta el desarrollo de una estrategia que a partir de actividades alrededor del Sol, los planetas, los cuerpos menores y la Luna, permite comunicar la astronomía a personas con diferentes tipos de discapacidad visual. La metodología se enfoca en la representación de características morfológicas, principios físicos y fenómenos astronómicos que tienen lugar en el Sistema Solar. La estrategia fue aplicada durante cuatro sesiones a una muestra compuesta por 15 personas en condición de ceguera y 15 con baja visión en la ciudad de Pereira, Colombia. La valoración realizada mostró un incremento significativo del porcentaje de acierto a las preguntas planteadas, lo cual evidenció la efectividad de la estrategia desarrollada. Palabras clave: Baja visión, discapacidad visual, divulgación de la ciencia, inclusión.
Article
Full-text available
Families have moved, or have been moved, from the streets into their homes or more specifically, into their bedrooms. Digital technology – computer games, mobiles & the internet and email are referred to as new technology when they are discussed in relation to Health & structure of families and Many parents, clinicians, researchers, and policy makers are concerned that electronic tools, especially those featuring violent content, may be harmful to individuals especially youths. This article is a documentary study, combination of multiple focus is chosen. In this article, Internet, Mobile and computer games. This article looks into the role of the technology on families. Specifically, it examines how the mediated space that this technology creates matters. Technological change often creates ungrounded fears but also overinflated hopes. In order to minimize risks and to seize chances, systematic, empirical, and ideally experimental research is crucial in all over the world. Major changes in family structure and environments might potentially severely disrupt family functioning, thus diminishing a family’s ability to cope with stress, so by the advent of a technology.
Article
Full-text available
The air traffic control industry is highly regulated, with stringent processes and procedures to ensure that IP (Intellectual Property) and workplaces are kept secure. The training of air traffic controllers (ATCs) and other roles relating to air traffic services is a lengthy and expensive process. The rate in which trainees can be trained is projected to fall significantly short of the demand for staff to work in the air traffic industry. This paper focuses on two prototype mobile training applications—Location Indicators (LI) and the Aircraft Control Positions Operator (ACPO) Starter Pack. LI and the ACPO Starter Pack have been produced to explore how air traffic control training could be improved and supported using digital applications. Each application explores a key learning area for trainees in the air traffic control industry and presents an alternative to the equivalent training that is currently in use. The two prototypes that have been designed focus on producing a succinct user experience alongside gamified elements to improve engagement. As part of this paper, usability testing has been undertaken with LI and the ACPO Starter Pack. A total of nine usability tests have been undertaken at four different locations. These usability tests consisted of participants from differing demographics, varying experience with the current training and differing amounts of time with both applications. The System Usability Scale (SUS) was adapted and used to quantify participant's reactions to the usability of each application. Usability scores for both applications were collected and then averaged to produce an overall score for each application. We can conclude from both usability scores and qualitative feedback that digital applications have the potential to engage future trainees in the air traffic services industry.
Article
This review examined articles on mobile apps for science learning published from 2007 to 2014. A qualitative content analysis was used to investigate the science mobile app research for its mobile app design, underlying theoretical foundations, and students' measured outcomes. This review found that mobile apps for science learning offered a number of similar design features, including technology-based scaffolding, location-aware functionality, visual/audio representations, digital knowledge-construction tools, digital knowledge-sharing mechanisms, and differentiated roles. Many of the studies cited a specific theoretical foundation, predominantly situated learning theory, and applied this to the design of the mobile learning environment. The most common measured outcome was students' basic scientific knowledge or conceptual understanding. A number of recommendations came out of this review. Future studies need to make use of newer, available technologies; isolate the testing of specific app features; and develop additional strategies around using mobile apps for collaboration. Researchers need to make more explicit connections between the instructional principles and the design features of their mobile learning environment in order to better integrate theory with practice. In addition, this review noted that stronger alignment is needed between the underlying theories and measured outcomes, and more studies are needed to assess students' higher-level cognitive outcomes, cognitive load, and skill-based outcomes such as problem solving. Finally, more research is needed on how science mobile apps can be used with more varied science topics and diverse audiences.
Conference Paper
Full-text available
In this paper, we describe design work with 36 children aged 9 and 10 in Bristol, United Kingdom. The design work was conducted using emerging mobile and wireless technology which has the potential to impact on the problematic issue of children's access to, use of, and safety within the wider urban environment. A series of workshops are described in which children were encouraged to think about their use of an outdoor space before their introduction to the technology. The children designed and created "soundscapes" in the outdoor environment. The future potential impact of the technology on children's spatial practice is discussed and the concept of children "tagging" environmental hazards is raised.
Conference Paper
Full-text available
This work presents the features and results of a problem-solving collaborative game for 8th graders science classes’ curriculum. Software for pocketPC was developed for this game, based on a complete framework methodology with students and teachers. From our point of view, the key to integrate mobile devices into school is the methodological framework which provides meaning; technology by itself does not contribute much to education. The evaluation study was focused on software usability and the results in the application of the methodology, observing their performance in problem-solving skills. A high degree of user satisfaction with the final product was found. They were motivated to participate actively in the proposed tasks. Results indicate that the experience contributed to the development of the student’s problem-solving skills obtaining positive gains as a result of this experience.
Article
Full-text available
Audio-based virtual environments have been increasingly used to foster cognitive and learning skills. A number of studies have also highlighted that the use of technology can help learners to develop effective skills such as motivation and self-esteem. This study presents the design and usability of 3D interactive environments for children with visual disabilities to help them solve problems in Chilean geography and culture. We introduce AudioChile, a virtual environment that can be navigated through 3D sound to enhance spatiality and immersion throughout the environment. 3D sound is used to orientate, avoid obstacles, and identify the positions of various characters and objects within the environment. We have found during the usability evaluation that sound can be fundamental for attention and motivation purposes during interaction. Learners identified and clearly discriminated environmental sounds to solve everyday problems, spatial orientation, and laterality.
A practical guide for teaching science to students with special needs in inclusive settings
  • M Mastropieri
  • T Scruggs
Mastropieri M, Scruggs T. (1992) A practical guide for teaching science to students with special needs in inclusive settings. West Lafayette, IN: Purdue Research Center, Purdue University.
Game accessibility case study: Terraformers-a real-time 3D graphic game
  • T Westin
Westin T. (2004) Game accessibility case study: Terraformers-a real-time 3D graphic game. Proceedings of the 3rd ICDVRAT 2004. Oxford, UK, pp. 95-100.
Science instruction for students with visual impairments
  • D Kumar
  • R Ramasamy
  • G Stefanich
Kumar D, Ramasamy R, Stefanich G. Science instruction for students with visual impairments. ERIC Digest 2001; pp. 1-2.
Audio and haptic access to math and science-audio graphs. Proceedings of the 15th IFIP WCC
  • S Sahyun
  • S Gardner
  • C Gardner
Sahyun S, Gardner S, Gardner C. (1998) Audio and haptic access to math and science-audio graphs. Proceedings of the 15th IFIP WCC. Vienna, pp. 78-86.