Conference PaperPDF Available

Blind Children Developing Mathematics Skills through Audio

Authors:

Figures

Content may be subject to copyright.
Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Oxford, UK, 2004
2002 ICDVRAT/University of Reading, UK; ISBN 07 049 11 44 2 183
AudioMath: blind children learning mathematics through audio
J H Sánchez and H E Flores
1Department of Computer Science, University of Chile,
Blanco Encalada 2120, Santiago, CHILE
jsanchez@dcc.uchile.cl, hflores@dcc.uchile.cl
ABSTRACT
Diverse studies using computer applications have been implemented to improve the learning of
children with visual disabilities. A growing line of research uses audio-based interactive
interfaces to enhance learning and cognition in these children. The development of short-term
memory and mathematics learning through virtual environments has not been emphasized in
these studies. This work presents the design, development, and usability of AudioMath, an
interactive virtual environment based on audio to develop and use short-term memory, and to
assist mathematics learning of children with visual disabilities. AudioMath was developed by
and for blind children. They participated in the design and usability tested the software during
and after implementation. Our results evidenced that sound can be a powerful interface to
develop and enhance memory and mathematics learning in blind children.
1. INTRODUCTION
Audio-based interfaces to foster cognition in blind children have been increasingly developed for people with
disabilities. A number of studies have focused on the design of audio-based interfaces and evaluated their
impact on learning and cognition (Baldis, 2001; Lumbreras, & Sánchez, 1998; McCrindle & Symons, 2000;
Mereu & Kazman, 1996; Sánchez, 2000; Sánchez, 2001; Tan, 2000, Winberg & Helltrom, 2000).
Most of these studies are based on interactive software that cannot be fully adapted to their needs and
requirements. Actually, most of them are fixed prototypes without enough flexibility and versatility.
A seminal study developed the first proof-of-concept application to navigate virtual environments through
audio by blind children (Lumbreras & Sánchez, 1998). The study used audio to enhance spatial cognitive
structures and found that spatialized sound can be used to develop spatial navigation skills in a virtual
environment.
Some studies designed experiences with audio stimuli to simulate visual cues for blind learners (Mereu &
Kazman, 1996). As a result, they found that by using 3D audio interfaces blind people can help to localize a
specific point in a 3D space. They performed with precision but slower than sighted people concluding that
navigating virtual environments with only sound can be more precise to blind people in comparison to
sighted persons (Mereu & Kazman, 1996).
Other studies describe the positive effect of 3D audio-based virtual environments (Cooper & Taylor,
1998). A study in the same line of research used sensory virtual environments through force feedback
joysticks simulating real places such as the school or work place. They probed the hypothesis that providing
appropriate spatial information through compensatory channels can improve the performance of blind people
(Lahav & Mioduser, 2000).
A research work in the same track of (Lumbreras & Sánchez, 1998) concluded that a traditional computer
game such as Space Invader can be replicated with 3D sound. Researchers used force feedback joysticks as
input interface by letting to play blind to sighted children to share the same experience (McCrindle &
Symons, 2000). An interesting study tested blind and sighted people with covered eyes across audio stimuli
by tracing specific places through sound. The skill to hold in mind the specific localization without
concurrent perceptual information or spatial update was evaluated (Loomis et al., 2002).
Studies focusing the development of computer applications to develop and enhance memory through
virtual environments in people with disabilities are scarce. Some memory processes such as spatial
distribution through virtual reality environments were developed by active participation of people with
Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Oxford, UK, 2004
2004 ICDVRAT/University of Reading, UK; ISBN 07 049 11 44 2
184
disabilities (Attree et al., 1996). As a result, active participation enhanced memory for spatial layout whereas
passive observation enhanced object memory.
Other studies have also used immersive virtual reality to analyze spatial memory in patients with brain
damage. They developed a test as proof-of-concept of the critical role that can be played by virtual reality as
experimental tool for memory purposes (Morris et al., 2000). The use of virtual reality was also found to be
relevant to improve the validity and reliability of neurological evaluation and rehabilitation (Rizzo et al.,
2002). The impact of spatial audio on memory was analyzed concluding that audio stimuli improves memory
and perceived comprehension by choosing spatial audio instead of non spatial (Baldis, 2001).
A trend in the literature is the absence of long-term usability studies. Increasing voices in the field are
sustaining that spatial audio may have a reduced impact when it is not associated to specific cognitive tasks.
Besides to the need for sound-based virtual environment studies there is a demand for more rigorous and
systematic usability studies by and for children with visual disabilities.
A most recent study have developed a sound-based virtual environment for children with visual
disabilities to enhance memory and fully tested the software with diverse children concluding that it can help
to enhance the development and practice of short-term memory in children with visual disabilities. (Sánchez
and Flores, 2003, 2004).
Learning the basis of mathematics has been a current issue in literacy literature. Most studies worldwide
agree that children do not learn mathematics adequately in the early grades. This has a tremendous impact on
further learning. In a world heavily based on science and technology children without understanding the
basics of math limit their role in the society. Children with visual disabilities are not the exception. Actually
in many respects this issue is radicalized in these children. When blindness is associated to social deprivation
the issue of learning primary school mathematics is really a critical issue (Edwards & Stevens, 1993; Sahyun
et al., 1998; Scadden, 1996). Thus one of the greatest challenges for children with visual disabilities has been
the learning of, and access to, mathematics information (Sahyun et al., 1998). Early learning and practice
may project a better construction of mathematics knowledge in visually impaired children (Edwards &
Stevens, 1993).
In this study we intended to foster learning and practice of mathematical concepts such as positional
value, sequences, additive decomposition, multiplication, and division. Through opening pairs of tokens in a
board with several levels of difficulty, the child has to find the corresponding pair of tokens in accordance
with the mathematical content. Thus AudioMath is used to assist learners with visual disabilities in learning
concepts such as establishing correspondence and equivalency relationships, memory development, and
differentiating tempo-spatial notions.
2. DESIGN OF AUDIOMATH
2.1 Model
AudioMath has different components. Specific content models the representation problem to generate a grid
with a pair of related tokens between them to be solved by the child. Random card generator is the editor that
allows setting the level of complexity and contents from a gallery. Computer model is the representation of
the real problem. It includes state system variables such as number of correct token pairs, time, and score as
well as parametric variables such as level, content, and user name. Projection implies transforming input
signals to changes perceived by blind users either audible or tactile. It bridges system and interfaces through
bidirectional feedback from and toward the user actions. Interface includes input/output interaction devices
such as audio, keyboard, mouse, force feedback joystick, and tablets.
The model is based on a matrix with rows and columns. There are four levels of complexity: Level 1 with
four tokens (two rows and two columns), level 2 with six tokens (three rows and two columns), level 3 with
twelve tokens (three rows and four columns), and level 4 with sixteen tokens (four rows and four columns).
Colours are used for children with residual vision (Rigden, 1999). This model meets the minimum standards
proposed by (Sánchez et al., 2004) for software design, and thus validating AudioMath as an appropriate
virtual environment to be used for the learning of children with visual disabilities.
2.2 Software and hardware Tools
AudioMath was developed by using Macromedia Director 8.5 and a library of routines for external joystick
control, Xtra RavJoystick. A joystick Side-Winder was used in conjunction with Force Feedback, mouse,
keyboard, and Wacon tablets.
Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Oxford, UK, 2004
2002 ICDVRAT/University of Reading, UK; ISBN 07 049 11 44 2 185
2.3 Interfaces
The principal interfaces of AudioMath for children with residual vision are displayed in Figure 1. For blind
children interfaces are only audio-based. A is the identification screen with two modes: facilitator or student
(two buttons). B considers the level of complexity (list box), content (list box), and input device (buttons).
Content can be filled, upgraded, and edited by using different media. C is the main interface of AudioMath
and includes options such as the position of the card grid, accumulated score, time elapsed (through speech),
restart, register, and exit (through buttons). D includes a logging actual use register (buttons) describing each
game and movements.
Figure 1. Interfaces of AudioMath
2.3 Interaction
AudioMath allows interaction with available interface elements such as buttons and text screens through
keyboard. Each interaction triggers an audio feedback and a high visual contrast screen to be perceived by
children with residual vision. The child has to move through a grid and open the corresponding token. Each
cell has associated music tones that identify the position in the grid. Sound is listened when moving through
cells. When open a token the associated element is visible by triggering an audio feedback. For example, if
the image is a car, a real traffic car sound is triggered. When open a correct token pair a feedback is set.
Finally, when all pairs are made the time used, the final score, and feedback are given.
AudioMath can be interacted through keyboard, joystick, and tablets. A few keystrokes are used with the
keyboard. These devices have been used in different applications developed by the research group after
testing with children with visual disabilities. The Microsoft SideWinder joysticks in conjunction with Xtra
RavJoystick for Macromedia Director allow grading the user position in the grid and give direct feedback
with different forces. Counter forces to the movement are generated per each token position change as well as
vibratory forces indicate to be near to the grid edge: up, down, left, and right. Force Feedback Joysticks allow
direct interaction with diverse degrees of freedom. A plastic graphic grid is posed on the tablet defining the
position of each token. A pen is used to point and select interface elements.
3. COGNITIVE EMPHASIS
AudioMath was designed to enhance memory in blind children and children with residual vision. For reading
and writing skills learners have to have developed visual and audio memory to discriminate graphemes and
phonemes. This memory prevents to be confused by graphemes for spatial orientation as well as by
phonemes with similar sounds. AudioMath allows children with visual disabilities to practice and rehearse
their short-term memory.
These children practice audio memory (blind and children with residual vision) and visual memory
(children with residual vision). The tasks implied to exercise audio/oral, visual/oral, audio/graphic, and
visual/graphic memory. The software also emphasizes learning concepts such as establishing correspondence
and equivalency relationships, memory development, and differentiating tempo-spatial notions.
Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Oxford, UK, 2004
2004 ICDVRAT/University of Reading, UK; ISBN 07 049 11 44 2
186
AudioMath was implemented to go further than just enhancing general domain skills such as memory and
tempo-spatial notions by integrating mathematics content based on the current school curriculum. We
embedded the software with mathematical concepts such as positional value, sequences, additive
decomposition, multiplication, and division. We wanted to observe how audio-based virtual environments
can foster the construction of mathematics learning in the mind of children with visual disabilities.
4. METHODOLOGY
4.1 Participants
The study was developed with ten children ages 8 to 15 who attend a school for blind children in Santiago,
Chile. The sample was conformed of 5 girls and 5 boys. Most of them have also added deficits such as
diverse intellectual development: normal, slow normal, border line, below to normal, and with mental deficit.
Four special education teachers also participated. All learners met the following prerequisites: to be legally
blind, to know the natural numbers, to express sequences orally, to order numbers, to decompose numbers
through audio means, to mentally estimate results of additions and subtractions, to mentally determine
products and coefficients, to mentally decompose numbers in additions, to manipulate multiplication tables
efficiently, and to have notions of fractions.
4.2 Evaluation Instruments
Three measurement tests were used to evaluate the impact of AudioMath on learning and practice of
mathematical concepts such as positional value, sequences, additive decomposition, multiplication, and
division. Immediate audio memory test (Cordero, 1977), evaluation of mathematics knowledge test
(Chadwick & Fuentes, 1980), and a usability evaluation test for end-users. The immediate audio memory test
has the purpose to measure logic, numeric memory, and associative memory from audio stimuli. The
evaluation of mathematics knowledge test measures: 1. The capacity to understand numbers (oral and
written); 2. The skills to make oral and written calculations; 3. The skills to count numeric series and graphic
elements; and 4. The skills for mathematic reasoning.
4.3 Procedure
Children were tested in the school from July to November 2003, twice a week, two one hour sessions per
week. They followed the steps of pre-testing (immediate audio memory and evaluation of mathematics
knowledge tests), interacting with AudioMath, solving cognitive tasks (see Figure 2 and 3), and post-testing
(immediate audio memory and evaluation of mathematics knowledge tests). Interacting with AudioMath and
solving cognitive tasks were the main steps of the study. During these steps children were observed and
assisted by four special education teachers filling check lists and registering observed behaviours. They also
applied a usability evaluation test for end-users developed by the authors.
Figure 2-3. Children solving cognitive tasks with AudioMath.
Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Oxford, UK, 2004
2002 ICDVRAT/University of Reading, UK; ISBN 07 049 11 44 2 187
5. COGNITIVE IMPACT
During the interactive sessions we realized that mathematical content used was appropriate to the educational
level of the sample. We analyzed the results case by case because the sample was not homogeneous in key
variables such as cognitive deficits and different degrees of blindness.
Children performed increasingly well in both tests: audio memory and mathematics knowledge. An
overall view of initial results shows pre-test – post-test gains in mathematics knowledge (see Figure 4), thus
indicating that interaction with AudioMath associated with cognitive tasks can improve mathematics learning
in these children. Audio memory pre-tests – post-tests show some gains after interacting with AudioMath,
thus indicating that short-term memory can be enhanced by using this software. Gains were higher in
mathematics knowledge than audio memory (see Figure 5).
Mathem atics Knowledge
0
10
20
30
40
50
60
70
80
90
100
12345678910
Children (Cases)
Achievement
Pretest
P o s tte s t
%
Figure 4. Pre-test – post-test gains in mathematics knowledge
Im m ediate Audio M em ory
0
10
20
30
40
50
60
70
80
90
100
12345678910
Ch ild ren (C ases)
Achievement
Pretest
P o s tte s t
%
Figure 5. Pre-test – post-test gains in short-term memory
In mathematics learning the results were promising. Most evaluated mathematics content was well attained
by learners with visual disabilities. The highest gains were in oral calculation (75%) and countdown of
numeric series (100%). We believe that these results were also partially due to a better attitude of the children
toward work and mathematics knowledge construction such as multiplication tables and use of mathematics
operations. They impacted positively on learners by increasing their certainty.
Children also obtained high gains in short-term memory. They improved their numeric memory (69th
percentile score), associative memory (65th percentile score), and logic memory (3rd percentile score). This
means that they performed better than, or received a higher score than, 69%, 65%, and 3% of other learners
that took the same tests.
Finally, we can conclude that thanks to the interaction with AudioMath and the associated cognitive tasks
learners with visual abilities developed mathematics skills and short-term memory. This is a major result in
Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Oxford, UK, 2004
2004 ICDVRAT/University of Reading, UK; ISBN 07 049 11 44 2
188
our research because we are initially observing that audio-based virtual environments can foster the
construction of mathematics learning in the mind of children with visual disabilities.
6. DISCUSSION
We have introduced AudioMath, a virtual environment to enhance memory and mathematics skills in
children with visual disabilities through audio. The software was made by and for children with visual
disabilities. They participated actively in the development of the software. We have also designed interfaces
for both blind and children with residual vision. A usability study was implemented with end-users,
facilitators, observers, and experts.
AudioMath was highly accepted by end-users. They liked, enjoyed, and were motivated when interacted
with the software. The flexibility of this application is also a plus. Teachers, children, and parents can include
new objects and sounds to adapt them to their needs. Thus, children with visual disabilities can choose
sounds to be interacted with and embed them into AudioMath. Content can be changed and updated easily.
AudioMath can be used to support memory when learning specific concepts and processes in a given subject
matter. AudioMath can be used for learning primary school mathematics.
The use of concrete materials was also a plus in this study. The children’s understanding was easier when
they first interacted with concrete materials and then with AudioMath. Parallel interaction with both concrete
material and AudioMath was also an advantage. Once they developed their own mental model of the
software the interaction with AudioMath was enriched.
Force Feedback Joysticks introduced a new scenario in virtual environment for blind children. They can
provide information and tactile sensations through force feedback. This can help to decrease audio stimuli
and relief possible acoustic pollution. Joysticks are devices with a high potential of use due to the availability
of many buttons.
Our model fits well the learning of primary school mathematics concepts such as positional value,
sequences, additive decomposition, multiplication, and division. Children performed increasingly well in
both tests: audio memory and mathematics knowledge. Oral calculation and countdown numeric series were
highly achieved as well as numeric and associative memory. Concrete cognitive tasks were crucial in this
achievement. We firmly conclude that interaction with AudioMath associated with cognitive tasks can help
to improve mathematic learning in these children.
More qualitative data are being analyzed. Most of them are case study because each child with visual
disabilities is a whole case that deserves a deep analysis to construct meaning about the role that can play
audio-based devices in learning general and specific domain skills.
Finally, we are convinced that further research studies we are implementing right now concerning
mathematic learning will reaffirm our hypothesis that audio-based virtual environment can foster the
construction of mathematics learning in the mind of children with visual disabilities.
Acknowledgements: This report was funded by the Chilean National Fund of Science and Technology,
Fondecyt, Project 1030158.
7. REFERENCES
A Cordero (1977), Immediate Audio Memory. 5th edition. TEA Editorial, Applied psychology publications,
Madrid, Spain, 1977.
A D N Edwards and R D Stevens (1993), Mathematical representations: Graphs, curves and formulas. In D.
Burger and J. C. Sperandio (ed.) Non-Visual Human-Computer Interactions: Prospects for the visually
handicapped. Paris: John Libbey Eurotext, (1993), pp. 181-194.
A Rizzo, T Bowerly, J Buckwalter, M Schultheis, R Matheis, C Shahabi, U Neumann, L Kim and M
Sharifzadeh (2002), Virtual environments for the assessment of attention and memory processes: the
virtual classroom and office. In Proceedings of the 4th International Conference on Disability, Virtual
Reality and Associated Technologies, ICDVRAT 2002, 18-20 September 2002, Veszprém, Hungary.
C Rigden (1999), The eye of the beholder-designing for colour blind. British Telecommunications
Engineering, Vol. 17, January.
C Sjostrom (2001), Using haptics in computer interfaces for blind people. Proceeding of the ACM CHI ´01,
Seattle, Washington, USA, March 31 – April 5, 2001, Vol 3, 1. pp. 245-246.
Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Oxford, UK, 2004
2002 ICDVRAT/University of Reading, UK; ISBN 07 049 11 44 2 189
E Attree, B Brooks, F Rose, T Andrews, A Leadbetter and B Clifford (1996), Memory processes and virtual
environments: I can’t remember what was there, but I can remember how I got there. Implications for
people with disabilities. In Proceedings of the 1st European Conference on Disability, Virtual Reality
and Associated Technologies, ECDVRAT 1996, 8-10 July 1996, Maidenhead, Berkshire, UK.
F Winberg and S Helltrom (2000), The quest for auditory manipulation: the sonified Towers of Hanoi. In
Proceedings of the Third International Conference on Disability, Virtual Reality and Associated
Technologies, ICDVRAT 2000, Sardinia Italia, 23-25 September, 2000. pp. 75-81.
H Tan (2000), Haptic interfaces. Communications of the ACM, 43(3). pp. 40-41.
J Baldis (2001), Effects of spatial audio on memory, comprehension, and preference during desktop
conferences. In Proceeding of the ACM CHI 2001, Seattle, Washington, USA, March 31 – April 5, 2001.
Vol 3, 1. pp. 166-173.
J Loomis, Y Lippa, R Klatzky amd R Golledge (2002), Spatial updating of locations specified by 3-D sound
and spatial language. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28(2), pp.
335-345. 2002
J Sánchez (2000), 3D interactive games for blind children. In Proceedings of Technology and Persons with
Disabilities, CSUN 2000. Los Angeles, USA.
J Sánchez (2000), Usability and cognitive impact of the interaction with 3D virtual interactive acoustic
environments by blind children. In Proceedings of the 3rd International Conference on Disability, Virtual
Reality and Associated Technologies, ICDVRAT 2000, Sardinia Italia, 23-25 September, 2000. pp. 67-73.
J Sánchez (2001), Interactive virtual acoustic environments for blind children. In Proceedings of ACM CHI
´2001, pp. 23-25. Seattle, Washington, USA, March 31 – April 5, 2001.
J Sánchez and H Flores (2003), AudioMemory: Developing memory in children with visual disabilities
through audio. Proceedings of the 8th International Workshop on Educational Software, TISE 2003,
Santiago, Chile, November, 2003.
J Sánchez and H Flores (2004), Memory enhancement through audio. To be published by in the Proceedings
of ACM Assets 2004, Atantla, Giorgia, USA, October 18-20.
J Sánchez, N Baloian and H Flores (2004), A methodology for developing audio-based interactive
environments for learners with visual disabilities, Proceedings of the World Conference on Educational
Multimedia, Hypermedia & Telecommunications, ED-MEDIA 2004, Lugano, Switzerland, June 21-26.
L Scadden (1996), Making mathematics and science accessible to blind students through technology.
Proceeding of Resna 96 Annual Conference, (1996), June 7-12, Salt Lake City, Utah, pp. 51-58.
M Chadwick and M Fuentes (1980), Evaluation of Mathematics Knowledge (adaptation of Benton-Luria
test). Santiago, 1980.
M Cooper and M E Taylor (1998), Ambisonic sound in virtual environments and applications for the blind
people. In Proceedings of the Second European Conference on Disability, Virtual Reality, and Associated
Technologies, ECDVRAT 1998, Skövde, Sweden, 10-11 September, 1998. pp. 113-118.
M Lumbreras and J Sánchez (1998), 3D aural interactive hyperstories for blind children. International
Journal of Virtual Reality 4(1. pp., 20-28.
O Lahav and D Mioduser (2000), Multisensory virtual environment for supporting blind persons’ acquisition
of spatial cognitive mapping, orientation, and mobility skills. In Proceedings of the Third International
Conference on Disability, Virtual Reality and Associated Technologies, ICDVRAT 2000, Sardinia Italia,
23-25 September, 2000. pp. 53-58.
R McCrindle and D Symons (2000), Audio space invaders. In Proceedings of the Third International
Conference on Disability, Virtual Reality and Associated Technologies, ICDVRAT 2000, Sardinia Italia,
23-25 September, 2000. pp. 59-65.
R Morris, D Parslow and M Recce (2000), Using immersive virtual reality to test allocentric spatial memory
impairment following unilateral temporal lobectomy. In Proceedings of he 3th International Conference
on Disability, Virtual Reality and Associated Technologies, ICDVRAT 2000, Sardinia Italia, 23-25
September, 2000.
S Mereu and R Kazman (1996), Audio enhanced 3D interfaces for visually impaired users. Proceedings of
CHI’96, ACM Press. (1996).
S Sahyun, S Gardner and C Gardner (1998), Audio and Haptic Access to Math and Science - Audio graphs,
Triangle, the MathPlus Toolbox, and the Tiger printer. Proceedings of the 15th IFIP World Computer
Congress, Vienna, September 1998, pp. 78-86.
... Some software previously designed in our research was modeled with this architecture (Sánchez and Sáenz 2005). For example, the software called Theo and Seth (see Fig. 3a) is an application to enhance mathematics skills in blind users comprising the following components: model, strategy, computer representation, model of learner, evaluation, learner, and interface. ...
Article
Full-text available
Current interactive learning environments cannot be accessed by learners with disabilities, in particular for students with vision disabilities. Modeling techniques are necessary to map real-world experiences to virtual worlds by using 3D auditory representations of objects for blind people. In this paper, a model to design multimedia software for blind learners is presented. The model was validated with existing educational software for these users. We describe the modeling of the real world including cognitive usability testing tasks by considering not only the representation of the real world but also modeling the learner’s knowledge of the virtual world. The software architecture is also described by using this model, displaying the components needed to impact learning. Finally, we analyze critical issues in designing software for learners with visual disabilities and propose some recommendations and guidelines.
... Through discrimination and localization of audio cues the auditory sense convey information to people with visual disabilities, becoming one of the most critical senses to them. Along with this, the auditory sense favors the perception of points of reference, the description of some physical places, the development of abstract thinking, the capacity of concentration, and the creation of mental schema of diverse physical environments [10,11,13,14,15,16,17]. ...
Chapter
Full-text available
In this study, we introduce software for blind users that represents a subway system in a desktop computer. A user can organize and prepare a travel with the assistance of the software before riding the subway. After a usability study and cognitive evaluation, we detected the need for creating a mobile solution with similar goals as the desktop application. This software for mobile devices has also the capacity to help the user to solve mobility and orientation problems in real subway stations. In order to design a handheld version it was necessary to consider new features such as travel duration, tickets fare, and the estimated time duration of the travel. Conclusions from the usability study revealed the importance of using interface elements such as the audio-based hierarchy menu, the travel simulation, and the information about the subway network, stations and their surroundings. The cognitive study results revealed important gains in the development of orientation and mobility skills to use the subway system in blind users, which help them to be more integrated to the society.
... Methodologies [10] and applications [11,12,13] have been developed to support learning in LWVD obtaining positive gains in the development of tempo-spatial skills, memory and mathematical logical analysis. Nevertheless, these applications restrict their implementation to a static context as the school classroom and home. ...
Conference Paper
Full-text available
Children tend to learn language conventions through processing environment stimuli. Thus, strategies for reading comprehension are commonly used for this purpose. This paper introduces AudioStoryTeller, a tool for pocketPC to support the development of reading and writing skills in learners with visual disabilities (LWVD) through storytelling, providing diverse evaluation tools to measure those skills. We implemented usability and cognitive evaluation to the AudioStoryTeller software. In the usability evaluation, the easiness of use of the proposed hardware by LWVD was established. The goal of the cognitive evaluation was to measure the development of reading skills through interactive audio narrations using a pocketPC device. Results indicate that users were able to utilize effortless the pocketPC device. AudioStoryTeller software together with cognitive tasks, can contribute to the development of cognitive skills in LWVD. This application allows LVD to have access to unlimited scope of books not available in printed Braille.
... This is mainly due to the kind of resources required and these students' lack of one of the most important senses for the process of observing natural phenomenon: vision [9,14]. Children with visual disability have more problems accessing information, learning and putting basic operations into practice, and solving problems than their sighted peers [23,13]. However, the use of technology that is correctly adapted for this kind of disablement can lead to an efficient solution. ...
Conference Paper
Full-text available
In this study we evaluate how the use of audio-based technology can facilitate school integration of blind learners through the interaction with a science videogame. This experience consisted of designing and implementing The Natomy’s Journey Game to be played by blind and sighted middle school students. The use of the videogame and its impact on the integration of blind learners into mainstream schools was also evaluated, through the participation of both teachers and learners. In the end, the goal was for blind learners to be able to interact and become socially integrated through active science video gaming and the application of specific science content. The results of this study provide initial data and evidence that the use of video games such as The Natomy’s Journey Game can improve the process for the school integration of learners with visual disabilities.
Article
Full-text available
A number of virtual environments have been developed during the last years. Among them there are some environments for blind people based on different type of audio, from simple sounds to 3D audio. In this study we pursued a different approach. We designed AudioMUD by using spoken text to describe the environment, navigation, and interaction. We have also introduced some collaborative aspects into the interaction between blind users. The core of a MUD game is a networked textual virtual environment and with this concept in mind, we developed AudioMUD by adding some collaborative aspects to the basic idea MUD and locating this virtual environment inside the human body. This study presents the design and usability evaluation of AudioMUD. Blind learners were motivated when interacted with AudioMUD and helped to improve the interaction through audio and interface design elements
Chapter
Full-text available
In this study we present AudioLink, an interactive audio-based virtual environment for children with visual disabilities to support their learning of science. AudioLink is a Role-Playing Game (RPG) for learning science concepts and scientific reasoning through audio. We analyzed how blind learners can learn science using audio as the main input/output interface and how to develop a challenging and engaging software. The usability of this software and a preliminary study of the cognitive impact were also evaluated. Results indicated that users considered the software was appealing, challenging and encouraging as a science learning tool. AudioLink promoted a free and independent interaction at the users’ own paces. Evidence indicates that blind children developed scientific thinking skills to identify a problem, build a strategy to solve it, and understand when it was solved.
Article
Full-text available
A number of virtual environments have been developed during the last years. Among them there are some applications for blind people based on different type of audio, from simple sounds to 3-D audio. In this study, we pursued a different approach. We designed AudioMUD by using spoken text to describe the environment, navigation, and interaction. We have also introduced some collaborative features into the interaction between blind users. The core of a multiuser MUD game is a networked textual virtual environment. We developed AudioMUD by adding some collaborative features to the basic idea of a MUD and placed a simulated virtual environment inside the human body. This paper presents the design and usability evaluation of AudioMUD. Blind learners were motivated when interacted with AudioMUD and helped to improve the interaction through audio and interface design elements.
Article
Full-text available
It is known that blind children represent spatial environments with cognitive difficulty. This can be decreased if they are exposed to interactive experiences with acoustic stimuli delivered through spatialized sound software. A few studies have approached this issue by using interactive applications that integrate virtual reality and cognitive tasks to enhance spatial orientation skills. The aim of this research was to implement a field study to detect and analyze cognitive and usability issues involved in the use of an aural environment and the issues of representing navigable structures with only spatial sound. This experimental study has arisen from a challenging pilot research project to a full fledged field-testing research with eleven children during six months in a Chilean school for blind children. The research was implemented by using a kit of cognitive representation tasks, which includes exposure to the 3D acoustic environment, corporal exercises, and experiences with concrete representation materials such as sand, clay, storyfoam, and Lego bricks. The cognitive kit also included activities to represent the perceived environment, the organization of the space, and problem solving related to the interactions with the software. The usability testing of the environment was an explicit issue in the research by using both qualitative and quantitative methods including interviews, survey methods, logging actual use, still pictures, and video tape recording session analysis. The idea was to motivate and engage blind children to explore and interact with virtual entities in challenge-action software and to construct invisibly cognitive spatial mental representations. The results of the study revealed that blind children can achieve the construction of mental structures rendered with only 3D sound and that spatial imagery is not purely visual by nature, but can be constructed and transferred through spatialized sound. Our hypothesis was fully confirmed revealing that each blind child passes four clear stages in their interaction with the sound environment and performing cognitive tasks: entry, exploration, adaptation, and appropriation. We also conclude that the child possesses both unique skills and pace referred to mental and spatial development, impacting directly on the quality of the topological features obtained in comparison to the ideal reference spatial structure embedded in the software.
Conference Paper
Full-text available
Blind children tend to represent spatial environments with cognitive difficulty. The absence of visual stimuli processing impedes the construction of rich representations of the surrounding space. This can be decreased if they are exposed to interactive experiences with acoustic stimuli delivered through spatialized sound software complemented with cognitive tasks tailored to accomplish richer representations. A few studies have approached this issue by using interactive applications that integrate virtual reality and cognitive tasks to enhance spatial orientation skills. The aim of this research has been to design interactive software based on spatialized sound to help blind learners to construct cognitive spatial structures.
Article
Full-text available
Whilst advances are underway in various areas to ease and encourage disabled uptake of new technology, very little emphasis to date has been placed on making the games market accessible to all. The aims of the described work have been twofold. Firstly, to prove that the standard features of a traditional space invader game can be replicated using a 3-D audio (ambisonic) environment. Secondly, through combining audio and visual interfaces with force feedback joystick movement that it is possible to produce a multi-modal game that can be played by both sighted and non-sighted users, thereby enabling them to share the same gaming experience. This paper describes the development and features of the resultant Audio Space Invaders game.
Article
Full-text available
Mental mapping of spaces, and of the possible paths for navigating through these spaces, is essential for the development of efficient orientation and mobility skills. The work reported here is based on the assumption that the supply of appropriate spatial information through compensatory channels (conceptual and perceptual), may contribute to the blind people's spatial performance. We developed a multi-sensory virtual environment simulating real-life spaces. This virtual environment comprises developer / teacher mode and learning mode.
Article
Full-text available
Interactive stories are commonly used for learning and entertaining purposes enhancing the development of several perceptual and cognitive skills. These experiences are not very common among blind children because most computer games and electronics toys do not have appropriate interfaces to be accessible by them. This study introduces the idea of interactive Hyperstories performed in a 3D acoustic virtual world. The hyperstory model enables us to build an application to help blind children to enrich their early world experiences through exploration of interactive virtual worlds by using 3D aural representations of the space. We have produced AudioDoom, interactive model-based software for blind children. The prototype was qualitative and quantitatively field-tested with several blind children in a Chilean school setting. Our preliminary results indicate that when acoustic-based entertainment applications are careful applied with an appropriate methodology can stimulate diminished cognitive skills. We also found that spatial sound experiences can create spatial navigable structures in the mind of blind children. Methodology and usability evaluation procedures and results appeared to be critical to the efectiveness of interactive Hyperstories performed in a 3D acoustic virtual world.
Article
Full-text available
Virtual Reality (VR) technology offers new options for neuropsychological assessment and cognitive rehabilitation. If empirical studies demonstrate effectiveness, virtual environments (VEs) could be of considerable benefit to persons with cognitive and functional impairments due to traumatic brain injury, neurological disorders, and learning disabilities. Testing and training scenarios that would be difficult, if not impossible, to deliver using conventional neuropsychological methods are now being developed that take advantage of the assets available with VR technology. These assets include the precise presentation and control of dynamic multi-sensory 3D stimulus environments, as well as advanced methods for recording behavioral responses. When combining these assets within the context of functionally relevant, ecologically valid VEs, a fundamental advancement emerges in how human cognition and functional behavior can be assessed and rehabilitated. This paper will focus on the progress of a collaborative VR research program at the University of Southern California and the Kessler Medical Rehabilitation Research and Education Corporation. These groups are developing and evaluating VR neuropsychological applications designed to target: 1. Attention processes in children with ADHD within a HMD virtual classroom and 2. Memory processes in persons with TBI within a HMD virtual office. Results from completed research, rationales and methodology of works in progress, and our plan for future work will be discussed. Our primary vision has been to develop VR systems that target cognitive processes and functional skills that are relevant to a wide range of patient populations with CNS dysfunction. We have also sought to select cognitive/functional targets that intuitively appear well matched to the specific assets available with the current state of VR technology.
Article
Colour-blind computer users see things differently from most people, but this is seldom considered in the design of software or Web pages. This article offers a simple technique for simulating the differences, to help designers and developers avoid disadvantaging their users.