Augmented Reality for Deaf Students: Can Mobile Devices Make It Possible?

To read the full-text of this research, you can request a copy directly from the authors.


Digital and real world events can be combined to create powerful learning opportunities for students, but time, tools, and expertise have been traditional barriers to teacher-created enhancements. This paper provides a rationale for using emerging, teacher-friendly tools, to merge real space and virtual space through video and 2D barcodes. The results of three pilot studies combine to illustrate the potential for using these tools. Results indicate that cell phones have the potential to facilitate augmented reality experiences for deaf students and adults. KeywordsAugmented Reality-Mobile Devices-Accessibility-Deaf

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... For instance, studies by [39] and [16] show that MAR enhances speech narration and conversion into readable text which makes communication easier for the HI community. Similarly, studies by [44], [34] and [42] reflect the importance of MAR in learning for the HI people. These studies suggest that MAR can provide a unique platform for HI interaction and stimulating learning environment. ...
... Engage, retention and learn Deaf Teaching and Learning [44] Learning opportunities for deaf students. ...
Many studies have begun to consider how to ensure a pleasant experience during visits to cultural heritage sites and museums. Although, when considering the populace of the visitors to these sites, the hearing impaired (HI) visitors which made up of a smaller percentage, have not been in the literature limelight as much as the normal hearing visitors. Thus, the hearing impaired tends to endure certain unpalatable experiences leading to dissatisfaction of their visits. Literature has shown that Mobile Augmented Reality (MAR) can improve the experiences of visitors to the museum in terms of engagement, enjoyment and learning. This is evident in a number of articles tailored towards normal hearing visitors. However, a recent study has taken into consideration the hearing impaired visitors by identifying the engagement elements of MAR for the HI museum visitors. The identified elements include; aesthetics, interaction, interest, usability, satisfaction, motivation, curiosity, enjoyment, perceived control, self-efficacy, and focused attention. This article thus takes a step further by introducing the MAR for the HI museum visitors’ engagement (MARHIME) conceptual model. These elements are derived from a review of literature which has been done comprehensively and are validated by a panel of experts. Altogether eleven elements went through the expert review process and only six elements were validated to be used for the construction of the MARHIME model. This article also further grounds the justification of these selected six elements in relation to engagement. Future work will include the development of the MARHIME prototype which will be used to validate the model among the hearing impaired visitors at a museum.
... With respect to the Disability category, 20% of the studies were focused on individuals with hearing impairments (DHH), given that the AR allows the use of mobile devices and the visual channel is often preferred for perceiving information. The applications developed for this population combine videos with other visual tools or interactive multimedia ( Parton et al., 2010), also promoting the use of glasses for AR and QR codes (Parton, 2017). On the other hand, 18% of the studies have also addressed the needs of individuals diagnosed with Autism Spectrum Disorder (ASD), since AR facilitates the creation of applications recognizing facial emotions, which represents a difficulty for individuals diagnosed with ASD ( . ...
Full-text available
The use of Augmented Reality (AR) to achieve educational inclusion has been not deeply explored. This systematic review describes the current state of using AR as an educational technology that takes into consideration the needs of all students including those with a disability. It is done through the analysis of factors, such as the advantages of AR, its limitations, uses, challenges, its scope in the educational field, the attended population and the positive or negative effects of its use in learning scenarios that involve students with diverse educational needs. A total of 50 studies between 2008 and 2018 were analyzed through searching in three interdisciplinary databases: Scopus, Web of Science, and Springer link. For this, the methodological stages considered were planning the review, search, analysis of literature and results report. After analyzing the results, it was possible to demonstrate that the use of AR for inclusive education in the field of sciences is where more studies have been conducted. In regard to the population with disabilities, among the most representative advantages reported were the motivation, interaction and generating interest on the part of the student. At the same time, an important methodological limitation identified was the size of the sample; some investigations were done with two or three subjects, some studies Single Subject Designs were found. In terms of the population attended, the studies generally included students with different impairments (hearing, visual, motor or cognitive), minorities (ethnic, vulnerable), leaving aside other groups excluded as exceptional talents and immigrants, which could be explored in the future. Despite different problems to be addressed, few frameworks to the diversity attention in education were reported, and there was no model and methodology in inclusive education considered in the studies. Finally, from this review we have identified open issues that could give rise to new research in the subject of using AR to favor the creation of inclusive learning scenarios.
Full-text available
THE STUDY examined the efficacy of an early intervention program to improve children's sequential time perception through virtual versus pictorial training in arranging episodes of temporal scripts. The researchers examined 65 deaf and hard of hearing children ages 4-7 years who were divided into two groups: (a) virtual environments technological intervention and (b) pictorial nontechnological intervention. Participants completed pretest and posttest measures. Both groups demonstrated significant improvement in sequential time achievement following intervention. However, the improvement was much more significant in the technological group.
Full-text available
This article presents a protype for a mobile learning solution using augmented reality. Aimed at promoting the digital inclusion of elementary school deaf students the project is entitled: "A minha rua"/ My street. Augmented reality; Mobile Learning; Digital Inclusion; Geographic Information Technologies, Special Needs. I. INTRODUÇÃO Actualmente, não obstante a ubiquidade e papel fundamental desempenhado pela tecnologia, continuam a verificar-se problemas ao nível do acesso e usabilidade de conteúdos por todos. No presente artigo é abordada a problemática dos alunos do primeiro ciclo com limitações auditivas, sendo apresentado uma proposta de utilização de dispositivos móveis e uma aplicação de Realidade Aumentada. Com esta proposta pretende demonstrar-se a importância deste tipo de soluções no desenvolvimento de competências funcionais e na promoção da inclusão. Ao levantamento do estado da arte, segue-se a descrição das diferentes fases projectuais, sendo posteriormente elencadas algumas das suas vantagens, potencialidades, dificuldades e áreas a explorar futuramente.
Full-text available
This paper describes research that investigates the use of a technology designed to support young children's collaborative artifact creation in outdoor environments. Collaboration while creating knowledge artifacts is an important part of children's learning, yet it can be limited while exploring outdoors. The construction of a joint representation often occurs in the classroom after the experience, where further investigation and observation of the environment is not possible. This paper describes a research study where collaborative technology was developed, used by children, and evaluated in an authentic setting — a U.S. National Park.
Conference Paper
Full-text available
The Shadow Box is a tangible computing project that exploits visual association and auditory clues to teach children the representational relationship between words and their meanings. The Shadow Box contains three major components: the main box, picture blocks and word blocks. The Shadow Box activates when a block or a matching pair of blocks is placed inside. The box prompts children to find matching blocks and combine them together. When children successfully combine the right word and picture, the box rewards them with an animated video as if they had made the objects come alive. An informal study shows that children responded positively to the concept of the box. They played with it for a length of time and engaged in a collaborative learning process with other children. project focuses on helping preschool aged children attain word-level skills. We propose a tangible toy for early literacy development called the "Shadow Box". The Shadow Box aims to teach the representational nature of written language to three to four year-old children while they are playing with tangible word and object blocks. The Shadow Box consists of three primary components: the shadow box equipped with an RFID reader, as well as picture blocks and word blocks with embedded RFID tags. In order to activate the Shadow Box, a single block or a matching pair of blocks must be placed inside. If the child inserts a word block, the box pronounces and spells the word and prompts the child to look for its matching picture by projecting a static shadow image of the object. A similar sequence is initiated when the child inserts a picture object and is prompted to find the matching word block. When the child successfully matches a picture block with its corresponding word block and places the pair inside the Shadow Box, they are rewarded with an animated movie clip.
Conference Paper
Full-text available
Tangibles, in the form of physical artefacts embedded with sensor technologies, offer the opportunity to exploit and build on our everyday interaction and experience with the world, enabling new forms of engagement and access to tools for supporting learning. The implications for learning are considerable, potentially bringing about a radical change in the way we conceptualise learning and learning activities. However, we know little about the specific learning benefits, and currently lack an effective structure within which to establish them. Although several frameworks have been proposed for conceptualizing tangible environments, none highlight the central role that external representations have in tangible environments. This paper argues for the importance of placing primary emphasis on representation, and the role that this might play in mediating interaction and cognition in tangible environments. The representation-tangible relationship is outlined, together with their differential potentials for learning. Based on this the paper then proposes a conceptual framework for systematically investigating how different ways of linking digital information with physical artefacts influence interaction and cognition, to gain a clearer understanding of their role for learning.
Clicker4 and More -multimedia and multiple support
  • L Rawlings
Rawlings, L. (n.d.) Clicker4 and More -multimedia and multiple support. Unpublished document retrieved November 10, 2003, from
Integrating technology and literacy: Digital video dictionary. Paper presented at the Technology and Education of the Deaf Symposium
  • R Stifter
Stifter, R. (2001). Integrating technology and literacy: Digital video dictionary. Paper presented at the Technology and Education of the Deaf Symposium, Rochester, NY.
Active learning through technology-creating a technologyinfused environment to engage Deaf students in the learning process. Paper presented at the Technology and Disabled Persons Conference
  • L Burik
  • W Kelly
Burik, L. & Kelly, W. (2003). Active learning through technology-creating a technologyinfused environment to engage Deaf students in the learning process. Paper presented at the Technology and Disabled Persons Conference, Las Angeles, CA.
A study of interactive media for Deaf learners in post 16 education. Paper presented at the Technology and Education of the Deaf Symposium
  • A Akhtar
Akhtar, A. (2003, June). A study of interactive media for Deaf learners in post 16 education. Paper presented at the Technology and Education of the Deaf Symposium, Rochester, NY..
A case study of tangible flags: a collaborative technology to enhance field trips
  • G Chipman
  • A Druin
  • D Beer
  • J Fails
  • M Guha
  • S Simms
Chipman, G., Druin, A., Beer, D., Fails, J., Guha, M., & Simms, S. (2006). A case study of tangible flags: a collaborative technology to enhance field trips. Paper presented at the 5th International Conference for Interactive Design and Children (IDC), Tampere, Finland.