No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Augmented Reality for Deaf Students: Can Mobile Devices Make It Possible?
Abstract
Digital and real world events can be combined to create powerful learning opportunities for students, but time, tools, and
expertise have been traditional barriers to teacher-created enhancements. This paper provides a rationale for using emerging,
teacher-friendly tools, to merge real space and virtual space through video and 2D barcodes. The results of three pilot studies
combine to illustrate the potential for using these tools. Results indicate that cell phones have the potential to facilitate
augmented reality experiences for deaf students and adults.
KeywordsAugmented Reality-Mobile Devices-Accessibility-Deaf
... This paper focuses specifically on AR studies for DHH people. An early study about mobile AR for DHH people was made by Parton et al (2010) using 2D barcodes and Youtube videos created by teachers themselves, to create an affordable and feasible AR-based teaching experience with early smartphones. Today, QR codes are more often used but the basic idea remains to link the physical world to virtual objects like online videos. ...
... To overcome some of the challenges with the five dimensions, civic intelligence (CI) is a promising opportunity where a community of trusted persons can provide content. One example was explored by Parton, Hancock, & Dawson (2010). Videos of trusted, DHH signers in different languages, who are knowledgeable in different subjects can potentially volunteer and upload explanations of concepts, especially those that do not have a direct language correspondence. ...
... The potential of trust can be harder to achieve, but a ranking system (Hoisl, B., Aigner, W. & Miksch, 2007;Ling et al, 2005) may help other users to know who are the best signers for a specific subject/topic and language, and promote more volunteers. An opportunity with image recognition is to link e.g., maths formulas to a set of CI videos explaining it similar to (Parton, Hancock, & Dawson, 2010) in the local sign language (Zainuddin, Zaman, & Ahmad, 2010a) for the student. Similar solutions can also be done for other subjects, for instance with books, questions and other material within curricula could be linked to CI videos. ...
Game-based learning has had a rapid development in the 21st century, attracting an increasing audience. However, inclusion of all is still not a reality in society, with accessibility for deaf and hard of hearing children as a remaining challenge. To be excluded from learning due to communication barriers can have severe consequences for further studies and work. Based on previous research Augmented Reality (AR) games can be joyful learning tools that include activities with different sign languages, but AR based learning games for deaf and hard of hearing lack research. This paper aims to present opportunities and challenges of designing inclusive AR games for education of deaf children. Methods involved conducting a scoping review of previous studies about AR for deaf people. Experts were involved as co-authors for in-depth understanding of sign languages and challenges for deaf people. A set of AR input and output techniques were analysed for appropriateness, and various AR based game mechanics were compared. Results indicate that inclusive AR gameplay for deaf people could be built on AR based image and object tracking, complemented with sign recognition. These technologies provide input from the user and the real-world environment typically via the camera to the app. Scene tracking and GPS can be used for location-based game mechanics. Output to the user can be done via local signed videos ideally, but also with images and animations. Moreover, a civic intelligence approach can be applied to overcome many of the challenges that have been identified in five dimensions for inclusion of deaf people i.e., cultural, educational, psycho-social, semantic, and multimodal. The input from trusted, educated signers and teachers can enable the connection between real world objects and signed videos to provide explanations of concepts. The conclusion is that the development of an inclusive, multi-language AR game for deaf people needs to be carried out as an international collaboration, addressing all five dimensions.
... This paper focuses specifically on AR studies for DHH people. An early study about mobile AR for DHH people was made by Parton et al (2010) using 2D barcodes and Youtube videos created by teachers themselves, to create an affordable and feasible AR-based teaching experience with early smartphones. Today, QR codes are more often used but the basic idea remains to link the physical world to virtual objects like online videos. ...
... To overcome some of the challenges with the five dimensions, civic intelligence (CI) is a promising opportunity where a community of trusted persons can provide content. One example was explored by Parton, Hancock, & Dawson (2010). Videos of trusted, DHH signers in different languages, who are knowledgeable in different subjects can potentially volunteer and upload explanations of concepts, especially those that do not have a direct language correspondence. ...
... The potential of trust can be harder to achieve, but a ranking system (Hoisl, B., Aigner, W. & Miksch, 2007;Ling et al, 2005) may help other users to know who are the best signers for a specific subject/topic and language, and promote more volunteers. An opportunity with image recognition is to link e.g., maths formulas to a set of CI videos explaining it similar to (Parton, Hancock, & Dawson, 2010) in the local sign language (Zainuddin, Zaman, & Ahmad, 2010a) for the student. Similar solutions can also be done for other subjects, for instance with books, questions and other material within curricula could be linked to CI videos. ...
Game-based learning has had a rapid development in the 21st century, attracting an increasing audience. However, inclusion of all is still not a reality in society, with accessibility for deaf and hard of hearing children as a remaining challenge. To be excluded from learning due to communication barriers can have severe consequences for further studies and work. Based on previous research Augmented Reality (AR) games can be joyful learning tools that include activities with different sign languages, but AR based learning games for deaf and hard of hearing lack research. This paper aims to present opportunities and challenges of designing inclusive AR games for education of deaf children. Methods involved conducting a scoping review of previous studies about AR for deaf people. Experts were involved as co-authors for in-depth understanding of sign languages and challenges for deaf people. A set of AR input and output techniques were analysed for appropriateness, and various AR based game mechanics were compared. Results indicate that inclusive AR gameplay for deaf people could be built on AR based image and object tracking, complemented with sign recognition. These technologies provide input from the user and the real-world environment typically via the camera to the app. Scene tracking and GPS can be used for location-based game mechanics. Output to the user can be done via local signed videos ideally, but also with images and animations. Moreover, a civic intelligence approach can be applied to overcome many of the challenges that have been identified in five dimensions for inclusion of deaf people i.e., cultural, educational, psycho-social, semantic, and multimodal. The input from trusted, educated signers and teachers can enable the connection between real world objects and signed videos to provide explanations of concepts. The conclusion is that the development of an inclusive, multi-language AR game for deaf people needs to be carried out as an international collaboration, addressing all five dimensions.
... For instance, studies by [39] and [16] show that MAR enhances speech narration and conversion into readable text which makes communication easier for the HI community. Similarly, studies by [44], [34] and [42] reflect the importance of MAR in learning for the HI people. These studies suggest that MAR can provide a unique platform for HI interaction and stimulating learning environment. ...
... Engage, retention and learn Deaf Teaching and Learning [44] Learning opportunities for deaf students. ...
Many studies have begun to consider how to ensure a pleasant experience during visits to cultural heritage sites and museums. Although, when considering the populace of the visitors to these sites, the hearing impaired (HI) visitors which made up of a smaller percentage, have not been in the literature limelight as much as the normal hearing visitors. Thus, the hearing impaired tends to endure certain unpalatable experiences leading to dissatisfaction of their visits. Literature has shown that Mobile Augmented Reality (MAR) can improve the experiences of visitors to the museum in terms of engagement, enjoyment and learning. This is evident in a number of articles tailored towards normal hearing visitors. However, a recent study has taken into consideration the hearing impaired visitors by identifying the engagement elements of MAR for the HI museum visitors. The identified elements include; aesthetics, interaction, interest, usability, satisfaction, motivation, curiosity, enjoyment, perceived control, self-efficacy, and focused attention. This article thus takes a step further by introducing the MAR for the HI museum visitors’ engagement (MARHIME) conceptual model. These elements are derived from a review of literature which has been done comprehensively and are validated by a panel of experts. Altogether eleven elements went through the expert review process and only six elements were validated to be used for the construction of the MARHIME model. This article also further grounds the justification of these selected six elements in relation to engagement. Future work will include the development of the MARHIME prototype which will be used to validate the model among the hearing impaired visitors at a museum.
... For instance, in museum or exhibition settings, the utilization of AR codes can exhibit historical facts or provide supplementary information directly on the screen of a smartphone or other device. This approach enables users to access pertinent information without relying on audio guides or seeking assistance from others [9]. ...
... With respect to the Disability category, 20% of the studies were focused on individuals with hearing impairments (DHH), given that the AR allows the use of mobile devices and the visual channel is often preferred for perceiving information. The applications developed for this population combine videos with other visual tools or interactive multimedia ( Parton et al., 2010), also promoting the use of glasses for AR and QR codes (Parton, 2017). On the other hand, 18% of the studies have also addressed the needs of individuals diagnosed with Autism Spectrum Disorder (ASD), since AR facilitates the creation of applications recognizing facial emotions, which represents a difficulty for individuals diagnosed with ASD ( . ...
The use of Augmented Reality (AR) to achieve educational inclusion has been not deeply explored. This systematic review describes the current state of using AR as an educational technology that takes into consideration the needs of all students including those with a disability. It is done through the analysis of factors, such as the advantages of AR, its limitations, uses, challenges, its scope in the educational field, the attended population and the positive or negative effects of its use in learning scenarios that involve students with diverse educational needs. A total of 50 studies between 2008 and 2018 were analyzed through searching in three interdisciplinary databases: Scopus, Web of Science, and Springer link. For this, the methodological stages considered were planning the review, search, analysis of literature and results report. After analyzing the results, it was possible to demonstrate that the use of AR for inclusive education in the field of sciences is where more studies have been conducted. In regard to the population with disabilities, among the most representative advantages reported were the motivation, interaction and generating interest on the part of the student. At the same time, an important methodological limitation identified was the size of the sample; some investigations were done with two or three subjects, some studies Single Subject Designs were found. In terms of the population attended, the studies generally included students with different impairments (hearing, visual, motor or cognitive), minorities (ethnic, vulnerable), leaving aside other groups excluded as exceptional talents and immigrants, which could be explored in the future. Despite different problems to be addressed, few frameworks to the diversity attention in education were reported, and there was no model and methodology in inclusive education considered in the studies. Finally, from this review we have identified open issues that could give rise to new research in the subject of using AR to favor the creation of inclusive learning scenarios.
... This method enables the child to become an active part of the environment, not solely a passive observer, as in the pictorial intervention (Barab, Hay, Barnett, & Squire, 2001;Harper, Hedberg, & Wright, 2000). Active learning has long been considered a proven method for increasing attention, motivation, and retention of concepts, especially among deaf children (Parton, Hancock, & Dawson, 2010). The acknowledged efficacy of active learning may suggest that children need a more active mode of representation and expression in order to attain a higher level of abstraction. ...
THE STUDY examined the efficacy of an early intervention program to improve children's sequential time perception through virtual versus pictorial training in arranging episodes of temporal scripts. The researchers examined 65 deaf and hard of hearing children ages 4-7 years who were divided into two groups: (a) virtual environments technological intervention and (b) pictorial nontechnological intervention. Participants completed pretest and posttest measures. Both groups demonstrated significant improvement in sequential time achievement following intervention. However, the improvement was much more significant in the technological group.
The systematic review investigates the effect of various educational technologies on the learning outcomes of diverse student populations, particularly focusing on assistive technology interventions for students with disabilities. The comprehensive analysis covers literature from 2012 to 2023. The study highlights the potential of AR and assistive technologies in fostering inclusive and engaging learning environments. Despite positive findings, the review emphasizes the imperative for further research to refine the implementation of these technologies and enhance their effectiveness. The systematic review of five databases provides crucial insights into the effectiveness of various assistive technologies. Mobile devices, iPads, and AR interventions emerge as frequently utilized tools. Research activity peaked in 2013 and 2018 and subsequently declined. Twelve studies focus on Autism Spectrum Disorder and emphasize the prioritization of ASD in assistive technology interventions. The research highlights the importance of adopting a holistic perspective on educational inclusion, emphasizing collaborative efforts among teachers, diverse teaching methods, and technology integration. Despite the promise shown by assistive technologies, the review acknowledges their limitations and advocates for ongoing research and innovation to refine their application across diverse educational contexts. The findings stress the importance of a nuanced interpretation of evidence, considering the challenges posed by the limited number of eligible studies. The review calls for careful consideration of future research directions to bolster the comprehensiveness and reliability of evidence synthesis in assistive technology interventions for students with disabilities.
This paper presents an empirical study of 33 hearing preschool children and 22 preschool children with a hearing impairment, examining the effects of Augmented Reality (AR) as a mode of representation to support perceptions of abstract time concepts in the solving of mathematical problems. The findings indicate that an AR representation mode has the potential to present more comprehensive information, enhance cognitive understanding and encourage the active learning experiences of preschool children. The effects were especially significant for those children with a hearing impairment. We have discussed how presenting information through an AR mode can aid the perception of time and sequence by preschool children, especially hearing-impaired children, resulting in improved skills for solving mathematical problems.
This article presents a protype for a mobile learning solution using augmented reality. Aimed at promoting the digital inclusion of elementary school deaf students the project is entitled: "A minha rua"/ My street. Augmented reality; Mobile Learning; Digital Inclusion; Geographic Information Technologies, Special Needs. I. INTRODUÇÃO Actualmente, não obstante a ubiquidade e papel fundamental desempenhado pela tecnologia, continuam a verificar-se problemas ao nível do acesso e usabilidade de conteúdos por todos. No presente artigo é abordada a problemática dos alunos do primeiro ciclo com limitações auditivas, sendo apresentado uma proposta de utilização de dispositivos móveis e uma aplicação de Realidade Aumentada. Com esta proposta pretende demonstrar-se a importância deste tipo de soluções no desenvolvimento de competências funcionais e na promoção da inclusão. Ao levantamento do estado da arte, segue-se a descrição das diferentes fases projectuais, sendo posteriormente elencadas algumas das suas vantagens, potencialidades, dificuldades e áreas a explorar futuramente.
This paper describes research that investigates the use of a technology designed to support young children's collaborative artifact creation in outdoor environments. Collaboration while creating knowledge artifacts is an important part of children's learning, yet it can be limited while exploring outdoors. The construction of a joint representation often occurs in the classroom after the experience, where further investigation and observation of the environment is not possible. This paper describes a research study where collaborative technology was developed, used by children, and evaluated in an authentic setting — a U.S. National Park.
The Shadow Box is a tangible computing project that exploits visual association and auditory clues to teach children the representational relationship between words and their meanings. The Shadow Box contains three major components: the main box, picture blocks and word blocks. The Shadow Box activates when a block or a matching pair of blocks is placed inside. The box prompts children to find matching blocks and combine them together. When children successfully combine the right word and picture, the box rewards them with an animated video as if they had made the objects come alive. An informal study shows that children responded positively to the concept of the box. They played with it for a length of time and engaged in a collaborative learning process with other children. project focuses on helping preschool aged children attain word-level skills. We propose a tangible toy for early literacy development called the "Shadow Box". The Shadow Box aims to teach the representational nature of written language to three to four year-old children while they are playing with tangible word and object blocks. The Shadow Box consists of three primary components: the shadow box equipped with an RFID reader, as well as picture blocks and word blocks with embedded RFID tags. In order to activate the Shadow Box, a single block or a matching pair of blocks must be placed inside. If the child inserts a word block, the box pronounces and spells the word and prompts the child to look for its matching picture by projecting a static shadow image of the object. A similar sequence is initiated when the child inserts a picture object and is prompted to find the matching word block. When the child successfully matches a picture block with its corresponding word block and places the pair inside the Shadow Box, they are rewarded with an animated movie clip.
Tangibles, in the form of physical artefacts embedded with sensor technologies, offer the opportunity to exploit and build on our everyday interaction and experience with the world, enabling new forms of engagement and access to tools for supporting learning. The implications for learning are considerable, potentially bringing about a radical change in the way we conceptualise learning and learning activities. However, we know little about the specific learning benefits, and currently lack an effective structure within which to establish them. Although several frameworks have been proposed for conceptualizing tangible environments, none highlight the central role that external representations have in tangible environments. This paper argues for the importance of placing primary emphasis on representation, and the role that this might play in mediating interaction and cognition in tangible environments. The representation-tangible relationship is outlined, together with their differential potentials for learning. Based on this the paper then proposes a conceptual framework for systematically investigating how different ways of linking digital information with physical artefacts influence interaction and cognition, to gain a clearer understanding of their role for learning.
Clicker4 and More - multimedia and multiple support
- L Rawlings
Rawlings, L. (n.d.) Clicker4 and More -multimedia and multiple support. Unpublished
document retrieved November 10, 2003, from http://education.qld.gov.au.
Integrating technology and literacy: Digital video dictionary. Paper Presented at the Technology and Education of the Deaf Symposium
- R Stifter
Stifter, R. (2001). Integrating technology and literacy: Digital video dictionary. Paper
presented at the Technology and Education of the Deaf Symposium, Rochester, NY.
Active learning through technology-creating a technology- infused environment to engage Deaf students in the learning process. Paper Presented at the Technology and Disabled Persons Conference
- L Burik
- W Kelly
Burik, L. & Kelly, W. (2003). Active learning through technology-creating a technologyinfused environment to engage Deaf students in the learning process. Paper presented at the
Technology and Disabled Persons Conference, Las Angeles, CA.
A study of interactive media for Deaf learners in post 16 education. Paper Presented at the Technology and Education of the Deaf Symposium
- A Akhtar
Akhtar, A. (2003, June). A study of interactive media for Deaf learners in post 16 education.
Paper presented at the Technology and Education of the Deaf Symposium, Rochester, NY..
A case study of tangible flags: a collaborative technology to enhance field trips
- G Chipman
- A Druin
- D Beer
- J Fails
- M Guha
- S Simms
Chipman, G., Druin, A., Beer, D., Fails, J., Guha, M., & Simms, S. (2006). A case study of
tangible flags: a collaborative technology to enhance field trips. Paper presented at the 5th
International Conference for Interactive Design and Children (IDC), Tampere, Finland.