PreprintPDF Available

Augmented Reality for Visually Impaired People (AR for VIPs)

  • May 2019
DOI:10.13140/RG.2.2.30196.78723
Authors:
Dylan Fox at University of California, Berkeley
Dylan Fox
Alyssa Li
Alyssa Li
Alyssa Li
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Anu Pandey at University of California, Berkeley
Anu Pandey
Rohan Kar at University of California, Berkeley
Rohan Kar
Show all 5 authorsHide
Preprints and early-stage research may not have been peer reviewed yet.
Download file PDFDownload file PDF
Read file
Download citation

Abstract

In Augmented Reality for Visually Impaired People, we delve into the possibilities of using the Microsoft HoloLens AR HMD for accessibility, creating a prototype to detect surfaces and capture and read text on them to assist visually impaired people. We test the prototype with two sighted and seven blind participants in finding a replica bus stop. See more at https://arvips.squarespace.com
finalPost
er.pdf
Content uploaded by Rajandeep Singh
Author content
All content in this area was uploaded by Rajandeep Singh on Nov 26, 2021
Content may be subject to copyright.
AR for VIPs Final Repo
rt.pdf
Content uploaded by Dylan Fox
Author content
All content in this area was uploaded by Dylan Fox on Oct 19, 2020
Content may be subject to copyright.
A preview of the PDF is not available

Supplementary resource (1)

Poster_AR for VIPs_Web.pdf
Data
October 2020
Dylan Fox · Alyssa Li · Anu Pandey · Rohan Kar
Augmented Reality based Furniture Application
Conference Paper
Full-text available
  • May 2023
In the modern digital transformation era, providing customers with confidence in the suitability of the furniture they choose for various situations in which it will be used is the major problem in online furniture sales. While buying furniture, purchasers must ensure the furniture's model, size, and color are appropriate for and complement the space in which they will be put. The flexibility and appropriateness of the furniture product when placed in a particular area may be seen by prospective buyers with suitable instruments. For furniture to be sold online correctly, these issues must be resolved. By using Augmented Reality, this is feasible, to do. When it comes to online product customization, certain application models in the furniture sector employ Augmented Reality and mobile technology. A 3D product visualization tool, product pricing, and an enhanced renderer are all included. For the purpose of creating AR-based furniture technology, Google AR is used in the development process. It provides a framework for people or teams to creatively and flexibly address challenging tasks and issues in order to generate and deliver the best performance. The study is finished using Android Studio and Google AR services. Based on the results of the studies, it is advised that this application be enhanced to have a greater degree of performance. This might be achieved by creating a variety of various creator patterns using barcodes or QR codes, classifying surfaces like ceilings and walls so that the program can accurately show 3D things on them, and improving 3D models to fit them in the projecting space.
View
Using an Augmented Reality Device as a Distance-based Vision Aid—Promise and Limitations
Article
Full-text available
  • Jun 2018
SIGNIFICANCE For people with limited vision, wearable displays hold the potential to digitally enhance visual function. As these display technologies advance, it is important to understand their promise and limitations as vision aids. PURPOSE The aim of this study was to test the potential of a consumer augmented reality (AR) device for improving the functional vision of people with near-complete vision loss. METHODS An AR application that translates spatial information into high-contrast visual patterns was developed. Two experiments assessed the efficacy of the application to improve vision: an exploratory study with four visually impaired participants and a main controlled study with participants with simulated vision loss (n = 48). In both studies, performance was tested on a range of visual tasks (identifying the location, pose and gesture of a person, identifying objects, and moving around in an unfamiliar space). Participants' accuracy and confidence were compared on these tasks with and without augmented vision, as well as their subjective responses about ease of mobility. RESULTS In the main study, the AR application was associated with substantially improved accuracy and confidence in object recognition (all P < .001) and to a lesser degree in gesture recognition (P < .05). There was no significant change in performance on identifying body poses or in subjective assessments of mobility, as compared with a control group. CONCLUSIONS Consumer AR devices may soon be able to support applications that improve the functional vision of users for some tasks. In our study, both artificially impaired participants and participants with near-complete vision loss performed tasks that they could not do without the AR system. Current limitations in system performance and form factor, as well as the risk of overconfidence, will need to be overcome.
View
Current Use and Future Perspectives of Spatial Audio Technologies in Electronic Travel Aids
Article
Full-text available
Electronic travel aids (ETAs) have been in focus since technology allowed designing relatively small, light, and mobile devices for assisting the visually impaired. Since visually impaired persons rely on spatial audio cues as their primary sense of orientation, providing an accurate virtual auditory representation of the environment is essential. This paper gives an overview of the current state of spatial audio technologies that can be incorporated in ETAs, with a focus on user requirements. Most currently available ETAs either fail to address user requirements or underestimate the potential of spatial sound itself, which may explain, among other reasons, why no single ETA has gained a widespread acceptance in the blind community. We believe there is ample space for applying the technologies presented in this paper, with the aim of progressively bridging the gap between accessibility and accuracy of spatial audio in ETAs.
View
AR4VI: AR as an Accessibility Tool for People with Visual Impairments
Conference Paper
Full-text available
  • Oct 2017
Although AR technology has been largely dominated by visual media, a number of AR tools using both visual and auditory feedback have been developed specifically to assist people with low vision or blindness - an application domain that we term Augmented Reality for Visual Impairment (AR4VI). We describe two AR4VI tools developed at Smith-Kettlewell, as well as a number of pre-existing examples. We emphasize that AR4VI is a powerful tool with the potential to remove or significantly reduce a range of accessibility barriers. Rather than being restricted to use by people with visual impairments, AR4VI is a compelling universal design approach offering benefits for mainstream applications as well.
View
Blind people are more sensitive than sighted people to binaural sound-location cues, particularly inter-aural level differences
Article
Full-text available
  • Oct 2015
Blind people use auditory information to locate sound sources and sound-reflecting objects (echolocation). Sound source localization benefits from the hearing system's ability to suppress distracting sound reflections, whereas echolocation would benefit from "unsuppressing" these reflections. To clarify how these potentially conflicting aspects of spatial hearing interact in blind versus sighted listeners, we measured discrimination thresholds for two binaural location cues: inter-aural level differences (ILDs) and inter-aural time differences (ITDs). The ILDs or ITDs were present in single clicks, in the leading component of click pairs, or in the lagging component of click pairs, exploiting processes related to both sound source localization and echolocation. We tested 23 blind (mean age = 54 y), 23 sighted-age-matched (mean age = 54 y), and 42 sighted-young (mean age = 26 y) listeners. The results suggested greater ILD sensitivity for blind than for sighted listeners. The blind group's superiority was particularly evident for ILD-lag-click discrimination, suggesting not only enhanced ILD sensitivity in general but also increased ability to unsuppress lagging clicks. This may be related to the blind person's experience of localizing reflected sounds, for which ILDs may be more efficient than ITDs. On the ITD-discrimination tasks, the blind listeners performed better than the sighted age-matched listeners, but not better than the sighted young listeners. ITD sensitivity declines with age, and the equal performance of the blind listeners compared to a group of substantially younger listeners is consistent with the notion that blind people's experience may offset age-related decline in ITD sensitivity.
View
Augmented Reality Powers a Cognitive Prosthesis for the Blind
Article
  • May 2018
To restore vision for the blind several prosthetic approaches have been explored that convey raw images to the brain. So far these schemes all suffer from a lack of bandwidth and the extensive training required to interpret unusual stimuli. Here we present an alternate approach that restores vision at the cognitive level, bypassing the need to convey sensory data. A wearable computer captures video and other data, extracts the important scene knowledge, and conveys that through auditory augmented reality. This system supports many aspects of visual cognition: from obstacle avoidance to formation and recall of spatial memories, to long-range navigation. Neither training nor modification of the physical environment are required: Blind subjects can navigate an unfamiliar multi-story building on their first attempt. The combination of unprecedented computing power in wearable devices with augmented reality technology promises a new era of non-invasive prostheses that are limited only by software.
View
View
View
View
Prentice Medal Lecture 2013: Visual Accessibility: A Challenge for Low-Vision Research
Article
Low vision may be defined as any chronic form of vision impairment, not correctable by glasses or contact lenses, that adversely affects everyday function. Visual accessibility refers to factors that make an environment, device, or display usable by vision. In this article, I discuss the concept of visual accessibility with special reference to low vision. What role can vision science play in enhancing visual accessibility for people with low vision? I propose that greater efforts to embed low-vision research in real-world contexts and collaboration with other disciplines will accelerate progress. I describe examples from my current research projects on architectural accessibility and reading accessibility.
View
Seeing Eye Phone: A smart phone-based indoor localization and guidance system for the visually impaired
Article
  • Apr 2014
In order to help the visually impaired as they navigate unfamiliar environment such as public buildings, this paper presents a novel smart phone, vision-based indoor localization, and guidance system, called Seeing Eye Phone. This system requires a smart phone from the user and a server. The smart phone captures and transmits images of the user facing forward to the server. The server processes the phone images to detect and describe 2D features by SURF and then matches them to the 2D features of the stored map images that include their corresponding 3D information of the building. After features are matched, Direct Linear Transform runs on a subset of correspondences to find a rough initial pose estimate and the Levenberg–Marquardt algorithm further refines the pose estimate to find a more optimal solution. With the estimated pose and the camera’s intrinsic parameters, the location and orientation of the user are calculated using 3D location correspondence data stored for features of each image. Positional information is then transmitted back to the smart phone and communicated to the user via text-to-speech. This indoor guiding system uses efficient algorithms such as SURF, homographs, multi-view geometry, and 3D to 2D reprojection to solve a very unique problem that will benefit the visually impaired. The experimental results demonstrate the feasibility of using a simple machine vision system design to accomplish a complex task and the potential of building a commercial product based on this design.
View