ArticlePDF Available
www.nature.com/eye
COMMENT
OPEN
The future of ophthalmology and vision science with the Apple
Vision Pro
Ethan Waisberg ]]]
1
, Joshua Ong
2
, Mouayad Masalkhi
1
, Nasif Zaman
3
, Prithul Sarker
]]]
3
, Andrew G. Lee
4,5,6,7,8,9,10,11
and
Alireza Tavakkoli
]]]
3
© The Author(s) 2023
Eye; https://doi.org/10.1038/s41433-023-02688-5
The Apple Vision Pro is an innovative gadget that integrates
sophisticated technology to deliver an innovative spatial comput-
ing experience [1]. This device provides a seamless integration of
digital media and the physical world and gives users the ability to
navigate using their eyes, hands, and voice [1]. The device
incorporates sophisticated cameras and sensors to facilitate
precise visual perception, environmental awareness, and hand
recognition [1]. The audio straps and speakers offer a superior
’Spatial Audio’ experience that integrates with surrounding
ambient sounds. Most importantly, the Apple Vision Pro is fitted
with personalized micro-OLED screens deliver a greater number
of pixels to each eye, providing unique clarity (Fig. 1). Paired with
eye-tracking technology and various other features, the Apple
Vision Pro offers exciting technical advances with hopes of further
improvements in performance and lower cost for extended reality
devices in the near future.
The advent of extended reality has revolutionized the field of
ophthalmology [2]. Current research includes utilizing extended
reality for surgical training, ophthalmic diagnosis, and even
monitoring of astronaut structural and functional vision changes
during spaceflight [24]. With aging populations worldwide, vision
screening will only become of greater importance in the coming
years. Extended reality vision screening will also enable more
frequent examinations of patients, which is particularly beneficial to
those with chronic conditions like glaucoma and age-related macular
degeneration (AMD) that have the potential to cause irreversi-
ble blindness if changes are left undetected [5]. Early detection,
monitoring, and treatment of such diseases can potentially be an
effective method prevent further vision loss [6]. The utilization of
Apple Vision Pro and other future extended reality headsets could
potentially be vital as a solution for vision screening globally, without
needing any additional specialized equipment. In underserved areas
and developing countries, there is a lack in trained ophthalmic
healthcare experts and specialized equipment to screen populations,
and extended reality visual screening can solve this deficit.
The potential integration of Apple Vision Pro’s sophisticated
display technology also presents as a promising method to
restore vision through augmented reality. For example, extended
reality has previously been used to successfully been used to
restore vision by reducing text metamorphopsia [7], visual field
expansion [8], visual acuity [8], to being used to treat mild cases
of strabismus [9]. All things considered, the Apple Vision Pro and
future work in the extended reality space represent an innovative
strategy for enhancing visual acuity and improving the overall
quality of life of individuals suffering from ophthalmic disorders.
Ophthalmologists and vision scientists may benefit from knowing
of the unique technical capabilities of such technology and the
advances in brings to the field.
REFERENCES
1. Apple Vision Pro. Apple. Accessed June, 2023. https://www.apple.com/apple-
vision-pro/.
2. Iskander M, Ogunsola T, Ramachandran R, McGowan R, Al-Aswad LA. Virtual reality
and augmented reality in ophthalmology: a contemporary prospective. Asia-Pac J
Ophthalmol. 2021;10:244–52. https://doi.org/10.1097/APO.0000000000000409.
3. Ong J, Tavakkoli A, Zaman N, Kamran SA, Waisberg E, Gautam N, et al. Terrestrial
health applications of visual assessment technology and machine learning in
spaceflight associated neuro-ocular syndrome. npj Microgravity. 2022;8:37.
https://doi.org/10.1038/s41526-022-00222-7.
4. Waisberg E, Ong J, Zaman N, Kamran SA, Lee AG, Tavakkoli A. Head-mounted
dynamic visual acuity for G-transition effects during interplanetary spaceflight:
technology development and results from an early validation study. Aerosp Med
Hum Perform. 2022;93:800–5. https://doi.org/10.3357/AMHP.6092.2022.
5. Heru: Virtual Eye Exam & Visual Field | AR Eye Machine. Heru. Accessed June, 2023.
https://www.seeheru.com/.
Fig. 1 The Apple Vision Pro headset.
1
University College Dublin School of Medicine, Belfield, Dublin, Ireland.
2
Michigan Medicine, University of Michigan, Ann Arbor, MI, USA.
3
Human-Machine Perception
Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, NV, USA.
4
Center for Space Medicine, Baylor College of Medicine, Houston, TX,
USA.
5
Department of Ophthalmology, Blanton Eye Institute, Houston Methodist Hospital, Houston, TX, USA.
6
The Houston Methodist Research Institute, Houston Methodist
Hospital, Houston, TX, USA.
7
Departments of Ophthalmology, Neurology, and Neurosurgery, Weill Cornell Medicine, New York, NY, USA.
8
Department of Ophthalmology,
University of Texas Medical Branch, Galveston, TX, USA.
9
University of Texas MD Anderson Cancer Center, Houston, TX, USA.
10
Texas A&M College of Medicine, Bryan, TX, USA.
11
Department of Ophthalmologys, The University of Iowa Hospitals and Clinics, Iowa City, IA, USA. email: ethan.waisberg@ucdconnect.ie
Received: 19 July 2023 Revised: 20 July 2023 Accepted: 27 July 2023
1234567890();,:
6. Waisberg E, Ong J, Paladugu P, Zaman N, Kamran SA, Lee AG, et al. Optimizing
screening for preventable blindness with head-mounted visual assessment
technology. J Vis Impairment Blindness. 2022;116:579–81. https://doi.org/10.1177/
0145482X221124186.
7. Ong J, Zaman N, Waisberg E, Kamran SA, Lee AG, Tavakkoli A. Head-mounted
digital metamorphopsia suppression as a countermeasure for macular-related
visual distortions for prolonged spaceflight missions and terrestrial health.
Wearable Technol. 2022;3:e26. https://doi.org/10.1017/wtc.2022.21.
8. Pur DR, Lee-Wing N, Bona MD. The use of augmented reality and virtual reality for
visual field expansion and visual acuity improvement in low vision rehabilitation:
a systematic review. Graefes Arch Clin Exp Ophthalmol. 2023;261:1743–55. https://
doi.org/10.1007/s00417-022-05972-4.
9. Waisberg E, Ong J, Zaman N, Kamran SA, Sarker P, Tavakkoli A, et al. Extended
reality for strabismus screening in developing countries. Eye. Published online
June, 2023. https://doi.org/10.1038/s41433-023-02649-y.
AUTHOR CONTRIBUTIONS
EW, JO, MM: conceptualization, writing. NZ, PS, AGL, AT: review, intellectual support.
FUNDING
NASA grant [80NSSC20K183]: a non-intrusive ocular monitoring framework to model
ocular structure and functional changes due to long-term spaceflight. Open Access
funding provided by the IReL Consortium.
COMPETING INTERESTS
The authors declare no competing interests.
ADDITIONAL INFORMATION
Correspondence and requests for materials should be addressed to Ethan Waisberg .
Reprints and permission information is available at http://www.nature.com/
reprints
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims
in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons
Attribution 4.0 International License, which permits use, sharing,
adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative
Commons licence, and indicate if changes were made. The images or other third party
material in this article are included in the article’s Creative Commons licence, unless
indicated otherwise in a credit line to the material. If material is not included in the
article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly
from the copyright holder. To view a copy of this licence, visit http://creativecom-
mons.org/licenses/by/4.0/.
© The Author(s) 2023
E. Waisberg et al.
2
Eye
... For AR to grow in use, some researchers suggest that the most important considerations are ergonomics and ease of use (Waisberg et al., 2023). As detailed above, some challenges in hardware due to the microphone and ergonomic fit were observed in this research as well. ...
... However, it is important to highlight the ongoing advancements in AR hardware, particularly in achieving a seamless integration of real and virtual worlds. Progress in this field, exemplified by products like Apple's Vision Pro (Tiwari et al., 2023) and Meta's Meta Quest Pro (Dempsey, 2023), have started to break barriers between physical and digital domains (Minaee et al., 2022;Waisberg et al., 2023). Another emerging technology is AR smart lenses, similar to contact lenses, which discreetly incorporate digital elements into a user's visual field, potentially enhancing visual capabilities (Waisberg et al., 2023). ...
... Progress in this field, exemplified by products like Apple's Vision Pro (Tiwari et al., 2023) and Meta's Meta Quest Pro (Dempsey, 2023), have started to break barriers between physical and digital domains (Minaee et al., 2022;Waisberg et al., 2023). Another emerging technology is AR smart lenses, similar to contact lenses, which discreetly incorporate digital elements into a user's visual field, potentially enhancing visual capabilities (Waisberg et al., 2023). Furthermore, AR holds the potential to introduce new sensory dimensions. ...
... Our initial research in the most commonly known database indexes (Google Scholar, Scopus, Web of Science, PsycINFO) revealed an increasing body of literature, concerning AVP, specifically in the general academic field of health. Such research has been conducted in the field of ophthalmology (Jonnakuti and Frankfort, 2023;Masalkhi et al., 2023) with (Waisberg et al., 2024a) addressing AVP as a mean that can enhance visual acuity and hence, improve individuals quality of life, surgery (Olexa et al., 2024;Waisberg et al., 2024b), medicine (Waisberg et al., 2024c), medical education (Armstrong et al., 2024;Waisberg et al., 2024d) and psychological research and therapy (Zhang et al., 2023). Besides healthcare, a significant number of references to AVP in the bibliography comes from the field of consumer behaviour. ...
Conference Paper
Full-text available
In the digital era, emerging technologies such as Vision Pro are crucial for businesses due to their transformative potential across various industries. As an amalgamation of augmented reality (AR), virtual reality (VR), computer vision, and machine learning, Vision Pro technology represents a frontier in the intersection of human-computer interaction, offering innovative solutions and opening up new avenues for value creation in business. Considering the primary stages of this technology, this study aims to explore the spectrum of reactions in Vision Pro, presenting a sentiment analysis of the 'VisionPro' subreddit, a community dedicated to discussing vision technologies. Through sentiment analysis, we could discern patterns that suggest the factors driving positive and negative reactions within the community. This paper sheds light on the specific sentiments prevalent in the 'VisionPro' subreddit and demonstrates the applicability of sentiment analysis in understanding community dynamics in technology-focused online forums. The findings contribute to the broader discourse on public sentiment towards emerging technologies, offering implications for developers, researchers, and enthusiasts engaged in vision technology.
... The latest research on the use of Apple Vision Pro also takes into account its potential in non-surgical applications. The high quality of the displays is being improved, which is indicated as a potential convenience for users with visual impairments [93,94]. Thanks to the appropriately designed interface, the user has the ability to track the movement of their own hands and gestures, which, unlike most devices of this class, allows for the elimination of physical controllers, touch screens or keyboards. ...
Article
The article is a systematic review of the latest information on the intensive development of virtual reality (VR) applications in various fields of medicine. The paper analyses in detail the impact of virtual reality on the user, taking into account both the advantages and disadvantages, as well as the limitations of the above technology. The aim of the study was to analyse and discuss the available information from the scientific literature on the use of virtual reality in education, intraoperative, diagnostic, therapeutic and rehabilitation applications. It summarises and discusses the potential directions of VR development in correlation with the discussed aspects of the medical area. A systematic review of the literature was performed using the databases: PubMed, Elsevier, and Google Scholar. The following keywords were used: “virtual reality”, “medicine”, “medical education”, “medical technology”, “virtual reality in medical training and education”, “virtual reality in diagnostic imaging and application in medical procedures”, “virtual reality in rehabilitation”, “intraoperative use of virtual reality”, “Apple Vision Pro in medical operations”, “virtual reality autism”, “virtual reality phobias”, “virtual reality PTSD”, “virtual reality pain”, “virtual reality and therapy”. A total of 1150 items were searched, and after removing duplicate items, applying exclusion criteria and filters such as clinical trial, meta-analysis, randomised control trial, systematic review, 320 articles were selected. After analysis, 121 articles were assessed for eligibility. Focusing on the characteristics of the materials and methods used and the results obtained and conclusions contained in each article, 101 research articles, reviews and meta-analyses were included in the study. Virtual reality, despite current limitations, is a dynamically developing field that finds more and more applications in medicine, including technology and medical education. With further development, the above implementations will inevitably be improved and tested, which leads to the conclusion that further discoveries can be expected confirming their potential in education and diagnostic and therapeutic processes.
... The video of the live broadcast can be played in VR glasses, which enhances the experience of the audience watching the sports event, so that the audience's senses receive greater stimulation, the audience has an immersive feeling so that the audience can better appreciate the athletes shooting and shooting action. Maintains the same convenience as live mobile phone broadcasting [10]. ...
Article
Full-text available
The evolution of broadcasting rights in the era of streaming media has ushered in a transformative phase, characterized by an exponential surge in their pricing. This paper delves into the historical progression of these rights, examining the multifaceted factors that have contributed to this significant escalation. It scrutinizes the business models of sports event organizers, who have traditionally relied on broadcasting rights, ticket sales, and commercial and brand partnerships as primary revenue streams. The trajectory of broadcasting rights’ value has seen a gradual ascent, culminating in a sharp increase in recent times. Key determinants of this price inflation include the intrinsic brand value of the events, the dynamics of audience demand, the intensity of competition among media outlets, and the inherent excitement and appeal of the sports events themselves. These elements collectively shape the economic landscape of broadcasting rights, reflecting the interplay of market forces and the growing commercialization of sports entertainment.
Article
Full-text available
Orthopedic trauma remains a critical challenge in modern healthcare, often resulting in severe mobility limitations, acute pain, and delayed recovery. Conventional rehabilitation techniques, though effective, fail to address the individualized, high-precision interventions needed for musculoskeletal injuries like fractures, joint instability, ligament tears, and muscular atrophy. Virtual reality (VR) technologies, such as Apple Vision Pro and HTC Vive Pro, offer a transformative approach by enhancing diagnostic precision, rehabilitation effectiveness, and patient engagement through interactive, immersive environments that improve clinical outcomes. These VR technologies provide real-time biomechanical data, such as joint mechanics, muscle coordination, and movement patterns, allowing clinicians to design personalized rehabilitation programs. These technologies can thus facilitate neuromuscular re-education, improve muscle proprioception, and enhance muscle coordination. Studies have shown that VR-based rehabilitation advances functional recovery, improves pain management, and reduces psychological barriers associated with immobility. VR also facilitates telemedicine, increasing accessibility for patients with geographic or mobility issues. However, while VR may provide biomechanical data, it is important to note that they fall short in accurate motion tracking, particularly in fine motor control tasks. This scoping review follows PRISMA guidelines to explore the potential of VR in orthopedic rehabilitation, analyzing its diagnostic capabilities, personalized interventions, and real-time feedback systems. Despite this, barriers remain, including regulatory challenges, limitations in haptic feedback, high cost, and patient compliance. By presenting a balanced perspective on the landscape of VR in orthopedic care, this paper emphasizes the need for rigorous clinical validation, regulatory advancements, and interdisciplinary collaboration. Ultimately, VR offers the potential to significantly improve recovery outcomes, enhance patient engagement, and streamline rehabilitation protocols, but its successful integration into clinical practice must be approached with both optimism and caution.
Article
With the rapid advancement of intelligent devices and cloud services, a novel edge-cloud computing paradigm is emerging, finding widespread adoption in numerous advanced applications. Despite its considerable convenience and benefits, edge-cloud computing raises security and privacy concerns. Although many cryptographic solutions have been proposed for the Internet of Things and cloud services, ensuring diverse access control in an untrusted edge-cloud environment and realizing flexible revocation and efficient outsourcing remain challenging. In this paper, we propose a certificateless attribute-based matchmaking encryption scheme (CRO-ABME) that supports fine-grained bilateral access control, attribute and identity revocation, and cryptographic workload outsourcing. Leveraging CRO-ABME, we design an edge-cloud data sharing system that ensures secure data uploading with privacy protection between end-users, such that only authorized matchers can access the data in edge-cloud computing. Furthermore, rigorous security proofs for CRO-ABME are provided, and experimental analyses demonstrate the efficiency and flexibility of our proposed scheme.
Article
Full-text available
Abstract Introduction Developments in image processing techniques and display technology have led to the emergence of augmented reality (AR) and virtual reality (VR)–based low vision devices (LVDs). However, their promise and limitations in low vision rehabilitation are poorly understood. The objective of this systematic review is to appraise the application of AR/VR LVDs aimed at visual field expansion and visual acuity improvement in low vision rehabilitation. Methods A systematic search of the literature was performed using MEDLINE, Embase, PsychInfo, HealthStar, and National Library of Medicine (PubMed) from inception to March 6, 2022. Articles were eligible if they included an AR or VR LVD tested on a sample of individuals with low vision and provided visual outcomes such as visual acuity, visual fields, and object recognition. Results Of the 652 articles identified, 16 studies comprising 382 individuals with a mean age of 52.17 (SD = 18.30) years, and with heterogeneous low vision etiologies (i.e., glaucoma, age-related macular degeneration, retinitis pigmentosa) were included in this systematic review. Most articles used AR (53%), VR (40%), and one article used both AR and VR. The main visual outcomes evaluated were visual fields (67%), visual acuity (65%), and contrast sensitivity (27%). Various visual enhancement techniques were employed including variable magnification using digital zoom (67%), contrast enhancements (53%), and minification (27%). AR LVDs were reported to expand the visual field from threefold to ninefold. On average, individuals using AR/VR LVDs experienced an improved in visual acuity from 0.9 to 0.2 logMAR. Ten articles were classified as high or moderate risk of bias. Conclusion AR/VR LVDs were found to afford visual field expansion and visual acuity improvement in low vision populations. Even though the results of this review are promising, the lack of controlled studies with well-defined populations, use of small, convenience samples, and incomplete reporting of inclusion and exclusion criteria among included studies makes it challenging to judge the true impact of these devices. Future studies should address these limitations and compare various AR/LVDs to determine what is the ideal LVD type and vision enhancement combination based on the user’s level of visual ability and lifestyle.
Article
Full-text available
INTRODUCTION: Dynamic visual acuity (DVA) refers to the ability of the eye to discern detail in a moving object and plays an important role whenever rapid physical responses to environmental changes are required, such as while performing tasks onboard a space shuttle. A significant decrease in DVA has previously been noted after astronauts returned from long-duration spaceflight (0.75 eye chart lines, 24 h after returning from space). As part of a NASA-funded, head-mounted multimodal visual assessment system for monitoring vision changes in spaceflight, we elaborate upon the technical development and engineering of dynamic visual acuity assessments with virtual reality (VR) technology as the first step in assessing astronaut performance when undergoing G-transitional effects. We also report results from an early validation study comparing VR DVA assessment with traditional computer based DVA assessment.METHODS: Various VR/AR headsets have been utilized to implement DVA tests. These headsets include HTC Vive Pro Eye system. Epic's game engine UnrealEngine 4 Version 4.24 was used to build the framework and SteamVR was used to experience virtual reality content. Eye tracking technology was used to maintain fixation of the participant. An early validation study with five participants was conducted comparing this technology versus traditional DVA with a laptop.RESULTS: The head-mounted technology developed for assessing DVA changes during G-transitions is fully functional. The results from the early validation study demonstrated that the two DVA tests (laptop-based and VR) indicated a strong association between both methods (Pearson correlation coefficient of 0.91). A Bland-Altman plot was employed to assess levels of agreement, with all data points falling within the limits of agreement.DISCUSSION: The results from this early validation study indicate that head-mounted DVA assessment performs similarly to traditional laptop-based methods and is a promising method for assessing DVA during spaceflight, particularly in G-transitions. Future studies are required for further assessment of validation and reliability of this technology. With its ease of use, accessibility, and portable design, VR DVA has the potential in the near-future to replace conventional methods of assessing DVA. The technology will likely be an important aspect to help monitor functionality and safety during interplanetary missions where astronauts are exposed to G-transitions.Waisberg E, Ong J, Zaman N, Kamran Sa, Lee AG, Tavakkoli A. Head-mounted dynamic visual acuity for G-transition effects during interplanetary spaceflight: technology development and results from an early validation study. Aerosp Med Hum Perform. 2022; 93(11):800-805.
Article
Full-text available
During long-duration spaceflight, astronauts are exposed to various risks including spaceflight-associated neuro-ocular syndrome, which serves as a risk to astronaut vision and a potential physiological barrier to future spaceflight. When considering exploration missions that may expose astronauts to longer periods of microgravity, radiation exposure, and natural aging processes during spaceflight, more severe changes to functional vision may occur. The macula plays a critical role in central vision and disruptions to this key area in the eye may compromise functional vision and mission performance. In this article, we describe the development of a countermeasure technique to digitally suppress monocular central visual distortion with head-mounted display technology. We report early validation studies with this noninvasive countermeasure in individuals with simulated metamorphopsia. When worn by these individuals, this emerging wearable countermeasure technology has demonstrated a suppression of monocular visual distortion. We describe the considerations and further directions of this head-mounted technology for both astronauts and aging individuals on Earth.
Article
Full-text available
The neuro-ocular effects of long-duration spaceflight have been termed Spaceflight Associated Neuro-Ocular Syndrome (SANS) and are a potential challenge for future, human space exploration. The underlying pathogenesis of SANS remains ill-defined, but several emerging translational applications of terrestrial head-mounted, visual assessment technology and machine learning frameworks are being studied for potential use in SANS. To develop such technology requires close consideration of the spaceflight environment which is limited in medical resources and imaging modalities. This austere environment necessitates the utilization of low mass, low footprint technology to build a visual assessment system that is comprehensive, accessible, and efficient. In this paper, we discuss the unique considerations for developing this technology for SANS and translational applications on Earth. Several key limitations observed in the austere spaceflight environment share similarities to barriers to care for underserved areas on Earth. We discuss common terrestrial ophthalmic diseases and how machine learning and visual assessment technology for SANS can help increase screening for early intervention. The foundational developments with this novel system may help protect the visual health of both astronauts and individuals on Earth.
Article
Full-text available
Purpose: Most published systematic reviews have focused on the use of virtual reality (VR)/augmented reality (AR) technology in ophthalmology as it relates to surgical training. To date, this is the first review that investigates the current state of VR/AR technology applied more broadly to the entire field of ophthalmology. Methods: PubMed, Embase, and CINAHL databases were searched systematically from January 2014 through December 1, 2020. Studies that discussed VR and/or AR as it relates to the field of ophthalmology and provided information on the technology used were considered. Abstracts, non-peer-reviewed literature, review articles, studies that reported only qualitative data, and studies without English translations were excluded. Results: A total of 77 studies were included in this review. Of these, 28 evaluated the use of VR/AR in ophthalmic surgical training/assessment and guidance, 7 in clinical training, 23 in diagnosis/screening, and 19 in treatment/therapy. 15 studies used AR, 61 used VR, and 1 used both. Most studies focused on the validity and usability of novel technologies. Conclusions: Ophthalmology is a field of medicine that is well suited for the use of VR/AR. However, further longitudinal studies examining the practical feasibility, efficacy, and safety of such novel technologies, the cost-effectiveness, and medical/legal considerations are still needed. We believe that time will indeed foster further technological advances and lead to widespread use of VR/AR in routine ophthalmic practice.