No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Immersive Virtual Reality Attacks and the Human Joystick
Abstract
This is one of the first accounts for the security analysis of consumer immersive Virtual Reality (VR) systems. This work breaks new ground, coins new terms, and constructs proof of concept implementations of attacks related to immersive VR. Our work used the two most widely adopted immersive VR systems, the HTC Vive, and the Oculus Rift. More specifically, we were able to create attacks that can potentially disorient users, turn their Head Mounted Display (HMD) camera on without their knowledge, overlay images in their field of vision, and modify VR environmental factors that force them into hitting physical objects and walls. Finally, we illustrate through a human participant deception study the success of being able to exploit VR systems to control immersed users and move them to a location in physical space without their knowledge. We term this the Human Joystick Attack. We conclude our work with future research directions and ways to enhance the security of these systems.
... Although existing works [2]- [4] highlight the importance of security and privacy issues in VR applications, there are a limited systematic efforts in evaluating the effect of various threat scenarios on such edge computing based collaborative systems with IoT devices. Specifically, VRLE applications are highly susceptible to Distributed Denial of Service (DDoS) attacks, due to the distributed IoT devices (i.e.,VR headsets) connecting to virtual classrooms through custom controlled This material is based upon work supported by the National Science Foundation under Award Number CNS-1647213. ...
... A recent study [1] on challenges in AR and VR discusses the threat vectors for educational initiatives without characterizing the attack impact. Survey articles [2]- [4], [14]- [16] are significant for understanding the concepts of threat taxonomy and attack surface area of IoT and fog computing. They highlight the need to go beyond specific components such as network, hardware or application, and propose end-to-end solutions that consider system and data vulnerabilities. ...
... To elucidate, critical VRLE applications such as flight simulations, military training exercises and vSocial (developed for children with ASD) are sensitive to information disclosure attacks that can cause significant disruption for the stakeholder participants. If an attacker compromises such sensitive information, it can be used to harm the participants in the form of e.g., a chaperone attack (to make a user run into walls) and other physical safety attacks detailed in [4]. ...
Social Virtual Reality Learning Environment (VRLE) is a novel edge computing platform for collaboration amongst distributed users. Given that VRLEs are used for critical applications (e.g., special education, public safety training), it is important to ensure security and privacy issues. In this paper, we present a novel framework to obtain quantitative assessments of threats and vulnerabilities for VRLEs. Based on the use cases from an actual social VRLE viz., vSocial, we first model the security and privacy using the attack trees. Subsequently, these attack trees are converted into stochastic timed automata representations that allow for rigorous statistical model checking. Such an analysis helps us adopt pertinent design principles such as hardening, diversity and principle of least privilege to enhance the resilience of social VRLEs. Through experiments in a vSocial case study, we demonstrate the effectiveness of our attack tree modeling with a reduction of 26% in probability of loss of integrity (security) and 80% in privacy leakage (privacy) in before and after scenarios pertaining to the adoption of the design principles.
... Owing to the inherent interconnectivity of the network-edge and the core cloud in the VRLE setup, the VR application is vulnerable to novel attacks known as immersion attacks. To elucidate, using SP issues as the vulnerabilities, an attacker can: (i) cause defacement of VRLE content with offensive images known as overlay attack [5], (ii) obstruct the user view or trigger noise attenuation during VRLE sessions known as occlusion attack [5], and (iii) create application issues i.e., reduction of graphical content or delays between both user and avatar movement. Failure to address such security, privacy and resulting safety (SPS) issues in VRLE results in alteration of instructional content, compromise of learning outcomes, abuse of access privileges leading to confidential student information disclosure and/or poor student engagement due to cybersickness. ...
... Owing to the inherent interconnectivity of the network-edge and the core cloud in the VRLE setup, the VR application is vulnerable to novel attacks known as immersion attacks. To elucidate, using SP issues as the vulnerabilities, an attacker can: (i) cause defacement of VRLE content with offensive images known as overlay attack [5], (ii) obstruct the user view or trigger noise attenuation during VRLE sessions known as occlusion attack [5], and (iii) create application issues i.e., reduction of graphical content or delays between both user and avatar movement. Failure to address such security, privacy and resulting safety (SPS) issues in VRLE results in alteration of instructional content, compromise of learning outcomes, abuse of access privileges leading to confidential student information disclosure and/or poor student engagement due to cybersickness. ...
... Survey articles such as [4], [6], [15]- [17] are significant for understanding the concepts of threat taxonomy and attack surface area of sensors and fog computing applications. They highlight the need to go beyond specific components such as network, hardware or user interface, and propose end-to-end solutions that consider system and data vulnerabilities [5]. An observation from the above state-of-art is that -there is a dearth of scholarly works on the quantitative evaluation for security and privacy threats in the context of VR applications. ...
Social Virtual Reality Learning Environments (VRLE) offer a new medium for flexible and immersive learning environments with geo-distributed users. Ensuring user safety in VRLE application domains such as education, flight simulations, military training is of utmost importance. Specifically, there is a need to study the impact of "immersion attacks" (e.g., chaperone attack, occlusion) and other types of attacks/faults (e.g., unauthorized access, network congestion) that may cause user safety issues (i.e., inducing of cybersickness). In this paper, we present a novel framework to quantify the security, privacy issues triggered via immersion attacks and other types of attacks/faults. By using a real-world social VRLE viz., vSocial and creating a novel attack-fault tree model, we show that such attacks can induce undesirable levels of cybersickness. Next, we convert these attack-fault trees into stochastic timed automata (STA) representations to perform statistical model checking for a given attacker profile. Using this model checking approach, we determine the most vulnerable threat scenarios that can trigger high occurrence cases of cybersickness for VRLE users. Lastly, we show the effectiveness of our attack-fault tree modeling by incorporating suitable design principles such as hardening, diversity, redundancy and principle of least privilege to ensure user safety in a VRLE session.
... 1) Threats to Personal Safety. In the metaverse, hackers can attack wearable devices, XR helmets, and other indoor sensors (e.g., cameras) to obtain the life routine and track the real-time position of users to facilitate burglary, which may threaten their safety [55]. Besides, due to the immersive realism of metaverse, hackers can suddenly display harmful and scary content (e.g., ghost pictures) in the virtual environment in front of the avatar, which may lead to the death of fright of the corresponding user. ...
... 2) Human Safety and Cyber syndromes: The full immersiveness in metaverse can also raise immersion concerns, e.g., occlusion and chaperone attack, as well as cybersickness. Casey et al. [55] investigate a new attack named human joystick attack in immersive VR systems such as Oculus Rift and HTC Vive. In their work, adversaries can modify VR environmental factors to deceive, disorient, and control immersed human players and move them to other physical locations without consciousness. ...
div>Metaverse, as an evolving paradigm of the next-generation Internet, aims to build a fully immersive, hyper spatiotemporal, and self-sustaining virtual shared space for humans to play, work, and socialize. Driven by recent advances in emerging technologies such as extended reality, artificial intelligence, and blockchain, metaverse is stepping from the science fiction to an upcoming reality. However, severe privacy invasions and security breaches (inherited from underlying technologies or emerged in the new digital ecology) of metaverse can impede its wide deployment. At the same time, a series of fundamental challenges (e.g., scalability and interoperability) can arise in metaverse security provisioning owing to the intrinsic characteristics of metaverse, such as immersive realism, hyper spatiotemporality, sustainability, and heterogeneity.
In this paper, we present a comprehensive survey of the fundamentals, security, and privacy of metaverse.
Specifically, we first investigate a novel distributed metaverse architecture and its key characteristics with ternary-world interactions. Then, we discuss the security and privacy threats, present the critical challenges of metaverse systems, and review the state-of-the-art countermeasures. Finally, we draw open research directions for building future metaverse systems.</div
... With the popularity of Virtual Reality (VR), numerous applications emerged to enable new experiences for traditional fields (i.e., games, shopping, etc.). However, most of VR application designers ignored one of crucial problems in designing VR applications, the security and privacy [3]. Especially, the traditional user authentication methods are no longer secure to meet the requirements of VR applications. ...
... From our observations, all three categories user authentication can be easily recorded when facing up with the Man-in-the-room attack [3]. For biometric-based authentication and behavioral biometric, even if the user authentication information is stolen by the MITR attacker, because the user's biometrics cannot be forged, the attacker is unable to impersonate the user to log in. ...
With the popularity of Virtual Reality (VR), most of VR applications focus on content creation and experience design, while the security and privacy of VR applications has been ignored. For example, the Man-in-the-room (MITR) attack is well known for tapping user interactions in VR, including the user’s behaviors, real-time conversations, screenshots and computer audio, etc. Therefore, to provide secure and usable user authentication become one of crucial problems for VR applications. In this work, we propose a multi-attribute user authentication method to against MITR attack in VR. Specifically, any combinations of these selected attributes and values can be password for the certain user. The presented objects that conform to the password from the random object group are provided by our designed principles for identity authentication. The user then can select any presented attributes and values for authentication. Through this process, the attacker could not easy to guess the password even via a MITR attack. We designed a pilot study and evaluated the effectiveness and security of the proposed authentication method. By imitating the behavior of the MITR attacker, the authentication schemes under different settings are used to test the influence of different variables in the proposed security scheme. The experimental result shows that the proposed scheme can effectively resist invisible attack. This work-in-progress can give preliminary suggestions to against MITR for securing VR applications.
... With MR systems blending together virtual and physical experiences, malware targeting these devices may have a slew of additional capabilities. An immersed user, who trusts both their vision and hearing to the VR device, is a perfect target for an attack that could even bring about physical harm (Casey et al., 2019;UNHcFREG, 2018). Should the VE be maliciously modified to disrupt or deceive an immersed user, the calibration between the physical and virtual space will inevitably be compromised. ...
... State-of-the art work has only scratched the surface on the forensics and security of VR. de Guzman et al. (2018) provided a survey of privacy and security research and approaches in MR. Most relevant to our work is the work by Yarramreddy et al. (2018) which considered disk and network artifacts of social VR applications, while Casey et al. (2019) evaluated the security of these systems and implemented proof-of-concept attacks. ...
Virtual Reality (VR) has become a reality. With the technology's increased use cases, comes its misuse. Malware affecting the Virtual Environment (VE) may prevent an investigator from ascertaining virtual information from a physical scene, or from traditional “dead” analysis. Following the trend of anti-forensics, evidence of an attack may only be found in memory, along with many other volatile data points. Our work provides the primary account for the memory forensics of Immersive VR systems, and in specific the HTC Vive. Our approach is capable of reconstituting artifacts from memory that are relevant to the VE, and is also capable of reconstructing a visualization of the room setup a VR player was immersed into. In specific, we demonstrate that the VE, location, state and class of VR devices can be extracted from memory. Our work resulted in the first open source VR memory forensics plugin for the Volatility Framework. We discuss our findings, and our replicable approach that may be used in future memory forensics research.
... For example, malicious software might add additional "objectives" or mechanics to games that induces the user to move in a particular direction. This has been dubbed the Human Joystick effect [10]. A primary use of this technology is to trick a user to enter certain types of biometric authentication. ...
... Researchers have been able to replicate this effect very consistently on the two most major VR systems, the HTC Vive and Oculus Rift by manipulation of files in the Steam software that they have in common. In doing so, they were able to not only implement overlays, but also set up independent sessions that allowed them to collect data on the user's actions [10]. Researchers have also examined and raised concerns about such technology being utilized in mixed reality (also known as augmented reality) which is an extension of virtual reality that allows the combination of virtual and realworld objects by means of either a partially clear VR screen or a video camera that has its feed combined with the necessary additions before being sent to the standard VR screen [11]. ...
There is a growing need for authentication methodology in virtual reality applications. Current systems assume that the immersive experience technology is a collection of peripheral devices connected to a personal computer or mobile device. Hence there is a complete reliance on the computing device with traditional authentication mechanisms to handle the authentication and authorization decisions. Using the virtual reality controllers and headset poses a different set of challenges as it is subject to unauthorized observation, unannounced to the user given the fact that the headset completely covers the field of vision in order to provide an immersive experience. As the need for virtual reality experiences in the commercial world increases, there is a need to provide other alternative mechanisms for secure authentication. In this paper, we analyze a few proposed authentication systems and reached a conclusion that a multidimensional approach to authentication is needed to address the granular nature of authentication and authorization needs of a commercial virtual reality applications in the commercial world.
... However, prior works lack in the knowledge to address both performance and security issues that can impact the user experience and user safety in VRLE sessions. Failure to address such impediments can lead to deface attacks on the VR content with offensive images [9] that can hamper user experience. They can also lead to application latency issues that degrade performance. ...
Social virtual reality learning environments (VRLEs) provide immersive experience to users with increased accessibility to remote learning. Lack of maintaining high-performance and secured data delivery in critical VRLE application domains (e.g., military training, manufacturing) can disrupt application functionality and induce cybersickness. In this paper, we present a novel rule-based 3QS-adaptation framework that performs risk and cost aware trade-off analysis to control cybersickness due to performance/security anomaly events during a VRLE session. Our framework implementation in a social VRLE viz., vSocial monitors performance/security anomaly events in network/session data. In the event of an anomaly, the framework features rule-based adaptations that are triggered by using various decision metrics. Based on our experimental results, we demonstrate the effectiveness of our rule-based 3QS-adaptation framework in reducing cybersickness levels, while maintaining application functionality. Using our key findings, we enlist suitable practices for addressing performance and security issues towards a more high-performing and robust social VRLE.
... These two aspects may be targets or facilitators of cyber attacks. Some excellent examples of such attacks were demonstrated by Casey et al. in [4] who exploited the OpenVR API to disorient users, turn their HMD camera on without their knowledge, overlay unwanted 2D images in their field of vision, and modify VR environmental factors that forced users into hitting physical objects and walls. They coined a proof of concept attack the "human joystick" where the user was deceived into moving to a target physical location without their knowledge. ...
Virtual Reality (VR) is expected to become an enabling technology for training in realistic conditions, data visualisation, education and many other applications. However, there is
still limited research on cyber threats to VR environments and even less on technical protections against them. We are currently developing a VR testbed specifically designed for
assessing different cyber threats, their impact to user experience and corresponding defences. In this work in progress, we demonstrate two novel approaches by which a cyber attack can potentially cause VR sickness on demand based on frame rate manipulation by taking advantage of GPU and network vulnerabilities. We further show that a simple unsupervised machine learning method using Isolation Forest can provide early warning of such attacks likely before they have significant impact on the VR system and its user.
... However, prior works lack in the knowledge to address both performance and security issues that can impact the user experience and user safety in VRLE sessions. Failure to address such impediments can lead to deface attacks on the VR content with offensive images [9] that can hamper user experience. They can also lead to application latency issues that degrade performance. ...
Social virtual reality learning environments (VRLEs) provide immersive experience to users with increased accessibility to remote learning. Lack of maintaining high-performance and secured data delivery in critical VRLE application domains (e.g., military training, manufacturing) can disrupt application functionality and induce cybersickness. In this paper, we present a novel rule-based 3QS-adaptation framework that performs risk and cost aware trade-off analysis to control cybersickness due to performance/security anomaly events during a VRLE session. Our framework implementation in a social VRLE viz., vSocial monitors performance/security anomaly events in network/session data. In the event of an anomaly, the framework features rule-based adaptations that are triggered by using various decision metrics. Based on our experimental results, we demonstrate the effectiveness of our rule-based 3QS-adaptation framework in reducing cybersickness levels, while maintaining application functionality. Using our key findings, we enlist suitable practices for addressing performance and security issues towards a more high-performing and robust social VRLE.
... Recently, security researchers started to explore the potential for immersive VR attacks. Casey et al. [11], presented a software vulnerability and were able to manipulate the visuals of the safety guardians of an HTC VIVE. Using this, the authors identified what they called the "Human Joystick Attack", which allows directing an immersed user's physical movement to a location without the user's knowledge. ...
"Virtual-Physical Perceptual Manipulations" (VPPMs) such as redirected walking and haptics expand the user's capacity to interact with Virtual Reality (VR) beyond what would ordinarily physically be possible. VPPMs leverage knowledge of the limits of human perception to effect changes in the user's physical movements, becoming able to (perceptibly and imperceptibly) nudge their physical actions to enhance interactivity in VR. We explore the risks posed by the malicious use of VPPMs. First, we define, conceptualize and demonstrate the existence of VPPMs. Next, using speculative design workshops, we explore and characterize the threats/risks posed, proposing mitigations and preventative recommendations against the malicious use of VPPMs. Finally, we implement two sample applications to demonstrate how existing VPPMs could be trivially subverted to create the potential for physical harm. This paper aims to raise awareness that the current way we apply and publish VPPMs can lead to malicious exploits of our perceptual vulnerabilities.
... This could be conceivably exacerbated within future extensions of technically already feasible "VR deepfakes" [9][10][11] by the particular aptness of VR to facilitate durable memories [12]. While such issues would already play a role regarding unintentional failure modes elicited by ethically aware actors in AIVR, recent research related to the security and safety of AI [13][14][15][16] and VR [17][18][19][20] respectively emphasizes the need to additionally consider the presence of unethical malicious actors. Thereby, to consider intentional malevolent design in AIVR could offer a worst-case scenario analysis [21] that can shed more light on the extent of potential consequences exhibited by the deployment of AIVR technology, but also by simpler cases in AI and VR separately. ...
In recent years, prevalent global societal issues related to fake news, fakery, misinformation, and disinformation were brought to the fore, leading to the construction of descriptive labels such as “post-truth” to refer to the supposedly new emerging era. Thereby, the (mis-)use of technologies such as AI and VR has been argued to potentially fuel this new loss of “ground-truth”, for instance, via the ethically relevant deepfakes phenomena and the creation of realistic fake worlds, presumably undermining experiential veracity. Indeed, unethical and malicious actors could harness tools at the intersection of AI and VR (AIVR) to craft what we call immersive falsehood, fake immersive reality landscapes deliberately constructed for malicious ends. This short paper analyzes the ethically relevant nature of the background against which such malicious designs in AIVR could exacerbate the intentional proliferation of deceptions and falsities. We offer a reappraisal expounding that while immersive falsehood could manipulate and severely jeopardize the inherently affective constructions of social reality and considerably complicate falsification processes, humans may neither inhabit a post-truth nor a post-falsification age. Finally, we provide incentives for future AIVR safety work, ideally contributing to a future era of technology-augmented critical thinking.
... The VR system usually consists of three elements, such as VR input device, VR application developing software, and output device. VR systems typically need at least one input device for the input device section, such as VR gloves [6], bodysuit [7], Phantom Omni [8], vive controllers [9], keyboard and mouse [10], touchscreen keyboard [11], joysticks [12], and voice recognition [13], and several other [14][15]. Major components of VR system need only one VR development software, such as Unreal, Unity 3D, Delta3D, OpenSimulator platform and many others. ...
Virtual reality (VR) technologies and computer-generated technologies are very useful for many industries, such as medical, education, gaming, mining, engineering, and other sectors. In engineering sector specifically for assembly department, the major problem they are facing was that traditional components of automotive engine are complex and very difficult to assemble using current method which use paper manual. The goal of this project is to create an interactive VR application such that users can quickly integrate and perform the work easily. In this work, the authors designed and developed an immersive VR application with several stages, such as 3D model creation, visual effects creation and gaming scene building. The application was built using Unreal and HTC vive headset. As this is an on-going project work, further functionality will be added to render the application immensely for users and assist the sense of pedagogy in terms of the digital method of learning.
Loosely coupled computing systems is an emerging class of parallel computing systems. They are capable of solving large computationally expensive problems at a relatively low cost. During the computational process one or more computing nodes can be turned off resulting into loss of data. In global optimization problems this loss of data can lead not only to increasing the computation time but also to decreasing the solution quality. This paper presents a new problem decomposition method for loosely coupled systems that splits the search domain into multiply connected subdomains. Such an approach allows minimizing the negative impact of node termination. Results of the comparative experimental investigation with a use of benchmark functions are presented in this paper which demonstrate the increase in solution quality comparing to the traditional decomposition methods.
The visual interfaces are the important part of almost any information system with decision support module. An operator makes his decisions based on the information he/she can get only from the visual interface of the system. The challenge here is that vulnerabilities in such interfaces can lead to the wrong decisions and, what is more important, hacked interface can even harm the operator. Even more challenging the situation becomes because of the transition from regular 2D graphical interfaces to the interfaces in the virtual and augmented reality, that are much less investigated.
The global pandemic of COVID-19 has turned the spotlight on video conferencing applications like never before. In this critical time, applications such as Zoom have experienced a surge in its user base jump over the 300 million daily mark (ZoomBlog, 2020). The increase in use has led malicious actors to exploit the application, and in many cases perform Zoom Bombings. Therefore forensically examining Zoom is inevitable. Our work details the primary disk, network, and memory forensic analysis of the Zoom video conferencing application. Results demonstrate it is possible to find users' critical information in plain text and/or encrypted/encoded, such as chat messages, names, email addresses, passwords, and much more through network captures, forensic imaging of digital devices, and memory forensics. Furthermore we elaborate on interesting anti-forensics techniques employed by the Zoom application when contacts are deleted from the Zoom application's contact list.
The global pandemic of COVID-19 has turned the spotlight on video conferencing applications like never before. In this critical time, applications such as Zoom have experienced a surge in its user base jump over the 300 million daily mark (ZoomBlog, 2020). The increase in use has led malicious actors to exploit the application, and in many cases perform Zoom Bombings. Therefore forensically examining Zoom is inevitable. Our work details the primary disk, network, and memory forensic analysis of the Zoom video conferencing application. Results demonstrate it is possible to find users' critical information in plain text and/or encrypted/encoded, such as chat messages, names, email addresses, passwords, and much more through network captures, forensic imaging of digital devices, and memory forensics. Furthermore we elaborate on interesting anti-forensics techniques employed by the Zoom application when contacts are deleted from the Zoom application's contact list.
Virtual Learning Environments (VLEs) are spaces designed to educate student groups remotely via online platforms. Although traditional VLEs have shown promise in educating students, they offer limited immersion that overall diminishes learning effectiveness. In this paper, we describe vSocial, a cloud-based virtual reality learning environment (VRLE) system that can be deployed over high-speed networks using the High Fidelity “social VR” platform. vSocial provides flexible control of group learning content and compliance with established VLE standards with improved immersive user experience for both instructor(s) and students. For our vSocial development, we build upon the use case of an existing special education VLE viz., iSocial that trains youth with Autism Spectrum Disorder by implementing the Social Competence Intervention (SCI) curriculum. The vSocial can be used to: (a) implement multiple learning modules using wearable VR technologies, (b) integrate cognitive state sensing devices, and (c) organize learning session data securely using web applications hosted on cloud resources. Our experiment results show that the VR mode of content delivery in vSocial better stimulates the generalization of lessons to the real world than non-VR lessons, and provides improved immersion when compared to an equivalent desktop version. Further, usability study results show that users can successfully use the web application features in vSocial for group learning activities with ease-of-use and consistency.
Mixed reality (MR) technology is now gaining ground due to advances in computer vision, sensor fusion, and realistic display technologies. With most of the research and development focused on delivering the promise of MR, there is only barely a few working on the privacy and security implications of this technology. This survey paper aims to put in to light these risks, and to look into the latest security and privacy work on MR. Specifically, we list and review the different protection approaches that have been proposed to ensure user and data security and privacy in MR. We extend the scope to include work on related technologies such as augmented reality (AR), virtual reality (VR), and human-computer interaction (HCI) as crucial components, if not the origins, of MR, as well as a number of work from the larger area of mobile devices, wearables, and Internet-of-Things (IoT). We highlight the lack of investigation, implementation, and evaluation of data protection approaches in MR. Further challenges and directions on MR security and privacy are also discussed.
Privacy mechanisms are important in mixed-presence (collocated and remote) collaborative systems. These systems try to achieve a sense of co-presence in order to promote fluid collaboration, yet it can be unclear how actions made in one location are manifested in the other. This ambiguity makes it difficult to share sensitive information with confidence, impacting the fluidity of the shared experience. In this paper, we focus on mixed reality approaches (blending physical and virtual spaces) for mixed presence collaboration. We present SecSpace, our software toolkit for usable privacy and security research in mixed reality collaborative environments. SecSpace permits privacy-related actions in either physical or virtual space to generate effects simultaneously in both spaces. These effects will be the same in terms of their impact on privacy but they may be functionally tailored to suit the requirements of each space. We detail the architecture of SecSpace and present three prototypes that illustrate the flexibility and capabilities of our approach. Author Keywords Usable privacy and security, mixed reality, mixed presence, software toolkit, smart room, framework
The visual interfaces of virtual environments such as video games often show scenes where objects are superimposed on a moving background. Three experiments were designed to better understand the impact of the complexity and/or overall motion of two types of visual backgrounds often used in video games on the detection and use of superimposed, stationary items. The impact of background complexity and motion was assessed during two typical video game tasks: a relatively complex visual search task and a classic, less demanding shooting task. Background motion impaired participants' performance only when they performed the shooting game task, and only when the simplest of the two backgrounds was used. In contrast, and independently of background motion, performance on both tasks was impaired when the complexity of the background increased. Eye movement recordings demonstrated that most of the findings reflected the impact of low-level features of the two backgrounds on gaze control.
Several Virtual Reality (VR) applications for the understanding, assessment and treatment of mental health problems have been developed in the last 15 years. Typically, in VR the patient learns to manipulate problematic situations related to his/her problem. In fact, VR can be described as an advanced form of human–computer interface that is able to induce a feeling of ‘presence’ in the computer-generated world experienced by the user. This feature transforms VR in an ‘empowering environment’, a sheltered setting where patients can start to explore and act without feeling threatened. With such assurance, they can freely explore, experiment, feel, live out and experience feelings and/or thoughts. The paper presents the current state of clinical research in this area. Furthermore, the open source ‘NeuroVR’ system and its potential clinical applications are presented and discussed.
As defenders, it is extremely dangerous to be ignorant of how attackers can disrupt our systems. Without a good understanding of the relative ease of certain attacks, it's easy to adopt poor policies and procedures. A good example of this is the tendency for some organizations to use invalid or "self-signed" certifications for SSL, an approach that both trains the user to ignore certificate warnings displayed by the browser and leaves connections vulnerable to man in the middle attacks. In this article, we illustrate how easy such attacks are to execute; we hope this will serve as an incentive to adopt defenses that not only seem secure, but actually are!
Although many virtual environment (VE) technologies such as the four-screen CAVE™ are described as immersive, users can still perceive distractions from the real world. This exposure to real- world distraction may reduce users' sense of presence, and if presence is correlated with performance as some have claimed, the real-world distractions may also hinder performance. Thus, VE designers may want to consider ways to reduce real-world distraction. This paper presents an experiment to investigate the effect of reduced visual stimulus in the peripheral area on user performance and the usability of an immersive VE. We carefully designed three tasks that cause different levels of awareness of the real-world distraction. Using these tasks, we evaluated users' performance and preference in two conditions. The low-stimulus condition was created by hanging a black cloth across the missing back wall of a CAVE. The high-stimulus condition was created by projected animations and real human motion outside the CAVE. The experiments show that reduced distraction may have a positive or negative effect on user performance, depending on the specific tasks and environments.
Permission-based security models provide controlled access to various system resources. The expressiveness of the permission set plays an important role in providing the right level of granularity in access control. In this work, we present a methodology for the empirical analysis of permission-based security models which makes novel use of the Self-Organizing Map (SOM) algorithm of Kohonen (2001). While the proposed methodology may be applicable to a wide range of architectures, we analyze 1,100 Android applications as a case study. Our methodology is of independent interest for visualization of permission-based systems beyond our present Android-specific empirical analysis. We offer some discussion identifying potential points of improvement for the Android permission model attempting to increase expressiveness where needed without increasing the total number of permissions or overall complexity.
Immersive virtual environments can break the deep, everyday connection between where our senses tell us we are and where we are actually located and whom we are with. The concept of 'presence' refers to the phenomenon of behaving and feeling as if we are in the virtual world created by computer displays. In this article, we argue that presence is worthy of study by neuroscientists, and that it might aid the study of perception and consciousness.
A common measure of the quality or effectiveness of a virtual environment (VE) is the amount of presence it evokes in users. Presence is often defined as the sense of being there in a VE. There has been much debate about the best way to measure presence, and presence researchers need, and have sought, a measure that is reliable, valid, sensitive, and objective. We hypothesized that to the degree that a VE seems real, it would evoke physiological responses similar to those evoked by the corresponding real environment, and that greater presence would evoke a greater response. To examine this, we conducted three experiments, the results of which support the use of physiological reaction as a reliable, valid, sensitive, and objective presence measure. The experiments compared participants' physiological reactions to a non-threatening virtual room and their reactions to a stressful virtual height situation. We found that change in heart rate satisfied our requirements for a measure of presence, change in skin conductance did to a lesser extent, and that change in skin temperature did not. Moreover, the results showed that inclusion of a passive haptic element in the VE significantly increased presence and that for presence evoked: 30FPS > 20FPS > 15FPS.
Augmented reality (AR) technologies, such as those in head-mounted displays like Microsoft HoloLens or in automotive windshields, are poised to change how people interact with their devices and the physical world. Though researchers have begun considering the security, privacy, and safety issues raised by these technologies, to date such efforts have focused on input, i.e., how to limit the amount of private information to which AR applications receive access. In this work, we focus on the challenge of output management: how can an AR operating system allow multiple concurrently running applications to safely augment the user's view of the world? That is, how can the OS prevent apps from (for example) interfering with content displayed by other apps or the user's perception of critical real-world context, while still allowing them sufficient flexibility to implement rich, immersive AR scenarios? We explore the design space for the management of visual AR output, propose a design that balances OS control with application flexibility, and lay out the research directions raised and enabled by this proposal.
Augmented reality (AR) browsers are an emerging category of mobile applications that add interactive virtual objects to the user's view of the physical world. This paper gives the first system-level evaluation of their security and privacy properties.
We start by analyzing the functional requirements that AR browsers must support in order to present AR content. We then investigate the security architecture of Junaio, Layar, and Wikitude browsers, which are running today on over 30 million mobile devices, and identify new categories of security and privacy vulnerabilities unique to AR browsers. Finally, we provide the first engineering guidelines for securely implementing AR functionality.
Augmented reality is poised to become a dominant computing paradigm over the next decade. With promises of three-dimensional graphics and interactive interfaces, augmented reality experiences will rival the very best science fiction novels. This breakthrough also brings in unique challenges on how users can authenticate one another to share rich content between augmented reality headsets. Traditional authentication protocols fall short when there is no common central entity or when access to the central authentication server is not available or desirable. Looks Good To Me (LGTM) is an authentication protocol that leverages the unique hardware and context provided with augmented reality headsets to bring innate human trust mechanisms into the digital world to solve authentication in a usable and secure way. LGTM works over point to point wireless communication so users can authenticate one another in a variety of circumstances and is designed with usability at its core, requiring users to perform only two actions: one to initiate and one to confirm. Users intuitively authenticate one another, using seemingly only each other's faces, but under the hood LGTM uses a combination of facial recognition and wireless localization to bootstrap trust from a wireless signal, to a location, to a face, for secure and usable authentication.
" The growth of mobile and wearable technologies has made it often difficult to understand what people in our surroundings are doing with their technology. In this paper, we introduce the concept of candid interaction: techniques for providing awareness about our mobile and wearable device usage to others in the vicinity. We motivate and ground this exploration through a survey on current attitudes toward device usage during interpersonal encounters. We then explore a design space for candid interaction through seven prototypes that leverage a wide range of technological enhancements, such as Augmented Reality, shape memory muscle wire, and wearable projection. Preliminary user feedback of our prototypes highlights the trade-offs between the benefits of sharing device activity and the need to protect user privacy.
As new technologies emerge such as wearables, it opens up for new challenges, especially related to security and privacy. One such recent technology is smart glasses. The use of glasses introduces security and privacy concerns for the general public but also for the user itself. In this paper we present work which focus on privacy of the user during authentication. We propose and analyze two methods, visual cryptography and obfuscation for protecting the user against HUD and camera logging adversaries as well as shoulder-surfing.
AR systems pose potential security concerns that should be addressed before the systems become widespread.
This is the first case report to demonstrate the efficacy of immersive computer-generated virtual reality (VR) and mixed reality (touching real objects which patients also saw in VR) for the treatment of spider phobia. The subject was a 37-yr-old female with severe and incapacitating fear of spiders. Twelve weekly 1-hr sessions were conducted over a 3-month period. Outcome was assessed on measures of anxiety, avoidance, and changes in behavior toward real spiders. VR graded exposure therapy was successful for reducing fear of spiders providing converging evidence for a growing literature showing the effectiveness of VR as a new medium for exposure therapy.
We introduce a novel class of intrusion: the hidden process, a type of intrusion that will not be detected by an intrusion detection system operating under the assumption that the underlying computing architecture is functioning as specified. A hidden process executes in a manner that is unobservable by many of the operating system's accounting and reporting functions. We present a mechanism to hide processes. Additionally, we show how a hidden process may communicate with an external entity by piggybacking onto a legitimate network connection. We have implemented a mechanism that detects hidden processes and make recommendations calling for the separation of critical operating system functions from more general operating system functions.
Profiles in innovation, virtual and augmented reality, understanding the race for the next computing platform
H. B. et al, "Profiles in innovation, virtual and augmented reality,
understanding the race for the next computing platform," Equity
Research, January 13 2016.
Virtual reality for consumer markets
- C W Mark Beccue
C. W. Mark Beccue, "Virtual reality for consumer markets," Tractica, vol. 4Q, 2016.
Worldwide spending on augmented and virtual reality expected to double or more every year through 2021, according to idc
- M T Shirer
M. T. Michael Shirer. (2017, August 07) Worldwide spending
on augmented and virtual reality expected to double or more
every year through 2021, according to idc. [Online]. Available:
https://www.idc.com/getdoc.jsp?containerId=prUS42959717
Super data report: 6.3 million virtual reality headsets shipped in 2016
- J Durbin
J.
Durbin.
(2017,
January
17)
Super
data
report:
6.3
million
virtual
reality
headsets
shipped
in
2016.
[Online].
Net of insecurity, a flaw in the design
- D C Craig Timberg
D. C. Craig Timberg, "Net of insecurity, a flaw in the design," May
30 2015.
Virtual reality headsets could put childrens health at risk
- F M Mckie
F.
M.
Robin
Mckie,
"Virtual
reality
headsets
could
put
childrens
health
at
risk,"
https://
www.theguardian.com/technology/2017/oct/28/
virtual-reality-headset-children-cognitive-problems, last accessed
2017-11-02.
How oculus rift works: Everything you need to know about the vr sensation
- D Nield
D. Nield, "How oculus rift works: Everything you need to know
about the vr sensation," March 29 2016, HowOculusRiftworks:
EverythingyouneedtoknowabouttheVRsensation,
last-accessed
2017-11-03.
Alan yates: Why valves lighthouse can't work
- E Williams
E. Williams. (2016, December 21) Alan yates: Why valves
lighthouse can't work. [Online]. Available: https://hackaday.
com/2016/12/21/alan-yates-why-valves-lighthouse-cant-work/
Steamvr will support microsoft vr headsets
- Road
- Vr
Road
to
VR,
"Steamvr
will
support
microsoft
vr
headsets,"
https://www.roadtovr.com/
windows-vr-headsets-mixed-reality-support-steamvr/,
last
accessed 2017-10-26.
Driver documentation
- V. Joe Ludwig
V. Joe Ludwig, "Driver documentation," https://github.
com/ValveSoftware/openvr/wiki/Driver-Documentation, last
accessed 2017-10-31.
Oculus quest hands-on and tech details
- B Lang
B. Lang, "Oculus quest hands-on and tech details," 2018,
https://www.roadtovr.com/oculus-quest-hands-specs-techdetails-oculus-connect-5/.
Enabling fine-grained permissions for augmented reality applications with recognizers
- S Jana
- D Molnar
- A Moshchuk
- A M Dunn
- B Livshits
- H J Wang
- E Ofek
S. Jana, D. Molnar, A. Moshchuk, A. M. Dunn, B. Livshits, H. J.
Wang, and E. Ofek, "Enabling fine-grained permissions for augmented reality applications with recognizers." in USENIX Security
Symposium, 2013, pp. 415-430.
Seamless and secure vr: Adapting and evaluating established authentication systems for virtual reality
- C Goerge
- M Khamis
- E Zezschwitz
- M Burger
- H Schmidt
- F Alt
- H Hussmann
C. Goerge, M. Khamis, E. von Zezschwitz, M. Burger, H. Schmidt,
F. Alt, and H. Hussmann, "Seamless and secure vr: Adapting and
evaluating established authentication systems for virtual reality,"
in Proceedings of the Network and Distributed System Security Symposium (USEC17). NDSS. DOI: http://dx. doi. org/10.14722/usec, 2017.
Usability of augmented reality for revealing secret messages to users but not their devices
- S J Andrabi
- M K Reiter
- C Sturton
S. J. Andrabi, M. K. Reiter, and C. Sturton, "Usability of augmented
reality for revealing secret messages to users but not their devices." in SOUPS, vol. 2015, 2015, pp. 89-102.
Teardown of htc vive highlights the headset's differences from oculus rift
- D Coldewey
D. Coldewey. (2016, Apr 26) Teardown of htc vive highlights
the headset's differences from oculus rift. Last updated -2016-04-27. [Online]. Available: https://techcrunch.com/2016/04/26/
teardown-of-htc-vive-highlights-the-headsets-differences-from-oculus-rift/