Gun A. Lee

University of South Australia | UniSA · IVE STEM

The present pilot study explored the research aim of understanding how independent-living older adults experience social isolation and loneliness and whether virtual tour digital technology can increase social connectedness (N = 10). Through triangulation of interviews, experiences, and feedback, this study contributes to the knowledge base on the...

In this paper, we explore how virtual replicas can enhance Mixed Reality (MR) remote collaboration with a 3D reconstruction of the task space. People in different locations may need to work together remotely on complicated tasks. For example, a local user could follow a remote expert's instructions to complete a physical task. However, it could be...

In this paper, we present a novel interface named GazeHand which is designed for distant object interaction in virtual reality. The GazeHand interface translates the virtual hands near to a distant object that the user is looking at and allows direct hand gesture interaction for manipulating an object. Either eye-gaze or head-gaze can be applied to...

In a remote collaboration involving a physical task, visualising gaze behaviours may compensate for other unavailable communication channels. In this paper, we report on a 360° panoramic Mixed Reality (MR) remote collaboration system that shares gaze behaviour visualisations between a local user in Augmented Reality and a remote collaborator in Vir...

Over the last several years, remote collaboration has been getting more attention in the research community because of the COVID-19 pandemic. In previous studies, researchers have investigated the effect of adding visual communication cues or shared views in collaboration, but there has not been any previous study exploring the influence between th...

Immersive virtual reality (VR)–based training has been widely proposed in different firms to improve the hazard recognition skills of their workforce and change their unsafe behavior. However, little is known about the impact of VR-based training on the user’s behavior and learning. With the use of structural equation modeling (SEM), this study inv...

Color selection and surface painting has been largely unexplored in head-worn Augmented Reality (AR) using hand gestures. In this demonstration we present 3DColAR: A system that implements several 2D and 3D techniques for color selection. We also implement two key approaches for painting a virtual 3D model using midair hand gestures. This includes...

This paper describes the user experience (UX) design for a smart-mirror-based personalized training system which aims to help people live a healthy life. A number of researchers and companies have developed fitness systems that use a virtual coach which shows the user with actions they should perform. However such systems can be difficult to accura...

Gaze is one of the predominant communication cues and can provide valuable implicit information such as intention or focus when performing collaborative tasks. However, little research has been done on how virtual gaze cues combining spatial and temporal characteristics impact real-life physical tasks during face to face collaboration. In this stud...

Researchers have employed hyperscanning, a technique used to simultaneously record neural activity from multiple participants, in real-world collaborations. However, to the best of our knowledge, there is no study that has used hyperscanning in Virtual Reality (VR). The aims of this study were; firstly, to replicate results of inter-brain synchrony...

In collaboration systems, collaborators often use hand gestures for showing information relating to a distant object. However, in viewer’s perspective, it is sometimes difficult to know where the conventional gesture cue (Hands-in-Air style: HiA) refers to because the conventional HiA gesture appears away from the distant object. In this paper, we...

Hyperscanning is a technique which simultaneously records the neural activity of two or more people. This is done using one of several neuroimaging methods, such as electroencephalography (EEG), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The use of hyperscanning has seen a dramatic rise in recen...

Optical-reflection type Augmented Reality (AR) mirror displays use half-silvered mirrors attached in front of a digital display to show virtual objects overlaid into the physical world reflected in the mirror. Prior works mostly displayed 2D images on the surface of the mirror hence suffered from visual depth mismatch between the optical reflection...

Remote collaboration using mixed reality (MR) enables two separated workers to collaborate by sharing visual cues. A local worker can share his/her environment to the remote worker for a better contextual understanding. However, prior techniques were using either 360 video sharing or a complicated 3D reconstruction configuration. This limits the in...

Many researchers have studied various visual communication cues (e.g. pointer, sketch, and hand gesture) in Mixed Reality remote collaboration systems for real-world tasks. However, the effect of combining them has not been so well explored. We studied the effect of these cues in four combinations: hand only, hand + pointer, hand + sketch, and hand...

Augmented reality (AR) games have provided positive, fun, and enjoyable user experiences in various ways. While these experiences are mostly oriented to single users, AR technology naturally supports face-to-face interaction, so it is valuable to take into account shared experience in sketching game scenarios, designing user interfaces and interact...

In this paper, we propose a novel concept of the gaze window, which follows the user's gaze to show relevant information close to their gaze point on a screen space. We explore the use of a gaze window, especially for two use cases: (1) a single‐user data entry task by displaying data‐source image close to the gaze point where the data are typed in...

Mixed Reality (MR) remote guidance has become a practical solution for collaboration that includes nonverbal communication. This research focuses on integrating different types of MR remote collaboration systems together allowing a new variety for remote collaboration to extend its features and user experience. In this demonstration, we present 360...

Research has shown that Mixed Presence (MP) systems are a valuable collaboration tool. However current research into MP systems is limited to a handful of tabletop and Virtual Reality (VR) systems with no exploration of Head-Mounted Display (HMD) based Mixed Reality (MR) solutions. In this paper, we present a prototype HMD based MR Mixed Presence s...

In this paper, we investigate the social aspects and effects of various visual cues in a 360 panorama-based Mixed Reality remote collaboration system. We conducted a series of user studies using a prototype system to compare the effects of different combinations of visual cues including hand gestures, ray pointing and drawing in a remote collaborat...

Augmented Reality (AR) mirrors can show virtual objects overlaid onto the physical world reflected in the mirror. Optical-reflection type AR mirror displays use half-silvered mirrors attached in front of a digital display. However, prior work suffered from visual depth mismatch between the optical reflection of the 3D physical space and 2D images d...

Mixed Reality (MR) remote collaboration provides an enhanced immersive experience where a remote user can provide verbal and nonverbal assistance to a local user to increase the efficiency and performance of the collaboration. This is usually achieved by sharing the local user's environment through live 360 video or a 3D scene, and using visual cue...

Collaborative Mixed Reality (MR) systems are at a critical point in time as they are soon to become more commonplace. However, MR technology has only recently matured to the point where researchers can focus deeply on the nuances of supporting collaboration, rather than needing to focus on creating the enabling technology. In parallel, but largely...

Remote Collaboration using Virtual Reality (VR) and Augmented Reality (AR) has recently become a popular way for people from different places to work together. Local workers can collaborate with remote helpers by sharing 360-degree live video or 3D virtual reconstruction of their surroundings. However, each of these techniques has benefits and draw...

Many researchers have studied various visual communication cues (e.g. pointer, sketching, and hand gesture) in Mixed Reality remote collaboration systems for real-world tasks. However, the effect of combining them has not been so well explored. We studied the effect of these cues in four combinations: hand only, hand + pointer, hand + sketch, and h...

We propose a multi-scale Mixed Reality (MR) collaboration between the Giant, a local Augmented Reality user, and the Miniature, a remote Virtual Reality user, in Giant-Miniature Collaboration (GMC). The Miniature is immersed in a 360-video shared by the Giant who can physically manipulate the Miniature through a tangible interface, a combined 360-c...

This paper investigates the effect of avatar appearance on Social Presence and users’ perception in an Augmented Reality (AR) telepresence system. Despite the development of various commercial 3D telepresence systems, there has been little evaluation and discussions about the appearance of the collaborator’s avatars. We conducted two user studies c...

Research has shown that Mixed Presence Groupware (MPG) systems are a valuable collaboration tool. However research into MPG systems is limited to a handful of tabletop and Virtual Reality (VR) systems with no exploration of Head-Mounted Display (HMD) based Augmented Reality (AR) solutions. We present a new system with two local users and one remote...

Augmented and Virtual Reality provide unique capabilities for Mixed Reality collaboration. This paper explores how different combinations of virtual awareness cues can provide users with valuable information about their collaborator's attention and actions. In a user study (n = 32, 16 pairs), we compared different combinations of three cues: Field-...

This paper describes two user studies in remote collaboration between two users with a video conferencing system where a remote user can draw annotations on the live video of the local user’s workspace. In these two studies, the local user had the control of the view when sharing the first-person view, but our interfaces provided instant control of...

In this paper, we investigate hand gestures and visual annotation cues overlaid in a live 360 panorama-based Mixed Reality remote collaboration. The prototype system captures 360 live panorama video of the surroundings of a local user and shares it with another person in a remote location. The two users wearing Augmented Reality or Virtual Reality...

Recent development of affordable head-mounted displays (HMDs) has led to accessible Virtual Reality (VR) solutions for watching 360-degree panoramic movies. While conventionally users cannot see their body while watching 360 movies, our prior work seamlessly blended a user's physical body into a 360 virtual movie scene. This paper extends this work...

This paper investigates how different collaboration styles and view independence affect remote collaboration. Our remote collaboration system shares a live video of a local user’s real-world task space with a remote user. The remote user can have an independent view or a dependent view of a shared real-world object manipulation task and can draw vi...

We present preliminary findings from sharing and augmenting facial expression in cooperative social Virtual Reality (VR) games. We implemented a prototype system for capturing and sharing facial expression between VR players through their avatar. We describe our current prototype system and how it could be assimilated into a system for enhancing so...

For our demonstration, we present two cooperative social Virtual Reality (VR) games that support sharing and augmenting facial expression between a VR player and a laptop player. The two games, "Bomb Defusal" and "Island Survivor", provide different and unique experiences that take advantages of sharing the player's facial expression. Our system ca...

Head-mounted displays are becoming increasingly popular as home entertainment devices for viewing 360° movies. This paper explores the effects of adding gesture interaction with virtual content and two different hand-visualisation modes for 360° movie watching experience. The system in the study comprises of a Leap Motion sensor to track the user's...

The advancements in Mixed Reality (MR), Unmanned Aerial Vehicle, and multi-scale collaborative virtual environments have led to new interface opportunities for remote collaboration. This paper explores a novel concept of flying telepresence for multi-scale mixed reality remote collaboration. This work could enable remote collaboration at a larger s...

Head and eye movement can be leveraged to improve the user's interaction repertoire for wearable displays. Head movements are deliberate and accurate, and provide the current state-of-the-art pointing technique. Eye gaze can potentially be faster and more ergonomic, but suffers from low accuracy due to calibration errors and drift of wearable eye-t...

We present Mini-Me, an adaptive avatar for enhancing Mixed Reality (MR) remote collaboration between a local Augmented Reality (AR) user and a remote Virtual Reality (VR) user. The Mini-Me avatar represents the VR user's gaze direction and body gestures while it transforms in size and orientation to stay within the AR user's field of view. A user s...

We describe a method and a prototype implementation for filtering shared social data (e.g., 360 video) in a wearable Augmented Reality (e.g., HoloLens) application. The data filtering is based on user-viewer relationships. For example, when sharing a 360 video, if the user has an intimate relationship with the viewer, then full fidelity (i.e. the 3...

We present Snow Dome, a Mixed Reality (MR) remote collaboration application that supports a multi-scale interaction for a Virtual Reality (VR) user. We share a local Augmented Reality (AR) user's reconstructed space with a remote VR user who has an ability to scale themselves up into a giant or down into a miniature for different perspectives and i...

We present CoVAR, a novel remote collaborative system combining Augmented Reality (AR), Virtual Reality (VR) and natural communication cues to create new types of collaboration. AR user can capture and share their local environment with a remote user in VR to collaborate on spatial tasks in shared space. COVAR supports various interaction methods t...

Sharing live 360 panorama video is becoming popular on social networking platforms. This research focuses on further improving shared live panorama based collaborative experiences by applying Mixed Reality (MR) technology. In this demonstration we present SharedSphere, an MR remote collaboration system which not only allows sharing of a live captur...

One of the key problems with representing social networks in Augmented Reality (AR) is how to differentiate between contacts. In this paper we explore how visual and spatial cues based on social relationships can be used to represent contacts in social AR applications, making it easier to distinguish between them. Previous implementations of social...