No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
XSpace: An Augmented Reality Toolkit for Enabling Spatially-Aware Distributed Collaboration
Abstract
Augmented Reality (AR) has the potential to leverage environmental information to better facilitate distributed collaboration, however, such applications are difficult to develop. We present XSpace, a toolkit for creating spatially-aware AR applications for distributed collaboration. Based on a review of existing applications and developer tools, we design XSpace to support three methods for creating shared virtual spaces, each emphasizing a different aspect: shared objects, user perspectives, and environmental meshes. XSpace implements these methods in a developer toolkit, and also provides a set of complimentary visual authoring tools to allow developers to preview a variety of configurations for a shared virtual space. We present five example applications to illustrate that XSpace can support the development of a rich set of collaborative AR experiences that are difficult to produce with current solutions. Through XSpace, we discuss implications for future application design, including user space customization and privacy and safety concerns when sharing users' environments.
... In this work, we leverage these developments to explore creating custom virtual environments for VR telepresence systems. Prior research demonstrated various benefits of incorporating users' familiar real-world context into virtual environments in remote collaboration scenarios, such as supporting deixis [25,48,51,61]), mutual awareness [29,67,69], and information recall [13,18,35,36]. Motivated by these findings, we explore a generative approach to creating spaces by blending together multiple users' environmental contexts. ...
... Motivated by these findings, we explore a generative approach to creating spaces by blending together multiple users' environmental contexts. This extends the body of work on aligning dissimilar remote spaces for mixed reality collaboration, e.g., via common object anchors [9,25,29,34] or mesh overlays [29,57]. ...
... Motivated by these findings, we explore a generative approach to creating spaces by blending together multiple users' environmental contexts. This extends the body of work on aligning dissimilar remote spaces for mixed reality collaboration, e.g., via common object anchors [9,25,29,34] or mesh overlays [29,57]. ...
There is increased interest in using generative AI to create 3D spaces for Virtual Reality (VR) applications. However, today's models produce artificial environments, falling short of supporting collaborative tasks that benefit from incorporating the user's physical context. To generate environments that support VR telepresence, we introduce SpaceBlender, a novel pipeline that utilizes generative AI techniques to blend users' physical surroundings into unified virtual spaces. This pipeline transforms user-provided 2D images into context-rich 3D environments through an iterative process consisting of depth estimation, mesh alignment, and diffusion-based space completion guided by geometric priors and adaptive text prompts. In a preliminary within-subjects study, where 20 participants performed a collaborative VR affinity diagramming task in pairs, we compared SpaceBlender with a generic virtual environment and a state-of-the-art scene generation framework, evaluating its ability to create virtual spaces suitable for collaboration. Participants appreciated the enhanced familiarity and context provided by SpaceBlender but also noted complexities in the generative environments that could detract from task focus. Drawing on participant feedback, we propose directions for improving the pipeline and discuss the value and design of blended spaces for different scenarios.
... While bi-directional approaches have been presented [51,55] the experience has been limited to only two users, with attention focused on remote interactions. For distributed teams using mixed reality, there are numerous examples demonstrating the potential of grounding communication around a shared environment to increase workspace awareness [12,19,24,63]. However, prior approaches are limited by the collaboration predominately relying on virtual elements, with remote collaborators represented as virtual cartoon avatars, or focus only on virtual interactions inside the shared space. ...
... Similar to our work are the concepts proposed in [12,19,24]. Herskovitz et al. [24] present a toolkit for facilitating distributed collaboration using portals, anchors, and world-in-miniature, however the provided interaction medium is only virtual and no discussion is provided on user presentation. ...
... Similar to our work are the concepts proposed in [12,19,24]. Herskovitz et al. [24] present a toolkit for facilitating distributed collaboration using portals, anchors, and world-in-miniature, however the provided interaction medium is only virtual and no discussion is provided on user presentation. Fink et al. [12] propose a dynamic mapping of interaction displays across incongruent remote spaces, however the collaborative interactions are mediated through traditional displays with users embodying featureless avatars. ...
... Since the check-in phase should give visiting users the ability to relate to the hosting user's current surroundings, the representation of the whole environment in the overview phase should be swapped for a more concentrated representation of the engaged user at this point in the collaboration. Similar to measures seen in AR/VR collaboration, this might happen by enlarging the previous overview representation of the virtual environment such that additional details are visible [47], [56] or by sharing the view of the hosting user with the visiting user to be able to discuss the hosting user's activities from the same point of view [57]. As check-ins require direct communication between the involved users, this would also be the point where a voice connection between users becomes more relevant and user representations might become more detailed to include gestures or even expressions, depending on what is offered by the technology in use. ...
... In addition, context information from the visiting user like their field of view [58], or additional ways of communication, like letting them place hints in the environment of their partner [49], might make the interaction more fluid. Depending on the intended interaction, the communication features can be chosen symmetrically, with both users receiving information about each other's environment, or asymmetrically, setting the focus on one user's surroundings that are to be discussed [56]. At the point where two users feel the need to collaborate at a level of directness that exceeds the interaction in the check-in phase, a visiting user could decide to temporarily travel to the hosting user's location. ...
Collaborative work in social virtual reality often requires an interplay of loosely coupled collaboration from different virtual locations and tightly coupled face-to-face collaboration. Without appropriate system mediation, however, transitioning between these phases requires high navigation and coordination efforts. In this paper, we present an interaction system that allows collaborators in virtual reality to seamlessly switch between different collaboration models known from related work. To this end, we present collaborators with functionalities that let them work on individual sub-tasks in different virtual locations, consult each other using asymmetric interaction patterns while keeping their current location, and temporarily or permanently join each other for face-to-face interaction. We evaluated our methods in a user study with 32 participants working in teams of two. Our quantitative results indicate that delegating the target selection process for a long-distance teleport significantly improves placement accuracy and decreases task load within the team. Our qualitative user feedback shows that our system can be applied to support flexible collaboration. In addition, the proposed interaction sequence received positive evaluations from teams with varying VR experiences.
... To this end, we introduce Blended Whiteboard-an MR system for remote collaboration that combines physical surfaces and virtual avatars into a blended whiteboard workspace, which can be reconfigured to suit users' collaborative needs. Prior related work has either focused on supporting physicality through MR [15,18,63] or workspace reconfigurability through other display types [14,17,30,62]. However, this work focuses on the tension that arises from using MR to combine the two. ...
... These systems enable new flexibility to blend remote spaces, but they are not reconfigurable. More recently, researchers have proposed partially blended spaces that can be dynamically defined [15,18]. The concept of Partially Blended Realities (PBR) by Grønbaek et al. [15] offers a solution, focusing on mapping relevant surfaces for tasks rather than aligning the whole space. ...
The whiteboard is essential for collaborative work. To preserve its physicality in remote collaboration, Mixed Reality (MR) can blend real whiteboards across distributed spaces.
Going beyond reality, MR can further enable interactions like panning and zooming in a virtually reconfigurable infinite whiteboard. However, this reconfigurability conflicts with the sense of physicality. To address this tension, we introduce Blended Whiteboard, a remote collaborative MR system enabling reconfigurable surface blending across distributed physical whiteboards. Blended Whiteboard supports a unique collaboration style, where users can sketch on their local whiteboards but also reconfigure the blended space to facilitate transitions between loosely and tightly coupled work. We describe design principles inspired by proxemics; supporting users in changing between facing each other and being side-by-side, and switching between navigating the whiteboard synchronously and independently. Our work shows exciting benefits and challenges of combining physicality and reconfigurability in the design of distributed MR whiteboards.
... Using 360-degree video capture, VR systems have been able to provide more complete pictures of local environments to remote users inhabiting a fixed perspective [56], moving based on local user control [39], or independently using telepresence robots [23]. With room-scale depth capture, projects such as XSpace [20] and Re-locations [14] enable the creation of blended locations for remote, multi-view collaboration. Systems such as Loki [44] and Holoportation [35] are also capable of streaming environmental depth information. ...
... These interactions have analogues in the space of immersive collaboration. Projects have discussed ways to merge [20,48], duplicate [54], and remix [29] immersive environments. Some of these ideas also emerged from participant reactions to the DualStream prototype. ...
In-person human interaction relies on our spatial perception of each
other and our surroundings. Current remote communication tools
partially address each of these aspects. Video calls convey real user
representations but without spatial interactions. Augmented and
Virtual Reality (AR/VR) experiences are immersive and spatial but
often use virtual environments and characters instead of real-life
representations. Bridging these gaps, we introduce DualStream,
a system for synchronous mobile AR remote communication that
captures, streams, and displays spatial representations of users and
their surroundings. DualStream supports transitions between user
and environment representations with different levels of visuospatial
fidelity, as well as the creation of persistent shared spaces using environment snapshots. We demonstrate how DualStream can enable spatial communication in real-world contexts, and support the creation of blended spaces for collaboration. A formative evaluation of DualStream revealed that users valued the ability to interact spatially and move between representations, and could see DualStream fitting into their own remote communication practices in the near future. Drawing from these findings, we discuss new opportunities for designing more widely accessible spatial communication tools, centered around the mobile phone.
... Using 360-degree video capture, VR systems have been able to provide more complete pictures of local environments to remote users inhabiting a fixed perspective [56], moving based on local user control [39], or independently using telepresence robots [23]. With room-scale depth capture, projects such as XSpace [20] and Re-locations [14] enable the creation of blended locations for remote, multi-view collaboration. Systems such as Loki [44] and Holoportation [35] are also capable of streaming environmental depth information. ...
... These interactions have analogues in the space of immersive collaboration. Projects have discussed ways to merge [20,48], duplicate [54], and remix [29] immersive environments. Some of these ideas also emerged from participant reactions to the DualStream prototype. ...
In-person human interaction relies on our spatial perception of each other and our surroundings. Current remote communication tools partially address each of these aspects. Video calls convey real user representations but without spatial interactions. Augmented and Virtual Reality (AR/VR) experiences are immersive and spatial but often use virtual environments and characters instead of real-life representations. Bridging these gaps, we introduce DualStream, a system for synchronous mobile AR remote communication that captures, streams, and displays spatial representations of users and their surroundings. DualStream supports transitions between user and environment representations with different levels of visuospatial fidelity, as well as the creation of persistent shared spaces using environment snapshots. We demonstrate how DualStream can enable spatial communication in real-world contexts, and support the creation of blended spaces for collaboration. A formative evaluation of DualStream revealed that users valued the ability to interact spatially and move between representations, and could see DualStream fitting into their own remote communication practices in the near future. Drawing from these findings, we discuss new opportunities for designing more widely accessible spatial communication tools, centered around the mobile phone.
... This way, the environments themselves do not need to be aligned, but rather a common anchor is established within each collaborator's environment. Herskovitz et al. provide a toolkit capable of displaying collaborators through a portal, world-in-miniature display, or by anchoring them to a common element of both rooms, such as a chair or table [9]. The idea of anchoring has also been explored in other works [6,7,10]. ...
Augmented Reality (AR) collaboration can benefit from a shared 2D surface, such as a whiteboard. However, many features of each collaborators physical environment must be considered in order to determine the best placement and shape of the shared surface. We explored the effects of three methods for beginning a collaborative whiteboarding session with varying levels of user control: MANUAL, DISCRETE CHOICE, and AUTOMATIC by conducting a simulated AR study within Virtual Reality (VR). In the MANUAL method, users draw their own surfaces directly in the environment until they agree on the placement; in the DISCRETE CHOICE method, the system provides three options for whiteboard size and location; and in the AUTOMATIC method, the system automatically creates a whiteboard that fits within each collaborators environment. We evaluate these three conditions in a study in which two collaborators used each method to begin collaboration sessions. After establishing a session, the users worked together to complete an affinity diagramming task using the shared whiteboard. We found that the majority of participants preferred to have direct control during the initialization of a new collaboration session, despite the additional workload induced by the Manual method.
... Spatial information refers to the geographical or spatiallyrelated data such as the location of certain objects or the occurrence of interactions. Spatial information is usually visualized by 3D models [34,41,48,130], overlaying data [40,54], and visual cues such as arrows and lines [10,64,112]. ...
Context-aware AR instruction enables adaptive and in-situ learning experiences. However, hardware limitations and expertise requirements constrain the creation of such instructions. With recent developments in Generative Artificial Intelligence (Gen-AI), current research tries to tackle these constraints by deploying AI-generated content (AIGC) in AR applications. However, our preliminary study with six AR practitioners revealed that the current AIGC lacks contextual information to adapt to varying application scenarios and is therefore limited in authoring. To utilize the strong generative power of GenAI to ease the authoring of AR instruction while capturing the context, we developed CARING-AI, an AR system to author context-aware humanoid-avatar-based instructions with GenAI. By navigating in the environment, users naturally provide contextual information to generate humanoid-avatar animation as AR instructions that blend in the context spatially and temporally. We showcased three application scenarios of CARING-AI: Asynchronous Instructions, Remote Instructions, and Ad Hoc Instructions based on a design space of AIGC in AR Instructions. With two user studies (N=12), we assessed the system usability of CARING-AI and demonstrated the easiness and effectiveness of authoring with Gen-AI.
... We propose an interoperable solution that enables users to create their personal AR environments that can be discovered in the physical world by other users using AR-enabled devices. Unlike other existing work such as XSpace [9], we aim for a synthetic and semantic interoperable solution that decentralises the collaborative aspect of AR environments while also enabling the sharing of crucial environmental information such as common reference spaces. Our interoperable solution should enable the unambiguous description of the virtual environment in a way that other applications can access and understand. ...
Augmented Reality (AR) environments are physical environments with virtual objects superimposed through AR-enabled devices. These virtual objects can range from simple aesthetic objects like pictures to superimposed contextual information about physical items. In most modern AR applications, the augmented spaces exist only for the user who created the environment or for proprietary applications that enable multi-user collaboration in the same environment. However, there is a lack of solutions that enable interoperable collaboration in these personal AR spaces, allowing users to share and contribute to an AR space. We propose a solution that enables users to create their personal AR space that can then be discovered by other users who are in physical proximity to this space, enabling them to view or contribute to the augmented space. In addition, we discuss a solution that utilises the same technique to create AR spaces that are bound to a specific room and can be discovered by users who are in close vicinity to these rooms.
... HoloLens, Tango devices, RoomAlive setup) into a single shared, co-located 3D space. Hershkovitz et al. later extend this work into a similar toolkit called XSpace [18]. ...
Real-time online interaction is the norm today. Tabletops and other dedicated interactive surface devices with direct input and tangible interaction can enhance remote collaboration, and open up new interaction scenarios based on mixed physical/virtual components. However, they are only available to a small subset of users, as they usually require identical bespoke hardware for every participant, are complex to setup, and need custom scenario-specific applications.
We present SurfaceCast, a software toolkit designed to merge multiple distributed, heterogeneous end-user devices into a single, shared mixed-reality surface. Supported devices include regular desktop and laptop computers, tablets, and mixed-reality headsets, as well as projector-camera setups and dedicated interactive tabletop systems. This device-agnostic approach provides a fundamental building block for exploration of a far wider range of usage scenarios than previously feasible, including future clients using our provided API.
In this paper, we discuss the software architecture of SurfaceCast, present a formative user study and a quantitative performance analysis of our framework, and introduce five example application scenarios which we enhance through the multi-user and multi-device features of the framework. Our results show that the hardware- and content-agnostic architecture of SurfaceCast can run on a wide variety of devices with sufficient performance and fidelity for real-time interaction.
... The registration of virtual spaces to physical spaces poses a significant challenge in VR experiences, especially when the configurations of the two spaces differ significantly. Previous avatar position mapping approaches aimed to connect every position between the two spaces, but this may not be feasible when the spaces are highly dissimilar [7,18,44,53]. Furthermore, avatar retargeting approaches have attempted to correlate user movement from his/her physical space to the target virtual space, but they resulted in unexpected shifts and compromised immersion [54]. ...
We propose a novel space-rescaling technique for registering dissimilar physical-virtual spaces by utilizing the effects of adjusting physical space with redirected walking. Achieving a seamless immersive Virtual Reality (VR) experience requires overcoming the spatial heterogeneities between the physical and virtual spaces and accurately aligning the VR environment with the user's tracked physical space. However, existing space-matching algorithms that rely on one-to-one scale mapping are inadequate when dealing with highly dissimilar physical and virtual spaces, and redirected walking controllers could not utilize basic geometric information from physical space in the virtual space due to coordinate distortion. To address these issues, we apply relative translation gains to partitioned space grids based on the main interactable object's edge, which enables space-adaptive modification effects of physical space without coordinate distortion. Our evaluation results demonstrate the effectiveness of our algorithm in aligning the main object's edge, surface, and wall, as well as securing the largest registered area compared to alternative methods under all conditions. These findings can be used to create an immersive play area for VR content where users can receive passive feedback from the plane and edge in their physical environment.
With the advents in geometry perception and Augmented Reality (AR), end-users can customize Tangible User Interfaces (TUIs) that control digital assets using intuitive and comfortable interactions with physical geometries (e.g., edges and surfaces). However, it remains challenging to adapt such TUIs in varied physical environments while maintaining the same spatial and ergonomic affordance. We propose AdapTUI, an end-to- end system that enables an end-user to author geometric-based TUIs and automatically adapts the TUIs when the user moves to a new environment. Leveraging a geometry detection module and the spatial awareness of AR, AdapTUI first lets users create custom mappings between geometric features and digital functions. Then, AdapTUI uses an optimization-based adaptation framework, which considers both the geometric variations and human-factor nuances, to dynamically adjust the attachment of the user-authored TUIs. We demonstrate three application scenarios where end-users can utilize TUIs at different locations, including portable car play, efficient AR workstation, and entertainment. We evaluated the effectiveness of the adaptation method as well as the overall usability through a comparison user study (N=12). The satisfactory adaptation of the user-authored TUIs and the positive qualitative feedback demonstrate the effectiveness of our system.
Shared Mixed Reality experiences allow two co-located users to collaborate on both physical and digital tasks with familiar social protocols. However, extending the same to remote collaboration is limited by cumbersome setups for aligning distinct physical environments and the lack of access to remote physical artifacts. We present SurfShare, a general-purpose symmetric remote collaboration system with mixed-reality head-mounted displays (HMDs). Our system shares a spatially consistent physical-virtual workspace between two remote users, anchored on a physical plane in each environment (e.g., a desk or wall). The video feed of each user's physical surface is overlaid virtually on the other side, creating a shared view of the physical space. We integrate the physical and virtual workspace through virtual replication. Users can transmute physical objects to the virtual space as virtual replicas. Our system is lightweight, implemented using only the capabilities of the headset, without requiring any modifications to the environment (e.g. cameras or motion tracking hardware). We discuss the design, implementation, and interaction capabilities of our prototype, and demonstrate the utility of SurfShare through four example applications. In a user experiment with a comprehensive prototyping task, we found that SurfShare provides a physical-virtual workspace that supports low-fi prototyping with flexible proxemics and fluid collaboration dynamics.
Programming by Demonstration (PbD) is a well-known technique that allows non-programmers to describe interactivity by performing examples of the expected behavior, but it has not been extensively explored for AR. We present Rapido, a novel early-stage prototyping tool to create fully interactive mobile AR prototypes from non-interactive video prototypes using PbD. In Rapido, designers use a mobile AR device to record a video prototype to capture context, sketch assets, and demonstrate interactions. They can demonstrate touch inputs, animation paths, and rules to, e.g., have a sketch follow the focus area of the device or the user's world-space touches. Simultaneously, a live website visualizes an editable overview of all the demonstrated examples and infers a state machine of the user flow. Our key contribution is a method that enables designers to turn a video prototype into an executable state machine through PbD. The designer switches between these representations to interactively refine the final interactive prototype. We illustrate the power of Rapido's approach by prototyping the main interactions of three popular AR mobile applications.
Real-time depth data is readily available on mobile phones with passive or active sensors and on VR/AR devices. However, this rich data about our environment is under-explored in mainstream AR applications. Slow adoption of depth information in the UX layer may be due to the complexity of processing depth data to simply render a mesh or detect interaction based on changes in the depth map. In this paper, we introduce DepthLab, a software library that encapsulates a variety of UI/UX features for depth, including geometry-aware rendering (occlusion, shadows), depth interactive behaviors (physically based collisions, avatar path planning), and visual effects (relighting, aperture effects). We break down the usage of depth map into point depth, surface depth, and per-pixel depth and introduce our real-time algorithms for different use cases. We present the design process, system, and components of DepthLab to streamline and centralize the development of interactive depth features. We open-sourced our software with external developers and collected both qualitative and quantitative feedback. Our results and feedback from engineers suggest that DepthLab help mobile AR developers unleash their creativity and effortlessly integrate depth in mobile AR experiences, and amplify their prototyping efforts.
We present SpaceState, a system for designing spatial user interfaces that react to changes of the physical layout of a room. SpaceState uses depth cameras to measure the physical environment and allows designers to interactively define global and local states of the room. After designers defined states, SpaceState can identify the current state of the physical environment in real-time. This allows applications to adapt the content to room states and to react to transitions between states. Other scenarios include analysis and optimizations of work flows in physical environments. We demonstrate SpaceState by showcasing various example states and interactions. Lastly, we implemented an example application: A projection mapping based tele-presence application, which projects a remote user in the local physical space according to the current layout of the space.
Today's virtual reality (VR) systems offer chaperone rendering techniques that prevent the user from colliding with physical objects. Without a detailed geometric model of the physical world, these techniques offer limited possibility for more advanced compositing between the real world and the virtual. We explore this using a realtime 3D reconstruction of the real world that can be combined with a virtual environment. RealityCheck allows users to freely move, manipulate, observe, and communicate with people and objects situated in their physical space without losing the sense of immersion or presence inside their virtual world. We demonstrate RealityCheck with seven existing VR titles, and describe compositing approaches that address the potential conflicts when rendering the real world and a virtual environment together. A study with frequent VR users demonstrate the affordances provided by our system and how it can be used to enhance current VR experiences.
360-degree video is increasingly used to create immersive user experiences; however, it is typically limited to a single user and not interactive. Recent studies have explored the potential of 360 video to support multi-user collaboration in remote settings. These studies identified several challenges with respect to 360 live streams, such as the lack of gaze awareness, out-of-sync views, and missed gestures. To address these challenges, we created 360Anywhere, a framework for 360 video-based multi-user collaboration that, in addition to allowing collaborators to view and annotate a 360 live stream, also supports projection of annotations in the 360 stream back into the real-world environment in real-time. This enables a range of collaborative augmented reality applications not supported with existing tools. We present the 360Anywhere framework and tools that allow users to generate applications tailored to specific collaboration and augmentation needs with support for remote collaboration. In a series of exploratory design sessions with users, we assess 360Anywhere's power and flexibility for three mobile ad-hoc scenarios. Using 360Anywhere, participants were able to set up and use fairly complex remote collaboration systems involving projective augmented reality in less than 10 minutes.
Augmented Reality (AR) developers face a proliferation of new platforms, devices, and frameworks. This often leads to applications being limited to a single platform and makes it hard to support collaborative AR scenarios involving multiple different devices. This paper presents XD-AR, a cross-device AR application development framework designed to unify input and output across hand-held, head-worn, and projective AR displays. XD-AR's design was informed by challenging scenarios for AR applications, a technical review of existing AR platforms, and a survey of 30 AR designers, developers, and users. Based on the results, we developed a taxonomy of AR system components and identified key challenges and opportunities in making them work together. We discuss how our taxonomy can guide the design of future AR platforms and applications and how cross-device interaction challenges could be addressed. We illustrate this when using XD-AR to implement two challenging AR applications from the literature in a device-agnostic way.
Virtual reality can help realize mediated social experiences where distance disappears and we interact as richly with those around the world as we do with those in the same room. The design of social virtual experiences presents a challenge for remotely located users with room-scale setups like those afforded by recent commodity virtual reality devices. Since users inhabit different physical spaces that may not be the same size, a mapping to a shared virtual space is needed for creating experiences that allow everyone to use real walking for locomotion. We designed three mapping techniques that enable users from diverse room-scale setups to interact together in virtual reality. Results from our user study (N = 26) show that our mapping techniques positively influence the perceived degree of togetherness and copresence while the size of each user's tracked space influences individual presence.
We present Remixed Reality, a novel form of mixed reality. In contrast to classical mixed reality approaches where users see a direct view or video feed of their environment, with Remixed Reality they see a live 3D reconstruction, gathered from multiple external depth cameras. This approach enables changing the environment as easily as geometry can be changed in virtual reality, while allowing users to view and interact with the actual physical world as they would in augmented reality. We characterize a taxonomy of manipulations that are possible with Remixed Reality: spatial changes such as erasing objects; appearance changes such as changing textures; temporal changes such as pausing time; and viewpoint changes that allow users to see the world from different points without changing their physical location. We contribute a method that uses an underlying voxel grid holding information like visibility and transformations, which is applied to live geometry in real time.
Context-aware pervasive applications can improve user experiences by tracking people in their surroundings. Such systems use multiple sensors to gather information regarding people and devices. However, when developing novel user experiences, researchers are left to building foundation code to support multiple network-connected sensors, a major hurdle to rapidly developing and testing new ideas.
We introduce Creepy Tracker, an open-source toolkit to ease prototyping with multiple commodity depth cameras. It automatically selects the best sensor to follow each person, handling occlusions and maximizing interaction space, while providing full-body tracking in scalable and extensible manners. It also keeps position and orientation of stationary interactive surfaces while offering continuously updated point-cloud user representations combining both depth and color data. Our performance evaluation shows that, although slightly less precise than marker-based optical systems, Creepy Tracker provides reliable multi-joint tracking without any wearable markers or special devices. Furthermore, implemented representative scenarios show that Creepy Tracker is well suited for deploying spatial and context-aware interactive experiences.
HCI research has demonstrated Mixed Reality (MR) as being beneficial for co-located collaborative work. For remote collaboration, however, the collaborators' visual contexts do not coincide due to their individual physical environments. The problem becomes apparent when collaborators refer to physical landmarks in their individual environments to guide each other's attention. In an experimental study with 16 dyads, we investigated how the provisioning of shared virtual landmarks (SVLs) influences communication behavior and user experience. A quantitative analysis revealed that participants used significantly less ambiguous spatial expressions and reported an improved user experience when SVLs were provided. Based on these findings and a qualitative video analysis we provide implications for the design of MRs to facilitate remote collaboration.
We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D ,almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.
Despite technical advances in CSCW over the past few years we still have relatively little understanding of the organisation of collaborative activity in real world, technologically supported, work environments. Indeed, it has been suggested that the failure of various technological applications may derive from its relative insensitivity to ordinary work practice and situated conduct. In this paper we discuss the possibility of utilising recent developments within social science, and in particular the naturalistic analysis of organisational conduct and interpersonal communication, as a basis for the design and development of tools and technologies to support collaborative work. Focussing on the Line Control Rooms on London Underground, a complex multimedia environment in transition, we begin to explicate the informal work practices and procedures whereby personnel systematically communicate information and coordinate a disparate collection of tasks and activities. These empirical investigations form the foundation to the design of new tools to support collaborative work in Line Control Rooms; technologies which will be sensitive to the ordinary conduct and practical skills of organisational personnel in the London Underground.
Shared phy sical workspaces allow people to maintain up - to-the minute knowledge about others' interaction with the workspace. This knowledge is workspace awareness, part of the glue that allows groups to collaborate effectively. In this paper, we present the concept of workspace awareness as a key for groupware systems that wish to support the fluid interaction evident i n face-to-face collaboration. We discuss why workspace awareness is difficult to support in groupware systems, and then present a conceptual framework that groupware designers can u se a s a starting point for thinking about and supporting awareness .
KinectFusion enables a user holding and moving a standard Kinect camera to rapidly create detailed 3D reconstructions of an indoor scene. Only the depth data from Kinect is used to track the 3D pose of the sensor and reconstruct, geometrically precise, 3D models of the physical scene in real-time. The capabilities of KinectFusion, as well as the novel GPU-based pipeline are described in full. Uses of the core system for low-cost handheld scanning, and geometry-aware augmented reality and physics-based interactions are shown. Novel extensions to the core GPU pipeline demonstrate object segmentation and user interaction directly in front of the sensor, without degrading camera tracking or reconstruction. These extensions are used to enable real-time multi-touch interactions anywhere, allowing any planar or non-planar reconstructed physical surface to be appropriated for touch.
People naturally understand and use proxemic relationships (e.g., their distance and orientation towards others) in everyday situations. However, only few ubiquitous computing (ubicomp) systems interpret such proxemic relationships to mediate interaction (proxemic interaction). A technical problem is that developers find it challenging and tedious to access proxemic information from sensors. Our Proximity Toolkit solves this problem. It simplifies the exploration of interaction techniques by supplying fine-grained proxemic information between people, portable devices, large interactive surfaces, and other non-digital objects in a room-sized environment. The toolkit offers three key features. 1) It facilitates rapid prototyping of proxemic-aware systems by supplying developers with the orientation, distance, motion, identity, and location information between entities. 2) It includes various tools, such as a visual monitoring tool, that allows developers to visually observe, record and explore proxemic relationships in 3D space. (3) Its flexible architecture separates sensing hardware from the proxemic data model derived from these sensors, which means that a variety of sensing technologies can be substituted or combined to derive proxemic information. We illustrate the versatility of the toolkit with proxemic-aware systems built by students.
In recent years there has been an introduction of sophisticated new video conferencing technologies (e.g., HP Halo, Cisco Telepresence) that have led to enhancements in the collaborative user experience over traditional video conferencing technologies. Traditional video conferencing set-ups often distort the shared spatial properties of action and communication due to screen and camera orientation disparities and other asymmetries. These distortions affect access to the common resources used to mutually organize action and communication. By contrast, new systems, such as Halo, are physically configured to reduce these asymmetries and orientation disparities, thereby minimizing these spatial distortions. By creating appropriate shared spatial geometries, the distributed spaces become “blended” where the spatial geometries of the local space continue coherently across the distributed boundary into the remote site, providing the illusion of a single unified space. Drawing on theories of embodied action and workplace design we discuss the importance of this geometric “blending” of space for distributed collaboration and how this is achieved in systems such as Halo. We then extend these arguments to explore the concept of Blended Interaction Spaces: blended spaces in which interactive groupware is incorporated in ways spatially consistent with the physical geometries of the video-mediated set-up. We illustrate this discussion through a system called BISi that introduces interactive horizontal and vertical multipoint surfaces into a blended video-mediated collaboration space. In presenting this system, we highlight some of the particular challenges of creating these systems arising from the spatial consequences of different interaction mechanisms (e.g., direct touch or remote control) and how they affect movement and spatial configuration of people in these spaces.
This paper surveys the current state-of-the-art of technology, systems and applications in Augmented Reality. It describes work performed by many different research groups, the purpose behind each new Augmented Reality system, and the difficulties and problems encountered when building some Augmented Reality applications. It surveys mobile augmented reality systems challenges and requirements for successful mobile systems. This paper summarizes the current applications of Augmented Reality and speculates on future applications and where current research will lead Augmented Reality’s development. Challenges augmented reality is facing in each of these applications to go from the laboratories to the industry, as well as the future challenges we can forecast are also discussed in this paper. Section 1 gives an introduction to what Augmented Reality is and the motivations for developing this technology. Section 2 discusses Augmented Reality Technologies with computer vision methods, AR devices, interfaces and systems, and visualization tools. The mobile and wireless systems for Augmented Reality are discussed in Section 3. Four classes of current applications that have been explored are described in Section 4. These applications were chosen as they are the most famous type of applications encountered when researching AR apps. The future of augmented reality and the challenges they will be facing are discussed in Section 5.
We propose an approach to creating shared mixed realities based on the construction of transparent boundaries between real and virtual spaces. First, we introduce a taxonomy that classifies current approaches to shared spaces according to the three dimensions of transportation, artificiality, and spatiality. Second, we discuss our experience of staging a poetry performance simultaneously within real and virtual theaters. This demonstrates the complexities involved in establishing social interaction between real and virtual spaces and motivates the development of a systematic approach to mixing realities. Third, we introduce and demonstrate the technique of mixed-reality boundaries as a way of joining real and virtual spaces together in order to address some of these problems.
In this paper we contribute a literature review and organization framework for classifying the collaboration needs and features that should be considered when designing headset-based augmented reality (AR) experiences for collocated settings. In recent years augmented reality technology has been experiencing significant growth through the emergence of headsets that allow gestural interaction, and AR designers are increasingly interested in using this technology to enhance collaborative activities in a variety of physical environments. However, collaborative AR applications need to contain features that enhance collaboration and satisfy needs that are present during the group activities. When AR designers lack an understanding of what collaborators need during an interaction, or what features have already been designed to solve those needs, then AR creators will spend time redesigning features that have already been created, or worse, create applications that do not contain necessary features. While much work has been done on designing virtual reality (VR) collaborative environments, AR environments are a relatively newer design space, and designers are lacking a comprehensive framework for describing needs that arise during collaborative activities and the features that could be designed into AR applications to satisfy those needs. In this paper we contribute a literature review of 92 papers in the areas of augmented reality and virtual reality, and we contribute a list of design features and needs that are helpful to consider when designing for headset-based collaborative AR experiences.
Studies on collaborative virtual environments (CVEs) have suggested capture and later replay of multimodal interactions (e.g., speech, body language, and scene manipulations), which we refer to as multimodal recordings, as an effective medium for time-distributed collaborators to discuss and review 3D content in an immersive, expressive, and asynchronous way. However, there exist gaps of empirical knowledge in understanding how this multimodal asynchronous VR collaboration (MAVRC) context impacts social behaviors in mediated-communication, workspace awareness in cooperative work, and user requirements for authoring and consuming multimedia recording. This study aims to address these gaps by conceptualizing MAVRC as a type of CSCW and by understanding the challenges and design considerations of MAVRC systems. To this end, we conducted an exploratory need-finding study where participants (N = 15) used an experimental MAVRC system to complete a representative spatial task in an asynchronously collaborative setting, involving both consumption and production of multimodal recordings. Qualitative analysis of interview and observation data from the study revealed unique, core design challenges of MAVRC in: (1) coordinating proxemic behaviors between asynchronous collaborators, (2) providing traceability and change awareness across different versions of 3D scenes, (3) accommodating viewpoint control to maintain workspace awareness, and (4) supporting navigation and editing of multimodal recordings. We discuss design implications, ideate on potential design solutions, and conclude the paper with a set of design recommendations for MAVRC systems.
Remotely instructing and guiding users in physical tasks has offered promise across a wide variety of domains. While it has been the subject of many research projects, current approaches are often limited in the communication bandwidth (lacking context, spatial information) or interactivity (unidirectional, asynchronous) between the expert and the learner. Systems that use Mixed-Reality systems for this purpose have rigid configurations for the expert and the learner. We explore the design space of bi-directional mixed-reality telepresence systems for teaching physical tasks, and present Loki, a novel system which explores the various dimensions of this space. Loki leverages video, audio and spatial capture along with mixed-reality presentation methods to allow users to explore and annotate the local and remote environments, and record and review their own performance as well as their peer's. The system design of Loki also enables easy transitions between different configurations within the explored design space. We validate its utility through a varied set of scenarios and a qualitative user study.
We present an optimization-based approach for Mixed Reality (MR) systems to automatically control when and where applications are shown, and how much information they display. Currently, content creators design applications, and users then manually adjust which applications are visible and how much information they show. This choice has to be adjusted every time users switch context, i.e., whenever they switch their task or environment. Since context switches happen many times a day, we believe that MR interfaces require automation to alleviate this problem. We propose a real-time approach to automate this process based on users' current cognitive load and knowledge about their task and environment. Our system adapts which applications are displayed, how much information they show, and where they are placed. We formulate this problem as a mix of rule-based decision making and combinatorial optimization which can be solved efficiently in real-time. We present a set of proof-of-concept applications showing that our approach is applicable in a wide range of scenarios. Finally, we show in a dual-task evaluation that our approach decreased secondary tasks interactions by 36%.
We introduce Blocks, a mobile application that enables people to co-create AR structures that persist in the physical environment. Using Blocks, end users can collaborate synchronously or asynchronously, whether they are colocated or remote. Additionally, the AR structures can be tied to a physical location or can be accessed from anywhere. We evaluated how people used Blocks through a series of lab and field deployment studies with over 160 participants, and explored the interplay between two collaborative dimensions: space and time. We found that participants preferred creating structures synchronously with colocated collaborators. Additionally, they were most active when they created structures that were not restricted by time or place. Unlike most of today's AR experiences, which focus on content consumption, this work outlines new design opportunities for persistent and collaborative AR experiences that empower anyone to collaborate and create AR content.
We explore 360 paper prototyping to rapidly create AR/VR prototypes from paper and bring them to life on AR/VR devices. Our approach is based on a set of emerging paper prototyping templates specifically for AR/VR. These templates resemble the key components of many AR/VR interfaces, including 2D representations of immersive environments, AR marker overlays and face masks, VR controller models and menus, and 2D screens and HUDs. To make prototyping with these templates effective, we developed 360proto, a suite of three novel physical--digital prototyping tools: (1) the 360proto Camera for capturing paper mockups of all components simply by taking a photo with a smartphone and seeing 360-degree panoramic previews on the phone or stereoscopic previews in Google Cardboard; (2) the 360proto Studio for organizing and editing captures, for composing AR/VR interfaces by layering the captures, and for making them interactive with Wizard of Oz via live video streaming; (3) the 360proto App for running and testing the interactive prototypes on AR/VR capable mobile devices and headsets. Through five student design jams with a total of 86 participants and our own design space explorations, we demonstrate that our approach with 360proto is useful to create relatively complex AR/VR applications.
In virtual reality a real walking interface limits the extent of a virtual environment to our local walkable space. As local spaces are specific to each user, sharing a virtual environment with others for collaborative work or games becomes complicated. It is not clear which user's walkable space to prefer, or whether that space will be navigable for both users.
This paper presents a technique which allows users to interact in virtual reality while each has a different walkable space. With this method mappings are created between pairs of environments. Remote users are then placed in the local environment as determined by the corresponding mapping.
A user study was conducted with 38 participants. Pairs of participants were invited to collaborate on a virtual reality puzzle-solving task while in two different virtual rooms. An avatar representing the remote user was mapped into the local user's space. The results suggest that collaborative systems can be based on local representations that are actually quite different.
Toolkit research plays an important role in the field of HCI, as it can heavily influence both the design and implementation of interactive systems. For publication, the HCI community typically expects toolkit research to include an evaluation component. The problem is that toolkit evaluation is challenging, as it is often unclear what 'evaluating' a toolkit means and what methods are appropriate. To address this problem, we analyzed 68 published toolkit papers. From our analysis, we provide an overview of, reflection on, and discussion of evaluation methods for toolkit contributions. We identify and discuss the value of four toolkit evaluation strategies, including the associated techniques that each employs. We offer a categorization of evaluation strategies for toolkit researchers, along with a discussion of the value, potential limitations, and trade-offs associated with each strategy.
The latest generations of smartphones with built-in AR capabilities enable a new class of mobile apps that merge digital and real-world content depending on a user's task, context, and preference. But even experienced mobile app designers face significant challenges: creating 2D/3D AR content remains difficult and time-consuming, and current mobile prototyping tools do not support AR views. There are separate tools for this; however, they require significant technical skill. This paper presents ProtoAR which supplements rapid physical prototyping using paper and Play-Doh with new mobile cross-device multi-layer authoring and interactive capture tools to generate mobile screens and AR overlays from paper sketches, and quasi-3D content from 360-degree captures of clay models. We describe how ProtoAR evolved over four design jams with students to enable interactive prototypes of mobile AR apps in less than 90 minutes, and discuss the advantages and insights ProtoAR can give designers.
We present Oasis, a novel system for automatically generating immersive and interactive virtual reality environments for single and multiuser experiences. Oasis enables real-walking in the generated virtual environment by capturing indoor scenes in 3D and mapping walkable areas. It makes use of available depth information for recognizing objects in the real environment which are paired with virtual counterparts to leverage the physicality of the real world, for a more immersive virtual experience. Oasis allows co-located and remotely located users to interact seamlessly and walk naturally in a shared virtual environment. Experiencing virtual reality with currently available devices can be cumbersome due to presence of objects and furniture which need to be removed every time the user wishes to use VR. Our approach is new, in that it allows casual users to easily create virtual reality environments in any indoor space without rearranging furniture or requiring specialized equipment, skill or training. We demonstrate our approach to overlay a virtual environment over an existing physical space through fully working single and multiuser systems implemented on a Tango tablet device.
MeetAlive combines multiple depth cameras and projectors to create a room-scale omni-directional display surface designed to support collaborative face-to-face group meetings. With MeetAlive, all participants may simultaneously display and share content from their personal laptop wirelessly anywhere in the room. MeetAlive gives each participant complete control over displayed content in the room. This is achieved by a perspective corrected mouse cursor that transcends the boundary of the laptop screen to position, resize, and edit their own and others' shared content. MeetAlive includes features to replicate content views to ensure that all participants may see the actions of other participants even as they are seated around a conference table. We report on observing six groups of three participants who worked on a collaborative task with minimal assistance. Participants' feedback highlighted the value of MeetAlive features for multi-user engagement in meetings involving brainstorming and content creation.
Room2Room is a life-size telepresence system that leverages projected augmented reality to enable co-present interaction between two remote participants. Our solution recreates the experience of a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their virtual copy into the remote space at life-size scale. This creates an illusion of the remote person’s physical presence in the local space, as well as a shared understanding of verbal and non-verbal cues (e.g., gaze, pointing) as if they were there. In addition to the technical details of our two prototype implementations, we contribute strategies for projecting remote participants onto physically plausible seating or standing locations, such that they form a natural and consistent conversational formation with the local participant. We also present observations and feedback from an evaluation with 7 pairs of participants on the usability of our solution for solving a collaborative, physical task.
Augmented Reality (AR) applications may require the precise alignment of virtual objects to the real world. We propose automatic alignment of virtual objects to physical constraints calculated from the real world in real time ("snapping to reality"). We demonstrate SnapToReality alignment techniques that allow users to position, rotate, and scale virtual content to dynamic, real world scenes. Our proof-of-concept prototype extracts 3D edge and planar surface constraints. We furthermore discuss the unique design challenges of snapping in AR, including the user's limited field of view, noise in constraint extraction, issues with changing the view in AR, visualizing constraints, and more. We also report the results of a user study evaluating SnapToReality, confirming that aligning objects to the real world is significantly faster when assisted by snapping to dynamically extracted constraints. Perhaps more importantly, we also found that snapping in AR enables a fresh and expressive form of AR content creation.
This research addresses architecture-scale problem-solving involving the design of living or working spaces, such as architecture and floor plans. Such design systems require two different viewpoints: a small-scale internal view, i.e., a first-person view of the space to see local details as an occupant of the space, and a large-scale external view, i.e., a top-down view of the entire space to make global decisions when designing the space. Architects or designers need to switch between these two viewpoints to make various decisions, but this can be inefficient and time-consuming. We present a system to address the problem, which facilitates asymmetric collaboration between users requiring these different viewpoints. One group of users comprises the designers of the space, who observe and manipulate the space from a top-down view using a large tabletop interface. The other group of users represents occupants of the space, who observe and manipulate the space based on internal views using head-mounted displays (HMDs). The system also supports a set of interaction techniques to facilitate communication between these two user groups. Our system can be used for the design of various spaces, such as offices, restaurants, operating rooms, parks, and kindergartens.
Modern applications increasingly rely on continuous monitoring of video, audio, or other sensor data to provide their functionality, particularly in platforms such as the Microsoft Kinect and Google Glass. Continuous sensing by untrusted applications poses significant privacy challenges for both device users and bystanders. Even honest users will struggle to manage application permissions using existing approaches.
We propose a general, extensible framework for controlling access to sensor data on multi-application continuous sensing platforms. Our approach, world-driven access control, allows real-world objects to explicitly specify access policies. This approach relieves the user's permission management burden while mediating access at the granularity of objects rather than full sensor streams. A trusted policy module on the platform senses policies in the world and modifies applications' "views" accordingly. For example, world-driven access control allows the system to automatically stop recording in bathrooms or remove bystanders from video frames,without the user prompted to specify or activate such policies. To convey and authenticate policies, we introduce passports, a new kind of certificate that includes both a policy and optionally the code for recognizing a real-world object.
We implement a prototype system and use it to study the feasibility of world-driven access control in practice. Our evaluation suggests that world-driven access control can effectively reduce the user's permission management burden in emerging continuous sensing systems. Our investigation also surfaces key challenges for future access control mechanisms for continuous sensing applications.
RoomAlive is a proof-of-concept prototype that transforms any room into an immersive, augmented entertainment experience. Our system enables new interactive projection mapping experiences that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment. The basic building blocks of RoomAlive are projector-depth camera units, which can be combined through a scalable, distributed framework. The projector-depth camera units are individually auto-calibrating, self-localizing, and create a unified model of the room with no user intervention. We investigate the design space of gaming experiences that are possible with RoomAlive and explore methods for dynamically mapping content based on room layout and user position. Finally we showcase four experience prototypes that demonstrate the novel interactive experiences that are possible with RoomAlive and discuss the design challenges of adapting any game to any room.
We propose a new approach to Physical Telepresence, based on shared workspaces with the ability to capture and remotely render the shapes of people and objects. In this paper, we describe the concept of shape transmission, and propose interaction techniques to manipulate remote physical objects and physical renderings of shared digital content. We investigate how the representation of user's body parts can be altered to amplify their capabilities for teleoperation. We also describe the details of building and testing prototype Physical Telepresence workspaces based on shape displays. A preliminary evaluation shows how users are able to manipulate remote objects, and we report on our observations of several different manipulation techniques that highlight the expressive nature of our system.
Instant access to computing, when and where we need it, has long been one of the aims of research areas such as ubiquitous computing. In this paper, we describe the WorldKit system, which makes use of a paired depth camera and projector to make ordinary surfaces instantly interactive. Using this system, touch-based interactivity can, without prior calibration, be placed on nearly any unmodified surface literally with a wave of the hand, as can other new forms of sensed interaction. From a user perspective, such interfaces are easy enough to instantiate that they could, if desired, be recreated or modified "each time we sat down" by "painting" them next to us. From the programmer's perspective, our system encapsulates these capabilities in a simple set of abstractions that make the creation of interfaces quick and easy. Further, it is extensible to new, custom interactors in a way that closely mimics conventional 2D graphical user interfaces, hiding much of the complexity of working in this new domain. We detail the hardware and software implementation of our system, and several example applications built using the library.
Photoportals build on digital photography as a unifying metaphor for reference-based interaction in 3D virtual environments. Virtual photos and videos serve as threedimensional references to objects, places, moments in time and activities of users. Our Photoportals also provide access to intermediate or alternative versions of a scenario and allow the review of recorded task sequences that include life-size representations of the captured users. We propose to exploit such references to structure collaborative activities of collocated and remote users. Photoportals offer additional access points for multiple users and encourage mutual support through the preparation and provision of references for manipulation and navigation tasks. They support the pattern of territoriality with configurable space representations that can be used for private interaction, as well as be shared and exchanged with others.
Instrumented with a single depth camera, a stereoscopic projector, and a curved screen, MirageTable is an interactive system designed to merge real and virtual worlds into a single spatially registered experience on top of a table. Our depth camera tracks the user's eyes and performs a real-time capture of both the shape and the appearance of any object placed in front of the camera (including user's body and hands). This real-time capture enables perspective stereoscopic 3D visualizations to a single user that account for deformations caused by physical objects on the table. In addition, the user can interact with virtual objects through physically-realistic freehand actions without any gloves, trackers, or instruments. We illustrate these unique capabilities through three application examples: virtual D model creation, interactive gaming with real and virtual objects, and a 3D teleconferencing experience that not only presents a 3D view of a remote person, but also a seamless 3D shared task space. We also evaluated the user's perception of projected 3D objects in our system, which confirmed that the users can correctly perceive such objects even when they are projected over different background colors and geometries (e.g., gaps, drops).
This paper presents a new interaction metaphor we have termed "god-like interaction". This is a metaphor for improved commu- nication of situational and navigational information between outdoor users, equipped with mobile augmented reality systems, and indoor users, equipped with tabletop projector display sys- tems. Physical objects are captured by a series of cameras view- ing a table surface indoors, the data is sent over a wireless net- work, and is then reconstructed at a real-world location for out- door augmented reality users. Our novel god-like interaction metaphor allows users to communicate information using physi- cal props as well as natural gestures. We have constructed a sys- tem that implements our god-like interaction metaphor as well as a series of novel applications to facilitate collaboration between indoor and outdoor users. We have extended a well-known video based rendering algorithm to make it suitable for use on outdoor wireless networks of limited bandwidth. This paper also describes the limitations and lessons learned during the design and con- struction of the hardware that supports this research.
In this paper we present a new approach to the use of computer-mediated communications technology to support distributed cooperative work. In contrast to most of the existing approaches to CSCW, we focus explicitly on tools to enable unplanned, informal social interaction. We describe a “social interface” which provides direct, low-cost access to other people through the use of multi-media communications channels. The design of the system centers around three basic concepts derived from the research literature and our own observations of the workplace: social browsing, a virtual workplace, and interaction protocols. We use these design properties to describe a new system concept, and examine the implications for CSCW of having automated social interaction available through the desktop workstation
The development of user interface systems has languished with the stability of desktop computing. Future systems, however, that are off-the-desktop, nomadic or physical in nature will involve new devices and new software systems for creating interactive applications. Simple usability testing is not adequate for evaluating complex systems. A set of criteria for evaluating new UI systems work is presented.
The technology of Collaborative Virtual Environment, (CVE) aims to transform today's computer networks into navigable and populated 3D spaces that support collaborative work and social play. CVEs are virtual worlds shared by participants across a computer network. Participants are provided with graphical embodiments called avatars that convey their identity, presence, location, and activities to others. They are able to use these avatars to interact with contents of the world and to communicate with one another using different media including audio, video, graphical gestures and texts. These worlds can take many forms and representations used for the virtual world and avatars can vary tremendously. In this article, the emergence of CVEs needs to be briefly reviewed and effort should be made to outline challenges and future for this significant technology. CVEs can be seen as the result of a convergence of research interests within the virtual reality and computer-supported cooperative work communities.