Evan Suma Rosenberg

University of Minnesota · Department of Computer Science and Engineering

Creating and animating realistic 3D human faces is an important element of virtual reality, video games, and other areas that involve interactive 3D graphics. In this paper, we propose a system to generate photorealistic 3D blendshape-based face models automatically using only a single consumer RGB-D sensor. The capture and processing requires no a...

Redirection techniques allow users to explore large virtual environments on foot while remaining within a limited physical space. However, research has primarily focused on redirection through manipulation of visual stimuli. We describe a within-subjects study (n=31) exploring if participants’ ability to detect differences between real and virtual...

Creating and animating a realistic 3D human face is an important task in computer graphics. The capability of capturing the 3D face of a human subject and reanimate it quickly will find many applications in games, training simulations, and interactive 3D graphics. We demonstrate a system to capture photorealistic 3D faces and generate the blendshap...

There is rapidly growing interest in the creation of rendered environments and content for tracked head-mounted stereoscopic displays for virtual reality. Currently, the most popular approaches include polygonal environments created with game engines, as well as 360 degree spherical cameras used to capture live action video. These tools were not or...

Over the past few years, virtual reality has experienced a resurgence. Fueled by a proliferation of consumer-level head-mounted display and motion tracking devices, an unprecedented quantity of immersive experiences and content are available for both desktop and mobile platforms. However, natural locomotion in immersive virtual environments remains...

Creating and animating a realistic 3D human face has been an important task in computer graphics. The capability of capturing the 3D face of a human subject and reanimate it quickly will find many applications in games, training simulations, and interactive 3D graphics. In this paper, we propose a system to capture photorealistic 3D faces and gener...

We develop and evaluate three interfaces for video search of articulated objects, specifically humans performing common actions. The three interfaces, (1) a freehand interface with motion cues (e.g., arrows), (2) an articulated human stick figure with motion cues, and (3) a keyframe interface, were designed to allow users to quickly generate motion...

Recent advances in scanning technology have enabled the widespread capture of 3D character models based on human subjects. However, in order to generate a recognizable 3D avatar, the movement and behavior of the human subject should be captured and replicated as well. We present a method of generating a 3D model from a scan, as well as a method to...

Depth-based gesture cameras provide a promising and novel way to interface with computers. Nevertheless, this type of interaction remains challenging due to the complexity of finger interactions and the under large viewpoint variations. Existing middleware such as Intel Perceptual Computing SDK (PCSDK) or SoftKinetic IISU can provide abundant hand...

We present a novel approach to integrating a touch screen device into the experience of a user wearing a Head Mounted Display (HMD) in an immersive virtual reality (VR) environment with tracked head and hands.

Redirected Walking is technique that leverages human perception characteristics to allow locomotion in virtual environments larger than the tracking area. Among the many redirection techniques, some strictly depend on the user's current position and orientation, while more recent algorithms also depend on the user's predicted behavior. This predict...

This demo captures a 3D model of a human subject using a single Microsoft Kinect, constructs a realistic 3D model from that subject, then animates and simulates it with a variety of different behaviors, all within a matter of minutes.

We demonstrate a method of acquiring a 3D model of a human using commodity scanning hardware and then controlling that 3D figure in a simulated environment in only a few minutes. The model acquisition requires four static poses taken at 90° angles relative to each other. The 3D model is then given a skeleton and smooth binding information necessary...

Redirected walking techniques enable natural locomotion through immersive virtual environments that are considerably larger than the available real world walking space. However, the most effective strategy for steering the user remains an open question, as most previously presented algorithms simply redirect toward the center of the physical space....

The USC Institute for Creative Technologies will demonstrate a pipline for automatic reconstruction and animation of lifelike 3D avatars acquired by rotating the user's body in front of a single Microsoft Kinect sensor. Based on a fusion of state-of-the-art techniques in computer vision, graphics, and animation, this approach can produce a fully ri...

To create effective immersive training experiences, it is important to provide intuitive interfaces that allow users to move around and interact with virtual content in a manner that replicates real world experiences. However, natural locomotion remains an implementation challenge because the dimensions of the physical tracking space restrict the s...

As sensing technology has evolved, spatial user interfaces have become increasingly popular platforms for interacting with video games and virtual environments. In particular, recent advances in consumer-level motion tracking devices such as the Microsoft Kinect have sparked a dramatic increase in user interfaces controlled directly by the user's h...

Redirected walking techniques enable natural locomotion through immersive virtual environments (VEs) that are larger than the real world workspace. Most existing techniques rely upon manipulating the mapping between physical and virtual motions while the layout of the environment remains constant. However, if the primary focus of the experience is...

form only given. The ICT Mixed Reality Lab will demonstrate a pipeline for rapidly generating personalized avatars from multiple depth and RGB scans of a user with a consumer level sensor such as a Microsoft Kinect. Based on a fusion of state-of-the-art techniques in graphics, surface reconstruction, and animation, our semi-automatic method can pro...

In this paper we introduce the concept of flexible spaces - a novel redirection technique that generalizes the use of overlapping (impossible) spaces and change blindness in an algorithm for dynamic layout generation. Flexible spaces is an impossible environment that violates the real world constancy in favor of providing the experience of seamless...

Children with autism experience significant positive behavioral and health benefits from exercise, though many of these children tend to lead sedentary lifestyles. Video games that incorporate physical activity, known as exergames, may help to motivate such children to engage in vigorous exercise, thus leading to more healthy lifestyles and reducin...

As wider field-of-view displays become more common, the question arises as to whether or not data collected on these displays are comparable to those collected with smaller field-of-view displays. This document describes a pilot study that aimed to address these concerns by comparing medium-field distance judgments in a 60° FOV display, a 150° FOV...

Purpose: This paper details a brief history and rationale for the use of virtual reality (VR) technology for clinical research and intervention, and then focuses on game-based VR applications in the area of rehabilitation. An analysis of the match between rehabilitation task requirements and the assets available with VR technology is presented. K...

Walking is only possible within immersive virtual environments that fit inside the boundaries of the user's physical workspace. To reduce the severity of the restrictions imposed by limited physical area, we introduce "impossible spaces," a new design mechanic for virtual environments that wish to maximize the size of the virtual environment that c...

Using video games in rehabilitation settings has the potential to provide patients with fun and motivating exercise tools. Within the Medical VR and MxR groups at the USC Institute for Creative Technologies, we have been leveraging the technology of the Microsoft Kinect 3D depth-sensing camera. Our Kinect-based rehabilitation game “JewelMine” consi...

Thin computing clients, such as smartphones and tablets, have exhibited recent growth in display resolutions, processing power, and graphical rendering speeds. In this poster, we show how we leveraged these trends to create virtual reality (VR) training games which run entirely on a commodity mobile computing platform. This platform consists of a c...

We describe two approaches for unobtrusively sensing subtle nonverbal behaviors using a consumer-level depth sensing camera. The first signal, respiratory rate, is estimated by measuring the visual expansion and contraction of the user's chest cavity during inhalation and exhalation. Additionally, we detect a specific type of fidgeting behavior, kn...

Natural walking can provide a compelling experience in immersive virtual environments, but it remains an implementation challenge due to the physical space constraints imposed on the size of the virtual world. The use of redirection techniques is a promising approach that relaxes the space requirements of natural walking by manipulating the user's...

Virtual humans are often presented as mixed reality characters projected onto screens that are blended into a physical setting. Stereo loudspeakers to the left and right of the screen are typically used for virtual human audio. Unfortunately, stereo pairs can produce an effect known as precedence, which causes users standing close to a particular l...

Our demonstration will exhibit a number of low cost virtual reality systems built using smartphones, inexpensive optics, and game engine software. These systems will demonstrate how VR researchers and developers can leverage current trends in commodity hardware and software to bring virtual reality to a wider audience. This demonstration dovetails...

The support of realistic and flexible training simulations for military, law enforcement, emergency response, and other domains has been an important motivator for the development of augmented reality technology. An important vision for achieving this goal has been the creation of a versatile “stage” for physical, emotional, and cognitive training...

The use of the commercial video games as rehabilitation tools, such as the Nintendo WiiFit, has recently gained much interest in the physical therapy arena. Motion tracking controllers such as the Nintendo Wiimote are not sensitive enough to accurately measure performance in all components of balance. Additionally, users can figure out how to "chea...

The use of commercial video games as rehabilitation tools, such as the Nintendo® Wii FitTM, has recently gained much interest in the physical therapy arena. However, physical rehabilitation requires accurate and appropriate tracking and feedback of performance, often not provided by existing commercial console devices or games. This paper describes...

New opportunities emerge when mixed reality environments are ­augmented with immersive displays and full body, real-time tracking. Such systems enable the creation of experiences where users “share space” with other virtual humans in the virtual environment. These systems can portray responsive 3D virtual humans that react to position, motion, and...

We present change blindness redirection, a novel technique for allowing the user to walk through an immersive virtual environment that is considerably larger than the available physical workspace. In contrast to previous redirection techniques, this approach, based on a dynamic environment model, does not introduce any visual-vestibular conflicts f...

Thin computing clients, such as smartphones and tablets, have experienced recent growth in display resolutions and graphics processing power. In this poster, we show how to leverage these trends to create an experimental wide field of view, 3D stereoscopic head mounted display (HMD), based on two high resolution smart-phones. This HMD prototype is...

We report a user study that investigated the effect of redirection in an immersive virtual environment on spatial orientation relative to both real world and virtual stimuli. Participants performed a series of spatial pointing tasks with real and virtual targets, during which they experienced three within-subjects conditions: rotation-based redirec...

We describe an approach for enabling people to share virtual space with a user that is fully immersed in a head-mounted display. By mounting a recently developed low-cost depth sensor to the user's head, depth maps can be generated in real-time based on the user's gaze direction, allowing us to create mixed reality experiences by merging real peopl...

The Flexible Action and Articulated Skeleton Toolkit (FAAST) is middleware to facilitate integration of full-body control with virtual reality applications and video games using OpenNI-compliant depth sensors (currently the PrimeSensor and the Microsoft Kinect). FAAST incorporates a VRPN server for streaming the user's skeleton joints over a networ...

The convergence of the exponential advances in virtual reality (VR)-enabling technologies with a growing body of clinical research and experience has fueled the evolution of the discipline of clinical VR. This article begins with a brief overview of methods for producing and delivering VR environments that can be accessed by users for a range of cl...

We present the design and evaluation of Astrojumper, an immersive virtual reality exergame developed to motivate players to engage in rigorous, full-body exercise. We performed a user study with 30 people between the ages of 6 and 50 who played the game for 15 min. Regardless of differences in age, gender, activity level, and video game experience,...

National Guard Civil Support Teams (CST) respond to a variety of situations involving dangerous materials. Many of these situations can be safely simulated for training purposes in the real world. Radiological threats, however, are difficult to simulate due to the lack of materials that can mimic radiation sources without the danger of the real rad...

Since its inception, the field of visual analytics has undergone tremendous growth in understanding how to create interactive visual tools to solve analytical problems. However, with few exceptions, most of these tools have been designed for single users in desktop environments. While often effective on their own, most single-user systems do not re...

We present a technique for exploiting change blindness to allow the user to walk through an immersive virtual environment that is much larger than the available physical workspace. This approach relies on subtle manipulations to the geometry of a dynamic environment model to redirect the user's walking path without becoming noticeable. We describe...

Children with autism show substantial benefits from rigorous physical activity, however it is often difficult to motivate these individuals to exercise due to their usually sedentary lifestyles. To address the problem of motivation, we have developed Astrojumper, a stereoscopic virtual reality exergame which was designed to fit the needs of childre...

We report a user study which compared four virtual environment travel techniques using a divided attention task. Participants used either real walking, gaze-directed, pointing-directed, or torso-directed travel to follow a target through an environment while simultaneously responding to auditory stimuli. In addition to travel technique, we investig...

Children with autism have shown substantial benefit s from rigorous exercise, however, it is often diffic ult to motivate these children to exercise due to their us ually sedentary lifestyles. To address the problem of motivation, we have developed Astrojumper, a stereoscopic virtual reality exergame which was designed to fit the needs of children...

This paper presents the design of the final stage of a new game currently in development, entitled cMotion, which will use virtual humans to teach emotion recognition and programming concepts to children. Having multiple facets, cMotion is designed to teach the intended users how to recognize facial expressions and manipulate an interactive virtual...

We report on a study in which we investigate the effects of travel technique on simulator sickness in a real and virtual environment. Participants explored either a real maze or a virtual maze using either natural walking or simulated walking. Reported scores for measures of overall simulator sickness, disorientation, nausea, and occulomotor discom...

We evaluated four selection techniques for volumetric data based on the four classes of bimanual action: symmetric-synchronous, asymmetric-synchronous, symmetric-asynchronous, and asymmetric-asynchronous. The purpose of this study was to determine the relative performance characteristics of each of these classes. In addition, we compared two types...

We present a method for detecting common human actions in video, common to athletics and surveillance, using intuitive sketches and motion cues. The framework presented in this paper is an automated end-to-end system which (1) interprets the sketch input, (2) generates a query video based on motion cues, and (3) incorporates a new content-based act...

Large financial institutions such as Bank of America handle hun- dreds of thousands of wire transactions per day. Although most transactions are legitimate, these institutions have legal and finan- cial obligations in discovering those that are suspicious. With the methods of fraudulent activities ever changing, searching on pre- defined patterns i...

Large financial institutions such as Bank of America handle hundreds of thousands of wire transactions per day. Although most transactions are legitimate, these institutions have legal and financial obligations in discovering those that are suspicious. With the methods of fraudulent activities ever changing, searching on predefined patterns is ofte...

This paper reports on a study that compares three different methods of travel in a complex, multi-level virtual environment using a between-subjects design. A real walking travel technique was compared to two common virtual travel techniques. Participants explored a two-story 3D maze at their own pace and completed four post-tests requiring them to...

In this paper, we describe our current work in developing a computer-based educational tool for Occupational Therapy students learning client evaluation techniques. The software is dialog-based and allows the student to interact with a virtual client. Students carry out an evaluation, following the appropriate procedures and assessing both the clie...

In this paper we present the results of a study to determine the effect and efficacy of a Virtual Reality game designed to elicit movements of the upper extremity. The study is part of an on-going research effort to explore the use of Virtual Reality as a means of improving the effectiveness of therapy for children with motor impairments. The curre...

Redirection techniques are a compelling way to enable natural walking in immersive virtual environments when physical space is limited. It is challenging to employ redirection in mixed reality environments, however, because these techniques disrupt the spa-tial alignment between virtual objects and corresponding real world props. In this paper, we...

We present an approach to redirect a user's walking path by dy-namically modifying the geometry of a virtual environment. This method allows real walking through environments that are much larger than the physical tracking area without requiring rotational or translational gains. We demonstrate this technique using a proof-of-concept example enviro...