Sai Keung Wong

National Yang Ming Chiao Tung University · Department of Computer Science

This paper investigated to what extent social interactions and empathy of users could be induced when different control mechanisms were used in an asymmetric collaboration. We conducted a user study to explore the user experience under one decentralized and two centralized control conditions via using the proposed two-player asymmetric collaborativ...

In this contribution, we empirically investigated the effect of small talk on the users' non-verbal behaviors and emotions when users interacted with a crowd of virtual humans (VHs) with positive behavioral dispositions. Users were tasked with collecting items in a virtual marketplace via natural speech-based dialogue with a crowd of virtual pedest...

In this article, we propose a framework to generate cart‐pulling animation using deep learning in a multiagent environment. Mainly, two workers pull a cart using ropes and interact with crowd agents which exhibit following, wandering, and evasion behaviors. The main idea is to train a policy to learn an individual behavior of the workers and crowd...

Lidars are commonly used on autonomous vehicles, but their performance can be significantly affected by adverse weather. A number of studies have been devoted to analyzing and improving lidars’ performance in rain, fog, and snow. Yet, relatively little attention has been paid to road spray which occurs when vehicles travel on wet surfaces at high s...

This paper proposes a framework that integrates reinforcement learning and blend‐trees to generate animation of multiple agents for object transportation. The main idea is that in the learning stage, policies are learned to control agents to perform specific skills, including navigation, pushing, and orientation adjustment. The policies determine t...

In this article, we present a framework which transfers video to dynamic relief‐structures with water interaction. Our method enables the interaction between video content and waterflows. The key idea is that the dynamic 2.5D structures of the objects in video are constructed and then waterflows are simulated to interact with the 2.5D structures. W...

We develop an interface that bridges crowd simulation and natural language processing techniques so that casual users can produce crowd animation by text input. The interface adopts a parser and a tagger to analyze simple English sentences to convert them into intermediate data structures that encapsulate the essential elements of crowds. There are...

In this article, we develop a system with a natural language interface to generate animation of small groups in environments with ambient crowds. Our system takes simple sentences of English as input and these sentences are parsed to obtain information about character attributes, behaviors, and locations for constructing situation nodes. These situ...

We present a system to help designers create icons that are widely used in banners, signboards, billboards, homepages, and mobile apps. Designers are tasked with drawing contours, whereas our system colorizes contours in different styles. This goal is achieved by training a dual conditional generative adversarial network (GAN) on our collected icon...

We present a system to help designers create icons that are widely used in banners, signboards, billboards, homepages, and mobile apps. Designers are tasked with drawing contours, whereas our system colorizes contours in different styles. This goal is achieved by training a dual conditional generative adversarial network (GAN) on our collected icon...

In this paper, we employ two approaches to perform cooperative transportation of boxes in virtual environments with pedestrians. Each box is pushed by two agents. The agents can change their formation while they push the boxes for collision avoidance with the pedestrians. The two approaches are sequence-based and sensor-based. We perform experiment...

This paper presents an approach to assist the generation of agent-based cooperative animation using reinforcement learning. We focus on manipulation skills for box-shaped objects, including pushing, pulling, and moving objects in a relay way. There are a learning process and an animation process. In the learning process, different kinds of agents a...

This paper studies users' learning perception in a virtual titration experiment with differentiated instruction. We develop a virtual reality chemistry lab and use leap motion to detect users' hand gestures for operations. Users wear a head‐mounted display and use their bare hands to interact with virtual objects to perform a titration experiment....

This paper studies the user collaboration experience with proactive and reactive agents in transporting boxes in virtual environments. Two main characters, the avatar and the agent, are controlled by a user and a controller, respectively. The user and the agent communicate with each other by voice. The agent can be proactive or reactive. The user f...

In this paper, we integrate a cellular automaton model with game theory to simulate crowd evacuation from a room with consideration of obstacle removal. The room has one or more exits, one of which is blocked by obstacles. The obstacles at the exit can be removed by volunteers. We investigate the cooperative and defective behaviors of pedestrians d...

This paper presents a novel agent-based approach to simulate cooperative tasks in crowd simulation. Our framework consists of two kinds of agents: pedestrians and workers. A cooperative task requires two or more workers to simultaneously perform actions on objects. The workers should cooperate with each other to finish tasks. This framework conside...

This paper focuses on simulating agents transporting passive objects in a virtual environment with pedestrians. The agents adopt the pushing manipulation patterns to transport the passive objects. We propose techniques to achieve two kinds of cooperation. First, the agents transport the passive objects to the goal positions. During the transportati...

We present an interactive system using one RGB-D sensor, which allows a user to use bare hands to perform sand drawing. Our system supports the common sand drawing functions, such as sand erosion, sand spilling, and sand leaking. To use hands to manipulate the virtual sand, we design four key hand gestures. The idea is that the gesture of one hand...

This paper studies the effects of a virtual crowd with emotion on task completion in a virtual market. The users have to interact with the virtual agents so that the virtual agents are willing to lead the users to specific vendors. Then, the users can buy the required items. The users communicate with the virtual agents via a conversation dialog. T...

Navigation fields are a popular data structure for directing the movement of virtual crowds. However, the congestion problem may occur at the corners of objects in using a navigation field because the agents move quite closely to each other. The agents may congest at the corners even though there is room for the agents to move away from the congest...

An evacuation plan helps people move away from an area or a building. To assist rapid evacuation, we present an algorithm to compute the optimal route for each local region. The idea is to reduce congestion and maximize the number of evacuees arriving at exits in each time span. Our system considers crowd distribution, exit locations, and corridor...

In this paper, we present an approach to use basis fires to design and synthesize fires in desired shapes and motions. In the preprocessing stage, each basis fire is simulated based on a physics-based fire simulator with a specific simulation configuration. The pathlines and temperatures of the basis fires are stored in a database. In the fire desi...

This paper presents a two-dimensional editing system which compromises the advantages of using hand gestures and real blocks to manipulate virtual objects. The system uses a RGB-D sensor to capture the color and depth images of the hands and real blocks. The raw images of the hands and blocks are segmented. Then we employ a contour based algorithm...

Crowd simulation has a wide range of applications, such as travel network planning, evacuation planning and layout design [Feng et al. 2016]. A critical issue in crowd simulation is to simulate crowds at different moments. In [Krontiris et al. 2016], an influence map based on attractors is used for authoring activity-centric crowd simulation. Howev...

We present a simple agent-based approach for crowd simulation in a two-dimensional space. Our approach computes a set of collision-free feasible movement directions for each agent. The movement directions of the agent are determined according to a radial view of the agent in the local region. Subsequently, the direction with the least effort is sel...

We present an agent-based approach to animate microscopic mixed traffic involving cars and motorcycles in complex scenarios, including signalized and nonsignalized road intersections, and traffic jams due to blockage. Based on our new car-following and lateral movement models, our method can reproduce lane-based and nonlane-based traffic behaviors...

Climbing plants with tendrils show search and coiling behaviour. A tendril searches for a host object and then twines around it. Subsequently, the tendril coils to pull the main stem of the climbing plant close to the host object. Furthermore, the stems may also twine around the host object. In this paper, we propose a procedural approach to increm...

In this paper, we developed a system which enables users to draw pictures of sand and water on a two-dimensional surface in a virtual environment. By mixing water and sand for artwork creation, we can create different styles of pictures. Our system has three major subsystems: water animation system, sand animation system and rendering system. In th...

We propose an Automatic Motorcycle Turn Signal (AMTS) system to assist motorcyclists to automatically signal turns. The AMTS system utilizes the gyroscope sensor of the Android phone to detect the turning direction of a motorcycle and then the system turns on the light and sound accordingly. We have evaluated the proposed AMTS system in indoor and...

In this paper, we propose a method for using particle swarm optimization (PSO) to compute optimal guidance paths for various crowd densities in an agent-based crowd simulation. The inputs of our system are guidance paths that provide hints for the movement directions of agents. Input guidance paths may not be located correctly (e.g., leading to con...

In this study, we develop a system for interactive snow simulation and rendering. Snow is modeled as a hybrid structure that handles movable snow and static snow separately. We develop a simple approach to convert between these two types of snow. Movable snow is represented by a set of particles, whereas static snow is modeled as grid cells. For a...

We propose a graphics processing unit-based approach to accelerate the radial view-based culling method for continuous self-collision detection of deformable surfaces. The deformable surfaces may have small round-shaped holes and ghost triangles are used to fill the holes. We identify the key processes of the radial view-based culling method, inclu...

The virtual world is currently limited in the graphic representation and visualization of material designs. The limitation is in part due to the limited range and the relative simplicity of spectral properties of the materials simulated in a virtual environment. Our work starts with a fast classification and retrieval for handling the large numbers...

In this paper, we present an interactive sand art drawing system using Kinect. Our system adopts a vision-based bare hand detection method which effectively detects the hand position and recognizes the hand gestures. Then, the corresponding sand manipulation actions are performed. Our system supports the common sand drawing functions, such as sand...

We present a skeleton-based approach for mapping consistently a segmentation of a source mesh to a target mesh. The source and target meshes are semantically similar but their geometries may be different. We assume that the segmentation of the source mesh and the skeleton correspondence between the meshes are provided. Our method first establishes...

In this paper, we propose a new approach based on a particle-based model for ice melting simulation. Each particle has an attribute called virtual water. The amount of the virtual water of an ice particle indicates the amount of water surrounding the ice particle. The transfer of the virtual water is performed between the exterior ice particles so...

In this paper, we propose a new data structure to perform continuous collision detection (CCD) for deformable triangular meshes. The critical component of this data structure is permissible clusters. At the preprocessing phase, the triangular meshes are divided into permissible clusters. Then, the features of the triangular meshes are assigned to t...

The radial view-based culling (RVBC) method has been presented for continuous self-collision detection to efficiently cull away non-colliding regions. While this technique mainly relies on the segmented clusters of the reference pose and the associated fixed observer points, it has several drawbacks during the animation and the reduced cost of exec...

In this paper, we propose a new continuous self-collision detection (CSCD) method for a deformable surface that interacts with a simple solid model. The method is developed based on the radial-view-based culling method. Our method is suitable for the deformable surface that has large contact region with the solid model. The deformable surface may c...

In management games, players enjoy developing the virtual objects, such as avatars, farms, villages, cities, resources, etc. Usually, the management games do not have a time limit to play. Players know little about how much time that they need to devote in order to quickly build up the virtual objects. This paper studies about the difficulty level...

Water drops and water flows exhibit interesting motion behavior. In this paper, we adopt a simple but effective approach for simulating this behavior on glass panes in a physically plausible manner. We combine a particle system and a height map to compute their movements and shapes. Our approach efficiently handles the merging of water drops and th...

We propose a new method for auto stylized sand art drawing. Our system is built on top of our sand drawing system which is mixed with a particle system and a height map. An image is input and the patterns of the image is analyzed. The contours of the image are computed and they are converted into strokes. In sand art drawing, artists can use multip...

Water drops and water flows exhibit interesting motion behaviors and amazing patterns on the surfaces of objects, such as leaves of plants and glass panes. Water drops and water flows are commonly seen in a rainy day. A water drop contains a small amount of water. The motion of a water drop is affected by various factors, including gravity, surface...

In this paper, we propose a hybrid method for simulating water droplets on the glass pane. We model the motion of water droplets based on a particle system. The shape of the water droplets is constructed by converting the particles into a height map according to the water amount of the water droplets. To realistically compute the shape of the water...

We present a novel skeleton-based approach for performing shape correspondence by exploiting the mesh geometric properties associated with the skeletons of the objects. The skeletal nodes which correspond to the similar parts of the two objects are matched. We compute a set of potential shape correspondences and then evaluate the best one. Our meth...

Game balancing affects the gaming experience of players in video-games. In this paper, we propose a novel system, team ability balancing system (TABS), which is developed for automatically evaluating the performance of two teams in a role-playing video game. TABS can be used for assisting game designers to improve team balance. In TABS, artificial...

We present a bounding volume hierarchy construction based on a culling technique for performing ray tracing on manycore GPUs. The objects are divided into view-sets of triangles with respect to a set of view targets (including the camera and the light sources). A bounding volume hierarchy, namely reduced bounding volume hierarchy (RBVH), is then co...

Physics simulation and character control are two important issues in computer games. In this paper, we propose two games which are tailored for investigating some aspects of these two issues. We study on the applications of neural network and the genetic algorithm techniques for building the controllers and the controllers should be able to finish...

We propose a novel adaptive pipeline for continuous collision detection in cloth simulation. The pipeline consists of four components: bounding volume hierarchy (BVH) update, BVH traversal, a skipping frame session, and elementary test processing. The skipping frame session is activated adaptively for skipping both BVH update and BVH traversal duri...

We study on artificial neural network-based controllers which are either trained or evolved by using the supervised or unsupervised learning approach. We employed backpropagation for the supervised method and the genetic algorithm for the unsupervised method. After training the controllers, we applied the controllers to our three newly designed min...

Continuous collision detection improves the computation of the contact information for interacting objects in dynamic virtual environments. The computation cost is relatively high in the phase of the elementary test processing. In virtual environments, such as crowds in large urban models, there is a large portion of feature pairs that do not colli...

Collision events between 3D objects in motion in computer animations or simulations are difficult to detect due to the difficulty of accurately sampling the motion paths of objects in space and time. One approach to this problem has been continuous collision detection but because the current approaches process potentially interacting primitive pair...

This paper presents a method for modelling the seam puckering of multi-layer fabrics by considering the thread shrinkage as well as seam structure and fabric compression properties. A theoretical investigation and numerical calculation of forces acting on the sewing thread were carried out. Based on this, the forces causing the thread tension pucke...

Dynamic objects in virtual environments are the source of some of the largest number of interaction events. These are mostly generated from object collisions. The collision detection methods that have been developed so far are based on geometrical object-space interference tests. In this chapter, we first discuss the development of conventional col...

Continuous collision detection techniques are applied extensively in the simulation of deformable surfaces, in particular for cloth simulation. Accurate contact information can be computed by using these techniques. Traditionally, for meshed surfaces, after collecting the triangle pairs that are potentially interacting, the feature pairs of these t...

An intrinsic collision detection unit (ICDU) forms the bottom-most layer of a collision detection pipeline. The ICDU performs collision detection and computes collision information for primitive feature pairs of objects in a 3D dynamic environment. A significant amount of time can be spent by the ICDU during the collision detection process. In this...

In this paper, we introduce new techniques that enhance the computational performance for the interactions between sharp objects and deformable surfaces. The new formulation is based on a time-domain predictor-corrector model. For this purpose, we define a new kind of (pi, beta, I)-surface. The partitioning of a deformable surface into a finite set...

We propose an architecture for the intrinsic collision detection unit to perform collision detection for deformable triangular meshes, accurately, robustly and efficiently. We adopt a cache scheme which greatly improves the feature (e.g. point, edge and triangle) collision detection and thus improves the overall collision detection process. Our met...

The interactive requirements of 3D games and physically driven virtual environments add strong constraints to the simulation of natural cloth collisions and self-collisions. In order to achieve interactive rates, we first define smoothness conditions over small patches of deformable surfaces and then resort to image-based collision-detection algori...

Modeling the natural interaction of cloth and garments with objects in a 3D environment is currently one of the most computationally demanding tasks. These highly deformable materials are subject to a very large number of contact points in the proximity of other moving objects. Furthermore, cloth objects often fold, roll, and drape within themselve...

We propose a positional constraint method to solve the multi-layered deformable surface problem based on a master-slave scheme. This allows two or more deformable surfaces to be attached together in any orientation relative to each other for the purpose of modeling cloth attachments and multi-layered clothing. The method does not require the mesh r...

Most collision detection methods developed so far are based on geometrical object-space interference tests. While this remains the basic mode of investigation for geometric algorithms, the requirements for interactive rates and complex geometry predominate in commercial applications. In this article, we propose a new mode of collision detection bas...

The natural behavior of garments and textile materials in the presence of changing object states is potentially the most computationally demanding task in a dynamic 3D virtual environment. Cloth materials are highly deformable inducing a very large number of contact points or regions with other objects. In a natural environment, cloth objects often...

In interactive virtual environments and dynamic simulations, collisions between complex objects and articulated bodies may occur simultaneously at multiple points or regions of interference. Many solutions to the collision response problem are formulated based on the local pair-wise contact dynamics. In this article, we present a new solution to th...

Object interactions are ubiquitous in interactive computer graphics, 3D object motion simulations, virtual reality and robotics applications. Most collision detection algorithms are based on geometrical object space interference tests. Some algorithms have employed an image space approach to the collision detection problem. We demonstrate an image...

Collision detection between complex objects using rasterizing graphics hardware provides a rich ground of exploration for speeding up the interference detection algorithms and computing points of collision. We show that despite the limitations imposed by the current graphics rendering hardware, it is still possible to perform collision detection at...