Daniel Thalmann

• PhD Computer Science, Univ. Geneva
• Professor at Swiss Federal Institute of Technology in Lausanne

Schizophrenia is a Xu, Shihao Yang, Zixu Chakraborty, Debsubhra that contributes to Tahir, Yasir Maszczyk, Tomasz Victoria, Chua Yi Han Dauwels, Justin Thalmann, Daniel life. Thalmann, Nadia Magnenat Tan, Bhing-Leet Keong, Jimmy Lee Chee to design objective assessment tools of schizophrenia. In earlier work, we investigated non-verbal quantitative...

Schizophrenia and depression are the two most common mental disorders associated with negative symptoms that contribute to poor functioning and quality of life for millions of patients globally. This study is part of a larger research project. The overall aim of the project is to develop an automated objective pipeline that aids clinical diagnosis...

Many on-line communities still rely on text messages as their privileged medium of communication and we believe it will remain so even if it can be complemented by images and video. Previous chapters have described how to infer emergent emotion dynamics from text corpus whereas we focus here on the instantaneous display of a detected emotion with a...

This paper presents an immersive volleyball game, where a player plays not only against virtual opponents but also with support on his/her side of virtual teammates. This volleyball game has been implemented for several 3D displays such as a stereoscopic display, an autostereoscopic display, Oculus Rift glasses, and a 320o Immersive Room. In this p...

Wearable devices such as Microsoft Hololens and Google glass are highly popular in recent years. As traditional input hardware is difficult to use on such platforms, vision-based hand pose tracking and gesture control techniques are more suitable alternatives. This demo shows the possibility to interact with 3D contents with bare hands on wearable...

This paper proposes a real time sociometric system to analyze social behavior from audio-visual recordings of two-person face-to-face conversations in English. The novelty of the proposed system lies in this automatic inference of ten social indicators in real time. The system comprises of a Microsoft kinect device that captures RGB and depth data...

Expanding the spectrum of agent social capabilities is an important challenge in agent-based simulation and other domains. While human-like emotionality has been vastly explored in the last 20years, little research addresses explicit, psychologically believable social situation modeling. Recently, some important elements have been underlined: hybri...

Human activity recognition in unconstrained RGB-D videos has extensive applications in surveillance, multimedia data analytics, human-computer interaction, etc, but remains a challenging prob-lem due to the background clutter, camera motion, viewpoint changes, etc. We develop a novel RGB-D activity recognition ap-proach that leverages the dense tra...

User ratings are the essence of recommender systems in e-commerce. Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of such systems and contribute to the well-known data sparsity and cold start problems. This article proposes a new information source for recommender systems, call...

Healthcare can be brought to a principally new level by creating virtual bodies of real patients and using them for diagnosis and treatment planning along with the real patients. It will help to improve the quality of healthcare services, and reduce the healthcare cost, especially when dealing with the rise in musculoskeletal disorders in the aging...

The precise evaluation of joint movements, using reconstructed three-dimensional models of the joint tissues, is an important issue in many medical diagnosis and surgery planning. In order to achieve appropriate results by analyzing 3D joint models, efficient and accurate evaluation of contact among the virtual tissues is critical. In this chapter,...

Hand pose tracking and gesture recognition are useful for human-computer interaction, while a major problem is the lack of discriminative features for compact hand representation. We present a robust hand parsing scheme to extract a high-level description of the hand from the depth image. A novel distance-adaptive selection method is proposed to ge...

In this paper, a human-robot interaction system based on a novel combination of sensors is proposed. It allows one person to interact with a humanoid social robot using natural body language. The robot understands the meaning of human upper body gestures and expresses itself by using a combination of body movements, facial expressions, and verbal l...

New technical innovations in telepresence allow people to attend meetings or gather together anywhere they are and to share the same virtual space using 3D avatars. More sophisticated systems have been developed with 3D capturing and rendering, leading to a true 3D telepresence experience. If the 3D avatar is remotely guided by the real participant...

Simulating a crowded scene like a busy shopping street requires tight packing of virtual characters. In such cases, collisions are likely to occur, and the choice in collision detection shape will influence how characters are allowed to intermingle. Full collision detection is too expensive for crowds, so simplifications are needed. The most common...

Trust has been extensively studied and its effectiveness demon-strated in recommender systems. Due to lack of explicit trust information in most systems, many trust metric ap-proaches have been proposed to infer implicit trust from user ratings. However, previous works have not compared these different approaches, and oftentimes focus only on the p...

In this chapter, we present a system that provides real-time feedback about an ongoing discussion. Various speech statistics, such as speaking length, speaker turns and speaking turn duration, are computed and displayed in real time. In social monitoring, such statistics have been used to interpret and deduce talking mannerisms of people and gain i...

Pompeii was a Roman city, destroyed and completely buried during an eruption of the volcano Mount Vesuvius. We have revived its past by creating a D model of its previous appearance and populated it with crowds of Virtual Romans. In this paper, we detail the process, based on archaeological data, to simulate ancient Pompeii life in real time. In a...

This paper presents a parallel texture synthesis over arbitrary surfaces, generating consistent and spatially-varying visual appearances. A novel scaling field is represented to measure the geometry-aware appearance or geometric deformation, therefore the generated textures locally agree with the geometric structure and maintain the coherence durin...

In this tutorial, we will describe both virtual characters and realistic humanoid social robots using the same high level models. Particularly, we will describe: 1. How to capture real-time gestures and facial emotions from real people, how to recognize any real person, how to recognize certain sounds. We will present a state of the art and some ne...

In neurofeedback, brain waves are transformed into sounds or music, graphics, and other representations, to provide real-time information on ongoing waves and patterns in the brain. Here we present various forms of neurofeedback, including sonification, sonification in combination with visualization, and at last, immersive neurofeedback, where audi...

In this paper, we propose a method to achieve effective facial emotional expressivity for embodied conversational agents by considering two types of asymmetry when exploiting the valence–arousal–dominance representation of emotions. Indeed, the asymmetry of facial expressions helps to convey complex emotional feelings such as conflicting and/or hid...

Content-based human motion retrieval is important for animators with the development of motion editing and synthesis, which need to search similar motions in large databases. Obtaining text-based representation from quantization of mocap data turned ...

A real-time system is proposed to quantitatively assess speaking mannerisms and social behavior from audio recordings of two-person dialogs. Speaking mannerisms are quantitatively assessed by low-level speech metrics such as volume, rate, and pitch of speech. The social behavior is quantified by sociometrics including level of interest, agreement,...

Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of recommender systems and contribute to the well-known data sparsity and cold start problems. This paper proposes a new information source for recommender systems, called prior ratings. Prior ratings are based on users' experience...

In this paper, a novel human-virtual human interaction system is proposed. This system supports a real human to communicate with a virtual human using natural body language. Meanwhile, the virtual human is capable of understanding the meaning of human upper body gestures and reacting with its own personality by the means of body action, facial expr...

In this paper we present a novel vision-based markerless hand pose estimation scheme with the input of depth image sequences. The proposed scheme exploits both temporal constraints and spatial features of the input sequence, and focuses on hand parsing and 3D fingertip localization for hand pose estimation. The hand parsing algorithm incorporates a...

Despite the demand for culturally placed agent models, an adequate simulation approach to the relationship between group-cultural and individual-psychological qualities, including culture emergence, is just appearing. It could be argued that we are at the beginning of a domain forming process, a dawn of generative, emergent artificial culture. In t...

We explore an approach to full-body motion editing with linear motion models, prioritized constraint-based optimization and latent-space interpolation. By exploiting the mathematical connections between linear motion models and prioritized inverse kinematics (PIK), we formulate and solve the motion editing problem as an optimization function whose...

The recent proliferation of virtual reality (VR) technology applications in the autism therapy to promote learning and positive behavior among such children has produced optimistic results in developing a variety of skills and abilities in them. Dolphin-assisted therapy has also become a topic of public and research interest for autism intervention...

Providing high quality recommendations is important for e-commerce systems to assist users in making effective selection decisions from a plethora of choices. Collaborative filtering is a widely accepted technique to generate recommendations based on the ratings of like-minded users. However, it suffers from several inherent issues such as data spa...

In this chapter, we focus on the technical aspect of real-time crowd simulation, and how an efficient architecture can be developed. But first, it is important to identify the main goals to achieve. We need: Quantity: real-time simulation of thousands of characters, Quality: state-of-the-art virtual humans, Efficiency: storage and real-time managem...

One of the important characteristics of behavioral animation is its ability to reduce the workload on the animators. This is achieved by letting a behavioral model automatically take care of the low-level details of the animation, freeing the animator to concentrate on the big picture. Freeing the animator from low-level animation details is even m...

While designing an animation engine usable in crowd simulations, several criteria have to be taken into account: animation computation should be efficient and scalable, it should allow for variability, and it should be compatible with levels of detail. To understand the problems, let us take the most important animation pattern used in crowd simula...

In this paper, we first present our crowd simulation method, Trajectory Variant Shift (TVS) based on real pedestrian trajectories re-use. We detail how to re-use and shift these trajectories to avoid collisions while retaining the liveliness of captured data. Second, we conducted a user study in a four-screen CAVE to compare our approach with three...

In this paper we present a model-based framework for hand pose estimation, which relies on the depth and color image sequence input. The proposed framework adopts a divide-and-conquer scheme, and combines fingertip tracking and articulated iterative closest point approach to restore the hand motion. The tracked fingertip positions are used to provi...

Traversing dense crowds can be challenging, but it is especially difficult in a virtual environment where the user is limited to simple input devices. Predicting near-future user movements with a hidden Markov model allows nearby autonomous agents to react politely, that is, to specifically avoid impeding the movement of the user. This predictive m...

We present a vision-based approach for robust 3D fingertip and palm tracking on depth images using a single Kinect sensor. First the hand is segmented in the depth images by applying depth and morphological constraints. The palm is located by performing distance transform to the hand contour and tracked with a Kalman filter. The fingertips are dete...

Robot localization is a key barrier to providing natural interaction between 3D virtual characters, human users and mobile robots. Knowing where the robot is, relative to a known world-frame, is essential to directed gestures, gazes and expressions between the robot and the other real and virtual participants in a telepresence system. The intrinsic...

This paper presents an interactive dynamic controller used to generate locomotion patterns for humanoid robots. The purpose of this work is to provide animators and artists easy and intuitive tools to design expressive motions for humanoid robots. A review of similar work in the computer animation community has guided our choices regarding the impl...

Providing high quality recommendations is important for online systems to assist users who face a vast number of choices in making effective selection decisions. Collaborative filtering is a widely accepted technique to provide recommendations based on ratings of similar users. But it suffers from several issues like data sparsity and cold start. T...

This paper proposes a new emotional model for Virtual Humans (VHs) in a conversational environment. As a part of a multi-users emotional 3D-chatting system, this paper focus on how to formulate and visualize the flow of emotional state defined by the Valence-Arousal-Dominance (VAD) parameters. From this flow of emotion over time, we successfully vi...

Non-verbal communication (NVC) makes up about two-thirds of all communication between two people or between one speaker and a group of listeners. However, this fundamental aspect of communicating is mostly omitted in 3D social forums or virtual world oriented games. This paper proposes an answer by presenting a multi-user 3D-chatting system enriche...

Autonomous virtual characters and social robots are meant to interact with humans. They should be able to communicate, express emotions and exhibit personality. Their social skills are highly dependant on their physical design, as well as on their motion capabilities. This paper presents a comparative survey of design choices and motion generation...

Since the identification of the first cases of Autism Spectrum Disorder (ASD) in 1943 by Dr Leo Kanner, the number of reported ASD cases as well as research studies expanding our understanding of the disorder has increased rapidly and significantly. In terms of ASD treatment, there is a wide range of conventional as well as complementary and altern...

We present an interface for navigating digital collections based on a one-dimensional analog control and a data visualization based on old analog radios. Our system takes advantage of inertial control to browse a large data collection in a compelling way, reducing the complexity of similar interfaces present in both desktop-based and portable media...

This paper presents an interactive multi-agent system based on a fully immersive virtual environment. A user can interact with the virtual characters in real time via an avatar by changing their moving behavior. Moreover, the user is allowed to select any character as the avatar to be controlled. A path planning algorithm is proposed to address the...

Autonomous virtual characters and social robots are meant to interact with humans. They should be able to communicate, express emotions and exhibit personality. Their social skills are highly dependent on their physical design, as well as on their motion capabilities. This paper presents a comparative survey of design choices and motion generation...

Computer-based simulations of human hip joints generally include investigating contacts happening among soft or hard tissues during hip movement. In many cases, hip movement is approximated as rotation about an estimated hip center. In this paper, we investigate the effect of different methods used for estimating hip joint center of rotation on the...

This paper essentially tries to examine all the roles that Virtual Humans can play in empowering human expression, and the research challenges we have to face to make this possible. It starts with a short history of Virtual Humans and how we contribute to the foundations of this field. We then define six typical Virtual Humans: the Performing Virtu...

Emotions are an important part of most societal dynamics. As with face to face meetings, Internet exchanges may not only include factual information but may also elicit emotional responses; how participants feel about the subject discussed or other group members. The development of automatic sentiment analysis has made large scale emotion detection...

Safety and efficiency are two major issues when it comes to long vehicle driving and operating, particularly those with very large lateral and longitudinal sizes. Trajectory planning for long vehicle tuning always plays an important role to secure safety and efficient operation, especially when turning in narrow spaces limited by surrounding object...

This paper is motivated by the objective of improving the realism of real-time simulated crowds by reducing short term collision avoidance through long term anticipation of pedestrian trajectories. For this aim, we choose to reuse outdoor pedestrian trajectories obtained with non-invasive means. This initial step is achieved by analyzing the record...

In order to be believable, embodied conversational agents (ECAs) must show expression of emotions in a consistent and natural looking way across modalities. The ECA has to be able to display coordinated signs of emotion during realistic emotional behaviour. Such a capability requires one to study and represent emotions and coordination of modalitie...

This paper presents a general-purpose simula-tion approach integrating a set of technological develop-ments and algorithmic methods in cellular automata (CA) domain. The approach provides a general-purpose comput-ing on graphics processor units (GPGPU) implementation for computing and multiple rendering of any direct-neighbor three-dimensional (3D)...

The acquisition of surround-view panoramas using a single hand-held or head-worn camera relies on robust real-time camera orientation tracking and relocalization. This paper presents robust methodology and evaluation for camera orientation relocalization, ...

Gesture recognition is becoming a popular way of interaction, but still suffers of important drawbacks to be integrated in everyday life devices. One of these drawbacks is the activation of the recognition system -- trigger gesture - which is generally tiring and unnatural. In this paper, we propose two natural solutions to easily activate the gest...

In this paper, we present an action selection mechanism to simulate the human behavior in realistic and believable virtual humans according to their personality and affective state taking into account their beliefs, desires, intentions, and the level of intensity of the events they perceive from their environment.

The use of pose and proportion to represent character traits is well established in art and psychology literature. However, there are no golden rules that quantify a generic design template for stylized character figure drawing. Given the wide variety ...

We present a visualization framework for exploring and analyzing data sets from biomechanical and neuromuscular simulations. These data sets describe versatile information related to the different stages of a motion analysis. In studying these data using a D visualization approach, interactive exploring is important, especially for supporting spati...

Today neuromuscular simulations are used in several fields, such as diagnostics and planing of surgery, to get a deeper understanding of the musculoskeletal system. During the last year, new models and datasets have been presented which can provide us with more in-depth simulations and results. The same kind of development has occurred in the field...

The last ten years have seen an explosion in the diversity of digital-life devices, e.g. music and video players. However, the interaction paradigm to use these devices has remained mostly unchanged. Remote controls are still the most common way to manage a digital-life device. Moreover, the interaction between devices themselves is still very limi...

The communication between avatar and agent has already been treated from different but specialized perspectives. In contrast, this paper gives a balanced view of every key architectural aspect: from text analysis to computer graphics, the chatting system and the emotional model. Non-verbal communication, such as facial expression, gaze, or head ori...

In this chapter, we present a three-layered model based on the Triune Brain Model to simulate human brain functioning and human beings’ behavior in realistic virtual humans. In order to implement this model, we use the ten personality scales defined by Minnesota Multiphasic Personality Inventory and the Emotional Competence Framework defined in the...

Digital media are more and more present in our lives, but we are still waiting for interfaces and devices completely adapted to this content. In this paper, we present the virtual shelf, an application dedicated to interact with our digital music collection. Our concept is to visualize our music collection in a familiar environment, a classic CD sh...

Our goal is to create dancing crowds in cyber worlds and to use this feature to support creative endeavors such as pre-visualization of choreography and actual stage performances. In this paper we present a method of motion planning using dance motion clips. We describe a trial algorithm of collision avoidance using a grid map. We also present meth...

In this paper, we present a fuzzy mechanism to update in a more natural way the emotional and mood states of virtual humans. To implement this mechanism, we take into account the ten personality scales defined by Minnesota Multiphasic Personality Inventory to endow virtual humans with a real personality. In this manner, we apply different sets of f...

In the recent years, usage of the third-person perspective (3PP) in virtual training methods has become increasingly viable and despite the growing interest in virtual reality and graphics underlying third-person perspective usage, not many studies have systematically looked at the dynamics and differences between the third and first-person perspec...

This paper presents a general-purpose simulation approach integrating a set of technological developments and algorithmic methods in cellular automata (CA) domain. The approach provides a general-purpose comput- ing on graphics processor units (GPGPU) implementation for computing and multiple rendering of any direct-neighbor three-dimensional (3D)...

Increased fatigue of pilots during long flights can place both humans and machine at high risk. In this paper, we describe our research on a SymBodic (SYMbiotic BODies) system designed to minimize pilot fatigue in a simulated 48 hour mission. The system detected the pilot's sleep breaks and used this information to plan future sleep breaks. When fa...

Over the years, different approaches have been explored to build effective learning methods in virtual reality but the design of effective 3D manipulation techniques still remains an important research problem. To this end, it is important to quantify behavioral and brain mechanisms underlying the geometrical mappings of the body with the environme...

We introduce a new method for computerized real-time evaluation of femoroacetabular impingement (FAI). In contrast to previously presented stress analyses, this method is based on two types of predictions of penetration depths for two rotating bodies: curvilinear and radial penetration depth. This intuitive method allows the analysis of both bony a...

This paper presents a novel concept: a graphical representation of human emotion extracted from text sentences. The major contributions of this paper are the following. First, we present a pipeline that extracts, processes, and renders emotion of 3D virtual human (VH). The extraction of emotion is based on data mining statistic of large cyberspace...

In this paper we propose a novel wearable architecture based on the popular and proven I2C protocol. Our goal is to develop a modular approach towards wearable computing in a sense that users simply attach specific modules to their garments depending on their requested functionalities. The I2C bus is embedded in the clothes and magnets are used as...

Overload on the hip joint is considered by medical experts as principal cause of hip joint osteo\-arthritis (OA) which is characterized by the degeneration of the articular cartilages. The aim of this study is to investigate the correlation between extreme movements and the development of early OA. Thus, the mechanical behavior of dancer's hip join...

In this paper we address the problem of creating accurate joint models from real motions while allowing scalability. We propose an automatic method to model, scale, and simulate non-idealized joints from the external motion of markers. We demonstrate the method on the human knee joint modeling for musculoskeletal analysis and for character animatio...

Model-based camera tracking is a technology that estimates a precise camera pose based on visual cues (e.g., feature points, edges) extracted from camera images given a 3D scene model and a rough camera pose. This paper proposes an automatic method for ...

In this paper, we present a new cognitive model based on Psychology for simulating human behavior in realistic virtual humans. To do this, we use the Minnesota Multiphasic Personality Inventory (MMPI), taking into account the personality scales defined in it to endow the virtual humans with a real personality and form a set of fuzzy rules used to o...

In this paper, we propose to increase realism and complexity of haptic applications for Virtual Reality and CAD applications. To achieve this goal, we use a Haptic Workstation which allows to acquire the posture and position of both hands, and to apply forces on the fingertips and wrists. We propose techniques to calibrate and improve the comfort o...

Motion LoD (Level of Detail) is a preprocessing technique that generates multiple details of captured motion by eliminating joints. This LoD technique is applied to movie, game or VR environments for the purpose of improving speed of crowd animation. So far, replacement techniques such as 'impostor' and 'rigid body motion' are widely used on real-t...