Daniel Thalmann

Daniel Thalmann
  • PhD Computer Science, Univ. Geneva
  • Professor at Swiss Federal Institute of Technology in Lausanne

About

1,003
Publications
180,184
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
21,858
Citations
Introduction
Daniel Thalmann is with the Institute for Media Innovation at the Nanyang Technological University in Singapore. He is a pioneer in research on Virtual Humans. His current research interests include Real-time Virtual Humans in Virtual Reality, crowd simulation, and 3D Interaction. Daniel Thalmann has been the Founder of The Virtual Reality Lab (VRlab) at EPFL. He is coeditor-in-chief of the Journal of Computer Animation and Virtual Worlds, and member of the editorial board of 6 other journals.
Current institution
Swiss Federal Institute of Technology in Lausanne
Current position
  • Professor
Additional affiliations
August 2009 - present
Nanyang Technological University
Position
  • Professor (Full)
Education
September 1972 - March 1977
University of Geneva
Field of study
  • Computer Science
October 1970 - June 1972
University of Geneva
Field of study
  • Numerical Analysis, Statistics
October 1965 - June 1970
University of Geneva
Field of study
  • Physics

Publications

Publications (1,003)
Chapter
Schizophrenia is a Xu, Shihao Yang, Zixu Chakraborty, Debsubhra that contributes to Tahir, Yasir Maszczyk, Tomasz Victoria, Chua Yi Han Dauwels, Justin Thalmann, Daniel life. Thalmann, Nadia Magnenat Tan, Bhing-Leet Keong, Jimmy Lee Chee to design objective assessment tools of schizophrenia. In earlier work, we investigated non-verbal quantitative...
Conference Paper
Schizophrenia and depression are the two most common mental disorders associated with negative symptoms that contribute to poor functioning and quality of life for millions of patients globally. This study is part of a larger research project. The overall aim of the project is to develop an automated objective pipeline that aids clinical diagnosis...
Chapter
Many on-line communities still rely on text messages as their privileged medium of communication and we believe it will remain so even if it can be complemented by images and video. Previous chapters have described how to infer emergent emotion dynamics from text corpus whereas we focus here on the instantaneous display of a detected emotion with a...
Conference Paper
This paper presents an immersive volleyball game, where a player plays not only against virtual opponents but also with support on his/her side of virtual teammates. This volleyball game has been implemented for several 3D displays such as a stereoscopic display, an autostereoscopic display, Oculus Rift glasses, and a 320o Immersive Room. In this p...
Conference Paper
Full-text available
Wearable devices such as Microsoft Hololens and Google glass are highly popular in recent years. As traditional input hardware is difficult to use on such platforms, vision-based hand pose tracking and gesture control techniques are more suitable alternatives. This demo shows the possibility to interact with 3D contents with bare hands on wearable...
Conference Paper
This paper proposes a real time sociometric system to analyze social behavior from audio-visual recordings of two-person face-to-face conversations in English. The novelty of the proposed system lies in this automatic inference of ten social indicators in real time. The system comprises of a Microsoft kinect device that captures RGB and depth data...
Article
Expanding the spectrum of agent social capabilities is an important challenge in agent-based simulation and other domains. While human-like emotionality has been vastly explored in the last 20years, little research addresses explicit, psychologically believable social situation modeling. Recently, some important elements have been underlined: hybri...
Conference Paper
Full-text available
Human activity recognition in unconstrained RGB-D videos has extensive applications in surveillance, multimedia data analytics, human-computer interaction, etc, but remains a challenging prob-lem due to the background clutter, camera motion, viewpoint changes, etc. We develop a novel RGB-D activity recognition ap-proach that leverages the dense tra...
Article
User ratings are the essence of recommender systems in e-commerce. Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of such systems and contribute to the well-known data sparsity and cold start problems. This article proposes a new information source for recommender systems, call...
Chapter
Healthcare can be brought to a principally new level by creating virtual bodies of real patients and using them for diagnosis and treatment planning along with the real patients. It will help to improve the quality of healthcare services, and reduce the healthcare cost, especially when dealing with the rise in musculoskeletal disorders in the aging...
Article
The precise evaluation of joint movements, using reconstructed three-dimensional models of the joint tissues, is an important issue in many medical diagnosis and surgery planning. In order to achieve appropriate results by analyzing 3D joint models, efficient and accurate evaluation of contact among the virtual tissues is critical. In this chapter,...
Article
Hand pose tracking and gesture recognition are useful for human-computer interaction, while a major problem is the lack of discriminative features for compact hand representation. We present a robust hand parsing scheme to extract a high-level description of the hand from the depth image. A novel distance-adaptive selection method is proposed to ge...
Article
In this paper, a human-robot interaction system based on a novel combination of sensors is proposed. It allows one person to interact with a humanoid social robot using natural body language. The robot understands the meaning of human upper body gestures and expresses itself by using a combination of body movements, facial expressions, and verbal l...
Article
Full-text available
New technical innovations in telepresence allow people to attend meetings or gather together anywhere they are and to share the same virtual space using 3D avatars. More sophisticated systems have been developed with 3D capturing and rendering, leading to a true 3D telepresence experience. If the 3D avatar is remotely guided by the real participant...
Article
Simulating a crowded scene like a busy shopping street requires tight packing of virtual characters. In such cases, collisions are likely to occur, and the choice in collision detection shape will influence how characters are allowed to intermingle. Full collision detection is too expensive for crowds, so simplifications are needed. The most common...
Conference Paper
Full-text available
Trust has been extensively studied and its effectiveness demon-strated in recommender systems. Due to lack of explicit trust information in most systems, many trust metric ap-proaches have been proposed to infer implicit trust from user ratings. However, previous works have not compared these different approaches, and oftentimes focus only on the p...
Chapter
Full-text available
In this chapter, we present a system that provides real-time feedback about an ongoing discussion. Various speech statistics, such as speaking length, speaker turns and speaking turn duration, are computed and displayed in real time. In social monitoring, such statistics have been used to interpret and deduce talking mannerisms of people and gain i...
Article
Full-text available
Pompeii was a Roman city, destroyed and completely buried during an eruption of the volcano Mount Vesuvius. We have revived its past by creating a D model of its previous appearance and populated it with crowds of Virtual Romans. In this paper, we detail the process, based on archaeological data, to simulate ancient Pompeii life in real time. In a...
Conference Paper
Full-text available
This paper presents a parallel texture synthesis over arbitrary surfaces, generating consistent and spatially-varying visual appearances. A novel scaling field is represented to measure the geometry-aware appearance or geometric deformation, therefore the generated textures locally agree with the geometric structure and maintain the coherence durin...
Conference Paper
In this tutorial, we will describe both virtual characters and realistic humanoid social robots using the same high level models. Particularly, we will describe: 1. How to capture real-time gestures and facial emotions from real people, how to recognize any real person, how to recognize certain sounds. We will present a state of the art and some ne...
Chapter
Full-text available
In neurofeedback, brain waves are transformed into sounds or music, graphics, and other representations, to provide real-time information on ongoing waves and patterns in the brain. Here we present various forms of neurofeedback, including sonification, sonification in combination with visualization, and at last, immersive neurofeedback, where audi...
Article
In this paper, we propose a method to achieve effective facial emotional expressivity for embodied conversational agents by considering two types of asymmetry when exploiting the valence–arousal–dominance representation of emotions. Indeed, the asymmetry of facial expressions helps to convey complex emotional feelings such as conflicting and/or hid...
Article
Content-based human motion retrieval is important for animators with the development of motion editing and synthesis, which need to search similar motions in large databases. Obtaining text-based representation from quantization of mocap data turned ...
Conference Paper
Full-text available
A real-time system is proposed to quantitatively assess speaking mannerisms and social behavior from audio recordings of two-person dialogs. Speaking mannerisms are quantitatively assessed by low-level speech metrics such as volume, rate, and pitch of speech. The social behavior is quantified by sociometrics including level of interest, agreement,...
Article
Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of recommender systems and contribute to the well-known data sparsity and cold start problems. This paper proposes a new information source for recommender systems, called prior ratings. Prior ratings are based on users' experience...
Conference Paper
Full-text available
In this paper, a novel human-virtual human interaction system is proposed. This system supports a real human to communicate with a virtual human using natural body language. Meanwhile, the virtual human is capable of understanding the meaning of human upper body gestures and reacting with its own personality by the means of body action, facial expr...
Article
In this paper we present a novel vision-based markerless hand pose estimation scheme with the input of depth image sequences. The proposed scheme exploits both temporal constraints and spatial features of the input sequence, and focuses on hand parsing and 3D fingertip localization for hand pose estimation. The hand parsing algorithm incorporates a...
Conference Paper
Full-text available
Despite the demand for culturally placed agent models, an adequate simulation approach to the relationship between group-cultural and individual-psychological qualities, including culture emergence, is just appearing. It could be argued that we are at the beginning of a domain forming process, a dawn of generative, emergent artificial culture. In t...
Article
Full-text available
We explore an approach to full-body motion editing with linear motion models, prioritized constraint-based optimization and latent-space interpolation. By exploiting the mathematical connections between linear motion models and prioritized inverse kinematics (PIK), we formulate and solve the motion editing problem as an optimization function whose...
Article
Full-text available
The recent proliferation of virtual reality (VR) technology applications in the autism therapy to promote learning and positive behavior among such children has produced optimistic results in developing a variety of skills and abilities in them. Dolphin-assisted therapy has also become a topic of public and research interest for autism intervention...
Article
Providing high quality recommendations is important for e-commerce systems to assist users in making effective selection decisions from a plethora of choices. Collaborative filtering is a widely accepted technique to generate recommendations based on the ratings of like-minded users. However, it suffers from several inherent issues such as data spa...
Chapter
In this chapter, we focus on the technical aspect of real-time crowd simulation, and how an efficient architecture can be developed. But first, it is important to identify the main goals to achieve. We need: Quantity: real-time simulation of thousands of characters, Quality: state-of-the-art virtual humans, Efficiency: storage and real-time managem...
Chapter
One of the important characteristics of behavioral animation is its ability to reduce the workload on the animators. This is achieved by letting a behavioral model automatically take care of the low-level details of the animation, freeing the animator to concentrate on the big picture. Freeing the animator from low-level animation details is even m...
Article
While designing an animation engine usable in crowd simulations, several criteria have to be taken into account: animation computation should be efficient and scalable, it should allow for variability, and it should be compatible with levels of detail. To understand the problems, let us take the most important animation pattern used in crowd simula...
Conference Paper
Full-text available
In this paper, we first present our crowd simulation method, Trajectory Variant Shift (TVS) based on real pedestrian trajectories re-use. We detail how to re-use and shift these trajectories to avoid collisions while retaining the liveliness of captured data. Second, we conducted a user study in a four-screen CAVE to compare our approach with three...
Conference Paper
In this paper we present a model-based framework for hand pose estimation, which relies on the depth and color image sequence input. The proposed framework adopts a divide-and-conquer scheme, and combines fingertip tracking and articulated iterative closest point approach to restore the hand motion. The tracked fingertip positions are used to provi...
Article
Traversing dense crowds can be challenging, but it is especially difficult in a virtual environment where the user is limited to simple input devices. Predicting near-future user movements with a hidden Markov model allows nearby autonomous agents to react politely, that is, to specifically avoid impeding the movement of the user. This predictive m...
Conference Paper
Full-text available
We present a vision-based approach for robust 3D fingertip and palm tracking on depth images using a single Kinect sensor. First the hand is segmented in the depth images by applying depth and morphological constraints. The palm is located by performing distance transform to the hand contour and tracked with a Kalman filter. The fingertips are dete...
Conference Paper
Full-text available
Robot localization is a key barrier to providing natural interaction between 3D virtual characters, human users and mobile robots. Knowing where the robot is, relative to a known world-frame, is essential to directed gestures, gazes and expressions between the robot and the other real and virtual participants in a telepresence system. The intrinsic...
Conference Paper
Full-text available
This paper presents an interactive dynamic controller used to generate locomotion patterns for humanoid robots. The purpose of this work is to provide animators and artists easy and intuitive tools to design expressive motions for humanoid robots. A review of similar work in the computer animation community has guided our choices regarding the impl...
Conference Paper
Providing high quality recommendations is important for online systems to assist users who face a vast number of choices in making effective selection decisions. Collaborative filtering is a widely accepted technique to provide recommendations based on ratings of similar users. But it suffers from several issues like data sparsity and cold start. T...
Conference Paper
Full-text available
This paper proposes a new emotional model for Virtual Humans (VHs) in a conversational environment. As a part of a multi-users emotional 3D-chatting system, this paper focus on how to formulate and visualize the flow of emotional state defined by the Valence-Arousal-Dominance (VAD) parameters. From this flow of emotion over time, we successfully vi...
Conference Paper
Full-text available
Non-verbal communication (NVC) makes up about two-thirds of all communication between two people or between one speaker and a group of listeners. However, this fundamental aspect of communicating is mostly omitted in 3D social forums or virtual world oriented games. This paper proposes an answer by presenting a multi-user 3D-chatting system enriche...
Conference Paper
Full-text available
Autonomous virtual characters and social robots are meant to interact with humans. They should be able to communicate, express emotions and exhibit personality. Their social skills are highly dependant on their physical design, as well as on their motion capabilities. This paper presents a comparative survey of design choices and motion generation...
Conference Paper
Full-text available
Since the identification of the first cases of Autism Spectrum Disorder (ASD) in 1943 by Dr Leo Kanner, the number of reported ASD cases as well as research studies expanding our understanding of the disorder has increased rapidly and significantly. In terms of ASD treatment, there is a wide range of conventional as well as complementary and altern...
Conference Paper
We present an interface for navigating digital collections based on a one-dimensional analog control and a data visualization based on old analog radios. Our system takes advantage of inertial control to browse a large data collection in a compelling way, reducing the complexity of similar interfaces present in both desktop-based and portable media...
Article
This paper presents an interactive multi-agent system based on a fully immersive virtual environment. A user can interact with the virtual characters in real time via an avatar by changing their moving behavior. Moreover, the user is allowed to select any character as the avatar to be controlled. A path planning algorithm is proposed to address the...
Conference Paper
Full-text available
Autonomous virtual characters and social robots are meant to interact with humans. They should be able to communicate, express emotions and exhibit personality. Their social skills are highly dependent on their physical design, as well as on their motion capabilities. This paper presents a comparative survey of design choices and motion generation...
Article
Computer-based simulations of human hip joints generally include investigating contacts happening among soft or hard tissues during hip movement. In many cases, hip movement is approximated as rotation about an estimated hip center. In this paper, we investigate the effect of different methods used for estimating hip joint center of rotation on the...
Article
This paper essentially tries to examine all the roles that Virtual Humans can play in empowering human expression, and the research challenges we have to face to make this possible. It starts with a short history of Virtual Humans and how we contribute to the foundations of this field. We then define six typical Virtual Humans: the Performing Virtu...
Conference Paper
Full-text available
Emotions are an important part of most societal dynamics. As with face to face meetings, Internet exchanges may not only include factual information but may also elicit emotional responses; how participants feel about the subject discussed or other group members. The development of automatic sentiment analysis has made large scale emotion detection...
Chapter
Safety and efficiency are two major issues when it comes to long vehicle driving and operating, particularly those with very large lateral and longitudinal sizes. Trajectory planning for long vehicle tuning always plays an important role to secure safety and efficient operation, especially when turning in narrow spaces limited by surrounding object...
Conference Paper
Full-text available
This paper is motivated by the objective of improving the realism of real-time simulated crowds by reducing short term collision avoidance through long term anticipation of pedestrian trajectories. For this aim, we choose to reuse outdoor pedestrian trajectories obtained with non-invasive means. This initial step is achieved by analyzing the record...
Chapter
Full-text available
In order to be believable, embodied conversational agents (ECAs) must show expression of emotions in a consistent and natural looking way across modalities. The ECA has to be able to display coordinated signs of emotion during realistic emotional behaviour. Such a capability requires one to study and represent emotions and coordination of modalitie...
Article
Full-text available
This paper presents a general-purpose simula-tion approach integrating a set of technological develop-ments and algorithmic methods in cellular automata (CA) domain. The approach provides a general-purpose comput-ing on graphics processor units (GPGPU) implementation for computing and multiple rendering of any direct-neighbor three-dimensional (3D)...
Article
Full-text available
The acquisition of surround-view panoramas using a single hand-held or head-worn camera relies on robust real-time camera orientation tracking and relocalization. This paper presents robust methodology and evaluation for camera orientation relocalization, ...
Conference Paper
Gesture recognition is becoming a popular way of interaction, but still suffers of important drawbacks to be integrated in everyday life devices. One of these drawbacks is the activation of the recognition system -- trigger gesture - which is generally tiring and unnatural. In this paper, we propose two natural solutions to easily activate the gest...
Article
Full-text available
In this paper, we present an action selection mechanism to simulate the human behavior in realistic and believable virtual humans according to their personality and affective state taking into account their beliefs, desires, intentions, and the level of intensity of the events they perceive from their environment.
Article
The use of pose and proportion to represent character traits is well established in art and psychology literature. However, there are no golden rules that quantify a generic design template for stylized character figure drawing. Given the wide variety ...
Article
Full-text available
We present a visualization framework for exploring and analyzing data sets from biomechanical and neuromuscular simulations. These data sets describe versatile information related to the different stages of a motion analysis. In studying these data using a D visualization approach, interactive exploring is important, especially for supporting spati...
Article
Full-text available
Today neuromuscular simulations are used in several fields, such as diagnostics and planing of surgery, to get a deeper understanding of the musculoskeletal system. During the last year, new models and datasets have been presented which can provide us with more in-depth simulations and results. The same kind of development has occurred in the field...
Article
Full-text available
The last ten years have seen an explosion in the diversity of digital-life devices, e.g. music and video players. However, the interaction paradigm to use these devices has remained mostly unchanged. Remote controls are still the most common way to manage a digital-life device. Moreover, the interaction between devices themselves is still very limi...
Conference Paper
Full-text available
The communication between avatar and agent has already been treated from different but specialized perspectives. In contrast, this paper gives a balanced view of every key architectural aspect: from text analysis to computer graphics, the chatting system and the emotional model. Non-verbal communication, such as facial expression, gaze, or head ori...
Chapter
In this chapter, we present a three-layered model based on the Triune Brain Model to simulate human brain functioning and human beings’ behavior in realistic virtual humans. In order to implement this model, we use the ten personality scales defined by Minnesota Multiphasic Personality Inventory and the Emotional Competence Framework defined in the...
Conference Paper
Digital media are more and more present in our lives, but we are still waiting for interfaces and devices completely adapted to this content. In this paper, we present the virtual shelf, an application dedicated to interact with our digital music collection. Our concept is to visualize our music collection in a familiar environment, a classic CD sh...
Conference Paper
Our goal is to create dancing crowds in cyber worlds and to use this feature to support creative endeavors such as pre-visualization of choreography and actual stage performances. In this paper we present a method of motion planning using dance motion clips. We describe a trial algorithm of collision avoidance using a grid map. We also present meth...
Conference Paper
In this paper, we present a fuzzy mechanism to update in a more natural way the emotional and mood states of virtual humans. To implement this mechanism, we take into account the ten personality scales defined by Minnesota Multiphasic Personality Inventory to endow virtual humans with a real personality. In this manner, we apply different sets of f...
Article
In the recent years, usage of the third-person perspective (3PP) in virtual training methods has become increasingly viable and despite the growing interest in virtual reality and graphics underlying third-person perspective usage, not many studies have systematically looked at the dynamics and differences between the third and first-person perspec...
Article
Full-text available
This paper presents a general-purpose simulation approach integrating a set of technological developments and algorithmic methods in cellular automata (CA) domain. The approach provides a general-purpose comput- ing on graphics processor units (GPGPU) implementation for computing and multiple rendering of any direct-neighbor three-dimensional (3D)...
Article
Increased fatigue of pilots during long flights can place both humans and machine at high risk. In this paper, we describe our research on a SymBodic (SYMbiotic BODies) system designed to minimize pilot fatigue in a simulated 48 hour mission. The system detected the pilot's sleep breaks and used this information to plan future sleep breaks. When fa...
Conference Paper
Over the years, different approaches have been explored to build effective learning methods in virtual reality but the design of effective 3D manipulation techniques still remains an important research problem. To this end, it is important to quantify behavioral and brain mechanisms underlying the geometrical mappings of the body with the environme...
Article
We introduce a new method for computerized real-time evaluation of femoroacetabular impingement (FAI). In contrast to previously presented stress analyses, this method is based on two types of predictions of penetration depths for two rotating bodies: curvilinear and radial penetration depth. This intuitive method allows the analysis of both bony a...
Article
Full-text available
This paper presents a novel concept: a graphical representation of human emotion extracted from text sentences. The major contributions of this paper are the following. First, we present a pipeline that extracts, processes, and renders emotion of 3D virtual human (VH). The extraction of emotion is based on data mining statistic of large cyberspace...
Conference Paper
In this paper we propose a novel wearable architecture based on the popular and proven I2C protocol. Our goal is to develop a modular approach towards wearable computing in a sense that users simply attach specific modules to their garments depending on their requested functionalities. The I2C bus is embedded in the clothes and magnets are used as...
Conference Paper
Full-text available
Overload on the hip joint is considered by medical experts as principal cause of hip joint osteo\-arthritis (OA) which is characterized by the degeneration of the articular cartilages. The aim of this study is to investigate the correlation between extreme movements and the development of early OA. Thus, the mechanical behavior of dancer's hip join...
Article
In this paper we address the problem of creating accurate joint models from real motions while allowing scalability. We propose an automatic method to model, scale, and simulate non-idealized joints from the external motion of markers. We demonstrate the method on the human knee joint modeling for musculoskeletal analysis and for character animatio...
Article
Model-based camera tracking is a technology that estimates a precise camera pose based on visual cues (e.g., feature points, edges) extracted from camera images given a 3D scene model and a rough camera pose. This paper proposes an automatic method for ...
Conference Paper
In this paper, we present a new cognitive model based on Psychology for simulating human behavior in realistic virtual humans. To do this, we use the Minnesota Multiphasic Personality Inventory (MMPI), taking into account the personality scales defined in it to endow the virtual humans with a real personality and form a set of fuzzy rules used to o...
Article
In this paper, we propose to increase realism and complexity of haptic applications for Virtual Reality and CAD applications. To achieve this goal, we use a Haptic Workstation which allows to acquire the posture and position of both hands, and to apply forces on the fingertips and wrists. We propose techniques to calibrate and improve the comfort o...
Article
Motion LoD (Level of Detail) is a preprocessing technique that generates multiple details of captured motion by eliminating joints. This LoD technique is applied to movie, game or VR environments for the purpose of improving speed of crowd animation. So far, replacement techniques such as 'impostor' and 'rigid body motion' are widely used on real-t...

Network

Cited By