Daniel Thalmann

Daniel Thalmann
École Polytechnique Fédérale de Lausanne | EPFL · Computer Science Section

PhD Computer Science, Univ. Geneva

About

1,002
Publications
131,211
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
20,151
Citations
Introduction
Daniel Thalmann is with the Institute for Media Innovation at the Nanyang Technological University in Singapore. He is a pioneer in research on Virtual Humans. His current research interests include Real-time Virtual Humans in Virtual Reality, crowd simulation, and 3D Interaction. Daniel Thalmann has been the Founder of The Virtual Reality Lab (VRlab) at EPFL. He is coeditor-in-chief of the Journal of Computer Animation and Virtual Worlds, and member of the editorial board of 6 other journals.
Additional affiliations
August 2009 - present
Nanyang Technological University
Position
  • Professor (Full)
Education
September 1972 - March 1977
University of Geneva
Field of study
  • Computer Science
October 1970 - June 1972
University of Geneva
Field of study
  • Numerical Analysis, Statistics
October 1965 - June 1970
University of Geneva
Field of study
  • Physics

Publications

Publications (1,002)
Conference Paper
Schizophrenia and depression are the two most common mental disorders associated with negative symptoms that contribute to poor functioning and quality of life for millions of patients globally. This study is part of a larger research project. The overall aim of the project is to develop an automated objective pipeline that aids clinical diagnosis...
Chapter
Many on-line communities still rely on text messages as their privileged medium of communication and we believe it will remain so even if it can be complemented by images and video. Previous chapters have described how to infer emergent emotion dynamics from text corpus whereas we focus here on the instantaneous display of a detected emotion with a...
Conference Paper
This paper presents an immersive volleyball game, where a player plays not only against virtual opponents but also with support on his/her side of virtual teammates. This volleyball game has been implemented for several 3D displays such as a stereoscopic display, an autostereoscopic display, Oculus Rift glasses, and a 320o Immersive Room. In this p...
Conference Paper
Full-text available
Wearable devices such as Microsoft Hololens and Google glass are highly popular in recent years. As traditional input hardware is difficult to use on such platforms, vision-based hand pose tracking and gesture control techniques are more suitable alternatives. This demo shows the possibility to interact with 3D contents with bare hands on wearable...
Conference Paper
This paper proposes a real time sociometric system to analyze social behavior from audio-visual recordings of two-person face-to-face conversations in English. The novelty of the proposed system lies in this automatic inference of ten social indicators in real time. The system comprises of a Microsoft kinect device that captures RGB and depth data...
Article
Expanding the spectrum of agent social capabilities is an important challenge in agent-based simulation and other domains. While human-like emotionality has been vastly explored in the last 20years, little research addresses explicit, psychologically believable social situation modeling. Recently, some important elements have been underlined: hybri...
Conference Paper
Full-text available
Human activity recognition in unconstrained RGB-D videos has extensive applications in surveillance, multimedia data analytics, human-computer interaction, etc, but remains a challenging prob-lem due to the background clutter, camera motion, viewpoint changes, etc. We develop a novel RGB-D activity recognition ap-proach that leverages the dense tra...
Article
User ratings are the essence of recommender systems in e-commerce. Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of such systems and contribute to the well-known data sparsity and cold start problems. This article proposes a new information source for recommender systems, call...
Chapter
Healthcare can be brought to a principally new level by creating virtual bodies of real patients and using them for diagnosis and treatment planning along with the real patients. It will help to improve the quality of healthcare services, and reduce the healthcare cost, especially when dealing with the rise in musculoskeletal disorders in the aging...
Article
The precise evaluation of joint movements, using reconstructed three-dimensional models of the joint tissues, is an important issue in many medical diagnosis and surgery planning. In order to achieve appropriate results by analyzing 3D joint models, efficient and accurate evaluation of contact among the virtual tissues is critical. In this chapter,...
Article
Hand pose tracking and gesture recognition are useful for human-computer interaction, while a major problem is the lack of discriminative features for compact hand representation. We present a robust hand parsing scheme to extract a high-level description of the hand from the depth image. A novel distance-adaptive selection method is proposed to ge...
Article
In this paper, a human-robot interaction system based on a novel combination of sensors is proposed. It allows one person to interact with a humanoid social robot using natural body language. The robot understands the meaning of human upper body gestures and expresses itself by using a combination of body movements, facial expressions, and verbal l...
Article
Full-text available
New technical innovations in telepresence allow people to attend meetings or gather together anywhere they are and to share the same virtual space using 3D avatars. More sophisticated systems have been developed with 3D capturing and rendering, leading to a true 3D telepresence experience. If the 3D avatar is remotely guided by the real participant...
Article
Simulating a crowded scene like a busy shopping street requires tight packing of virtual characters. In such cases, collisions are likely to occur, and the choice in collision detection shape will influence how characters are allowed to intermingle. Full collision detection is too expensive for crowds, so simplifications are needed. The most common...
Conference Paper
Full-text available
Trust has been extensively studied and its effectiveness demon-strated in recommender systems. Due to lack of explicit trust information in most systems, many trust metric ap-proaches have been proposed to infer implicit trust from user ratings. However, previous works have not compared these different approaches, and oftentimes focus only on the p...
Chapter
Full-text available
In this chapter, we present a system that provides real-time feedback about an ongoing discussion. Various speech statistics, such as speaking length, speaker turns and speaking turn duration, are computed and displayed in real time. In social monitoring, such statistics have been used to interpret and deduce talking mannerisms of people and gain i...
Article
Full-text available
Pompeii was a Roman city, destroyed and completely buried during an eruption of the volcano Mount Vesuvius. We have revived its past by creating a D model of its previous appearance and populated it with crowds of Virtual Romans. In this paper, we detail the process, based on archaeological data, to simulate ancient Pompeii life in real time. In a...
Conference Paper
Full-text available
This paper presents a parallel texture synthesis over arbitrary surfaces, generating consistent and spatially-varying visual appearances. A novel scaling field is represented to measure the geometry-aware appearance or geometric deformation, therefore the generated textures locally agree with the geometric structure and maintain the coherence durin...
Conference Paper
In this tutorial, we will describe both virtual characters and realistic humanoid social robots using the same high level models. Particularly, we will describe: 1. How to capture real-time gestures and facial emotions from real people, how to recognize any real person, how to recognize certain sounds. We will present a state of the art and some ne...
Chapter
Full-text available
In neurofeedback, brain waves are transformed into sounds or music, graphics, and other representations, to provide real-time information on ongoing waves and patterns in the brain. Here we present various forms of neurofeedback, including sonification, sonification in combination with visualization, and at last, immersive neurofeedback, where audi...
Article
In this paper, we propose a method to achieve effective facial emotional expressivity for embodied conversational agents by considering two types of asymmetry when exploiting the valence–arousal–dominance representation of emotions. Indeed, the asymmetry of facial expressions helps to convey complex emotional feelings such as conflicting and/or hid...
Article
Content-based human motion retrieval is important for animators with the development of motion editing and synthesis, which need to search similar motions in large databases. Obtaining text-based representation from quantization of mocap data turned ...
Conference Paper
Full-text available
A real-time system is proposed to quantitatively assess speaking mannerisms and social behavior from audio recordings of two-person dialogs. Speaking mannerisms are quantitatively assessed by low-level speech metrics such as volume, rate, and pitch of speech. The social behavior is quantified by sociometrics including level of interest, agreement,...
Article
Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of recommender systems and contribute to the well-known data sparsity and cold start problems. This paper proposes a new information source for recommender systems, called prior ratings. Prior ratings are based on users' experience...
Conference Paper
Full-text available
In this paper, a novel human-virtual human interaction system is proposed. This system supports a real human to communicate with a virtual human using natural body language. Meanwhile, the virtual human is capable of understanding the meaning of human upper body gestures and reacting with its own personality by the means of body action, facial expr...
Article
In this paper we present a novel vision-based markerless hand pose estimation scheme with the input of depth image sequences. The proposed scheme exploits both temporal constraints and spatial features of the input sequence, and focuses on hand parsing and 3D fingertip localization for hand pose estimation. The hand parsing algorithm incorporates a...
Conference Paper
Full-text available
Despite the demand for culturally placed agent models, an adequate simulation approach to the relationship between group-cultural and individual-psychological qualities, including culture emergence, is just appearing. It could be argued that we are at the beginning of a domain forming process, a dawn of generative, emergent artificial culture. In t...
Article
Full-text available
We explore an approach to full-body motion editing with linear motion models, prioritized constraint-based optimization and latent-space interpolation. By exploiting the mathematical connections between linear motion models and prioritized inverse kinematics (PIK), we formulate and solve the motion editing problem as an optimization function whose...
Article
Full-text available
The recent proliferation of virtual reality (VR) technology applications in the autism therapy to promote learning and positive behavior among such children has produced optimistic results in developing a variety of skills and abilities in them. Dolphin-assisted therapy has also become a topic of public and research interest for autism intervention...
Article
Providing high quality recommendations is important for e-commerce systems to assist users in making effective selection decisions from a plethora of choices. Collaborative filtering is a widely accepted technique to generate recommendations based on the ratings of like-minded users. However, it suffers from several inherent issues such as data spa...
Chapter
In this chapter, we focus on the technical aspect of real-time crowd simulation, and how an efficient architecture can be developed. But first, it is important to identify the main goals to achieve. We need: Quantity: real-time simulation of thousands of characters, Quality: state-of-the-art virtual humans, Efficiency: storage and real-time managem...
Chapter
One of the important characteristics of behavioral animation is its ability to reduce the workload on the animators. This is achieved by letting a behavioral model automatically take care of the low-level details of the animation, freeing the animator to concentrate on the big picture. Freeing the animator from low-level animation details is even m...
Article
While designing an animation engine usable in crowd simulations, several criteria have to be taken into account: animation computation should be efficient and scalable, it should allow for variability, and it should be compatible with levels of detail. To understand the problems, let us take the most important animation pattern used in crowd simula...
Conference Paper
Full-text available
In this paper, we first present our crowd simulation method, Trajectory Variant Shift (TVS) based on real pedestrian trajectories re-use. We detail how to re-use and shift these trajectories to avoid collisions while retaining the liveliness of captured data. Second, we conducted a user study in a four-screen CAVE to compare our approach with three...
Conference Paper
In this paper we present a model-based framework for hand pose estimation, which relies on the depth and color image sequence input. The proposed framework adopts a divide-and-conquer scheme, and combines fingertip tracking and articulated iterative closest point approach to restore the hand motion. The tracked fingertip positions are used to provi...
Article
Traversing dense crowds can be challenging, but it is especially difficult in a virtual environment where the user is limited to simple input devices. Predicting near-future user movements with a hidden Markov model allows nearby autonomous agents to react politely, that is, to specifically avoid impeding the movement of the user. This predictive m...
Conference Paper
Full-text available
We present a vision-based approach for robust 3D fingertip and palm tracking on depth images using a single Kinect sensor. First the hand is segmented in the depth images by applying depth and morphological constraints. The palm is located by performing distance transform to the hand contour and tracked with a Kalman filter. The fingertips are dete...
Conference Paper
Full-text available
Robot localization is a key barrier to providing natural interaction between 3D virtual characters, human users and mobile robots. Knowing where the robot is, relative to a known world-frame, is essential to directed gestures, gazes and expressions between the robot and the other real and virtual participants in a telepresence system. The intrinsic...
Conference Paper
Full-text available
This paper presents an interactive dynamic controller used to generate locomotion patterns for humanoid robots. The purpose of this work is to provide animators and artists easy and intuitive tools to design expressive motions for humanoid robots. A review of similar work in the computer animation community has guided our choices regarding the impl...
Conference Paper
Providing high quality recommendations is important for online systems to assist users who face a vast number of choices in making effective selection decisions. Collaborative filtering is a widely accepted technique to provide recommendations based on ratings of similar users. But it suffers from several issues like data sparsity and cold start. T...
Conference Paper
Full-text available
This paper proposes a new emotional model for Virtual Humans (VHs) in a conversational environment. As a part of a multi-users emotional 3D-chatting system, this paper focus on how to formulate and visualize the flow of emotional state defined by the Valence-Arousal-Dominance (VAD) parameters. From this flow of emotion over time, we successfully vi...
Conference Paper
Full-text available
Non-verbal communication (NVC) makes up about two-thirds of all communication between two people or between one speaker and a group of listeners. However, this fundamental aspect of communicating is mostly omitted in 3D social forums or virtual world oriented games. This paper proposes an answer by presenting a multi-user 3D-chatting system enriche...
Conference Paper
Full-text available
Autonomous virtual characters and social robots are meant to interact with humans. They should be able to communicate, express emotions and exhibit personality. Their social skills are highly dependant on their physical design, as well as on their motion capabilities. This paper presents a comparative survey of design choices and motion generation...
Conference Paper
Since the identification of the first cases of Autism Spectrum Disorder (ASD) in 1943 by Dr Leo Kanner, the number of reported ASD cases as well as research studies expanding our understanding of the disorder has increased rapidly and significantly. In terms of ASD treatment, there is a wide range of conventional as well as complementary and altern...
Conference Paper
We present an interface for navigating digital collections based on a one-dimensional analog control and a data visualization based on old analog radios. Our system takes advantage of inertial control to browse a large data collection in a compelling way, reducing the complexity of similar interfaces present in both desktop-based and portable media...
Article
This paper presents an interactive multi-agent system based on a fully immersive virtual environment. A user can interact with the virtual characters in real time via an avatar by changing their moving behavior. Moreover, the user is allowed to select any character as the avatar to be controlled. A path planning algorithm is proposed to address the...
Conference Paper
Full-text available
Autonomous virtual characters and social robots are meant to interact with humans. They should be able to communicate, express emotions and exhibit personality. Their social skills are highly dependent on their physical design, as well as on their motion capabilities. This paper presents a comparative survey of design choices and motion generation...
Article
Computer-based simulations of human hip joints generally include investigating contacts happening among soft or hard tissues during hip movement. In many cases, hip movement is approximated as rotation about an estimated hip center. In this paper, we investigate the effect of different methods used for estimating hip joint center of rotation on the...
Article
This paper essentially tries to examine all the roles that Virtual Humans can play in empowering human expression, and the research challenges we have to face to make this possible. It starts with a short history of Virtual Humans and how we contribute to the foundations of this field. We then define six typical Virtual Humans: the Performing Virtu...
Conference Paper
Full-text available
Emotions are an important part of most societal dynamics. As with face to face meetings, Internet exchanges may not only include factual information but may also elicit emotional responses; how participants feel about the subject discussed or other group members. The development of automatic sentiment analysis has made large scale emotion detection...
Chapter
Safety and efficiency are two major issues when it comes to long vehicle driving and operating, particularly those with very large lateral and longitudinal sizes. Trajectory planning for long vehicle tuning always plays an important role to secure safety and efficient operation, especially when turning in narrow spaces limited by surrounding object...
Conference Paper
Full-text available
This paper is motivated by the objective of improving the realism of real-time simulated crowds by reducing short term collision avoidance through long term anticipation of pedestrian trajectories. For this aim, we choose to reuse outdoor pedestrian trajectories obtained with non-invasive means. This initial step is achieved by analyzing the record...
Chapter
In order to be believable, embodied conversational agents (ECAs) must show expression of emotions in a consistent and natural looking way across modalities. The ECA has to be able to display coordinated signs of emotion during realistic emotional behaviour. Such a capability requires one to study and represent emotions and coordination of modalitie...
Article
Full-text available
This paper presents a general-purpose simula-tion approach integrating a set of technological develop-ments and algorithmic methods in cellular automata (CA) domain. The approach provides a general-purpose comput-ing on graphics processor units (GPGPU) implementation for computing and multiple rendering of any direct-neighbor three-dimensional (3D)...
Article
Full-text available
The acquisition of surround-view panoramas using a single hand-held or head-worn camera relies on robust real-time camera orientation tracking and relocalization. This paper presents robust methodology and evaluation for camera orientation relocalization, ...