Nadia Magnenat Thalmann

Nadia Magnenat Thalmann
University of Geneva | UNIGE · Centre Universitaire d'informatique

PhD Quantum Physics, UniGE
Director MIRALab, Universite de Geneve

About

1,350
Publications
237,196
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
22,288
Citations
Additional affiliations
July 2009 - July 2017
Nanyang Technological University
Position
  • Research Director
Description
  • I work at this place since 7 years sharing my time with MIRALab University of Geneva

Publications

Publications (1,350)
Article
Full-text available
Motivated by how humans perceive geometry and color to recognize objects, we propose a boundary embedded attentional convolution (BEACon) network for point cloud instance segmentation. At the core of BEACon, we introduce the attentional weight in the convolution layer to adjust the neighboring features, with the weight being adapted to the relation...
Chapter
Using intelligent virtual assistants for controlling employee population in workspaces is a research area that remains unexplored. This paper presents a novel application of virtual humans to enforce Covid-19 safety measures in a corporate workplace. For this purpose, we develop a virtual assistant platform, Chloe, equipped with automatic temperatu...
Chapter
More and more companies have started to use nonhuman agents for employment interviews, making the selection process easier, faster, and unbiased. To assess the effectiveness of the above, in this paper, we systematically analyzed, reviewed, and compared human interaction with a social robot, a digital human, and another human under the same scenari...
Article
Full-text available
Intangible heritage is often linked to human actions and performances. The use of digital humans (DHs) for its digital representation and therefore its preservation, allows reframing the way to transmit and deal with content that is difficult to visualize. To that end, the digital human becomes an important element establishing the connection betwe...
Article
Full-text available
The publication of this article unfortunately contained mistakes. The articles mentioned in the text were not the correct ones.
Article
Full-text available
A protocol for the representation of traditional crafts and the tools to implement this are proposed. The proposed protocol is a method for the systematic collection and organization of digital assets and knowledge, their representation into a formal model, and their utilization for research, education, and preservation. A set of digital tools acco...
Article
Sketch recognition relies on two types of information, namely, spatial contexts like the local structures in images and temporal contexts like the orders of strokes. Existing methods usually adopt convolutional neural networks (CNNs) to model spatial contexts, and recurrent neural networks (RNNs) for temporal contexts. However, most of them combine...
Article
Full-text available
Optical see-through near-eye display (NED) technologies for augmented reality (AR) have achieved significant advancements recently with investments from both academia and industry. Although various AR NED products have been successfully commercialized and even deployed into applications, there are still challenges with present AR NED technologies (...
Article
Full-text available
In this paper, a representation based on digital assets and semantic annotations is established for Traditional Craft instances, in a way that captures their socio-historic context and preserves both their tangible and intangible Cultural Heritage dimensions. These meaningful and documented experiential presentations are delivered to the target aud...
Article
Full-text available
This work regards the representation of handicrafts for craft training and demonstration in the environment of an ethnographic heritage museum. The craft of mastic cultivation is chosen as a use case. This paper presents the process of representation and presentation of this craft, following an articulated pipeline approach for data collection, ann...
Article
Full-text available
Traditional crafts exhibit tangible and intangible dimensions. Intangible dimensions include the practitioner’s gestural know-how in craft practice and have received smaller attention than tangible dimensions in digitization projects. This work presents the process of representation and presentation of the glasswork and is exemplified in the re-cre...
Chapter
Full-text available
From antiquity to today, some scientists, writers, and artists are passionate about representing humans not only as beautiful statues but as automatons that can perform actions. Already in ancient Greece, we can find some examples of automatons that replaced servants. In this chapter, we go through the development of automatons until the social rob...
Article
Full-text available
Social Assistive Robotics is increasingly being used in care settings to provide psychosocial support and interventions for the elderly with cognitive impairments. Most of these social robots have provided timely stimuli to the elderly at home and in care centres, including keeping them active and boosting their mood. However, previous investigatio...
Preprint
Full-text available
In this paper, we present the design and development of MIDAS. MIDAS is intended to be a portable immersive system inclusive of an exoskeleton, a smell releasing device, and a VR game that will help stroke patients feel more motivated during a hand rehabilitation session. MIDAS engages four out of five senses during rehabilitation. MIDAS uses EMG s...
Preprint
Full-text available
In this paper, we present the design and development of MIDAS. MIDAS is intended to be a portable immersive system inclusive of an exoskeleton, a smell releasing device, and a VR game that will help stroke patients feel more motivated during a hand rehabilitation session. MIDAS engages four out of five senses during rehabilitation. MIDAS uses EMG s...
Chapter
This paper describes three case studies performed in everyday life with our social humanoid robot, Nadine. The development of AI, vision, and NLP over the last few years has made it possible to improve social robots’ awareness ability of their environment and has enabled them to understand spoken interactions. We have developed a software platform...
Article
Full-text available
Virtual Humans are becoming a commodity in computing technology and lately have been utilized in the context of interactive presentations in Virtual Cultural Heritage environments and exhibitions. To this end, this research work underlines the importance of aligning and fine-tuning Virtual Humans’ appearance to their roles and highlights the import...
Chapter
There are considerable advancements in medical health care in recent years, resulting in a rising older population. As the workforce for such a population is not keeping pace, there is an urgent need to address this problem. Having robots to stimulating recreational activities for older adults can reduce the workload for caretakers and give them ti...
Article
Full-text available
Current research of designing prosthetic robotic hands mainly focuses on improving their functionality by devising new mechanical structures and actuation systems. Most of existing work relies on a single structure/system (e.g., bone-only or tissue-only) and ignores the fact that the human hand is composed of multiple functional structures (e.g., s...
Article
Full-text available
In the field of robotic hand design, soft body and anthropomorphic design are two trends with a promising future. Designing soft body anthropomorphic robotic hands with human-like grasping ability, but with a simple and reliable structure, is a challenge that still has not been not fully solved. In this paper, we present an anatomically correct rob...
Article
Recent methods including CoViAR and DMC-Net provide a new paradigm for action recognition since they are directly targeted at compressed videos (e.g., MPEG4 files). It avoids the cumbersome decoding procedure of traditional methods, and leverages the pre-encoded motion vectors and residuals in compressed videos to complete recognition efficiently....
Chapter
The current chapter serves as a state-of-the-art, presenting the limitations of the existing technology used in the broad area of human–computer interaction up to now. Although different kind of agents have been used to contribute to several domains, like education, health, entertainment, both in virtual and physical environments, the virtual chara...
Chapter
As one of the fundamental problems of scene understanding and modeling, object detection has attracted extensive attention in the research communities of computer vision and artificial intelligence. Recently, inspired by the success of deep learning, various deep neural network-based models have been proposed and become the de facto solution for ob...
Chapter
Socially intelligent robots are dedicated mostly for social interaction with humans. They can assume two useful roles: a functional and an affective one. They aim to serve as an interface between humans and technology, and to increase their quality of life by providing companionship and assisting them in everyday tasks and routines. Although there...
Chapter
Full-text available
Visualization techniques for the usage of tools, handicrafts, and assembly operations are employed for demonstrating processes (e.g., assembly instructions). Until today, most commonly used techniques include written information, sketches in manuals, video instructions, etc. The evolution of technology nowadays has generated mature methods for tran...
Chapter
With recent developments in the field of artificial intelligence, machine learning, and deep learning, the field of social robotics has gained momentum. Any social robot requires to interact with human and its environment. Any human–robot interaction involves two aspects, speech-based and speechless interactions. Among the two, the latter is an ess...
Article
Virtual reality (VR) volleyball games can provide an immersive entertainment experience and facilitate users to get familiar with game rules. In real world, players often experience emotions from scores, strokes, passes, and the influence from their teammates or opponents, and so forth. However, seldom studies are carried out for this emotional exp...
Preprint
Full-text available
There are considerable advancements in medical health care in recent years, resulting in rising older population. As the workforce for such a population is not keeping pace, there is an urgent need to address this problem. Having robots to stimulating recreational activities for older adults can reduce the workload for caretakers and give them time...
Book
This edited book is one of the first to describe how Autonomous Virtual Humans and Social Robots can interact with real people and be aware of the surrounding world using machine learning and AI. It includes: · Many algorithms related to the awareness of the surrounding world such as the recognition of objects, the interpretation of various sources...
Book
This book constitutes the refereed proceedings of the 38th Computer Graphics International Conference, CGI 2021, held virtually in September 2021. The 44 full papers presented together with 9 short papers were carefully reviewed and selected from 131 submissions. The papers are organized in the following topics: computer animation; computer vision;...
Preprint
Most current anthropomorphic robotic hands can realize part of the human hand functions, particularly for object grasping. However, due to the complexity of the human hand, few current designs target at daily object manipulations, even for simple actions like rotating a pen. To tackle this problem, we introduce a gesture based framework, which adop...
Conference Paper
Despite the great progress in human motion prediction, it remains a challenging task due to the complicated structural dynamics of human behaviors. In this paper, we address this problem in three aspects. First, to capture the long-range spatial correlations and temporal dependencies, we apply a transformer-based architecture with the global attent...
Preprint
Current anthropomorphic robotic hands mainly focus on improving their dexterity by devising new mechanical structures and actuation systems. However, most of them rely on a single structure/system (e.g., bone-only) and ignore the fact that the human hand is composed of multiple functional structures (e.g., skin, bones, muscles, and tendons). This n...
Article
Full-text available
In this work we have enhanced the perception of a humanoid robot by integrating it with a social state estimation system. We present a user study of the humanoid Nao robot as a social mediator, comprising two sets of experiments. In the first sets of experiments, the participants rate their understanding of feedback messages delivered via the Nao r...
Article
Full-text available
Learning couple dance such as salsa is challenging as it requires to understand and assimilate all the dance skills (guidance, rhythm, style) correctly. Salsa is traditionally learned by attending a dancing class with a teacher and practice with a partner, the difficulty to access such classes though, and the variability of dance environment can im...
Article
Compared with depth-based 3D hand pose estimation, it is more challenging to infer 3D hand pose from monocular RGB images, due to the substantial depth ambiguity and the difficulty of obtaining fully-annotated training data. Different from the existing learning-based monocular RGB-input approaches that require accurate 3D annotations for training,...
Book
The LNCS journal Transactions on Computational Science reflects recent developments in the field of Computational Science, conceiving the field not as a mere ancillary science but rather as an innovative approach supporting many other scientific disciplines. The journal focuses on original high-quality research in the realm of computational science...
Book
This book constitutes the refereed proceedings of the 37th Computer Graphics International Conference, CGI 2020, held in Geneva, Switzerland, in October 2020. The conference was held virtually. The 43 full papers presented together with 3 short papers were carefully reviewed and selected from 189 submissions. The papers address topics such as: virt...
Book
This book constitutes the revised selected papers of the 33rd International Conference on Computer Animation and Social Agents, CASA 2020, held in Bournemouth, UK*, in October 2020. The 1 full paper and 13 short papers presented were carefully reviewed and selected from a total of 86 submissions. The papers are organized in topical sections of mode...
Conference Paper
Full-text available
In this work, we address the challenging issue of scene segmentation. To increase the feature similarity of the same object while keeping the feature discrimination of different objects, we explore to propagate information throughout the image under the control of objects' boundaries. To this end, we first propose to learn the boundary as an additi...
Chapter
In recent years, we have witnessed a steady growth of the research in real-time 3D hand pose estimation with depth cameras, since this technology plays an important role in various human–computer interaction applications. In this chapter, we first review existing techniques and systems for real-time 3D hand pose estimation. Then, we will discuss tw...
Preprint
Full-text available
In this work, we address the challenging issue of scene segmentation. To increase the feature similarity of the same object while keeping the feature discrimination of different objects, we explore to propagate information throughout the image under the control of objects' boundaries. To this end, we first propose to learn the boundary as an additi...
Conference Paper
Schizophrenia and depression are the two most common mental disorders associated with negative symptoms that contribute to poor functioning and quality of life for millions of patients globally. This study is part of a larger research project. The overall aim of the project is to develop an automated objective pipeline that aids clinical diagnosis...
Chapter
Full-text available
The current preliminary study concerns the identification of the effects human-humanoid interaction can have on human emotional states and behaviors, through a physical interaction. Thus, we have used three cases where people face three different types of physical interaction with a neutral person, Nadine social robot and the person on which Nadine...