Patrick Gebhard

Patrick Gebhard
Deutsches Forschungszentrum für Künstliche Intelligenz | DFKI · Cognitive Assistants

Dr.

About

98
Publications
32,521
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
2,265
Citations
Citations since 2017
37 Research Items
807 Citations
2017201820192020202120222023050100150
2017201820192020202120222023050100150
2017201820192020202120222023050100150
2017201820192020202120222023050100150
Introduction
With his team, he researches the adaptation of psychological theories for the simulation of human emotions and social behavior. The aim is to enable a human-like interaction with computer systems and evaluate socio-emotional interactive human-agent systems. His group developed the internationally recognized research frameworks ALMA and MARSSI for computationally simulating affect and the Visual SceneMaker authoring tool to model and execute interactive behavior for socially interactive agents.
Additional affiliations
June 2007 - present
Universität des Saarlandes
April 1996 - April 1999
Universität des Saarlandes
Position
  • Researcher

Publications

Publications (98)
Preprint
Backchannels, i.e. short interjections of the listener, serve important meta-conversational purposes like signifying attention or indicating agreement. Despite their key role, automatic analysis of backchannels in group interactions has been largely neglected so far. The MultiMediate challenge addresses, for the first time, the tasks of backchannel...
Conference Paper
Full-text available
Backlash is a form of social penalty occurring when people behave counter-stereotypically. When promoting themselves, dominant females compared to males are typically liked less and paid worse, because dominance is associated with males, and proscribed for females. Such backlash effects have been shown in human-human interactions, but attempts to r...
Poster
Full-text available
Computational modeling of human emotions is daunting, as emotions have a behavioral component represented by social signals and an internal component mainly reflecting individual experiences (Schneeberger et al., 2021). The challenge is how these two components are connected? We propose a terminology of emotions aiming to facilitate an understandin...
Article
Cover Caption: The cover image is based on the Review Article Physical and mental well‐being of cobot workers: A scoping review using the Software‐Hardware‐Environment‐Liveware‐Liveware‐Organization model by Fabio A. Storm et al., https://doi.org/10.1002/hfm.20952
Preprint
Full-text available
Job interviews are usually high-stakes social situations where professional and behavioral skills are required for a satisfactory outcome. Professional job interview trainers give educative feedback about the shown behavior according to common standards. This feedback can be helpful concerning the improvement of behavioral skills needed for job int...
Preprint
Full-text available
In today's world, many patients with cognitive impairments and motor dysfunction seek the attention of experts to perform specific conventional therapies to improve their situation. However, due to a lack of neurorehabilitation professionals, patients suffer from severe effects that worsen their condition. In this paper, we present a technological...
Preprint
Full-text available
Highly automated vehicles represent one of the most crucial development efforts in the automotive industry. In addition to the use of research vehicles, production vehicles for the general public are realistic in the near future. However, to fully exploit the benefits of these systems, it is fundamental that users have an appropriate level of trust...
Article
The present scoping review investigated the current state of the art concerning factors affecting physical and mental health and well‐being of workers using collaborative robots (cobots) in manufacturing industries. Each identified factor was classified using the SHELLO (Software‐Hardware‐Environment‐Liveware‐Liveware‐Organization) conceptual model...
Conference Paper
Full-text available
In the last decade, empirical sciences have faced a tremendous change in the way of conducting research. As a broad interdisciplinary field, research in Affective Computing often employs empirical user studies. The current paper analyzes research practices in Affective Computing and deduces recommendations for improving the quality of methods and r...
Chapter
This chapter examines how notions of trust and empathy can be applied to human–robot interaction and how it can be used to create the next generation of emphatic agents, which address some of the pressing issues in multicultural aging societies: to harness cutting-edge AI research to develop solutions and new services. First, some examples of possi...
Conference Paper
Full-text available
Understanding emotions is key to Affective Computing. Emotion recognition focuses on the communicative component of emotions encoded in social signals. This view alone is insufficient for deeper understanding and computational representation of the internal, subjectively experienced component of emotions. This paper presents the DEEP method as a st...
Conference Paper
Full-text available
Computational emotion recognition focuses on observable expressions. In the case of highly unpleasant emotions that are rarely displayed openly and mostly unconsciously regulated -- such as shame -- this approach can be difficult. In previous studies, we found participants to smile and laugh while experiencing shame. Most current approaches interpr...
Conference Paper
Full-text available
This paper presents an overview of AVASAG; an ongoing applied-research project developing a text-to-sign-language translation system for public services. We describe the scientific innovation points (geometry-based SL-description, 3D animation and video corpus, simplified annotation scheme, motion capture strategy) and the overall translation pipel...
Conference Paper
Full-text available
Co-regulation in groups is essential for learning and may hold the key to training regulation skills. However, there is little possibility to train social co-regulation skills in authentic social situations. Existing systems mostly concentrate on training para-and nonverbal skills. While co-regulation in scripted scenarios may work at the behaviora...
Conference Paper
Full-text available
The acceptance of driving automation is influenced by user trust. Integrating virtual agents into the interface between automation and user has been found to increase trust, also in driving automation. In this context, however, it remains unclear what virtual agents’ characteristics influence trust. This paper presents a pilot study about how inexp...
Preprint
Full-text available
Using Social Agents as health-care assistants or trainers is one focus area of IVA research. While their use as physical health-care agents is well established, their employment in the field of psychotherapeutic care comes with daunting challenges. This paper presents our mobile Social Agent EmmA in the role of a vocational reintegration assistant...
Article
In this paper, we focus on experience-based role play with virtual agents to provide young adults at the risk of exclusion with social skill training. We present a scenario-based serious game simulation platform. It comes with a social signal interpretation component, a scripted and autonomous agent dialog and social interaction behavior model, and...
Conference Paper
Using Social Agents as health-care assistants or trainers is one focus area of IVA research. This paper presents a concept of our mobile Social Agent EmmA in the role of a vocational reintegration assistant for burn-out outpatient treatment. We follow a typical par- ticipatory design approach including experts and patients in order to address requi...
Conference Paper
Full-text available
Virtual agents usually come in a virtual environment that can be designed in various ways which might affect users. This paper presents a study that examines whether the design of the virtual environment has an impact on the assessment of the virtual agent and the interaction. In a virtual job interview training, participants interacted with a virt...
Conference Paper
Full-text available
Human interaction partners can deal with interruptions and then resume the interaction. This ability should be emulated by social agents. How fast interruptions are handled might influence the overall impression of an agent. In this paper, we present the results of a user study on how a human dialog partner perceives the be- havior of a virtual age...
Conference Paper
Full-text available
In this demo, we present PARLEY, an interactive system to train difficult social situations in a safe environment with a Virtual Agent. The system realizes different phenomena studied by psychology research that are known to create a natural interaction. Moreover, we include an open learner model to ensure an explainable user experience.
Conference Paper
Virtual agents play an important role when we interact with machines. They are in the role of assistants or companions with less or more human-like appearance. Such agents influence our behavior. With an increasing and broader distribution, their influence might become stronger, and at some point, they might even adopt roles with a degree of author...
Conference Paper
Full-text available
Understanding emotions of others requires a theory of mind ap- proach providing knowledge of internal appraisal and regulation processes of emotions. Multi-modal social signal classification is insufficient for understanding emotional expressions. Mainly, be- cause many communicative, emotional expressions are not directly related to internal emoti...
Article
Full-text available
Expanding research on employment interview training, this study introduces virtual employment interview (VI) training with focus on nonverbal behavior. In VI training, participants took part in a simulated interview with a virtual character. Simultaneously, the computer analyzed participants’ nonverbal behavior and provided real-time feedback for i...
Data
Full-text available
Conference Paper
Full-text available
Technology-enhanced learning environments are designed to help users practise social skills. In this paper, we present and evaluate a virtual job interview training game which has been adapted to the special requirements of young people with low chances on the job market. The evaluation spanned three days, during which we compared the technology-en...
Article
Full-text available
The outcome of interpersonal interactions depends not only on the contents that we communicate verbally, but also on nonverbal social signals. As a lack of social skills is a common problem for a significant number of people, serious games and other training environments have recently become the focus of research. In this work we present NovA (Nonv...
Conference Paper
Full-text available
Grounding is an important process that underlies all human interaction. Hence, it is crucial for building social robots that are expected to collaborate effectively with humans. Gaze behavior plays versatile roles in establishing, maintaining and repairing the common ground. Integrating all these roles in a computational dialog model is a comple...
Conference Paper
Full-text available
Grounding is essential in human interaction and crucial for social robots collaborating with humans. Gaze plays versatile roles for establishing, maintaining and repairing the common ground. It is combined with parallel modalities and involved in several pro-cesses for behavior generation and recognition. We present a uniform modeling approach focu...
Conference Paper
Full-text available
Grounding is essential in human interaction and crucial for social robots collaborating with humans. Gaze plays versatile roles for establishing, maintaining and repairing the common ground. It is combined with parallel modalities and involved in several pro-cesses for behavior generation and recognition. We present a uniform modeling approach focu...
Conference Paper
Full-text available
Job interviews come with a number of challenges, especially for young people who are out of employment, education, or training (NEETs). This paper presents an approach to a job training simulation environment that employs two virtual characters and social cue recognition techniques to create an immersive interactive job interview. The two virtual c...
Conference Paper
Full-text available
The TARDIS project aims to build a scenario-based serious-game simulation platform for NEETs and job-inclusion associations that supports so-cial training and coaching in the context of job interviews. This paper presents the general architecture of the TARDIS job interview simulator, and the serious game paradigm that we are developing.
Conference Paper
Full-text available
This paper presents an approach that makes use of a virtual character and social signal processing techniques to create an immersive job interview simulation environment. In this environment, the virtual character plays the role of a recruiter which reacts and adapts to the user's behavior thanks to a component for the automatic recognition of soci...
Chapter
In this chapter, the utilization of Digital Product Memories (DPMs) in the retail domain is discussed. We demonstrate a complete retail-related usage scenario from a consumer perspective that begins with the preparation of a shopping list before the physical in-store shopping activity. Throughout the entire shopping process in the retail store, the...
Chapter
Creating intelligent interactive applications based on DPMs is a challenging task. The definition of system reactions, independent interface behaviors, and necessary context data requires expert programmers at various representation levels and often results in hard-to-maintain code. We use the Visual SceneMaker authoring tool, a visual authoring ap...
Article
Full-text available
Creating interactive applications with multiple virtual characters comes along with many challenges that are related to different areas of expertise. The definition of context-sensitive interactive behavior requires expert pro- grammers and often results in hard-to-maintain code. To tackle these challenges, we suggest a visual authoring ap- proach...
Conference Paper
Full-text available
In this paper, we present a visual authoring approach for the management of highly interactive, mixed-initiative, multi-party dialogues. Our approach enforces the separation of dialog-content and-logic and is based on a statechart language enfolding concepts for hierarchy , concurrency, variable scoping and runtime history. These concepts facilitat...
Article
Full-text available
While current virtual characters may look photorealistic they often lack behavioral complexity. Emotion may be the key ingredient to create behavioral variety, social adaptivity and thus believability. While various models of emotion have been suggested, the concrete parametrization must often be designed by the implementer. We propose to enhance a...
Article
Full-text available
Digital Object Memories, DOM, comprise concepts and technologies to physically and conceptually associate digital information with physical objects in an application-independent manner. By storing information about an object's properties, state, and history of use in its digital memory, objects become self-representative, which allows for novel kin...
Conference Paper
Full-text available
Auto-ID as well as traditional identification technologies such as barcodes allow for linking physical products with digital data. Thus products become “smart items”, which may contribute to the consumer's retail experience in future retail environments. In this article, we discuss how digital assistants can utilize so-called digital product memori...
Conference Paper
Full-text available
Generating coordinated multimodal behavior for an embodied agent (speech, gesture, facial expression...) is challenging. It requires a high degree of animation control, in particular when reactive behaviors are required. We suggest to distinguish realization planning, where gesture and speech are processed symbolically using the behavior markup lan...
Chapter
Full-text available
What would the shopping experience of tomorrow look like? In this chapter we propose several human-centered and resource-adaptive ideas to this question. Throughout the whole chapter we explain our ideas with the recurrent theme of a shop that consists of instrumented shelves, public displays, audio systems, and mobile devices for each user.
Chapter
Full-text available
Recently, a number of research projects have been started to create virtual agents that do not just serve as assistants to which tasks may be delegated, but that may even take on the role of a companion. Such agents require a great deal of social intelligence, such as the ability to detect the user's affective state and to respond to it in an empat...
Thesis
Full-text available
Die vorliegende Arbeit beschreibt eine Erklärungskomponente einer natürlichsprachlichen Schnittstelle, die in Robotersystemen für die Kommunikation mit Menschen eingesetzt werden kann. Die speziell dafür benötigten Bereiche des Roboterwissens bestehen einerseits aus Plänen, in denen Roboteranweisungen formuliert sind, und andererseits aus Diagnose-...
Conference Paper
This paper presents the results of a field study of the COHIBIT museum exhibit. Its purpose is to convey knowledge about car-technology and virtual characters in an entertaining way. Two life-sized virtual characters interact with the visitors and support them in constructing car models with specific tangible car modules. The evaluation should reve...
Conference Paper
Full-text available
This paper describes a system to enhance the interaction between humans and virtual characters with emotional mimicry and role-taking. Such system increases the believability of virtual agents. Mimicking necessitates a model of emotional intelligence to understand and display user's emotions. A more complex processing is however necessary for a rea...
Conference Paper
Full-text available
The continuous collection of digital information via smart labels attached to physical objects is a promising way to support information availability across all stages of a product's lifecycle. Since such "digital product memories" may contain a vast amount of heterogeneous data, we expect a strong demand for user support in tasks related to inform...
Conference Paper
In this paper we present an interactive poker game in which one human user plays against two animated agents using RFID-tagged poker cards. The game is used as a showcase to illustrate how current AI technologies can be used for providing new features to computer games. A powerful and easy-to-use multimodal dialog authoring tool is used for modelin...
Conference Paper
Full-text available
We present IGaze, a semi-immersive human-avatar interaction system. Using head tracking and an illusionistic 3D effect we let users interact with a talking avatar in an application interview scenario. The avatar features reactive gaze behavior that adapts to the user position according to exchangeable gaze strategies. In user studies we showed that...