Koen V. Hindriks

Koen V. Hindriks
Vrije Universiteit Amsterdam | VU · Department of Computer Science

PhD

About

298
Publications
77,542
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
6,078
Citations
Introduction
Koen Hindriks is Full Professor Artificial Intelligence at the Vrije Universiteit (VU) Amsterdam and CIO of the spin-off Interactive Robotics. He is chair of the Special Interest Group on Artificial Intelligence in The Netherlands (An IPN member). His work on Artificial Intelligence focuses on socio-cognitive robotics. He has published more than 150 papers on cognitive agent technology. Interaction between man and robot is at the heart of his research and also drives the goals of his company. He has developed robots as companions and assistants for healthcare and education, and is developing service robots for business solutions.
Additional affiliations
November 2018 - present
Vrije Universiteit Amsterdam
Position
  • Professor (Full)
January 2015 - present
Delft University of Technology
Position
  • Professor (Associate)
December 2014 - July 2016
Delft University of Technology
Position
  • Interim Section Lead
Education
September 1990 - August 1996
University of Groningen
Field of study
  • Computer Science

Publications

Publications (298)
Preprint
Full-text available
Spectral subtraction, widely used for its simplicity, has been employed to address the Robot Ego Speech Filtering (RESF) problem for detecting speech contents of human interruption from robot's single-channel microphone recordings when it is speaking. However, this approach suffers from oversubtraction in the fundamental frequency range (FFR), lead...
Conference Paper
Full-text available
AIRSI2024 is an international conference focused on the application and effects of technologies that are part of the so called Industry 4.0 (artificial intelligence, robots, virtual assistants, avatars, metaverse, augmented reality, virtual reality, big data, blockchain, NFTs, etc.). Specifically, the aim of this conference is to deepen and broaden...
Preprint
Full-text available
Reinforcement Learning (RL) has achieved great success in sequential decision-making problems, but often at the cost of a large number of agent-environment interactions. To improve sample efficiency, methods like Reinforcement Learning from Expert Demonstrations (RLED) introduce external expert demonstrations to facilitate agent exploration during...
Preprint
Full-text available
With current state-of-the-art automatic speech recognition (ASR) systems, it is not possible to transcribe overlapping speech audio streams separately. Consequently, when these ASR systems are used as part of a social robot like Pepper for interaction with a human, it is common practice to close the robot's microphone while it is talking itself. Th...
Conference Paper
In this paper, we study how well human speech can automatically be filtered when this overlaps with the voice and fan noise of a social robot, Pepper. We ultimately aim for an HRI scenario where the microphone can remain open when the robot is speaking, enabling a more natural turn-taking scheme where the human can interrupt the robot. To respond a...
Article
As social robots see increasing deployment within the general public, improving the interaction with those robots is essential. Spoken language offers an intuitive interface for the human-robot interaction (HRI), with dialogue management (DM) being a key component in those interactive systems. Yet, to overcome current challenges and manage smooth,...
Chapter
We designed a wine recommendation robot and deployed it in a small supermarket. In a study aimed to evaluate our design we found that people with no intent to buy wine were interacting with the robot rather than the intended audience of wine-buying customers. Behavioural data, moreover, suggests a very different evaluation of the robot than the sur...
Preprint
Full-text available
As social robots see increasing deployment within the general public, improving the interaction with those robots is essential. Spoken language offers an intuitive interface for the human-robot interaction (HRI), with dialogue management (DM) being a key component in those interactive systems. Yet, to overcome current challenges and manage smooth,...
Article
Full-text available
While interacting with a social robot, children have a need to express themselves and have their expressions acknowledged by the robot. A need that is often unaddressed by the robot, due to its limitations in understanding the expressions of children. To keep the child-robot interaction manageable the robot takes control, undermining children’s abi...
Article
Full-text available
Young adulthood is a period of high risk for the development of mental health concerns. Increasing well-being among young adults is important to prevent mental health concerns and their consequences. Self-compassion has been identified as a modifiable trait with the potential to protect against mental health concerns. An online self-guided mental h...
Article
Full-text available
Emotions are known to spread among people, a process known as emotion contagion. Both positive and negative emotions are believed to be contagious, but the mass spread of negative emotions has attracted the most attention due to its danger to society. The use of agent-based techniques to simulate emotion contagion in crowds has grown over the last...
Article
Smart devices, such as smart phones, voice assistants and social robots, provide users with a range of input modalities, e.g., speech, touch, gestures, and vision. In recent years, advancements in processing of these input channels enable more natural interaction (e.g., automated speech, face, and gesture recognition, dialog generation, emotion exp...
Chapter
We propose SUPPLE, a new class of dialogue management systems that takes the core concept of a dialogue sequence as its main starting point. SUPPLE is inspired by the conversation patterns from the Natural Conversation Framework (NCF). While NCF primarily provides a design framework, we propose to automate the selection and updating of dialogue seq...
Chapter
Full-text available
We propose SUPPLE (Sequence-Update Pattern-Based Processing with Logical Expansions), a new dialog management framework that takes the core concept of a dialog sequence as its main starting point. SUPPLE naturally enables the integration of the flexible and re-usable conversation patterns from the Natural Conversation Framework (NCF). Whereas NCF p...
Article
Full-text available
In this article we discuss two studies of children getting acquainted with an autonomous socially assistive robot. The success of the first encounter is key for a sustainable long-term supportive relationship. We provide four validated behavior design elements that enable the robot to robustly get acquainted with the child. The first are five conve...
Article
Full-text available
Objectives: Children with cancer often experience sleep problems, which are associated with many negative physical and psychological health outcomes, as well as with a lower quality of life. Therefore, interventions are strongly required to improve sleep in this population. We evaluated interactive education with respect to sleep hygiene with a so...
Conference Paper
Both social group detection and group emotion recognition in images are growing fields of interest, but never before have they been combined. In this work we aim to detect emotional subgroups in images, which can be of great importance for crowd surveillance or event analysis. To this end, human annotators are instructed to label a set of 171 image...
Article
Full-text available
A key challenge in human-robot interaction (HRI) design is to create and sustain engaging social interactions. This paper argues that improvisational techniques from the performing arts can address this challenge. Contrary to the ways in which improvisation is generally used in social robotics, we propose an understanding of improvisational techniq...
Article
Full-text available
The robot rights debate has thus far proceeded without any reliable data concerning the public opinion about robots and the rights they should have. We have administered an online survey ( n = 439) that investigates layman’s attitudes toward granting particular rights to robots. Furthermore, we have asked them the reasons for their willingness to g...
Conference Paper
Full-text available
Service robots provide retailers with new opportunities to innovate their in-store service offerings. Despite advances made in the fields of human-robot interaction, information systems, and marketing, there is relatively little known about how to apply a service robot in retailing. In this paper we aim to shed light on this issue by exploring the...
Preprint
Full-text available
We present a discovery-based, first version, explicit model of social interaction that provides a basis for measuring the quality of interaction of a human user with a social robot. The two core elements of the social interaction model are engagement and co-regulation. Engagement emphasizes the \textit{qualitative nature} of social interaction and...
Conference Paper
Full-text available
Existing agent-based models of emotion contagion that account for the emotional diversity in groups have mostly focussed on the spread of categorical emotions (happy, sad, angry). In practice this raises problems with regard to how the spread of different emotions should interact. Can one be both very happy and very angry at the same time, or shift...
Chapter
Effective use of negotiation support systems depends on the systems capability of explaining itself to the user. This paper introduces the notion of an explanation matrix and an aberration detection mechanism for bidding strategies. The aberration detection is a mechanism that detects if one of the negotiating parties deviates from their expected b...
Article
Full-text available
This review aims to summarize and describe research on the topic of automatic group emotion recognition. In recent years, the topic of emotion analysis of groups or crowds has gained interest, with studies performing emotion detection in different contexts, using different datasets and modalities (such as images, video, audio, social media messages...
Article
Full-text available
Serious games and gamification is a popular and growing field, commercially and for academic research. This paper aims to give an overview of a specific domain within the field of serious gaming and gamification; the field of serious games and gamification to empower vulnerable target groups. This overview contributes to a better understanding of t...
Article
Full-text available
We define hybrid intelligence (HI) as the combination of human and machine intelligence, augmenting human intellect and capabilities instead of replacing them and achieving goals that were unreachable by either humans or machines. HI is an important new research focus for artificial intelligence, and we set a research agenda for HI by formulating f...
Conference Paper
Emotion expression is an important part of human-robot interaction. Previous studies typically focused on a small set of emotions and a single channel to express them. We developed an emotion expression model that modulates motion, poses and LED features parametrically, using valence and arousal values. This model does not interrupt the task or ges...
Chapter
The issue of explainability for autonomous systems is becoming increasingly prominent. Several researchers and organisations have advocated the provision of a “Why did you do that?” button which allows a user to interrogate a robot about its choices and actions. We take previous work on debugging cognitive agent programs and apply it to the questio...
Chapter
The commercial availability of robots and voice-operated smart devices such as Alexa or Google Home have some companies wondering whether they can replace some current human interactions by using these devices. One such area of interaction is at the reception desk. While both platforms can offer the necessary interaction features to take on the tas...
Article
It is claimed that, in the nascent ‘Cognitive Era’, intelligent systems will be trained using machine learning techniques rather than programmed by software developers. A contrary point of view argues that machine learning has limitations, and, taken in isolation, cannot form the basis of autonomous systems capable of intelligent behaviour in compl...
Article
Full-text available
It is claimed that, in the nascent ‘Cognitive Era’, intelligent systems will be trained using machine learning techniques rather than programmed by software developers. A contrary point of view argues that machine learning has limitations, and, taken in isolation, cannot form the basis of autonomous systems capable of intelligent behaviour in compl...
Preprint
BACKGROUND Society is facing a global shortage of 17 million healthcare workers, along with increasing healthcare demands from a growing number of older adults. Social robots are being considered as solutions to part of this problem. OBJECTIVE To evaluate the quality of care perceived by patients and caregivers for an integrated care pathway in an...
Article
Full-text available
Background: Society is facing a global shortage of 17 million health care workers, along with increasing health care demands from a growing number of older adults. Social robots are being considered as solutions to part of this problem. Objective: Our objective is to evaluate the quality of care perceived by patients and caregivers for an integr...
Article
Full-text available
Patient reported outcome measures (PROMs) are an essential means for collecting information on the effectiveness of hospital care as perceived by the patients themselves. Especially older adult patients often require help from nursing staff to successfully complete PROMs, but this staff already has a high work load. Therefore, a social robot is int...
Conference Paper
Full-text available
This paper presents the design and evaluation of human-like welcoming behaviors for a humanoid robot to draw the attention of passersby by following a three-step model: (1) selecting a target (person) to engage, (2) executing behaviors to draw the target's attention, and (3) monitoring the attentive response. A computer vision algorithm was develop...
Chapter
We are developing a social robot that should autonomously interact long-term with pediatric oncology patients. The child and the robot need to get acquainted with one another before a long-term interaction can take place. We designed five interaction design patterns and two sets of robot behaviors to structure a getting acquainted interaction. We d...
Chapter
We report on the exploratory design and study of a robot math tutor that can provide feedback on specific errors made by children solving basic addition and subtraction problems up to 100. We discuss two interaction design patterns, one for speech recognition of answers when children think aloud, and one for providing error-specific feedback. We ev...
Article
Agents in teamwork may be highly interdependent on each other, the awareness of interdependence relationships is an important requirement for designing and consequently implementing a multi-agent system. In this work, we propose a formal graphical and domain-independent language that can facilitate the identification of comprehensive interdependenc...
Article
Full-text available
Artificial Intelligence (AI) is becoming more and more ubiquitous, and AI also invades health care and related fields ever more. We believe this is a good thing. Inspired by work that we have done in various projects at the Alan Turing Institute Almere (ATIA), Delft University of Technology (TUD), Utrecht University (UU), and Vrije University (VU),...
Chapter
Full-text available
The evaluation of cognitive agent systems, which have been advocated as the next generation model for engineering complex, distributed systems, requires more benchmark environments that offer more features and involve controlling more units. One issue that needs to be addressed time and again is how to create a connector for interfacing cognitive a...
Article
The continuous integration of software-intensive systems together with the ever-increasing computing power offer a breeding ground for intelligent agents and multi-agent systems (MAS) more than ever before. Over the past two decades, a wide variety of languages, models, techniques and methodologies have been proposed to engineer agents and MAS. Des...
Conference Paper
Drawing the attention of passersby is a basic task of a social robot to initiate an interaction in a public environment (e.g., shopping malls, museums or hospitals). Humans use several social cues, both verbal and nonverbal, to draw the attention of others. In this study, we investigate whether similar behaviors can also be effectively used by a so...
Article
Full-text available
Background /Objectives Healthcare professionals (HCP) are confronted with an increased demand for assessments of important health status measures, such as patient-reported outcome measurements (PROM), and the time this requires. The aim of this study was to investigate the effectiveness and acceptability of using an HCP robot assistant, and to test...
Article
Full-text available
We are designing a social robot to collect patient data in hospitals by interviewing patients. This task is crucial for improving and providing value-based care. Currently, professional caretakers administer self-reported outcome questionnaires called patient reported outcome measures (PROMs) to collect this data. By delegating this task to a robot...
Conference Paper
Full-text available
It has been argued that the evaluation of cognitive agent systems requires richer benchmark problems. We think that real-time strategy (RTS) games can offer such a testbed, as AI for RTS requires the design of complicated strategies for coordinating hundreds of units that need to solve a range of challenges. Therefore, in this paper, we report on t...
Conference Paper
Full-text available
Patient Reported Outcome Measures (PROMs) are a means of collecting information on the effectiveness of care delivered to patients as perceived by the patients themselves. A patient's pain level is a typical parameter only a patient him/herself can describe. It is an important measure for a person?s quality of life. When a patient stays in a Dutch...
Conference Paper
We report on a field exercise in which a team of human fire-fighters used robots to enact a realistic disaster response mission in an industrial environment. In this exercise we evaluated the technical working of an integrated robotic system and gained insights concerning the manner in which robots and information streams can be utilized effectivel...
Conference Paper
Full-text available
Today, off-the-shelf social robots are used increasingly in the HRI community to research social interactions with different target user groups across a range of domains (e.g. healthcare, education, retail and other public spaces). We invite everyone doing HRI studies with end users, in the lab or in the wild, to collect past experiences of methods...
Preprint
See https://goalapl.atlassian.net/wiki/spaces/GOAL/pages/33043/Tutorials+Documentation+and+Education for the latest version of the GOAL Programming Guide.
Preprint
Full-text available
See https://goalapl.atlassian.net/wiki/spaces/GOAL/pages/33043/Tutorials+Documentation+and+Education for the latest version.
Conference Paper
Full-text available
Pediatric oncology patients could benefit from bonding with a social robot and talking about their day in the hospital. With our research we aim to contribute to the development of a robot that is able to facilitate a child-robot bond autonomously and long-term. We propose to use robot-disclosure and a shared interaction history to create a child-r...
Conference Paper
Full-text available
Medical staff uses Patient Reported Outcome Measurement (PROM) questionnaires as a means of collecting information on the effectiveness of care delivered to patients as perceived by the patients themselves. Especially for the older patient group, the PROM questioning poses an undesirable workload on the staff. This proof of concept paper investigat...
Conference Paper
Full-text available
Task allocation and management is crucial for human-robot collaboration in Urban Search And Rescue response efforts. The job of a mission team leader in managing tasks becomes complicated when adding multiple and different types of robots to the team. Therefore, to effectively accomplish mission objectives, shared situation awareness and task manag...
Conference Paper
Exploration games are games where agents (or robots) need to search resources and retrieve these resources. In principle, performance in such games can be improved either by adding more agents or by exchanging more messages. However, both measures are not free of cost and it is important to be able to assess the trade-off between these costs and th...
Conference Paper
Full-text available
Artificial Intelligence (AI) systems, including intelligent agents, are becoming increasingly complex. Explainable AI (XAI) is the capability of these systems to explain their behaviour, in a for humans understandable manner. Cognitive agents, a type of intelligent agents, typically explain their actions with their beliefs and desires. However, hum...
Conference Paper
Full-text available
This paper presents a cognitive (belief-desire-intention based) agent that can self-explain its behaviour based on its goals and emotions. We implement a cognitive agent, embodied by a nao-robot or virtual avatar thereof, to play a quiz with its user. During the interaction the agent intelligently selects questions to optimally educate the user. We...
Article
Full-text available
When an agent program exhibits unexpected behaviour, a developer needs to locate the fault by debugging the agent’s source code. The process of fault localisation requires an understanding of how code relates to the observed agent behaviour. The main aim of this paper is to design a source-level debugger that supports single-step execution of a cog...