Conference PaperPDF Available

Knowledge-based Intelligence and Strategy Learning for Personalised Virtual Assistance in the Healthcare Domain

Authors:

Abstract and Figures

This paper introduces a virtual assistant framework that combines knowledge-based and statistical techniques to produce meaningful task-oriented conversations that are enhanced by "chatty" style dialogues in order to increase system's naturalness and user engagement. The paper describes how appropriate ontologies, semantic reasoning, dialogue management and policy learning techniques can be linked together and integrated through the dialogue process to enable a) the internal representation of the conversational state, b) the conversational awareness that drives the retrieval of appropriate information from the Knowledge Base (KB) and the inference of unrelated system actions with the current conversational state, and c) the dynamic selection of the most appropriate strategy at each dialogue turn, tackling both informational and social-related needs of individuals. The framework is exemplified by a use case from the healthcare domain where companionship and supportive care-related services are prerequisites for an efficient human-system interaction through a conversational agent.
Content may be subject to copyright.
Knowledge-based Intelligence and Strategy Learning for Personalised Virtual
Assistance in the Healthcare Domain
Eleni Kamateri, Georgios Meditskos, Spyridon Symeonidis, Stefanos Vrochidis,
Ioannis Kompatsiarisand Wolfgang Minker
Information Technologies Institute
Centre for Research and Technology Hellas, 6th Km Charilaou-Thermi Road, Thessaloniki, Greece
Email: {ekamater, gmeditsk, spyridons, stefanos, ikom}@iti.gr
Institute of Communications Engineering
Ulm University, 89081 Ulm, Germany
Email: wolfgang.minker@uni-ulm.de
Abstract—This paper introduces a virtual assistant framework
that combines knowledge-based and statistical techniques to pro-
duce meaningful task-oriented conversations that are enhanced by
”chatty” style dialogues in order to increase system’s naturalness
and user engagement. The paper describes how appropriate
ontologies, semantic reasoning, dialogue management and policy
learning techniques can be linked together and integrated through
the dialogue process to enable a) the internal representation of the
conversational state, b) the conversational awareness that drives
the retrieval of appropriate information from the Knowledge
Base (KB) and the inference of unrelated system actions with
the current conversational state, and c) the dynamic selection of
the most appropriate strategy at each dialogue turn, tackling both
informational and social-related needs of individuals. The frame-
work is exemplified by a use case from the healthcare domain
where companionship and supportive care-related services are
prerequisites for an efficient human-system interaction through
a conversational agent.
KeywordsDialogue management; Knowledge representations;
Reasoning; Strategy learning; Virtual assistance.
I. INTRODUCTION
Nowadays, there is an increasing demand for intelligent
agents. A challenging domain includes personalised virtual
assistants that carry out human-like conversations taking into
account the latest user’s utterance, the dialogue history, as well
as the background knowledge about the user. The development
of such personalised systems requires a knowledge represen-
tation model for describing the semantics of various contexts
and structuring the background knowledge about individuals.
Current task-oriented dialogue systems focus on one task at
a time using frame-based [1] or agenda-based [2] mechanisms,
while it was only recently, when some ontology-based dialogue
systems (such as [3] and [4]) have been proposed using
semantic models for the representation of user’s utterance
and the generation of the system’s response. Access to a
rich domain model and the conversation memory can deal
with complex task-oriented dialogues. However, the typical
problem of task-oriented dialogue solutions remains that is the
difficulty of tackling user utterances that go beyond the agent’s
representational model and the smooth transition between task-
oriented and ”chatty” style dialogues.
To succeed this, we propose a hybrid dialogue framework
that can be placed at the heart of any personalised virtual
assistant to enhance its model-driven operation by ”chatty”
style responses. The proposed approach, which is an on-going
work, combines knowledge representation and reasoning with
statistical learning for the smooth transition between strategies,
discussion topics and available knowledge with the aim to
impose social skills in the personalised virtual assistants in
order to efficiently realise meaningful task-oriented conversa-
tions, recover breakdowns in a natural way, and increase user
engagement.
Our major contributions are summarised as follows:
1) a domain and a dialogue representation model are
proposed and populated with local semantics coming
from the language analysis of the user’s utterance
by means of semantic similarity and disambiguation
techniques,
2) a dialogue history representation model is pro-
posed and populated with global semantics of the
entire dialogue session at each dialogue turn,
3) semantic reasoning techniques are applied on top
of the semantically structured data with the aim to
generate dynamically-inferred insights and actions,
4) a dialogue management technique analyses the
system’s confidence regarding the task-oriented re-
sponse and produces a set of social-oriented action
candidates, and
5) a strategy selection technique is used to select the
appropriate strategy, i.e., action.
Such personalised virtual assistants can have many appli-
cations in the healthcare domain and provide a mixture of
companionship and supportive care-related services, improving
the quality of life of individuals. We selected to apply our
framework in a rehabilitation setting, which involves people
with motor, cognitive and behavioural disorders being in a
clinical environment or after returning home.
The rest of the paper is structured as follows: Section
II presents related work on dialogue systems. Section III
describes the specifics of the proposed framework, elaborating
on the representation, reasoning and dialogue management
capabilities. Section IV presents an example use case in the
rehabilitation domain, where the framework is currently being
used. Finally, Section V concludes our work, mentioning future
research directions.
II. RE LATE D WOR K
First conversational systems were mainly task-oriented
(e.g., [5] realises restaurant reservations) lacking social compe-
tences. More recent personal assistants, such as the commercial
platforms of Alexa, Siri, Google Assistant and Contana, have
started to incorporate social features and support non-task-
oriented dialogues as well, where users do not have a clear
goal or intention. However, these systems are usually model-
less, constrained to accessing the parameters of the last users
utterance and thus, they are acceptable only for simple tasks
that do not need to sustain the whole conversation memory.
On the other hand, non-task-oriented dialogue systems do
not have a specific goal and are capable of addressing a wide
range of topics. To succeed this, they are based on data-driven
methods, such as the retrieval-based response selection [6]
and the sequence-to-sequence recurrent neural networks [7].
Like most data-driven systems, they produce utterances that
are incoherent or inappropriate from time to time and they
require a big volume of data that may not be always available.
The combination of the two types of dialogue systems has
only recently studied. Zu et al. [8] address the problems of
task-oriented dialogue systems when the user’s intention is
not clear with a framework that incorporates non-task-oriented
strategies to keep users interest in the conversation. Similarly,
Papaioannou et al. [9] propose a system that combines task-
oriented and chat-style dialogues. Both systems apply a re-
inforcement learning mechanism for selecting the appropriate
strategy at each dialogue turn. Coronado et al. [10] propose a
hybrid dialogue system that combines a Question Answering
system with a conversational agent dealing with rest (small
talk) phrases giving a social aspect to the system.
Although current works introduce social aspects through
non-task-oriented strategies, we noticed that they mainly use
retrieval-based methods with only exception the [11], which
incorporates an extension of OwlSpeak dialogue manager [12]
and decides whether to consult a knowledge-based module
or react on its own. To the best of our knowledge, this is
the first approach to combine knowledge-based and statistical
techniques to produce task-oriented dialogues that will be used
interchangeably with chatty style dialogues exploiting a rich
domain model and sustaining the whole conversation memory.
III. FRA ME WORK OVERVI EW
Our framework has four major components: (a) a Con-
textual Modelling and Representation (CMR) module, (b) a
Semantic Intelligence (SI) module, (c) a Dialogue Management
(DM) module and (d) a Strategy Selection (SS) module. Figure
1 shows the information flow among these components.
A user utterance is sent to the language understanding
module that extracts useful information to help the CMR
represent the parsed key entities and identify the discussion
topic. Based on the CMR outcome, the SI updates the system’s
conversational picture, correlates it with background knowl-
edge (e.g., the dialogue history) and infer unrelated insights
and actions. Simultaneously, the DM accesses the discussion
topic and produces topic-oriented action(s) along with a set
of social-oriented actions. Finally, the SS selects among all
the actions the most appropriate one and forwards it (along
with relevant information from KB, if needed) to the language
generation module to produce a system response.
A. Contextual Modelling and Representation
The module semantically represents and interlinks the user
utterance against the system’s cognitive models considering the
information passed from the language understanding module.
To achieve this, the module employs existing ontologies
and vocabularies. Existing ontologies form the basis of our
domain model extended with application-specific aspects. Al-
though there is a significant number of ontologies representing
the domain knowledge, we found only few examples of
respective ontologies for capturing the different features of
the dialogue process. From these, we selected to reuse the
well-established OwlSpeak ontology [12] extending it with
domain-retrieved knowledge communicated within the user’s
utterance, exploiting the framework proposed in [4]. The
dialogue turn, which is modelled by the Move concept, was
extended with two new subclasses, the UserMove and the
SystemMove, and each of them is broken down into a set
of ”generic” actions, which are common for both edges. For
these actions, we used the list of typical actions for multi-
agent dialogues presented in [13], including: Open/Greeting,
Close/Goodbye, Pause, Resume, Ask, Inform, Affirm, Assert,
Remind, and Alert, and extended them with ”Repeat” and the
”Recommend” action.
Each action is further specialised by a set of topic-oriented
actions, which constitute the ”discussion topics” that can be
covered by the agent. Each topic might be associated with
domain knowledge by means of a dialogue entity (dialogueEn-
tity) which consist the target entity of each discussion topic.
Additional entities extracted from the user’s utterance might
be associated with the dialogueEntity to further specify the
requested entity.
The module semantically represents a user utterance using
state-of-the-art disambiguation tools (e.g., UKB [14] or Ba-
belfy [15]) that assign key entities extracted from the language
understanding module to resource categories (i.e., synsets).
These resource categories are then used to identify entities
(synonyms) and topics against the domain and the dialogue
ontology, respectively.
With respect to domain-driven mapping, we assume that
label(r), is the label of resource rK B,syn(k)is the synset
of key entity kK and σis a similarity function, the set S(k)
of all the relevant resources to kis defined as:
S(k) = argmaxk K σ(k, label(r)) (1)
The UMBC Semantic Similarity Service [16] is used to
calculate the semantic similarity σbetween kand label(r)
combining Latent Semantic Analysis (LSA) word similarity
and WordNet knowledge.
With respect to dialogue-driven mapping, a simple clas-
sification algorithm calculates the conditional probability of
each discussion topic for all parsed resources, given that each
discussion topic is described by means of a set of similar
resources:
P(T opici|tx) = P(T opicitx)
P(tx)(2)
where T opiciis a topic defined by a set of resources
t1, t2, ...tk, while txis assumed to be a parsed resource
from user utterance. This probability is then multiplied with
respective probabilities for all parsed resources.
When a discussion topic is identified, the dialogue session
is informed with the dialogue details including the dialogue
topic, the dialogueEntity and associated entities populated with
knowledge coming from the analysis of user utterance.
Figure 1. Framework Architecture.
B. Semantic Intelligence
The module utilises pattern-based models [17] to update
domain models with new information communicated through
the human-system interaction and inform the dialogue his-
tory with identified entities and topics at each dialogue turn.
Moreover, it translates the system actions into actionable rules
(SPARQL queries), which are then used to retrieve pertinent
information from the underlying KB.
SPARQL Inferencing Notation (SPIN) are also applied to
generate alerts, reminders and recommendations, which are
triggered by the knowledge of the preceding discourse and the
specific user profile. By this way, motivational or interventional
actions are forwarded to the SS module, which might interrupt
the usual flow and impose situation-oriented system responses.
These actions consists of: (1) alert, (2) remind, (3) recommend
and (4) repeat action.
C. Dialogue Management
The module processes the outcome of topic identification
and decides the topic-oriented action to follow, selecting
among: (5) predefined topic-based (re-)action, when the match-
ing score of a topic exceeds a specific threshold, (6) clarifi-
cation action, in case of partial topic identification with more
than one topics receiving a significant matching score, and (7)
say-again action, in case of incomplete topic identification.
Simultaneously, the module formulates a set of social-
oriented action candidates considering the information received
from the CMR and supportive information extracted from the
KB. The social-oriented actions include (8) switch topic (a new
topic is suggested based on user’s preferences), (9) initiate
a relevant topic, (10) end current topic and make an open
question, (11) suggest to provide more info about the current
topic, and (12) elicit more information.
D. Strategy Selection
This module chooses among all action candidates the most
appropriate one with the aim to optimise the conversational
flow towards natural and meaningful interaction. Different
learning algorithms can be applied to train the strategy se-
lection, such as Q-learning [8] and policy gradient [18]. Our
strategy selection was implemented based on a simplified
version of the reinforcement learning algorithm presented in
[8]. The algorithm has a function that calculates the quantity
of a state-action combination Q:SxA> R, called Q table.
Qt+1 (st, at)Qt(st, at) + at(st, at)·
(Rt+1 +γmaxQt(st+1 , a)Qt(st, at))
For the reward function, we used domain experts’ knowl-
edge provided in [19] and [8]. According to them, the reward
is calculated based on: turn index, number of times each
strategy executed, sentiment polarity of previous utterances,
most recently used strategy and coherence confidence of the
response.
IV. A US E CAS E EXA MP LE I N REHABILITATION
As depicted in Figure 2, the system starts a conversation
saying ”Hello, what can I do for you?”. Let us assume that
the user replying ”Can you tell me my workout exercises for
today?”.
For domain modelling, we reused COPDology [20], an
ontology which was designed to facilitate the systematic mon-
itoring of Chronic Obstructive Pulmonary Disease (COPD) pa-
tients, containing concepts pertinent to an individual’s profile,
the conditions they suffer from, and the medications/workout
exercises they receive. We extended it with new properties,
such as the hasExecutionDay,hasExecutionSets and hasExe-
cutionRepetitions, to describe the execution guidelines for the
scheduled workout exercises. Moreover, we assume that there
is a AskActivityForSpecificDay topic, with the Activity being
the target entity and the Day specifying the topic receiving a
specific value, e.g., Monday.
The CMR annotates the key entities parsed from the
language understanding module and identifies the ”discussion
topic”. The incoming information ”workout exercises” and ”to-
day” are associated with the Activity concept and the Monday
instance of Day concept, while the AskActivityForSpecificDay
topic is identified with a matching score of 0.8.
The SI module updates the dialogue history and enforces
predefined rules. Emergency situations can be detected, for
example, if the user asks more than a couple of times about
the same topic, the system initially conceives it as repetition
but if it happens more than a predefined amount of times (e.g.,
three times) the system enforces an emergency situation.
The DM evaluates the matching score of identified dis-
cussion topic and decides that a ”predefined topic-based (re-
)action” will be followed. This means that the InformActivity-
ForSpecificDay system action, which is one-by-one associated
Figure 2. Use case example.
with the user’s action, will be enforced. In the meantime,
the SI (upon DM’s request) translates the system action and
dialogue entities into SPARQL queries to retrieve instances
of the ”Activity” concept for Monday. Simultaneously, the
module formulates a set of social-oriented action candidates.
Based on the learned Q table, the SS selects the most
appropriate action and forwards it to language generation to
produce the system response content.
V. CONCLUSION
The proposed framework combines dynamic knowledge-
based features with social competences which are orchestrated
by the means of a statistical policy learning that selects
among action candidates the most appropriate one to opti-
mise conversational effectiveness. The framework is currently
validated in a running project involving clinicians and staff
of a rehabilitation clinic. Our next steps is to establish an
experimental set-up and evaluate it with real data. In addition,
we plan to enrich the context understanding capabilities of
the agent by integrating and fusing multimodal information,
such as home activities and gestures, increasing the situational
awareness of the agent.
ACKNOWLEDGMENT
This research has been cofinanced by the European Re-
gional Development Fund of the European Union and Greek
national funds through the Operational Program Compet-
itiveness, Entrepreneurship and Innovation, under the call
RESEARCH-CREATE-INNOVATE (project code: T1EDK-
00686)
REFERENCES
[1] D. G. Bobrow, R. M. Kaplan, M. Kay, D. A. Norman, H. Thompson,
and T. Winograd, “Gus, a frame-driven dialog system,” Artificial
intelligence, vol. 8, no. 2, 1977, pp. 155–173.
[2] A. Rudnicky and W. Xu, “An agenda-based dialog management ar-
chitecture for spoken language systems,” in IEEE Automatic Speech
Recognition and Understanding Workshop, vol. 13, no. 4, 1999.
[3] D. Altinok, “An ontology-based dialogue management system for
banking and finance dialogue systems,” CoRR, vol. abs/1804.04838,
2018. [Online]. Available: http://arxiv.org/abs/1804.04838
[4] M. Wessel, G. Acharya, J. Carpenter, and M. Yin, OntoVPA—An
Ontology-Based Dialogue Management System for Virtual Personal
Assistants. Cham: Springer International Publishing, 2019, pp. 219–
233.
[5] F. Jurc´
ıcek, S. Keizer, M. Gasic, F. Mairesse, B. Thomson, K. Yu,
and S. J. Young, “Real user evaluation of spoken dialogue systems
using amazon mechanical turk,” in INTERSPEECH 2011, 12th Annual
Conference of the International Speech Communication Association,
Florence, Italy, August 27-31, 2011, 2011, pp. 3061–3064.
[6] R. E. Banchs and H. Li, “Iris: a chat-oriented dialogue system based
on the vector space model,” in Proc. of the ACL 2012 System Demon-
strations, 2012, pp. 37–42.
[7] O. Vinyals and Q. V. Le, “A neural conversational
model,” CoRR, vol. abs/1506.05869, 2015. [Online]. Available:
http://arxiv.org/abs/1506.05869
[8] Z. Yu, Z. Xu, A. W. Black, and A. Rudnicky, “Strategy and policy
learning for non-task-oriented conversational systems,” in Proc. of the
17th annual meeting of the special interest group on discourse and
dialogue, 2016, pp. 404–412.
[9] I. Papaioannou, C. Dondrup, J. Novikova, and O. Lemon, “Hybrid chat
and task dialogue for more engaging hri using reinforcement learning,”
in 26th IEEE Int. Symposium on Robot and Human Interactive Com-
munication (RO-MAN), 2017, pp. 593–598.
[10] M. Coronado, C. A. Iglesias, and A. Mardomingo, “A personal agents
hybrid architecture for question answering featuring social dialog,” in
Int. Symposium on Innovations in Intelligent SysTems and Applications
(INISTA), 2015, pp. 1–8.
[11] L. Pragst, J. Miehle, W. Minker, and S. Ultes, “Challenges for
adaptive dialogue management in the kristina project,” in Proceedings
of the 1st ACM SIGCHI International Workshop on Investigating
Social Interactions with Artificial Agents, ser. ISIAA 2017. New
York, NY, USA: ACM, 2017, pp. 11–14. [Online]. Available:
http://doi.acm.org/10.1145/3139491.3139508
[12] S. Ultes and W. Minker, “Managing adaptive spoken dialogue for
intelligent environments,” Journal of Ambient Intelligence and Smart
Environments, vol. 6, no. 5, 2014, pp. 523–539.
[13] J. Baskar and H. Lindgren, “Human-agent dialogues and their pur-
poses,” in Proceedings of the European Conference on Cognitive
Ergonomics 2017, ser. ECCE 2017. New York, NY, USA: ACM,
2017, pp. 101–104.
[14] [Online]. Available: http://ixa2.si.ehu.es/ukb/
[15] [Online]. Available: http://babelfy.org
[16] L. Han, A. L. Kashyap, T. Finin, J. Mayfield, and J. Weese,
“Umbc ebiquity-core: Semantic textual similarity systems,” in 2nd Joint
Conf. on Lexical and Computational Semantics (* SEM), Volume
1: Proc. of the Main Conf. and the Shared Task: Semantic Textual
Similarity, 2013, pp. 44–52.
[17] G. Meditskos, S. Dasiopoulou, S. Vrochidis, L. Wanner, and I. Kompat-
siaris, “Question answering over pattern-based user models,” in Proc.
of the 12th Int Conf. on Semantic Systems, 2016, pp. 153–160.
[18] J. Li, W. Monroe, A. Ritter, M. Galley, J. Gao, and D. Jurafsky,
“Deep reinforcement learning for dialogue generation,” arXiv preprint
arXiv:1606.01541, 2016.
[19] F. Sukno and other, “A multimodal annotation schema for non-verbal
affective analysis in the health-care domain,” in Proc. of the 1st
Int. Workshop on Multimedia Analysis and Retrieval for Multimodal
Interaction, 2016, pp. 9–14.
[20] H. Ajami and H. Mcheick, “Ontology-based model to support ubiqui-
tous healthcare systems for copd patients,” Electronics, vol. 7, no. 12,
2018, p. 371.
... The Dialogue Management system is based on the hybrid dialogue framework proposed in Kamateri et al. (2019), which consists of five major components: a) the contextual modelling and representation, b) the topic detection, c) the dialogue coordination, d) the semantic intelligence and e) the strategy selection. Figure 4 shows the components and the way they interact. ...
Article
Full-text available
The details presented in this article revolve around a sophisticated monitoring framework equipped with knowledge representation and computer vision capabilities, that aims to provide innovative solutions and support services in the healthcare sector, with a focus on clinical and non-clinical rehabilitation and care environments for people with mobility problems. In contemporary pervasive systems most modern virtual agents have specific reactions when interacting with humans and usually lack extended dialogue and cognitive competences. The presented tool aims to provide natural human-computer multi-modal interaction via exploitation of state-of-the-art technologies in computer vision, speech recognition and synthesis, knowledge representation, sensor data analysis, and by leveraging prior clinical knowledge and patient history through an intelligent, ontology-driven, dialogue manager with reasoning capabilities, which can also access a web search and retrieval engine module. The framework’s main contribution lies in its versatility to combine different technologies, while its inherent capability to monitor patient behaviour allows doctors and caregivers to spend less time collecting patient-related information and focus on healthcare. Moreover, by capitalising on voice, sensor and camera data, it may bolster patients’ confidence levels and encourage them to naturally interact with the virtual agent, drastically improving their moral during a recuperation process.
... Another similar approach was implemented for the healthcare domain and the goal was to produce a framework that can assist patients by providing them an AI chatbot with strong conversational skills and a robust Knowledge Base source [20,21]. ...
Chapter
The paper presents recent work on the design and development of AI chatbots for museums using Knowledge Graphs (KGs). The utilization of KGs as a key technology for implementing chatbots raises not only issues related to the representation and structuring of exhibits’ knowledge in suitable formalism and models, but also issues related to the translation of natural language dialogues to and from the selected technology for the formal representation and structuring of information and knowledge. Moreover, such a translation must be as transparent as possible to visitors, towards a realistic human-like question-answering process. The paper reviews and evaluates a number of recent approaches for the use of KGs in developing AI chatbots, as well as key tools that provide solutions for natural language translation and the querying of Knowledge Bases and Linked Open Data sources. This evaluation aims to provide answers to issues that are identified within the proposed MuBot approach for designing and implementing AI chatbots for museums. The paper also presents Cretan MuBot, the first experimental KG/Ontology-based AI chatbot of the MuBot Platform, which is under development in the Heracleum Archaeological Museum.
Article
Full-text available
Over the past 30 years, information technology has gradually transformed the way health care is provisioned for patients. Chronic Obstructive Pulmonary Disease (COPD) is an incurable malady that threatens the lives of millions around the world. The huge amount of medical information in terms of complex interdependence between progression of health problems and various other factors makes the representation of data more challenging. This study investigated how formal semantic standards could be used for building an ontology knowledge repository to provide ubiquitous healthcare and medical recommendations for COPD patient to reduce preventable harm. The novel contribution of the suggested framework resides in the patient-centered monitoring approach, as we work to create dynamic adaptive protection services according to the current context of patient. This work executes a sequential modular approach consisting of patient, disease, location, devices, activities, environment and services to deliver personalized real-time medical care for COPD patients. The main benefits of this project are: (1) adhering to dynamic safe boundaries for the vital signs, which may vary depending on multiple factors; (2) assessing environmental risk factors; and (3) evaluating the patient’s daily activities through scheduled events to avoid potentially dangerous situations. This solution implements an interrelated set of ontologies with a logical base of Semantic Web Rule Language (SWRL) rules derived from the medical guidelines and expert pneumologists to handle all contextual situations.
Article
Full-text available
Keeping the dialogue state in dialogue systems is a notoriously difficult task. We introduce an ontology-based dialogue manage(OntoDM), a dialogue manager that keeps the state of the conversation, provides a basis for anaphora resolution and drives the conversation via domain ontologies. The banking and finance area promises great potential for disambiguating the context via a rich set of products and specificity of proper nouns, named entities and verbs. We used ontologies both as a knowledge base and a basis for the dialogue manager; the knowledge base component and dialogue manager components coalesce in a sense. Domain knowledge is used to track Entities of Interest, i.e. nodes (classes) of the ontology which happen to be products and services. In this way we also introduced conversation memory and attention in a sense. We finely blended linguistic methods, domain-driven keyword ranking and domain ontologies to create ways of domain-driven conversation. Proposed framework is used in our in-house German language banking and finance chatbots. General challenges of German language processing and finance-banking domain chatbot language models and lexicons are also introduced. This work is still in progress, hence no success metrics have been introduced yet.
Conference Paper
Full-text available
Most of today's task-based spoken dialogue systems perform poorly if the user goal is not within the system's task domain. On the other hand, chatbots cannot perform tasks involving robot actions but are able to deal with unforeseen user input. To overcome the limitations of each of these separate approaches and be able to exploit their strengths, we present and evaluate a fully autonomous robotic system using a novel combination of task-based and chat-style dialogue in order to enhance the user experience with human-robot dialogue systems. We employ Reinforcement Learning (RL) to create a scalable and extensible approach to combining chat and task-based dialogue for multimodal systems. In an evaluation with real users, the combined system was rated as significantly more " pleasant " and better met the users' expectations in a hybrid task+chat condition, compared to the task-only condition, without suffering any significant loss in task completion.
Conference Paper
Full-text available
In this paper we present an ontology-driven framework for natural language question analysis and answering over user models (e.g. preferences, habits and health problems of individuals) that are formally captured using ontology design patterns. Pattern-based modelling is extremely useful for capturing n-ary relations in a well-defined and axiomatised manner, but it introduces additional challenges in building NL interfaces for accessing the underlying content. This is mainly due to the encapsulation of domain semantics inside conceptual layers of abstraction (e.g. using reification or container classes) that demand flexible, context-aware approaches for query analysis and interpretation. We describe the coupling of a frame-based formalisation of natural language user utterances with a context-aware query interpretation towards question answering over pattern-based RDF knowledge bases. The proposed framework is part of a human-like socially communicative agent that acts as an intermediate between elderly migrants and care personnel, assisting the latter to solicit personal information about care recipients (e.g. medical history, care needs, preferences, routines, habits, etc.).
Chapter
Dialogue management (DM) is a difficult problem. We present OntoVPA, an Ontology-Based Dialogue Management System (DMS) for Virtual Personal Assistants (VPAs). The features of OntoVPA are offered as generic solutions to core DM problems, such as dialogue state tracking, anaphora and coreference resolution, etc. To the best of our knowledge, OntoVPA is the first commercially available, fully implemented DMS that employs ontologies and ontology-based rules for (a) domain model representation and reasoning, (b) dialogue representation and state tracking, and (c) response generation. OntoVPA is a declarative, knowledge-based system which can be customized to a new VPA domain by modifying and exchanging ontologies and rule bases, with very little to no conventional programming required.
Conference Paper
Access to health care related information can be vital and should be easily accessible. However, immigrants often have difficulties to obtain the relevant information due to language barriers and cultural differences. In the KRISTINA project, we address those difficulties by creating a socially competent multimodal dialogue system that can assist immigrants in getting information about health care related questions. Dialogue management, as core component responsible for the system behaviour, has a significant impact on the successful reception of such a system. Hence, this work presents the specific challenges of the KRISTINA project to adaptive dialogue management, namely the handling of a large dialogue domain and the cultural adaptability required by the envisioned dialogue system, and our approach to handling them.
Conference Paper
A common conversation between an older adult and a nurse about health-related issues includes topics such as troubles with sleep, reasons for walking around nighttime, pain conditions, etc. Such a dialogue can be regarded as a "natural" dialogue emerging from the participating agents' lines of thinking, their roles, needs and motives, while switching between topics as the dialogue unfolds. The purpose of this work is to define a generic conceptual model of purposeful human-agent dialogue activity including different types of argumentation dialogues, suitable for health-related topics. This is done based on analyses of a scenario, persona and models of human behaviour. The model will be shared between the human and the agent, allowing for adaptation to the human's reasoning, needs and motives.
Conference Paper
There exists a growing trend in using NLIs (Natural Language Interfaces) that ranges from research to commercial products. Conversational agents beneath these interfaces have become more sophisticated, being able to either perform a task in behalf of the user or give a precise response to a question as Question Answering systems do. When combining Conversational Agents with QA capabilities the maintenance cost exponentially increases. In this paper we propose a hybrid architecture for a Question Answering system that features social dialog. We claim that including social dialog in QA systems increases users satisfaction and makes them easily engage with the system. Finally, we present an evaluation that supports these hypotheses.