Roberta Catizone’s research while affiliated with Florida Institute for Human and Machine Cognition and other places

The paper describes a prototype Embodied Conversational Agent or Companion, called CALONIS, for a brain-injured Veteran with severe cognitive impairment. The CALONIS project is a sub-project of the larger Tampa VA SmartHome implementation initiative. CALONIS is intended to provide increased engagement, diversion and assistance beyond the usual mechanisms of providing assistance through text based prompting and interactions. We hope to eventually integrate CALONIS fully into the next generation of the Tampa VA SmartHome in which the SmartHome itself becomes a fully interactive and intelligent electronic caregiver. The project began with a Wizard-of-Oz version of CALONIS but even at this early stage we appear to have achieved high levels of patient engagement as well as in relation to the caregiver. The full CALONIS prototype is based on the Senior Companion project, originally developed as part of a large-scale EU project [1].

This paper describes an initial prototype of the Companions project (www.companions-project.org): the Senior Companion (SC), designed to be a platform to display novel approaches to:(1)The use of Information Extraction (IE) techniques to extract the content of incoming dialogue utterances after an ASR phase.(2)The conversion of the input to RDF form to allow the generation of new facts from existing ones, under the control of a Dialogue Manager (DM), that also has access to stored knowledge and knowledge accessed in real time from the web, all in RDF form.(3)A DM expressed as a stack and network virtual machine that models mixed initiative in dialogue control.(4)A tuned dialogue act detector based on corpus evidence.The prototype platform was evaluated, and we describe this; it is also designed to support more extensive forms of emotion detection carried by both speech and lexical content, as well as extended forms of machine learning. We describe preliminary studies and results for these, in particular a novel approach to enabling reinforcement learning for open dialogue systems through the detection of emotion in the speech signal and its deployment as a form of a learned DM, at a higher level than the DM virtual machine and able to direct the SC's responses to a more emotionally appropriate part of its repertoire.

Several dialogue management (DM) architectures and conversational speech for dialogue systems are presented. Basic types of DM systems include dialogue grammars and frames, plan-based and collaborative systems, and conversational games theory. DM architectures include SmartKom, Trindi, WITAS, CONVERSE, COMIC, agent-based dialogue management, and DM and automatic speech recognition (ASR) language modeling. All data collection tasks should be tailored for the conversational scenario under consideration as each scenario can present different properties. It is shown in the multimodal dialogue system that turn taking can usually be achieved by a fusion of gesture, gaze, and intonation. Intonation within the speech signal informs the dialogue manager when new information is introduced into the current conversation. By placing established emotion detection methods within the recursive nature of conversation we can consider discourse as the exploitation of the shared set of interaction affordances.

COMPANIONS is a concept that aims to change the way we think about the relationships of people to computers and the Internet by developing a virtual 'Companion' to stand between individuals and the torrent of data on the Internet, including their own life information, which will soon be too large for people to handle easily without some new form of assistance. The Companion is intended as an agent or 'presence' that stays with a user for periods of time, longer than in conventional task-based dialogue systems, developing a relationship and 'knowing' and assisting its owner's experiences, preferences, plans, and wishes. The Companions concept aims to model a fuller range of conversation than has been done hitherto, both task and non-task based, and discusses what properties people will want in a long term computer Companion that is also an Internet agent in a new form.

Companions are agents devised to accompany users day by day building long-term relationships with them. They do not only assist users for particular tasks in sporadic times, but they provide more support and have more information to adapt themselves to each users' needs. Currently, these agents and their possibilities are being researched as a part of an EU project, which is described in this chapter.

This paper presents an overview of a cognitive architecture, OntoAgent, that supports the creation and deployment of intelligent agents capable of simulating human-like abilities. The agents, which have a simulated mind and, if applicable, a simulated body, are intended to operate as members of multi-agent teams featuring both artificial and human agents. The agent architecture and its underlying knowledge resources and processors are being developed in a sufficiently generic way to support a variety of applications. In this paper we briefly describe the architecture and two applications being configured within it: the Maryland Virtual Patient (MVP) system for training medical personnel and the CLinician's ADvisor (CLAD). We organize the discussion around four aspects of agent modeling and how they are utilized in the two applications: physiological simulation, modeling an agent's knowledge and learning, decision-making and language processing.

What will it be like to admit Artificial Companions into our society? How will they change our relations with each other? How important will they be in the emotional and practical lives of their owners – since we know that people became emotionally dependent even on simple devices like the Tamagotchi? How much social life might they have in contacting each other? The contributors to this book discuss the possibility and desirability of some form of long-term computer Companions now being a certainty in the coming years. It is a good moment to consider, from a set of wide interdisciplinary perspectives, both how we shall construct them technically as well as their personal philosophical and social consequences. By Companions we mean conversationalists or confidants – not robots – but rather computer software agents whose function will be to get to know their owners over a long period. Those may well be elderly or lonely, and the contributions in the book focus not only on assistance via the internet (contacts, travel, doctors etc.) but also on providing company and Companionship, by offering aspects of real personalization.

This paper presents a case study showing how hybrid methods of knowledge elicitation can be used to build models in support of the functioning of intelligent agents. What facilitates both the elicitation of knowledge and its conversion into actionable models is the use of a unified representational knowledge scheme - specifically, an unambiguous, ontologically grounded metalanguage that serves as the language of all recorded knowledge as well as the language in which agents remember and reason.

This paper describes an initial prototype demonstrator of a Companion, designed as a platform for novel approaches to the following: 1) The use of Information Extraction (IE) techniques to extract the content of incoming dialogue utterances after an Automatic Speech Recognition (ASR) phase, 2) The conversion of the input to Resource Descriptor Format (RDF) to allow the generation of new facts from existing ones, under the control of a Dialogue Manger (DM), that also has access to stored knowledge and to open knowledge accessed in real time from the web, all in RDF form, 3) A DM implemented as a stack and network virtual machine that models mixed initiative in dialogue control, and 4) A tuned dialogue act detector based on corpus evidence. The prototype platform was evaluated, and we describe this briefly; it is also designed to support more extensive forms of emotion detection carried by both speech and lexical content, as well as extended forms of machine learning.