Article

A Transparent and Decentralized Model of Perception and Action for Intelligent Virtual Agents

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Intelligent virtual agent behaviour is a crucial element of any virtual environment application as it essentially brings the environment to life, introduces believability and realism and enables complex interactions and evolution over time. However, the development of mechanisms for virtual agent perception and action is neither a trivial nor a straightforward task. In this paper we present a model of perception and action for intelligent virtual agents that meets specific requirements and can as such be systematically implemented, can seamlessly and transparently integrate with knowledge representation and intelligent reasoning mechanisms, is highly independent of virtual world implementation specifics, and enables virtual agent portability and reuse.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Other recent essays in this direction are Goertzel et al., 2013, McRorie et al. (2012, Gratch et al. (2013), Faur et al., (2013), Kang and Tan (2013), Anastassakis and Panayiotopoulos (2014), Haubrich et al., (2015). For more advanced issues, see Bostrom (2012), Hughes et al. (2012), Schönbrodt and Asendorpf (2011). ...
Chapter
This introductory essay aims to introduce the chapters in the book presenting some aspects of the theoretical and conceptual framework necessary to consider the advantages computer simulation techniques and technologies offer to historical disciplines, but also quoting from the hundreds of examples in current scientific literature to give a context within which the individual contributions can be understood better. We argue that historical simulations should be much more than vivid illustrations of what scholars believe in the present existed in the past. A simulation is basically the computer representation of a “mechanism”, representing how social intentions, goals and behaviors were causally connected in the past. This can be done by formulating a “generative model”, that is, a model of a set of mechanisms. In this chapter, it is suggested that computer simulation may act as a Virtual Laboratory to help studying how human societies have experimented relevant transformations and in which way the consequences of those transformations in technology, activities, behavior, organization or knowledge were transmitted to other social agents or groups of social agents. Building artificial societies inside a computer allows us to understand that social reality is not capricious. It has been produced somehow, although not always the same cause produces the same effect, because social actions are not performed in isolation, but in complex and dialectical frameworks, which favor, prevent, or modify the capacity, propensity, or tendency the action has to produce or to determine a concrete effect. This way of studying social dynamics in the past by means of computer simulations is beginning to abandon its infancy. Archaeologists and historians have started to convert social theories in computer programs trying to simulate social process and experiment with different explanations about known archaeological societies. Our book is just one additional example of a current trend among archaeologists and historians: historical events occurred only once and many years ago but within a computer surrogates of those events can be artificially repeated here and now for understanding how and why they happened.
... However, the FIPA provides the agent specifications only. There are still no standards for developing agent-based systems in manufacturing domain [39]. Also, for security and practical reasons, industrial companies cannot accept GPL or LGPL licenses of open source tools and commercial tools that are not suitable for industrial application developments given the cost-effective analysis. ...
Article
In a ubiquitous environment, various enterprise information systems (EISs) are used for supporting daily operations. Besides, with the support with Auto-ID technology, enterprises are able to collect real-time operation data but these data are continuously pushed to different EISs. Data without processing are meaningless and not able to support managerial decision-makings. The wide use of EISs and Auto-ID devices within an enterprise increases the difficulties in data interoperability among difficult data sources. Since data sources from applications and devices are characterized by multiple types of heterogeneities, such as communication protocols, blinding methods, and developing environments, the difficulty in managing heterogeneous data sources is highly raised. Information integration in ubiquitous enterprises is critical because it has a significant influence on the efficiency and effectiveness of decisions. This article presents an innovative Data-Source Interoperability Service (DSIS) that serves as a middleware for providing a querying and information integration service for heterogeneous data sources. The DSIS applies software agent technology that is capable of accomplishing tasks in an autonomous way without human intervention. Operations provided by DSIS Agent are converted into standard web services. The agent-based services are managed by DSIS Universal Description, Discovery and Integration registry (DSIS-UDDI) which facilitates collaboration among agents. Additionally, the DSIS platform also provides set of visual tools for users to manage and (re)configure different data sources within enterprises.
Chapter
This article describes how management education is engaged in significant programmatic reforms in response to the business community's call for web-savvy, problem-solving graduates. Web-based intelligent tutors provide a readily accessible vehicle for enhancing business students' learning performance as well as prepare them for the rigors of the global marketplace. A primary goal of these AI-based systems is to approach Bloom's two-sigma learning performance standard. Bloom found that average students tutored one-to-one with mastery learning techniques performed two standard deviations better than students who learned via conventional teaching methods. Intelligent tutors can also be used to identify students at risk, to formulate appropriate intervention plans, and to support team learning. The purpose of this article is to highlight the growing potential for using intelligent tutors to enhance student and team learning opportunities and outcomes and to outline strategies for implementing this revolutionary process throughout the management education community of practice.
Article
This article describes how management education is engaged in significant programmatic reforms in response to the business community's call for web-savvy, problem-solving graduates. Web-based intelligent tutors provide a readily accessible vehicle for enhancing business students' learning performance as well as prepare them for the rigors of the global marketplace. A primary goal of these AI-based systems is to approach Bloom's two-sigma learning performance standard. Bloom found that average students tutored one-to-one with mastery learning techniques performed two standard deviations better than students who learned via conventional teaching methods. Intelligent tutors can also be used to identify students at risk, to formulate appropriate intervention plans, and to support team learning. The purpose of this article is to highlight the growing potential for using intelligent tutors to enhance student and team learning opportunities and outcomes and to outline strategies for implementing this revolutionary process throughout the management education community of practice.
Article
This paper develops artificial life patterned after animals as evolved as those in the superclass Pisces. It demonstrates a virtual marine world inhabited by realistic artificial fishes. Our algorithms emulate not only the appearance, movement, and behavior of individual animals, but also the complex group behaviors evident in many aquatic ecosystems. We model each animal holistically. An artificial fish is an autonomous agent situated in a simulated physical world. The agent has (i) a three-dimensional body with internal muscle actuators and functional fins, which deforms and locomotes in accordance with biomechanic and hydrodynamic principles, (ii) sensors, including eyes that can image the environment, and (iii) a brain with motor, perception, behavior, and learning centers. Artificial fishes exhibit a repertoire of piscine behaviors that rely on their perceptual awareness of their dynamic habitat. Individual and emergent collective behaviors include caudal and pectoral loc...
Conference Paper
Full-text available
Modern virtual agents require knowledge about their environment, the interaction itself, and their interlocutors’ behavior in order to be able to show appropriate nonverbal behavior as well as to adapt dialog policies accordingly. Recent achievements in the area of automatic behavior recognition and understanding can provide information about the interactants’ multimodal nonverbal behavior and subsequently their affective states. In this paper, we introduce a perception markup language (PML) which is a first step towards a standardized representation of perceived nonverbal behaviors. PML follows several design concepts, namely compatibility and synergy, modeling uncertainty, multiple interpretative layers, and extensibility, in order to maximize its usefulness for the research community. We show how we can successfully integrate PML in a fully automated virtual agent system for healthcare applications.
Article
Full-text available
Believability has been a perennial goal for the intelligent virtual agent community. One important aspect of believability largely consists in demonstrating autonomous behavior, consistent with the agent's personality and motivational state, as well as the world conditions. Autonomy, on behalf of the agent, implies the existence of an internal structure and mechanism that allows the agent to have its own needs and interests, based on which the agent will dynamically select and generate goals that will in turn lead to self-determined behavior. Intrinsic motivation allows the agent to function and demonstrate behavior, even when no external stimulus is present, due to the constant change of its internal emotional and physiological state. The concept of motivation has already been investigated by research works on intelligent agents, trying to achieve autonomy. The current work presents an architecture and model to represent and manage internal driving factors in intelligent virtual agents, using the concept of motivations. Based on Maslow and Alderfer's bio-psychological needs theories, we present a motivational approach to represent human needs and produce emergent behavior through motivation synthesis. Particular attention is given to basic, physiological level needs, which are the basis of behavior and can produce tendency to action even when there is no other interaction with the environment.
Chapter
Full-text available
Perception and attention mechanisms are of great importance for entities situated within complex dynamic environments. With roles extending greatly beyond passive information services about the external environment, such mechanisms actively prioritise, augment and expedite information to ensure that the potentially relevant is made available so appropriate action can take place. Here, we describe the rationale behind endowing artificial entities, or virtual agents, with real-time perception and attention systems. We cover the fundamentals of designing and building such systems. Once equipped, the resulting agents can achieve a more substantial connection with their environment for the purposes of reacting, planning, decision making and, ultimately, behaving.
Conference Paper
Full-text available
The development of complex interactive 3D systems raises the need for representations supporting more abstract descriptions of world objects, their behaviour and the world dynamics. The inclusion of Artificial Intelligence representations and their use within 3D graphic worlds face both fundamental and technical issues due to the difference in representational logic between computer graphics and knowledge-based systems. We present a framework for such an integration illustrated by a first prototype.
Conference Paper
Full-text available
In this paper we discuss a modular multi-sense perception system for DIVAs virtual agents. It is based on the idea that local environmental influences are constantly sensed by one or more of an agent's senses (sight, hearing, smell, etc.). The perception system is extensible and modifiable and allows users to dynamically modify how an agent perceives its environment at execution time. This paper focuses on the modularity of the agent perception architecture rather than the perception algorithms themselves.
Conference Paper
Full-text available
Nowadays, more Virtual Environments (VEs) are becoming available on the Web. This means that VEs are becoming accessible to a larger and more diverse audience. It also means that it is more likely that the use of these VEs (i.e. how to interact with the virtual environment and the meanings of the associated virtual objects) may be different for different groups of persons. In order for a VE to be a success on the Web, end-users should easily get familiar with the VE and understand the meanings of its virtual objects. Otherwise, the end-user may be tempted to quit the VE. Therefore, annotations and the creation of navigation paths for virtual tour guides become important to ease the use of VEs. Most of the time, this is done by VR-experts and the annotations are very poor and often only text based. This paper describes an approach and associated tool that allows a layman to add or update annotations to existing VEs. In addition, annotations are not limited to text but may also be multimedia elements, i.e. images, videos, sounds. Furthermore, the approach (and the tool) also allows easy creation of navigation paths and tour guides, which can be used to adapt a VE to the needs of a user. The paper illustrates the results by means of a real case, which is a reconstruction of a coalmine site for a museum.
Article
Full-text available
We introduce an interface for connecting agent platforms to environments. This interface provides generic functionality for executing actions and for perceiving changes in an agent’s environment. It also provides support for managing an environment, e.g., for starting, pausing and terminating it. Among the benefits of such an interface are (1) standard functionality is provided by the interface implementation itself, and (2) agent platforms that support the interface can connect to any environment that implements the interface. This significantly reduces effort required from agent and environment programmers as the environment code needed to implement the interface needs to be written only once. We propose that the interface presented may be used as a standard that enables agents to control entities in environments. Our starting point for designing such a generic interface is based on a careful study of the various interfaces used by different agent programming languages to connect agent programs to environments. We discuss several case studies that use our interface (an elevator simulator, the well-known agent contest, and an implementation of the interface to connect agents to bots in Unreal Tournament 2004).
Article
Full-text available
The simulation of synthetic humans inhabiting virtual environments is a current research topic with a great number of behavioral problems to be tackled. Semantical Virtual Environments (SVEs) have recently been proposed not only to ease world modelling but also to enhance agent-object and agent-agent interaction. Thus, we propose the use of ontologies to define the world's knowledge base and to introduce semantic levels of detail that help the sensorization of complex scenes - containing lots of interactive objects. The object taxonomy also helps to create general and reusable operativity for autonomous characters - for example, liquids can be poured from containers such as bottles. On the other hand, we use the ontology to define social relations among agents within an artificial society. These relations must be taken into account in order to display socially acceptable decisions. Therefore, we have implemented a market-based social model that reaches coordination and sociability by means of task exchanges. This paper presents a multi-agent framework oriented to simulate socially intelligent characters in Semantic virtual environments. The framework has been successfully tested in 3D dynamic scenarios while simulating a virtual university bar, where groups of waiters and customers interact with both the objects in the scene and the other virtual agents finally displaying complex social behaviors.
Article
Full-text available
In this paper we present an object representation based on the semantics and functionality of interactive digital items - virtual objects - within a virtual environment (VE). We consider every object participating in a VE application not only as a 3D shape, but as a dynamic entity with multiple visual representations and functionalities. This allows for dynamically scaling and adapting the object's geometry and functions to different scenarios. Some test applications are described to demonstrate the benefits of the semantics-based representation of interactive virtual environments, including autonomous characters and collaborative environments
Article
Full-text available
A close coupling of perception and action processes is assumed to play an important role in basic capabilities of social interaction, such as guiding attention and observation of others' behavior, coordinating the form and functions of behavior, or grounding the understanding of others' behavior in one's own experiences. In the attempt to endow artificial embodied agents with similar abilities, we present a probabilistic model for the integration of perception and generation of hand-arm gestures via a hierarchy of shared motor representations, allowing for combined bottom-up and top-down processing. Results from human-agent interactions are reported demonstrating the model's performance in learning, observation, imitation, and generation of gestures.
Article
Full-text available
This research proposes a computational framework for generating visual attending behavior in an embodied simulated human agent. Such behaviors directly control eye and head motions, and guide other actions such as locomotion and reach. The implementation of these concepts, referred to as the AVA, draws on empirical and qualitative observations known from psychology, human factors and computer vision. Deliberate behaviors, the analogs of scanpaths in visual psychology, compete with involuntary attention capture and lapses into idling or free viewing. Insights provided by implementing this framework are: a defined set of parameters that impact the observable effects of attention, a defined vocabulary of looking behaviors for certain motor and cognitive activity, a defined hierarchy of three levels of eye behavior (endogenous, exogenous and idling) and a proposed method of how these types interact.
Article
World representation is a crucial aspect of any design and development effort in the field of Virtual Environments today. Approaches presented so far don't seem to address all relevant needs. In this paper, we propose a model for the representation of physical properties, functionality and semantics of objects in virtual environments. Our proposed model aims to be uniform, pragmatic, systematic and extendible. We demonstrate practical usage of the proposed model through a complete example. Also, we briefly describe two case studies of fully-implemented virtual environment applications that encompass the proposed model in the context of diverse virtual agent interactions.
Conference Paper
In order to create realistic simulations,virtual agents need to learn about their environment through perception. To date, most multi-agent simulation systems that implement some form of perception have focused heavily on a single sense, vision. In this paper we discuss a multi-sense perception system for virtual agents situated in large scale open environments. The perception system consists of modules (i.e., sensors) for visual, audible and olfactory senses. It also includes a perception combination module that combines data received from the multiple sensors into useful knowledge.
Article
Real-time 3D graphics are being extensively used to build interactive virtual environments for a number of different applications. In many situations, virtual objects are required to exhibit complex behaviors and the use of a versatile behavioral programming methodology is required. This paper presents a feature modeling approach to define behavioral information of virtual objects, giving special attention to their capabilities of interaction with virtual human actors. The proposed method is based on a complete definition and representation of interactive objects, which are called smart objects. This representation is based on the description of interaction features: parts, movements, graspable sites, functionalities, purposes, etc., including a behavioral automata based on scripts and graphical state machines.
Article
An important property of Intelligent Virtual Agents (IVAs) is their capability to acquire/perceive information from their environment. Bearing in mind some studies on situational awareness – where sensitive perception can be understood as the first level of awareness – and taking into account one of the most successful awareness models in Computer Supported Cooperative Work (CSCW) – the Spatial Model of Interaction (SMI), we have developed a human-like perceptual model for IVAs. This perceptual model extends and reinterprets the key concepts of the SMI and also introduces some factors typical from human being perception with the aim of making perception, in this kind of systems, more human-like.
Article
This paper describes our proposal for a new representation model for virtual worlds. Our model provides a layer of semantic information for virtual environments that does not intend to substitute application dependant approaches, but to constitute a common lower level for all of them. Our model is simple in that it consists of a reduced number of different elements and most of them can be automatically annotated. The paper shows the ideal annotation process. In this sense, we show our own experiences in annotating worlds. The model has already been successfully used in two different applications
Article
This paper develops artificial life patterned after animals as evolved as those in the superclass Pisces. It demonstrates a virtual marine world inhabited by realistic artificial fishes. Our algorithms emulate not only the appearance, movement, and behavior of individual animals, but also the complex group behaviors evident in many aquatic ecosystems. We model each animal holistically. An artificial fish is an autonomous agent situated in a simulated physical world. The agent has (i) a three-dimensional body with internal muscle actuators and functional fins, which deforms and locomotes in accordance with biomechanic and hydrodynamic principles, (ii) sensors, including eyes that can image the environment, and (iii) a brain with motor, perception, behavior, and learning centers. Artificial fishes exhibit a repertoire of piscine behaviors that rely on their perceptual awareness of their dynamic habitat. Individual and emergent collective behaviors include caudal and pectoral loc...