Article

State Space Construction for Behavior Acquisition in Multi Agent Environments with Vision and Action

Dept. of Adaptive Machine Syst., Osaka Univ.
11/1998; DOI: 10.1109/ICCV.1998.710819
Source: IEEE Xplore

ABSTRACT

This paper proposes a method which estimates the relationships between learner's behaviors and other agents' ones in the environment through interactions (observation and action) using the method of system identication. In order to identify the model of each agent, Akaike's Information Criterion is applied to the results of Canonical Variate Analysis for the relationship between the observed data in terms of action and future observation. Next, reinforcement learning based on the estimated state vectors is performed to obtain the optimal behavior. The proposed method is applied to a soccer playing situation, where a rolling ball and other moving agents are well modeled and the learner's behaviors are successfully acquired by the method. Computer simulations and real experiments are shown and a discussion is given. 1 Introduction Building a robot that learns to accomplish a task through visual information has been acknowledged as one of the major challenges facing vision, robotics, a...

Download full-text

Full-text

Available from: Koh Hosoda, Jan 30, 2015
  • Source
    • "Robot soccer is a good domain for researchers to study the multi-agent cooperation problem. Under the robot soccer simulation environment, Stone and Veloso [2] proposed a layered learning method that consists of two levels of learned behaviors; Uchibe [3] et al proposed a scheme in which the relationship between a learner's behaviors and those of other robots is estimated based on the method of system identification and the cooperative behaviors is acquired by reinforcement learning. These methods have developed efficient action selections of individual robot. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In a multi robots environment, the overlap of actions selected by each robot makes the acquisition of cooperation behaviors less efficient. An approach is proposed to determine the action selection priority level based on which the cooperative behaviors can be readily controlled. First, eight levels are defined for the action selection priority, which can be correspondingly mapped to eight subspaces of actions. Second, using the local potential field method, the action selection priority level for each robot is calculated and thus its action subspace is obtained. Then, Reinforcement learning (RL) is utilized to choose a proper action for each robot in its action subspace. Finally, the proposed method is implemented in a soccer game and the high efficiency of the proposed scheme was verified by the result of both the computer simulation and the real experiments.
    Preview · Article · Sep 2002
  • Source
    • "In order to construct the local predictive model of other agents, Akaike's Information Criterion(AIC) [1] is applied to the result of Canonical Variate Analysis(CVA) [5]. We just briefly explained the method (for the details of the local predictive model, see [9] [10]). CVA uses a discrete time, linear, state space model as follows: "
    [Show abstract] [Hide abstract]
    ABSTRACT: Discusses how a robot can develop its state vector according to the complexity of the interactions with its environment. A method for controlling the complexity is proposed for a vision-based mobile robot whose task is to shoot a ball into a goal avoiding collisions with a goalkeeper. First, we provide the most difficult situation (the maximum speed of the goalkeeper with chasing-a-ball behavior), and the robot estimates the full set of state vectors with the order of the major vector components by a method of system identification. The environmental complexity is defined in terms of the speed of the goalkeeper while the complexity of the state vector is the number of the dimensions of the state vector. According to the increase of the speed of the goalkeeper, the dimension of the state vector is increased by taking a trade-off between the size of the state space (the dimension) and the learning time. Simulations are shown, and other issues for the complexity control are discussed
    Full-text · Conference Paper · Jun 1998
  • Source
    • "The rest of this article is structured as follows: at first we show our basic idea, then we give brief explanation of the local predictive model and reinforcement learning. The details of the local predictive model and learning algorithms are described in [8] and [7], respectively. Finally, we show simulation results and real experiments and give a discussion. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper proposes a method that acquires robots' behaviors based on the estimation of the state vectors. In order to acquire the cooperative behaviors in multi-robot environments, each learning robot estimates the local predictive model between the learner and the other objects separately. Based on the local predictive models, the robots learn the desired behaviors using reinforcement learning. The proposed method is applied to a soccer playing situation, where a rolling ball and other moving robots are well modeled and the learner's behaviors are successfully acquired by the method. Computer simulations and real experiments are shown and a discussion is given
    Full-text · Conference Paper · Jun 1998
Show more