No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
An Overview on Opponent Modeling in RoboCup Soccer Simulation 2D
Abstract
This paper reviews the proposed opponent modeling algorithms within the soccer simulation domain. RoboCup soccer simulation 2D is a rich multi agent environment where opponent modeling plays a crucial role. In multi agent systems with adversarial and cooperative agents, team agents should be adapted to the current environment and opponent in order to propose appropriate and effective counteractions. Predicting the opponent's future behaviors during competition allows for more informed decisions. We divide opponent modeling into two categories of individual agent behaviors and team behaviors. Individual behaviors concern modeling the low-level behaviors of individual opponent agents, however in team behaviors, the high-level strategy of the entire team like formation, offensive and defensive system, is recognized. Several methods have been proposed to create different models of opponents to improve the performance of teams in an essential aspect. In this paper, we review the approaches to the problem of opponent modeling published from 2000 to 2010.
... Високорівневі навички є комбінацією низькорівневих команд та допоміжних алгоритмів, таких як: розрахунок прискорення об'єкту, швидкості, кута до цільового об'єкта та інші. Також навички діляться на два типи: дії з м'ячем і без нього (рис. 1) [5]. ...
У даній статті наведено основні принципи та підходи до командної взаємодії інтелектуальних агентів, що використовуються у всесвітніх змаганнях 2D RoboCup simulation league. Відповідно до підходу – «Drop-in player challenge», розроблено та описано базові моделі поведінки, що були протестовані в середовищі моделювання 2D RoboCup simulator. Стаття стисло описує суть моделей поведінки супроводжуючи ілюстраціями та хід проведення експериментального тестування з результатами у вигляді таблиць. Підходом «Drop-in player challenge» почали займатися відносно недавно. Багато відомих команд, такі як Austrian Kangaroos, B-Human, Cerberus, що змагаються восновних лігах почали брати участь у змаганнях «змішаних» гравців. Це свідчить про потребу саме в такому типі взаємодії. Основна задача роботи – це вдосконалення взаємодії з союзними роботами, що мають інші кодові бази. Такі роботи не мають змоги комунікувати – тільки аналізувати дії союзних гравців і бути корисними для досягнення спільної мети. Також присутній елемент внутрішньо-командної конкуренції, коли задача агенту не тільки максимально допомогти союзним роботам, а і стати найрезультативнішим гравцем в команді. Тема основної роботи охоплює проблематику окремих випадків тактики гри команди, на кшталт, гри з застосуванням тільки нападаючих моделей поведінки, гри з застосуванням тільки оборонних моделей, різних комбінацій моделей поведінки для визначення глобального оптимуму виграшної тактики. За результатами тестування було визначено ефективність розроблених моделей поведінки, найкращі співвідношення застосувань моделей на полі, а також протестовані пограничні випадки застосування моделей, що підтвердило теоретичні передбачення. Бібл. 6, іл. 7, табл. 4.
... Unsurprisingly, these properties have attracted the attention of many researchers studying opponent modeling, and the RoboCup simulation league in particular has supported a variety of opponent modeling research (Pourmehr & Dadkhah, 2011). From here on we will illustrate concepts using RoboCup-based examples, and because of this grounding we will refer to the set of variables relevant to decision making and prediction, both fully and partially observable, as the 'game state', Y . ...
Opponent modeling is the ability to use prior knowledge and observations in order to predict the behavior of an opponent. This survey presents a comprehensive overview of existing opponent modeling techniques for adversarial domains, many of which must address stochastic, continuous, or concurrent actions, and sparse, partially observable payoff structures. We discuss all the components of opponent modeling systems, including feature extraction, learning algorithms, and strategy abstractions. These discussions lead us to propose a new form of analysis for describing and predicting the evolution of game states over time. We then introduce a new framework that facilitates method comparison, analyze a representative selection of techniques using the proposed framework, and highlight common trends among recently proposed methods. Finally, we list several open problems and discuss future research directions inspired by AI research on opponent modeling and related research in other disciplines.
... Other studies and improvements considering our methodology may be developed. For example, once the set of important variables has been identified, it could be possible to recreate possible scenarios using agents simulations [26][27][28] , not only to forecast but also to have a better grasp of the football variables, the variability of the possible outcomes, and suggest ways to design and organize a team to confront a given opposing network. Such analysis will be included in future manuscripts. ...
We study the relevance of considering social network analysis in determining soccer results. As a benchmark, we start using a simple regression model based on past performance to try to determine the main trends of a soccer match based on probabilities of winning, losing or tying, as home or visiting teams. The success of this simple model, based on historical performance, is improved by the addition of network descriptors of both teams in a game. Therefore, such network measures do offer additional useful information in determining match outcomes. We validate our approach using the data of the Spanish League (La Liga) 2012–2013. We observe that betweenness centrality seems to provide additional relevance information related to the performance of a team during the tournament.
... Bakkes et al. (2012) survey methods for player modelling in commercial video games, where the purpose of modelling is to improve the playing strength of game AI as well as player satisfaction. Pourmehr and Dadkhah (2012) provide an overview of modelling methods used in 2D simulated robot soccer, in which two teams of agents compete in a soccer match. Rubin and Watson (2011) survey research in Poker playing agents and dedicate a section to opponent modelling methods. ...
Much research in artificial intelligence is concerned with the development of autonomous agents that can interact effectively with other agents. An important aspect of such agents is the ability to reason about the behaviours of other agents, by constructing models which make predictions about various properties of interest (such as actions, goals, beliefs) of the modelled agents. A variety of modelling approaches now exist which vary widely in their methodology and underlying assumptions, catering to the needs of the different sub-communities within which they were developed and reflecting the different practical uses for which they are intended. The purpose of the present article is to provide a comprehensive survey of the salient modelling methods which can be found in the literature. The article concludes with a discussion of open problems which may form the basis for fruitful future research.
... For example, it is possible to identify task identification with the literature on plan recognition [37], inverse reinforcement learning [1] and related topics, where an agent must identify a target task from the observed behavior of other agent(s). Similarly, it is possible to identify teammate identification with the large body of work on learning in games [20] and opponent modeling [21,48], where an agent must predict the behavior of other agents from observing their actions. Finally, planning is clearly related with the growing body of work on decentralized/distributed planning [27,52]. ...
This paper addresses the problem of ad hoc teamwork, where a learning agent engages in a cooperative task with other (unknown) agents. The agent must effectively coordinate with the other agents towards completion of the intended task, not relying on any pre-defined coordination strategy. We contribute a new perspective on the ad hoc teamwork problem and propose that, in general, the learning agent should not only identify (and coordinate with) the teammates’ strategy but also identify the task to be completed. In our approach to the ad hoc teamwork problem, we represent tasks as fully cooperative matrix games. Relying exclusively on observations of the behavior of the teammates, the learning agent must identify the task at hand (namely, the corresponding payoff function) from a set of possible tasks and adapt to the teammates’ behavior. Teammates are assumed to follow a bounded-rationality best-response model and thus also adapt their behavior to that of the learning agent. We formalize the ad hoc teamwork problem as a sequential decision problem and propose two novel approaches to address it. In particular, we propose (i) the use of an online learning approach that considers the different tasks depending on their ability to predict the behavior of the teammate; and (ii) a decision-theoretic approach that models the ad hoc teamwork problem as a partially observable Markov decision problem. We provide theoretical bounds of the performance of both approaches and evaluate their performance in several domains of different complexity.
Making a correct decision is a difficult task in a Soccer Simulation 2D environment due to the fact that there is a lack of information for each agent. Therefore, coach agent can take role as a mediator for agents to analyze data and inform players about crucial events by sending command messages. This paper proposes a new method to detect the formation of opponents which is not still possible for agents to extract. In the experimental results of this paper, we show that team formation is successfully learned by various well-known classification algorithms.
The paper presents an approach to perform post-hoc analysis of RoboCup Soccer Simulation 3D teams via log files of their matches and to learn a model to classify them not only as being strong, medium or weak but also through their game playing styles such as frequent kickers, frequent dribblers, heavy/lean attackers, etc. The learned model can then be used to further cluster teams to predict game style of similar opponents. We have applied the presented approach to 22 teams from RoboCup 2011 in a fully automated fashion and the results show the validity of our approach.
Opponent modeling [1] is a research area that focuses on the analysis and interpretation of adversary’s actions. This modeling, although, is easy for humans but is a challenging task for autonomous multi-agents [2]. In a multi-agent environment, an opponent typically represents a team of agents who are capable of doing decentralized decision making. Thus, an opponent model consists of a collective set of tactical behaviors, termed as strategy, that are exhibited by agents in a dynamic and uncertain environment. RoboCup Soccer [3] provides such an environment and serves as a test bed for analysis and application of new intelligent algorithms. In the context of RoboCup Soccer Simulation League 2D, opponent modeling has been exhaustively researched and various machine learning techniques have been devised for strategy prediction and identification. However, when it comes to RoboCup Soccer Simulation League 3D the problem is much more sophisticated as identification of basic actions that are being performed by a single agent is itself a complex task. This research aims to develop a framework that takes a team of agents as input, analyzes their set of individual actions as well as coordinated team behavior using a set of pre-defined rules, interprets their strategic pattern and suggests a counter strategy. The framework would involve both offline and online learning. The major benefit of this approach is that if we are able to judge our opponent within the first thirty seconds (or so) then we can apply specific rules for exclusively dealing with it. For example, certain teams make excessive use of kicking the ball while many others have the tendency to dribble the ball to maintain possession of it. If the opponent has strong dribbling skill then our best strategy would be to block the path of the opponent and put most of our players to this job. On the other hand, if the opponent team has a tendency to frequently kick the ball then we would like to keep most of our players around our goal area in order to save the goal (especially if the ball is in our half). Thus, if we are able to classify our opponent based upon certain key attributes then we will be able to devise opponent-specific strategy. The motivation behind this approach is that our team KarachiKoalas [4] is applying this technique and utilizing the features obtained from offline team analysis to build a model of the opponent team within initial seconds of the game.
In dynamic multiagent domains with adversary agents, an agent has to adapt its behavior to the opponent actions in order to in-crease its ability to compete. A frequently used opponent modeling ap-proach in these domains is to rely on an omniscient agent (e.g., a coach in a soccer environment) to classify the opponent and to communicate the opponent's model (or a counter-strategy for that model) to other agents. In this paper, we propose an alternative opponent modeling ap-proach where each agent observes and classifies online the adversaries it encounters into automatically learned models. Thus, our approach re-quires neither an omniscient agent nor pre-defined models. Empirical re-sults obtained in a simulated robotic soccer environment promises a high suitability of this approach for real-time, dynamic, multiagent domains.
By analyzing play history, it is possible to gain critical in-sights about future plays. Plays are sequences of actions to be undertaken by a collection of agents, or teammates. The success of a play depends on a number of factors including, perhaps most importantly, the opponent's play. In this paper, we present an approach for online opponent modeling and il-lustrate how it can be used to improve offensive performance in the Rush 2008 football simulator. In football, team behav-iors have an observable spatio-temporal structure, defined by the relative physical positions of team members over time. We demonstrate that this structure can be exploited to recog-nize football plays at a very early stage of the play using a supervised learning method. Using the recognized defensive play, knowledge about expected outcomes, and spatial simi-larity between offensive plays, we retrieve an offensive play from case base. This play is then reused to improve an in progress offensive play. We call this process a play switch. Empirical results indicate that spatial similarity is central to play retrieval, and that, modifying only a subset of the current play with the retrieved play yields greater improvement.
The area of agent modeling deals with the task of observing other agents and modeling their behavior, in order to predict
their future behavior, coordinate with them, assist them, or counter their actions. Typically, agent modeling techniques assume
the availability of a plan- or behavior-library, which encodes the full repertoire of expected observed behavior. However,
recent applications areas of agent modeling raise challenges to the assumption of such a library, as agent modeling systems
are increasingly used in open and/or adversarial settings, where the behavioral repertoire of the observed agents is unknown at design time. This paper focuses on the challenge
of the unsupervised autonomous learning of the sequential behaviors of agents, from observations of their behavior. The techniques
we present translate observations of the dynamic, complex, continuous multi-variate world state into a time-series of recognized
atomic behaviors. This time-series is then analyzed to find repeating subsequences characterizing each team. We compare two
alternative approaches to extracting such characteristic sequences, based on frequency counts and statistical dependencies.
Our results indicate that both techniques are able to extract meaningful sequences, and do significantly better than random
predictions. However, the statistical dependency approach is able to correctly reject sequences that are frequent, but are
due to random co-occurrence of behaviors, rather than to a true sequential dependency between them.
Opponent modeling is one of the most attractive and practical arenas in multi agent system (MAS) for predicting and identifying the future behaviors of opponent. This paper introduces an approach towards opponent modeling in RoboCup Soccer Coach Simulation. In this scene, an autonomous coach agent is able to identify the weaknesses or patterns of the opponent by analyzing the opponent's past games and advising own players. To gain this goal, we introduce a 3-tier learning architecture. At first, by gathering data from the environment, sequential events of the players are identified. Then the weaknesses or patterns of the opponent are predicted using statistical calculations. Eventually, by comparing the opponent patterns with the rest of team's behavior, a model of the opponent is constructed. According to this architecture, coach models the opponent and to simplify pattern recognition, provides an appropriate strategy to play against the opponent. This structure is tested in RoboCup Soccer Coach Simulation and MRLCoach was the champion at Iran Open 2006
The fall of chess as a main domain for Artificial Intelligence (AI) research, after the victory of Deep Blue over Kasparov, put forward robotic soccer as a new challenging domain for AI researchers. Robotic Soccer is very appropriate for researching in Distributed AI problems, e.g. coordination methodologies, whose results can later be applied to other domains. Coordination can be defined as the capability of several agents to work together as a team, in order to accomplish a common goal. In the context of robotic soccer, several aspects may benefit from appropriate coordination methodologies, since team play is decisive for winning the games. This paper presents a methodology to enable a team of robotic soccer agents to coordinate their positioning in the field, and a methodology that enables them to cooperatively execute predefined set plays (flexible plans). Both coordination methods are used in real soccer and sports in general, commonly known as formations and set plays respectively. However, its use in a team of autonomous agents is still unexplored and is a major innovation of this work. A graphical tool to define formations and set plays was fully developed in the context of this work offering a quicker and more appealing way to define both formations and set plays. The tool's interface was inspired on ”blackboard” applications, used nowadays for the same purpose in real sports. The tool enables very easy definition of set plays that may then be executed in real-time during the matches. Examples of complete set plays and their practical execution are also described, showing the usefulness of the set play concept and the utility of the developed tool.
Machine Learning (ML) and Knowledge Discovery (KD) are research areas with several different applications but that share a common objective of acquiring more and new information from data. This paper presents an application of several ML techniques in the identification of the opponent team and also on the classification of robotic soccer formations in the context of RoboCup international robotic soccer competition. RoboCup international project includes several distinct leagues were teams composed by different types of real or simulated robots play soccer games following a set of pre-established rules. The simulated 2D league uses simulated robots encouraging research on artificial intelligence methodologies like high-level coordination and machine learning techniques. The experimental tests performed, using four distinct datasets, enabled us to conclude that the Support Vector Machines (SVM) technique has higher accuracy than the k-Nearest Neighbor, Neural Networks and Kernel Naïve Bayes in terms of adaptation to a new kind of data. Also, the experimental results enable to conclude that using the Principal Component Analysis SVM achieves worse results than using simpler methods that have as primary assumption the distance between samples, like k-NN.
In Multi-Agent System, observing other agents and modelling their behaviour represents an essential task: agents must be able
to quickly adapt to the environment and infer knowledge from other agents’ deportment. The observed data from this kind of
environments are inherently sequential. We present a relational model to characterise adversary teams based on its behaviour
using a set of relational sequences in order to classify them. We propose to use a relational learning algorithm to mine meaningful
features as frequent patterns among the relational sequences and use these features to construct a feature vector for each
sequence and then to compute a similarity value between sequences. The sequence extraction and classification are implemented
in the domain of simulated robotic soccer, and experimental results are presented.
The prediction of the future states in Multi-Agent Systems has been a challenging problem since the begining of MAS. Robotic soccer is a MAS environment in which the predictions of the opponents strategy, or opponent modeling, plays an important role. In this paper, a novel case-based architecture is applied in the soccer coach that learns and predicts opponent movements. Case-Based Reasoning(CBR) is a powerful and a frequently applied way to solve problems for humans. However, using CBR in highly dynamic environments results in a large number of cases to be retained, leading to high computational costs for subsequent case management(selection, composition and adaptation). The novel two-layered CBR learning architecture consists of an additional layer of cases to an ordinary layer which provides representation, adaptation and similarity measurement parameters for it. In this way, the complexity of the problem will go back to two subproblems, each of which works with a relatively small number of cases, so preventing the mentioned side effects of an ordinary CBR. We will provide and compare the performance statistics of both the ordinary and the two-layerd CBR learning systems and discuss features of this domain that makes the proposed learning approach perform well.
In this paper we will describe our research in case of using Expert System as a decision-making system. We made our attempt
to expose a base strategy from past log files and implement an online learning system which receives information from the
environment. In developing the coach, the main research effort comprises two complementary parts: (a) Design a rule-based
expert system in which its task is to analyze the game (b) Employing the decision-making trees for generating advice. Considering
these two methods, coach learns to predict agent behavior and automatically generates advice to improve team’s performance.
This structure is tested previously in RoboCup Soccer Coach Simulation League. Using this approach, the MRLCoach2004 took
first place in the competition held in 2004.
Agents in dynamic environments have to deal with world rep- To appear in: RoboCup 2005: Robot Soccer World Cup IX, c Springer-Verlag, 2006 resentations that change over time. In order to allow agents to act au- tonomously and to make their decisions on a solid basis an interpretation of the current scene is necessary. If intentions of other agents or events that are likely to happen in the future can be recognized the agent's performance can be improved as it can adapt the behavior to the situ- ation. In this work we present an approach which applies unsupervised symbolic learning o-line to a qualitative abstraction in order to create frequent patterns in dynamic scenes. These patterns can be later applied during runtime in order to predict future situations and behaviors. The pattern mining approach was applied to two games of the 2D RoboCup simulation league.
Opponent Modeling is one of the most attractive and practical arenas in Multi Agent System (MAS) for predicting and identifying
the future behaviors of opponent. This paper introduces a novel approach using rule based expert system towards opponent modeling
in RoboCup Soccer Coach Simulation. In this scene, an autonomous coach agent is able to identify the patterns of the opponent
by analyzing the opponent’s past games and advising own players. For this purpose, the main goal of our research comprises
two complementary parts: (a) developing a 3-tier learning architecture for classifying opponent behaviors. To achieve this
objective, sequential events of the game are identified using environmental data. Then the patterns of the opponent are predicted
using statistical calculations. Eventually, by comparing the opponent patterns with the rest of team’s behavior, a model of
the opponent is constructed. (b) designing a rule based expert system containing provocation strategies to expedite detection
of opponent patterns. These items mentioned are used by coach, to model the opponent and generate an appropriate strategy
to play against the opponent. This structure is tested in RoboCup Soccer Coach Simulation and MRLCoach was the champion at
RoboCup 2006 in Germany.
WWW home page:http://www.virtualwerder.de Abstract. The online coach within the simulation league has become more pow- erful over the last few years. Therefore, new options with regard to the recognition of the opponents strategy are possible. For example, the online coach is the only player who gets the information of all the objects on the field. This leads to the idea determine the opponents play system by the online coach and then choose an effective counter-strategy. This has been done with the help of an artificial neural network and will be discussed in this paper. All soccer-clients are initialized with a specific behavior and can change their behavior to an appropriate mode depend- ing on the coach's commands. The result is a flexible and effective game played by the eleven soccer-clients.
This article describes Soccer Server, a simulator of the game of soccer designed as a benchmark for evaluating multiagent systems and cooperative algorithms. In real life, successful soccer teams require many qualities, such as basic ball control skills, the ability to carry out strategies, and teamwork. We believe that simulating such behaviors is a significant challenge for computer science, artificial intelligence, and robotics technologies. It is to promote the development of such technologies, and to help define a new standard problem for research, that we have developed Soccer Server. We demonstrate the potential of Soccer Server by reporting an experiment that uses the system to compare the performance of a neural network architecture and a decision tree algorithm at learning the selection of soccer play plans. Other researchers using Soccer Server to investigate the nature of cooperative behavior in a multiagent environment will have the chance to assess their progress at RoboCup-97, an international competition of robotic soccer to be held in conjunction with IJCAI-97. Soccer Server has been chosen as the official server for this contest.
Agent technology represents a very interesting new means for analyzing, designing and building complex software systems. Nowadays, agent modelling in multi-agent systems is increasingly becoming more complex and significant. RoboCup Coach Campetition. is an exciting competition in the RoboCup Soccer League and its main goal is to encourage research in multiagent modelling. This paper describes a novel method used by the team CAOS (CAOS Coach 2006 Simulation Team) in this competition. The objective of the team is to model successfully the behaviour of a multi-agent system.
The main goal of agent modelling is to extract and represent the knowledge about the behaviour of other agents. Nowadays, modelling an agent in multi-agent systems is increasingly becoming more complex and significant. Also, robotic soccer domain is an interesting environment where agent modelling can be used. In this paper, we present an approach to classify and compare the behaviour of a multi-agent system using a coach in the soccer simulation domain of the RoboCup. Proceeding of: Third International Conference on Modeling Decisions for Artificial Intelligence, MDAI 2006, Tarragona, Spain, April 3-5, 2006.
In order to make a good decision, humans usually try to predict the behavior of others. By this prediction, many different tasks can be performed, such as to coordinate with them, to assist them or to predict their future behavior. In competitive domains, to recognize the behavior of the opponent can be very advantageous. In this paper, an approach for creating automatically the model of the behavior of a soccer team is presented. This approach is an effective and notable improvement of a previous work. As the actions performed by a soccer team are sequential, this sequentiality should be considered in the modeling process. Therefore, the observations of a soccer team in a dynamic, complex and continuous multi-variate world state are transformed into a sequence of atomic behaviors. Then, this sequence is analyzed in order to find out a model that defines the team behavior. Finally, the classification of an observed team is done by using a statistical test. Proceeding of: International Conference on Intelligent Autonomous Systems (IAS 2008), Baden Baden, Germany, July 23-25th, 2008. This work has been supported by the Spanish Ministry of Education and Science under project TRA-2007-67374-C02-02.
In competitive domains, some knowledge about the opponent can give players a clear advantage. This idea led many people to propose approaches that automatically acquire models of opponents, based only on the observation of their input–output behavior. If opponent outputs could be accessed directly, a model can be constructed by feeding a machine learning method with traces of the behavior of the opponent. However, that is not the case in the RoboCup domain where an agent does not have direct access to the opponent inputs and outputs. Rather, the agent sees the opponent behavior from its own point of view and inputs and outputs (actions) have to be inferred from observation. In this paper, we present an approach to model low-level behavior of individual opponent agents. First, we build a classifier to infer and label opponent actions based on observation. Second, our agent observes an opponent and labels its actions using the previous classifier. From these observations, machine learning techniques generate a model that predicts the opponent actions. Finally, the agent uses the model to anticipate opponent actions. In order to test our ideas, we have created an architecture called OMBO (Opponent Modeling Based on Observation). Using OMBO, a striker agent can anticipate goalie actions. Results show that in this striker-goalie scenario, scores are significantly higher using the acquired opponent's model of actions. This work has been partially supported by the Spanish MCyT under projects TRA2007-67374- C02-02 and TIN-2005-08818-C04.Also, it has been supported under MEC grant by TIN2005-08945- C06-05. We thank anonymous reviewers for their helpful comments. Publicado
A very important issue in multi-agent systems is that of adaptability to other agents, be it to cooperate or to compete. In competitive domains, the knowledge about the opponent can give any player a clear advantage. In previous work, we acquired models of another agent (the opponent) based only on the observation of its inputs and outputs (its behavior) by formulating the problem as a classification task. In this paper we extend this previous work to the RoboCup domain. However, we have found that models based on a single classifier have bad accuracy, To solve this problem, In this paper we propose to decompose the learning task into two tasks: learning the action name (i.e. kick or dash) and learning the parameter of that action. By using this hierarchical learning approach accuracy results improve, and at worst, the agent can know what action the opponent will carry out, even if there is no high accuracy on the action parameter.
This paper illustrates the prediction of opponent behaviour in a competitive, highly dynamic, multi-agent and partially observable environment, namely RoboCup small size league robot soccer. The performance is illustrated in the context of the highly successful robot soccer team, the RoboRoos. The project is broken into three tasks; classification of behaviours, modelling and prediction of behaviours and integration of the predictions into the existing planning system. A probabilistic approach is taken to dealing with the uncertainty in the observations and with representing the uncertainty in the prediction of the behaviours.
In competitive domains, the knowledge about the opponent can give players a clear advantage. This idea lead us in the past to propose an approach to acquire models of opponents, based only on the observation of their input-output behavior. If opponent outputs could be accessed directly, a model can be constructed by feeding a machine learning method with traces of the opponent. However, that is not the case in the Robocup domain. To overcome this problem, in this paper we present a three phases approach to model low-level behavior of individual opponent agents. First, we build a classifier to label opponent actions based on observation. Second, our agent observes an opponent and labels its actions using the previous classifier. From these observations, a model is constructed to predict the opponent actions. Finally, the agent uses the model to anticipate opponent reactions. In this paper, we have presented a proof-of-principle of our approach, termed OMBO (Opponent Modeling Based on Observation), so that a striker agent can anticipate a goalie. Results show that scores are significantly higher using the acquired opponent's model of actions.
RoboCup Challenge offers a set of challenges for intelligent agent researchers using a friendly competition in a dynamic, real-time, multi-agent domain: synthetic Soccer. While RoboCup in general envisions longer range challenges over the next few decades, RoboCup Challenge presents three specific challenges for the next two years: (i) learning of individual agents and teams; (ii) multi-agent team planning and plan-execution in service of teamwork; and (iii) opponent modeling. RoboCup Challenge provides a novel opportunity for researchers in planning and multi-agent arenas --- it not only supplies them a concrete domain to evalute their techniques, but also challenges them to evolve these techniques to face key constraints fundamental to this domain: real-time and teamwork.
This paper describes opuCI 2D , our soccer team that has been submitted to the qualification for the competition of the soccer 2D simulation league of RoboCup 2008. The main characteristic feature of this team is to use neural networks for team formation. The position of our players who are not in ball possession is determined by a neural network based on the current ball position. The position of opponent players are also estimated from the ball position by neural networks. We show a series of computational experiments to show that the use of neural networks gives a beneficial effect on our team.
Planning plays an important role in the domain of multi-agent sys-tems. Recognizing opponent' s plans and taking appropriate countermeasures will improve a team' s adaptability and cooperation in dynamic, adversarial envi-ronments. This paper describes our work on establishing a mechanism to recog-nize and retrieve team plans based on observations of agents' coordinative be-haviors.
In multi-agent system, agents have to analyze several features in order to adapt their behavior to the current situation. This extracted information is usually related to the environment and other agents influence. In this paper we present a method that compare two different agent models in order to extract the qualitative differences between them. This proposed comparative method captures several features of the two agent models and model them considering its behavior.
This paper describes opuCI 2D , our soccer team that has been submitted to the qualification for the competition of the soccer 2D simulation league of RoboCup 2010. The main characteristic feature of this team is to use neural networks for pass prediction. Neural networks are trained so that a pass receiver is successfully predicted from a situa-tion when our team is attacking. First we present the introduction of our team. Then the pass prediction task by neural networks is shown with experimental results.
In RoboCup Soccer Simulation 2D League, it is often difficult for players to make a correct decision because of the uncertainty in the field information. We particularly focus on the unpredictability of the opponent player's position. This paper presents a method that learns opponent team formation. Neural networks are used for this purpose. In the computational experiments of this paper, we show that team formation is successfully learned by neural networks. We also show the application of the learned neural networks to the prediction of successful passes.
The Robocup 2D simulation competition [13] proposes a dy-namic environment where two opponent teams are confronted in a sim-plified soccer game. As of 2009, all major teams use a fixed algorithm to control its players. An unexpected opponent strategy, not previously considered by the developers, might result in winning all matches. The impossibility to adapt to new strategies is a recurring problem in com-petitive games. To improve this we use ILP to learn action descriptions of opponent players. These descriptions can be used to plan for desired field states. To show this we start with a simplified scenario where we learn the behaviour of a goalkeeper based on the actions of a shooter player. The induced description is used to plan for states where a goal can be scored. This result can directly be extended to a multiplayer environment. For learning on dynamic domains, we have to deal with the frame problem. These descriptions can be used to dynamically modify the behaviour of the controlled team according to the opponent's strategies.
Behaviors in soccer-agent domains can involve individual plays, several players involved in tactical plays or the whole team trying to follow strategies supported by specific formations. The discovery of such behaviors needs the tracking of both the positions of players at any instant of the game and relevant relations able to represent particular interactions between players. Nevertheless, the tracking task becomes very complicated because the dynamic conditions of the game implying drastic changes of positions and interactions between players. We propose in this work a model able to manage the constant changes occurring in the game, which consists in building topological structures based on triangular planar graphs. Thus, based on this model tactical behavior patterns have been discovered even the dynamic conditions. Experimental results show that the proposed model is able to manage the constant changes of the world and discover tactical behaviors patterns. For that, an important number of matches have been analyzed from the RoboCup Simulation league.
In order for robotic systems to be successful in domains with other agents possibly interfering with the accomplishing of goals, the agents must be able to adapt to the opponents ' behavior. The more quickly the agents can respond to a new situation, the better they will perform. We present an approach to doing adaptation which relies on classification of the current adversary into predefined adversary classes. For feature extraction, we present a windowing technique to abstract useful but not overly complicated features. The feature extraction and classification steps are fully implemented in the domain of simulated robotic soccer, and experimental results are presented.
This paper proposes a similarity-based ap- proach for opponent modelling in multi-agent games. The classification accuracy is increased by adding de- rived attributes from imperfect domain theories to the similarity measure. The main contributions are to show how different forms of domain knowledge can be in- corporated into similarity measures for opponent mod- elling, and to show that the situation space of the oppo- nent modelling approach is not required to be the same as the situation space of the opponent players. Our ap- proach has been implemented and evaluated in the do- main of simulated soccer.
We describe a framework for planning in dy- namic environments. A central question is how to focus the sensing performed by such a sys- tem, so that it responds appropriately to rel- evant changes, but does not attempt to moni- tor all the changes that could possibly occur in the world. To achieve the required balance, we introduce rationale.based monitors, which repre- sent the features of the world state that are in- cluded in the plan rationale, i.e., the reasons for the plannln~ decisions so far made. Rationale- based monitors capture information both about the plan currently under development and the al- ternative choices that were found but not pur- sued. We discuss the plan transformations that may result from the firing of a rationale-based monitor, for example when an alternative choice is detected. We have implemented the genera- tion of and response to rationale-based monitor- ing within the Prodigy planner, and we describe experiments that show the feasibility of our ap- proach.
The main purpose of this work is the recognition of soccer team formations by considering a dynamic structural analysis. Traditionally,
the recognition of team formations is carried-out without taking into account an expressive representation of relations between
players. This kind of approaches are not able to manage the constant changes occurring in the soccer domain, which results
in an inefficient way of recognizing formations immerse in dynamic environments. It is presented in this work an efficient
model to recognize formations based on a representation that takes into account multiple relations among defender, midfielder
and forward players. The proposed model has been tested with different teams in off-line mode showing that it is able to recognize
the different main formations used by a team during a match.
The UT Austin Villa 2003 simulated online soccer coach was a first time entry in the RoboCup Coach Competition. In developing the coach, the main research focus was placed on treating advice-giving as a machine learning problem. Competing against a field of mostly hand- coded coaches, the UT Austin Villa coach earned first place in the com- petition. In this paper, we present the multi-faceted learning strategy that our coach used and examine which aspects contributed most to the coach's success.
The area of agent modeling deals with the task of observing other agents and modeling their behavior, in order to predict their future behavior, coordinate with them, assist them, or counter their actions. Typically, agent modeling techniques assume the availability of a plan- or behavior-library, which encodes the full repertoire of expected observed behavior. However, recent applications areas of agent modeling raise challenges to the assumption of such a library, as agent modeling systems are increasingly used in open and/or adversarial settings, where the behavioral repertoire of the observed agents is unknown at design time. This paper focuses on the challenge of the unsupervised autonomous learning of the sequential behaviors of agents, from observations of their behavior. The techniques we present translate observations of the dynamic, complex, continuous multi-variate world state into a time-series of recognized atomic behaviors. This time-series is then analyzed to find repeating subsequences characterizing each team. We compare two alternative approaches to extracting such characteristic sequences, based on frequency counts and statistical dependencies. Our results indicate that both techniques are able to extract meaningful sequences, and do significantly better than random predictions. However, the statistical dependency approach is able to correctly reject sequences that are frequent, but are due to random co-occurrence of behaviors, rather than to a true sequential dependency between them.
This paper proposes a classification approach to identify the team’s formation (formation means the strategical layout of
the players in the field) in the robotic soccer domain for the two dimensional (2D) simulation league. It is a tool for decision
support that allows the coach to understand the strategy of the opponent. To reach that goal we employ Data Mining classification
techniques. To understand the simulated robotic soccer domain we briefly describe the simulation system, some related work
and the use of Data Mining techniques for the detection of formations. In order to perform a robotic soccer match with different
formations we develop a way to configure the formations in a training base team (FC Portugal) and a data preparation process.
The paper describes the base team and the test teams used and the respective configuration process. After the matches between
test teams the data is subjected to a reduction process taking into account the players’ position in the field given the collective.
In the modeling stage appropriate learning algorithms were selected. In the solution analysis, the error rate (% incorrectly
classify instances) with the statistic test t-Student for paired samples were selected, as the evaluation measure. Experimental
results show that it is possible to automatically identify the formations used by the base team (FC Portugal) in distinct
matches against different opponents, using Data Mining techniques. The experimental results also show that the SMO (Sequential
Minimal Optimization) learning algorithm has the best performance.
In multiagent adversarial domains, team agents should adapt to the environment and opponent. We introduce a model representation as part of a planning process for a simulated soccer domain. The planning is centralized, but the plans are executed in a multi-agent environment, with teammate and opponent agents. Further, we present a recognition algorithm where the model which most closely matches the behavior of the opponents can be selected from few observations of the opponent. Empirical results are presented to verify that important information is maintained through the abstraction the models provide.
to predict what other agents are going to do in the future.
In the growing area of multi-agent-systems (MAS) also the diversity of the types of agents within these systems grows. Agent designers can no longer hard-code all possible interaction situations into their software, because there are many types of agents to be encountered. Thus, agents have to adapt their behavior online depending on the encountered agents. This paper proposes that agent behavior can be classified by distinct and stable tactical moves, called features, on different levels of granularity. The classification is used to select appropriate counter-strategies. While the overall framework is aimed to be applicable in a wide range of domains, the feature-representation in the case-base and the counter-strategies is done in a domain-specific language. In the RoboCup domain the standard coach-language is used. The approach has been successfully evaluated in a number of experiments.
In simple terms, one can say that team coaching in adversarial domains consists of providing advice to distributed players to help the team to respond eectively to an adversary. We have been researching this problem to nd that creating an autonomous coach is indeed a very challenging and fascinating endeavor. This paper reports on our extensive empirical study of coaching in simulated robotic soccer. We can view our coach as a special agent in our team. However, our coach is also capable of coaching other teams other than our own, as we use a recently developed universal coach language for simulated robotic soccer with a set of prede ned primitives. We present three methods that extract models from past games and respond to an ongoing game: (i) formation learning, in which the coach captures a team's formation by analyzing logs of past play; (ii) set-play planning, in which the coach uses a model of the adversary to direct the players to execute a speci c plan; (iii) passing rule learning, in which the coach learns clusters in space and conditions that de ne passing behaviors. We discuss these techniques within the context of experimental results with dierent teams. We show that the techniques can impact the performance of teams and our results further illustrate the complexity of the coaching problem.
In multiagent domains with adversarial and cooperative agents, team agents should be adaptive to the current environment and opponent. We introduce an online method to provide the agents with team plans that a "coach" agent generates in response to the specific opponents. The coach agent is equipped with a number of pre-defined opponent models. The coach is then able to quickly select between different models online by using a naive Bayes style algorithm, making the planning adaptive to the current adversary. The coach uses a Simple Temporal Network to represent team plans as coordinated movements among the multiple agents and it searches for an opponent-dependent plan for its teammates. This plan is then communicated to the agents, who execute the plan in a distributed fashion. The system is fully implemented in a simulated robotic soccer domain.
In multiagent domains with adversarial and cooperative team agents, team agents should be adaptive to the current environment and opponent. We introduce an online method to provide the agents with team plans that a "coach" agent generates in response to the specific opponents. The coach agent can observe the agents' behaviors but it has only periodic communication with the rest of the team. The coach uses a Simple Temporal Network to represent team plans as coordinated movements among the multiple agents and the coach searches for an opponent-dependent plan for its teammates. This plan is then communicated to the agents, who execute the plan in a distributed fashion, using information from the plan to maintain consistency among the team members. In order for these plans to be effective and adaptive, models of opponent movement are used in the planning. The coach is then able to quickly select between different models online by using a Bayesian style update on a probability distribution over the models. Planning then uses the model which is found to be the most likely. The system is fully implemented in a simulated robotic soccer environment. In several recent games with completely unknown adversarial teams, the approach demonstrated a visible adaptation to the different teams.
Weka Machine Learning Project
- Weka
Using expert system in robocup soccer coach simulation: An opponent modeling approach
- R Fathzadeh
- V Mokhtari
- A T Haghighat
- M Mousakhani
Feature-Based Declarative Opponent-Modelling
- T. Steffens
- D. Polani
- B. Browning
- A. Bonarini
- K. Yoshida