Content uploaded by Sebastian Hübner
Author content
All content in this area was uploaded by Sebastian Hübner on Apr 20, 2016
Content may be subject to copyright.
Recognizing Formations in Opponent Teams
Ubbo Visser, Christian Dr¨
ucker, Sebastian H¨
ubner,
Esko Schmidt, Hans-Georg Weland
TZI - Center for Computing Technologies
University of Bremen
D-28334 Bremen, Germany
{visser|druecker|huebner|esko|weland}@tzi.de
WWW home page: http://www.virtualwerder.de
Abstract. The online coach within the simulation league has become more pow-
erful over the last few years. Therefore, new options with regard to the recognition
of the opponents strategy are possible. For example, the online coach is the only
player who gets the information of all the objects on the field. This leads to the
idea determine the opponents play system by the online coach and then choose an
effective counter-strategy. This has been done with the help of an artificial neural
network and will be discussed in this paper. All soccer-clients are initialized with
a specific behavior and can change their behavior to an appropriate mode depend-
ing on the coach’s commands. The result is a flexible and effective game played
by the eleven soccer-clients.
1 Introduction
Our team is based on the sources that were released by the CMUnited99 team [10]. We
decided to do so because it would have been to time consuming to reinvent all basic
skills.1Instead, we focus on research w.r.t. high level functions which will hopefully
lead to new ideas and results for the RoboCup community. Our long run plan is to use
part of the provided functions from the CMU-client to construct a more sophisticated
team with individual players.
Over the last few years several attempts have been made in learning of team be-
haviour. Similar approacheshave been developed from numerous research groups. These
studies have the focus on learning team behavior within the simulation and middle size
league (see [11], [12], [13], [9]). Raines et al. [7], e.g., describe a new approach to
automate assistants to aid humans in understanding team behaviours for the simulation
league. This approaches are designed for the analysis of games, off-line after playing, to
gain new experiences for the next games. Frank et al. ([4]) present a real time approach
which is based on statistical methods. A team will be evaluated statistically but there is
no recognition of team strategies.
While conducting our research for this project we obtained support from real-life
soccer experts. In an interview, Thomas Schaaf, the manager of SV Werder Bremen
pointed out the importance of the strategy recognition of the opponent team. While the
1We would like to give special thanks to the original authors
coach-client has been able to participate through analysis and control in real matches
since 1998 [3], the idea of general strategic planning becomes possible. Like in real
life matches, the coach is able to give strategic commands depending on the opponent’s
system and the current score. We presume that the performance of our team can be
improved by analyzing the opponent’s strategy.
2 Agents
The Virtual Werder team consists of individual players which have different behaviors.
Players: 22 types of players have been developed with a variety of characteristics to
ensure the flexibility and variability of actions and reactions within a game. There are
different types of forwards, defenders, mid-fielders and goal keepers. Therefore, our
clients have the option to change their behavior from one style to another at any stage
of the game. This concept is an important key feature to carry out changes in a forma-
tion while playing against teams that are switching between different play systems. We
plan to use the online-coach, so that our clients have the ability to receive the messages,
parse them, and change their behavior accordingly. On the other hand, the low-level
skills of the agents are based on the sources provided by CMUnited99 [2]. These skills
include functions to locate the ball, the other team members and the opponent players.
In addition, methods to communicate via UDP-sockets, a parser for soccer server mes-
sages and other utilities such as the memory structures, have been used. Our clients are
initialized with a certain behavior and the desired formation when connecting to the
soccer server. Furthermore, they have the ability to choose other behaviors and switch
to them immediately. These high-level functions include methods to carry out different
defense systems such as man-to-man marking and zone defense. We also included new
mid-level functions, e.g. finding a teammate able to catch a pass. Another new skill in-
cludes the player moving to a certain point on the field, while keeping track of the ball.
This is accomplished through the turn neck command. The characteristics of these 22
types of players are described in the Virtual Werder team description.
Coach: The coach observes the game continually, analyzes the formation of the op-
ponent team at given points in time with an artificial neural network (ANN) ([5], [8])
and broadcasts an adequate counter formation to the players during the next interrup-
tion. The current evaluation takes place twice per second. The positions of the players
serve as inputs for an ANN, that is trained with the formations most commonly played
in our test games and the log-files (see section 3). In order to get a reliable impression
only outputs with high ratings are used. Whenever the play mode switches to another
state than PLAY ON, the coach generates a message for his team. It instructs the play-
ers which formation the opponent is currently using and gives information about the
appropriate counter attack.
3 Approach
We observed that teams in the last RoboCup-tournaments typically relied on their strat-
egy and team formation and often didn’t change it within a game. When changes were
Fig.1. Positions of opponent players and bounding box; marked cells define the input
vector for the neural network.
made they depended on the score. A common practice was to switch from a offensive
to a defensive formation if the team lead with more than ngoals. On the other hand
some teams remained on the same system regardless of the opponent’s strategy. This
is the point where the online-coach diagnoses what to do depending on the opponent’s
formation. The coach-client had to be fed with formations and information on how to
analyze them. Part of the integrated knowledge was obtained from an interview with a
real expert, the Werder Bremen’s head coach Thomas Schaaf, other parts from litera-
ture, e. g. [1] and from games played in the last RoboCup tournaments. This knowledge
can be used by the online-coach within the decision process.
The observation and analysis of the opponents team is processed in several steps
which are described below. The CMUnited99 sources provide the communication with
the soccer server and the parsing of messages. Furthermore, it supplies the coach with
collected information in data structures which are easily accessible. This information is
then prepared and will act as an input vector for an ANN. In order to prepare the data
we first have to decide which variables should be used. Our model consists of 64 binary
input variables. For this purpose a bounding box is placed around the positions of the
opponent players. The box is currently divided into a grid of eight by eight cells which
leads to an arrangement of 64 fields (Fig. 1). There must be at least one player inside a
specific field with the value of this field set to 1. Otherwise, the value must be set to 0.
The sum of the fields defines the input vector for the ANN. The network classifies the
vector in one of 16 output classes, each representing a specific play system.
The network is implemented as a C-function2and is called to calculate the ratings
at 16 possible output neurons which represent the opponent’s formations (section 2). If
the output neuron with the highest value exceeds a demanded threshold, this class will
be chosen as the result of this function. This result and other information in addition,
e.g. the size and position of the bounding box or the current score of the game, deter-
2snns2c by Bernward Kett was used to transform the trained network from an internal SNNS
representation to a usable C-function [14].
mines the appropriate counter formation. During the next interruption of the game, this
information is broadcasted to the team.
Training results: Our coach uses an ANN to analyze the formation of the opponent’s
team. The network itself was trained to recognize 16 different formations. The coach-
client uses this knowledge about the classified formation to evaluate a proper counter-
attack. Soccer formations are typically noted as a combination of defense, mid-field
and forward players, e.g. a 5-3-2 represents a team with five defense players (and a
goalie), three players in the middle field and two forward players. However, there are
some special systems that do not fit into this pattern, these are referred to by name, e. g.
the Catenaccio system [1].
We developed a tool called “ExportPlayer” to obtain formation examples for our
network. This program is based on the log-player and takes automatic snapshots of ex-
isting log-files. It extracts the positions of the players (not including the goal keeper) and
normalizes the coordinates to a grid of eight by eight cells. Cells labeled ’0’ do not con-
tain players, cells labeled ’1’ contain at least one player. The ExportPlayer returns a pat-
tern file which contains the values of all input and output neurons to serve the ANN (see
also team description). We used the Stuttgart Neural Network Simulator (SNNSv4.1)
[14] to train the examples and to create the code for a feedforward-network. We used
standard backpropagation as learning method with the learning parameter η= 1.0and
the maximum difference dmax = 0.3between the teaching value and the output.
The choice of the threshold mentioned in section 3is a very important factor for the
efficiency of the online-coach. Table 1shows the correlation between different thresh-
olds which results in the amount of permitted input patterns and their correctness. The
inquiry is based on 680 patterns obtained from log-files and test games. These have
been previously classified by us. On ten separate occasions, these patterns were ran-
domly divided into 612 training and 68 test sets (≈10%) and were processed by the ne.
The average of these ten different results have been calculated for validity.
Due to the large quantity of test patterns (currently, a snapshot is made twice per
second), the relatively high amount of rejections is not problematic in this environment.
Furthermore, it is similar to a real soccer game where distinct formation occurs infre-
quently.
threshold classified classified correctly
0% 100% 48.37%
80% 49.67% 65.53%
85% 41.34% 67.88%
90% 32.52% 69.30%
95% 20.10% 72.27%
Table 1. Relation between output threshold and correctness
4 Results
Our hypothesis was that we improve the performance of our team by detecting op-
ponents strategies and obtaining the appropriate counter formation. Our test environ-
ment consists of our team and the teams of CMUnited99 and last years’ Mainz Rolling
Brains. We carried out ten games against each team with two different play systems.
We decided that the formations 5-4-1 (defensive) and 3-4-3 (offensive) were the most
promising for our demonstration.
Against CMUnited99 Virtual Werder performs better with a defensive formation.
The average loss against the CMU-team was 0:14 with the 3-4-3 and 0.1:9 with the
defensive strategy (we might add that this is an unacceptable situation in total). Table 2
shows the average results. Against Mainz Rolling Brains on the other hand we can see
that Virtual Werder performs better with the offensive formation. The average score of
3.1:0.7 was better than the 0.5:0.9 score with the defensive formation. We believe that
the understaffed mid-field caused this situation (see table 2).
We come to the conclusion that the online-coach can help to detect the opponent’s
strategy. Once a team knows the play system of the opponent, appropriate counter ac-
tions can be carried out. However, later experiments have shown that we cannot exclude
that the score is caused by other skills such as the individual play style. We think that
further investigations with a ’standard team’ would be helpful to make a clear point
on this issue. In summary, we have seen that the Virtual Werder team performs better
with this new information. The average score depends upon the chosen play system and
whether the team can change their system online.
Mainz RB CMU-99
VW Def. 5-4-1 0.5:0.9 0.1:9
VW Off. 3-4-3 3.1:0.7 0:14
Table 2. Relation between formation and score
The technology of strategy detection could be useful for other application areas.
Firstly, the quality of action predictions of physical agents can be improved which plays
an important role within the control mechanisms of autonomous agents. Secondly, it is
important to improve the robustness and security issues of electronic markets within the
area of electronic commerce.
5 Future Work
Further work can be done in the following areas:
Keeping track of changes: This means that the coach-client consists of internal states.
With internal states a list of ’scenes’ describing the current play system of the oppo-
nent’s can be stored. The next step is to detect changes in the opponent strategy. A low
pass filter can then be used to determine whether the play system changed temporary or
for a longer period.
Evaluation of counter attacks: The evaluation of a fitting counter-attack is another
issue in our research. Therefore, we will focus on new criteria, such as play cycle and
score. The idea is to change formations in situations that do not depend on the oppo-
nent’s play.
Captain: Looking at formations is a first step to a more strategic play. The next step to
improve the team performance will be the transfer of the coach knowledge to a key-
player, which can give commands to the team members during the game, not only
within a break. This ”captain”-concept could also be extended with the concept of a
key defense player, which is responsible for the guidance of the defense.
References
1. Christoph Biermann and Ulrich Fuchs. Der Ball ist rund, damit das Spiel die Richtung
¨
andern kann. Kiepenheuer & Wisch, 1999.
2. CMU. Sources of low-level skills. http://www.cs.cmu.edu/afs/cs/usr/pstone/mosaic/
RoboCup/CMUnited99-sim.html.
3. Emiel Corten, Klaus Dorer, Fredrik Heintz, Kosta Kostiadis, Johan Kummeneje, Helmut
Myritz, Itsuki Noda, Jukka Riekki, Ratrick Riley, Peter Stone, and Tralvey Yeap. Soc-
cerserver manual ver5.1 release. Manual, 1999.
4. Ian Frank, Kumiko Tanaka-Ishi, Katsuto Arai, and Hitoshi Matsubara. The statistics proxy
server. In Tucker Balch, Peter Stone, and Gerhard Kraetschmar, editors, 4th International
Workshop on RoboCup, pages 199–204, Melbourne, Australia, 2000. Carnegie Mellum Uni-
versity Press.
5. J. Hertz, A. Krogh, and R.G. Palmer. Introduction to the theory of neural computation,
volume 1. Addison-Wesley Publishing Company, Redwood City, California, 1991.
6. H. Kitano, Y. Kuniyoshi, I. Noda, M. Asada, H. Matsubara, and E. Osawa. Robocup: A
challenge problem for ai. Artificial Intelligence Magazine, 18(1):73–85, 1997.
7. Taylor Raines, Millind Tambe, and Stacy Marsella. Automated assistants to aid humans in
understanding team behaviors. In Fourth International Conference on Autonomous Agents
(Agents 2000), Barcelona, Spain, 2000.
8. Raul Rojas. Neural Networks - A Systematic Introduction. Springer-Verlag, Berlin, 1996.
9. Peter Stone. Layered Learning in Multiagent Systems: A Winning Approach to Robotic Soc-
cer. Intelligent Robotics and Autonomous Agents. MIT Press, Cambridge, Massachusetts,
2000.
10. Peter Stone, Patrick Riley, and Manuela Veloso. The CMUnited-99 champion simulator
team. In Manuela Veloso, Enrico Pagello, and Hiroaki Kitano, editors, RoboCup-99: Robot
Soccer World Cup III, Berlin, 2000. Springer-Verlag.
11. Peter Stone and Manuela Veloso. A layered approach to learning client behaviors in the
robocup soccer server. Applied Artificial Intelligence, 12(3):165–188, 1998.
12. Peter Stone and Manuela Veloso. Towards collaborative and adversarial learning: A case
study in robotic soccer. International Journal of Human-Computer Studies, 48, 1998.
13. Peter Stone and Manuela Veloso. Team-partitioned, opaque-transition reinforcement learn-
ing. In M. Asada and H. Kitano, editors, Robot Soccer World Cup II, Berlin, 1999. Springer-
Verlag.
14. A. Zell, G. Mamier, M. Vogt, N. Mache, R. Hubner, K.U. Herrmann, T. Soyez, M. Schmalzl,
T. Sommer, A. Hatzigeorgiou, S. D¨
oring, and D. Posselt. Snns: Stuttgart neural network
simulator, user manual version 3.2. Technical Report 3/94, Universit¨
at Stuttgart, 1994.
