A simple method for decision making in RoboCup soccer simulation 3D environment

Abstract

In this paper new hierarchical hybrid fuzzy-crisp methods for decision making and action selection of an agent in soccer simulation 3D environment are presented. First, the skills of an agent are introduced, implemented and classified in two layers, the basic skills and the high level skills. In the second layer, a two phase mechanism for decision making is introduced. In phase one, some useful methods are implemented which check the agent's situation for performing required skills. In the next phase, the team strategy, team formation, agent's role and the agent's positioning system are introduced. A fuzzy logical approach is employed to recognize the team strategy and furthermore to
tell the player the best position to move. At last, we comprised our implemented algorithm in the Robocup Soccer Simulation 3D environment and results showed the efficiency of the introduced methodology.

Full text

Disponible en: http://www.redalyc.org/src/inicio/ArtPdfRed.jsp?iCve=133117550013
Redalyc
Sistema de Información Científica
Red de Revistas Científicas de América Latina, el Caribe, España y Portugal
Niki Maleki, Khashayar; Hadi Valipour, Mohammad; Yeylaghi Ashrafi, Roohollah;
Mokari, Sadegh; Jamali, M. R.; Lucas, Caro
A simple method for decision making in RoboCup soccer simulation 3D environment
Revista Avances en Sistemas e Informática, vol. 5, núm. 3, diciembre, 2008, pp. 109-
116
Universidad Nacional de Colombia
Medellín, Colombia
¿Cómo citar? Número completo Más información del artículo Página de la revista
Revista Avances en Sistemas e Informática
ISSN (Versión impresa): 1657-7663
mprada@unalmed.edu.co;avances@unalmed.ed
u.co
Universidad Nacional de Colombia
Colombia
www.redalyc.org
Proyecto académico sin fines de lucro, desarrollado bajo la iniciativa de acceso abierto

RevistaAvancesenSistemaseInformática,Vol.5No.3,Diciembrede2008,Medellín,ISSN16577663
AsimplemethodfordecisionmakinginRoboCup
soccersimulation3Denvironment
Unmétodosimpleparalatomadedecisionesen
ambientes3DdesimulacióndefútbolRoboCup
Recibidopara revisión:3 0 deSeptiembrede 2008, Aceptado:28 de Noviembrede 2008 ,Versión final:9 de Diciembrede 2008

Abstract
—I n th i s p a p er  n ew h ier a r ch ica l h y br id  f uzzy cr isp 
meth ods for  decision ma kin g an d a ction selection  of an  agen t in 
soccer simula tion  3D envir onmen t a re p resen ted . Fir st, th e skills
of a n  age nt  ar e in tr od uced , imp leme nt ed a n d cla ssifi ed in  t wo
layer s,th ebasicskillsan dthehighlevelskills.Int hesecondla yer,
a t wop ha se mech an ism fo r d ecision  ma king  is int rod u ced. I n
p ha seo ne, some u sefu l meth ods a re  implem ent ed whi ch ch eck
th e a gent ’ssit u ati on for  p erf orm ing r equ ir ed ski lls. In t he n ext
p ha se, t he t eam  str a teg y,t eam for ma tio n, a gen t’s r ole a nd  th e
agen t’sposition ingsystemareintrodu ced.Afuzzylogicalap proa ch
is emp loyed  to r ecogn izet he  tea m str a te gy a nd  fu rt h er more t o
tell th e pl ayer  th e b est p ositi on t o move.A t la st, we comp ri sed
our  imp lemen ted  alg orit hm in  th e Rob ocup  Soccer  Simu lat ion
3D  en v ir o n me n t  a n d  r e su l ts  sh o wed  t h e  e ff i cie n cy  o f t h e
in tro du ced met hod ology.

Keywords—
M ul tiA gen t  syste ms, M a ch in e lea r n in g, Ar t ific ial
in t e ll ig en ce , R e in f or c em en t  l ea r n i n g , F u zzy L o gi c,  F u zzy
re infor cemen t lear n ing , Rob oCu p soccer  simu lat ion.

Resumen
—E n este a r tícu los e pr esent a u n n u evomét odo d ifu so
hib r ido pa r a la t oma d ed ecisiones yselección d ea cciones de u n
ag ent ee n u n a mbi ent ed e simu lación  de f út bol 3 D. Pr imer o, se
introd ucen, implementa ny clasifican las competenciasdel agente
en  d os ca p a s:  com p ete nc ia s b á si ca s y com pe te n cia s d e n ivel
sup erior. E n la  segund a  cap a se int rod uce un  mecan ismod e dos
fa sesp a r a la t oma d e decision es.E n l a pr imer a  fa se,se a plica n
a lgun os mét od os út iles  qu e p er mit en ver ifica r  la  situ ac ión d el
ag ente p ar a la  ejecu ción de la scomp eten cias r equ eri das .E n la 
siguien te fase, se ad iciona n la est ra tegia  y forma ción d eleq uip o,
y elr ol y sistema d e posición d e losa gen tes. Se emplea la  lógica
KhashayarNikiMaleki
1
,MohammadHadiValipour
1
,RoohollahYeylaghiAshrafi
1
,SadeghMokari
1
,M.R.Jamali
1,2
,CaroLucas
2
1.DepartmentofElectricalandComputerEngineering,ShahidRajaeeUniversity,Tehran,Iran
2.ControlandIntelligentProcessingCenterofExcellence,SchoolofElectricalandComputerEngineering,
UniversityofTehran,Tehran,Iran
{kh.niki,m.h.valipour,r.yeylaghi,s.mokari}@sru.ac.ir, m.jamali@ece.ut.ac.ir,lucas@ut.ac.ir
I.  I NT RO D UC T IO N
R
obocupiscontinuingAIresearchinitiativethatusesthe
gameofsoccerasaunifyingandmotivatingdomain[1,
2]. T he Robocup si mulati on comp etition  pits team s of 11
independentlycontrolledautonomousagentsagainsteachother
inRobocupsimulator,orSoccerServer,arealtime,dynamic
environment[3].TheonlyRobocupsoccersimulatorusedto
be«2D»foryears,manyresearchesinAIhaveperformedin
tha t full cha llengin g envir onment for  functiona lity of the
developedalgorithminamultiagentenvironment[4].Butas
ithasbeensetbyRoboCupFederation[RoboCup,2004], the
ultimategoalcanbeexpressedbyseveraldiscussion.«Bymid
21stcentury,ateamoffullyautonomoushumanoidrobotsoccer
playersshallwinasoccergame,complyingwith theofficial
rulesoftheFIFA,againstthewinnerofthemostrecentworld
difu sapa r ar econocerla estr a tegia deeq uipo yad emásin dicar le
aljugador, lamejor p osiciónpa r amover se.Porúltimo,seincluye
el alg orit mo implemen ta d o en el a mbien te 3D d e Simu lación  de
Fú tb olRoboCupylosresulta dosque demuestr a nla eficiencia de
la met odología  int ro du cida .

Pa labra sClave
—Sistemasmu ltia gente,Ap rend izaje demá quina,
In telig encia  a r tif icial , Ref uer zo d el a pr en diza je, L ógica  d ifu sa ,
Refu er zo del ap r end izaj ed ifu so,Simu lación  de fú tb olR oboC up .

RevistaAvancesenSistemaseInformática,Vol.5No.3,Diciembrede2008,Medellín,ISSN16577663
110
cupforhumanplayers.»[Kitano&Asada,1998]Sotherewasa
needforamorerealisticplatformratherthantheprevious2D
environmentwhichismorefamiliarwithrealsoccergame.That
leadedtoRobocup3DSoccerSimulator.
AstheoptimalscoringproblemiswellsuitedforMachine
Learn ing (ML) t echniques [ 5], Nowadays m any powerful
methodswithdifferentroots,hasbeenintroducedinMLsuch
as: neural networks [5], genetic algorithms [6], genetic
programming[6], fuzzylogic[7],coordinationgraphs[8,9]
andalsomanyhybridapproachesasacombination ofsome
aforementionedones[4]MLtechniqueshavebeenappliedto
a variety of problems in cl uding data min ing , patt er n
recognition , classification problems, e.g. roa d condition
classifier [Ferdowsizadeh, 2004], adaptive control, robot
control,combinatorialoptimizationandgameplaying.There
arelotsofpublicationsinapplyingthesemethodsinRoboCup
Soccer Si mul ated tea ms mostl y in 2D (and NO T 3D)
environment[1,2, 3,4,5,6,8,and9](therearemuchmore
thanreferencedresearches).Itmayhavedifferentreasons,but
havingproblemwithlowlevelskillsthatleadstodisabilityin
controlling agent s, complica ted dynamics which  makes
predictionsnottoworkwellandavastrangeofperformingsoccer
skills in comparisonwith2D simulatedenvironmentfor sure
arethemostimportanteffectingfactors.Therearenewfeatures
and also limitations in this environment which make some
distinctions in decision making process. (For more detailed
informationyoucanreferto[10,11])
In this paper a  newmeth odology for decision making i n
Rob oCup s occe r s imul ated 3D envi ron ment  wit h a ll
abovement ioned problem s consid erat ion is introduced.
Implementationofrealsoccerskillsintwolayersandutilizing
both fuzzy and non fuzzy algorith ms in different layers of
decisionmakingaretheessentialkeysfortheaccomplishment
of this system. Section two reviews the state of ar t of thi s
methodologyinordertoapplyitindecisionmakingprocess.In
thissectionthebasicskillsandtheirfunctionalityandalsosome
ofsoccer(highlevel)skillsareintroducedand classified.The
firstlayerofdecisionmakingprocessisdiscussedinsectionthree
andthesecondlayerinsectionfour.Sectionfivecomprisesour
resultswhichareimplementedonScorpiusSoccerSimulation
Team
1 andfinallysectionsixconcludesthepaper.
II .LAYERSOF SK ILL S ANDDEC ISIO NMAKI NG PR OCE SS
In t he proposed meth odology some applica ble skill s are
introduced and the decision makin g policy i sdeveloped by
consideringthefeaturesandlimitationsofthisenvironment.
Theskillsareclassifiedintwolayers,inthefirstlayerthere
arethesimpleactionswhicharealreadyimplementedbyserver
(basic skills) and  in the second layer the a ctions are more
complicatedand sometimesa combination of«basicskills»
are used (highlevel skills). Besides, th e decision making
processhastwosteps;firststepconsiderstheagent’sabilities
ofperformingahighlevelskillaccordingtohiscircumstances
and the secon dstep con siders the agen tpositioning and
choosingthebestactionregardingtothefirststepresults.

A.Basicskills

Thebasicskillsaredefinedastheactions,whicharealready
implemented by server [10, 11]. They can be employed by
sending the pr oper comma nds to soccer s erver
2
. Th ese
commandsare:
∙
 A(drive x  y  z)
: movesthe agent by applying the force
vector(x,y,z)tocenterofit.
∙
 A(kickalphaf)
:agentkickstheballbyapplyingtheforce
f

withtheangle
alpha
toitiftheballisinkickabledistance.
∙
 A(pantiltangle1angle2)
:Thiscommandchangestheview
directi on of an agen t where «angle1»a nd «angle2» a re
changes(indegrees)ofthepanandtiltangle,respectively.
∙
 A(say(«say message here»))
:sends a message to all  the
playersthatarelocatedin50meterfromsender.
∙
 A(catch)
:(forgoalkeeperonly)holdsandfreezestheball
iftheballisinthecatchablearea.
(See [ 10, 12 ] for mor e detai led in format ion about  the
commands)

B.Highlevelskills

High level ski lls ar e those in  which th e world m odel
3
informationandthebasicskillsarebeingapplied.Theseskills
consistoftheactionswithballlike:pass, shoot,dribble,etc.
andactionswithoutballlike:mark,findobjectandinformation
broadcasting.
1)Actionswithball
Toperformtheseactions,basicallyweneedsomeinformation
abouttheballtreats,whenaforcewithaparticularangleis
applied,consideringtheenvironmentparameters(e.g.airforce,
friction,collision,etc).Anumberofcomplicatedmechanical
formulascouldhelptopredicttheballmovements(Figure1
showstheball predictedmotionincomparisonwiththereal
ballpositioninformationgivenfromthemonitor
4
).Someof
themostapplicableskillswithballareasfollows:
1. htt p://www.scorpius.ir
2. These skillsare implemented in rcsoccersim3D_0.5 .5
3. World Modelis a data bank in wh ich the information about the
environment is stored
4.Wewrotea programtoparsethemonitorlogfile.Weassumethat
thisinformationisthemostaccurateonewe mayhavefromwhat
really happens in server.

Asimplemethod fordecision making inRoboCup soccersimulation 3D environment  –  Malekiet al111
∙
 Shoot

Shootingforsureisthemostimportantskillinsoccerand
allotherskillslikepass,dribble,etcarebasedonit.Aplayer
inthisenvironmentcanonlykicktheballinfrontofhimself,
soheneedstogetbehindballinacorrectpositiontoshootthe
ballinhisdesireddirection(seeFigure4).
∙
 Pass

The agent kick st he ball so that the othe r teammate can
receiveit.Thiscanbeasimpledefinitionfortheskill«pass».
As this skill can be usedi n different situations, the
type
 is
definedforit.Wehavethreekindsofpass:

1.Securepass
:Theplayeriscompletelysurethat this
pass will arr ive to the player h e wishes. The most
popularusageofthistypeofpassiswhentheballisin
dangerzone
5
.

2.Normalpass
:theprobabilityofsuccessinpassismore
thanitsfailure.Mostlyusedinthemiddleofthefield.

3.Riskypass
:In thiscase,theprobabilityofballarrival
exists, but the possibility of failure is more than its
success.T his type is used in order to create a good
situationforteam.
∙
 Dribble(runwithball)

Theagentusuallyusestheskill«dribble»whenheownsthe
ballinanalmostfreespaceandcan’tfindtheotheragentswith
betterpositionsorcan’tpassthemtheball(seeFigure7b).

∙ Clearball

Thisisanactionthatagentchoosestodo,whenheownsthe
ball an d can per form no other a ction or  the pla yer is i n
dangeroussituation, mostlyhappens in
 defense
.Depending
ontheoccurredcircumstance,theagentmaykicktheballout
ofthefield,towardtheopponentgoalorotherpositions.
5.Dangerzone isdefinedinfuzzymemberfunction(see Fig.4)
Figure1.Thecomparisonbetweenthemechanicalformulausedforball
motionpredictionandtherealdatagivenfromthemonitor.
2)Actionswithoutball
Theseskillsmakeagentstobearrangedinpositionssothat
theywouldhavethemostchancetocreateopportunitiesfor
teamortogettheopponentsopportunities.
∙
 Mark

Markskillapproachestwopurposes:
1.Nottolettheballreachestheopponents(markplayer).
2.Nottolettheopponentsshoottotheirdesiredposition
(markball).
Accordingtothepurposetheplayergetsneartotheopponent
uptothe
MarkSecureDistance
andmarkshim.
∙
 Pantilt(objectfindingskill)

Theagentusesthisskilltofindanobjectand/ortoupdate
hisworldmodel.Therearetwodifferentconditions:
1.  The agent seesan object butwants to keep it in the
centerofhisvisionnottoletthatobjectgetsoutsight
easily.Thisusuallyhappensforballbecauseitslocation
variesveryfastandeasilymayget outoftheagent’s
sight(Figure2).
2.Theagentdoesn’tseethe object;inthiscasehepans
with
MaxPanAngle
withthedirectioninwhichthelast
timethatobjectwasseen.
∙
 Say(informationbroadcastingoralertingskill)

Thisskillisbeingusedforalerting theagents and alsoto
updatetheirworldmodels.Dependingonhowmanycharacters
percycleanagentcantalk,hecanusetheminordertoupdate
theotheragents’worldmodels.Oneofthemostrecommended
usagesofthis skillis utilizedin mark.Whenthedefenders
«
markball
»theopponentstheydonotseetheopponentswhich
are marked. Sotheyneedtochange their viewineach few
cycles,butwhentheballgetsneartheymayfocusonballand
forgetabouttheagent.In thiscaseorsimilareventswecan
usesayskilltoupdateandalsoalerttheagents.Figure2shows
thisevent.
Figure2. Thegoalkeepersaysamessagetoupdatethedefendersworldmodel.
Theycan’tseetheopponents’displacementbehindthem.

RevistaAvancesenSistemaseInformática,Vol.5No.3,Diciembrede2008,Medellín,ISSN16577663
112
II I.DECISI ON MAK INGPH ASE1
In t his pha se, the ag ent ability of per forming an  action
accordin g to the en vironment  situation (e.g. opponents’
positions/speed,ballposition/speedandtheirpredictedstates)
istestedbyDecisionMakers(DM) toconfirmifthataction
can be done by the agent in that condition. Some of these
decisionmakersareexplainedbelow:
∙
 DMforshootingtoaposition

Todetermineifanagentcanshoottoapositionornot,firstthe
agentcalculatestheminimumdegree,forshooting,ifthedegree
couldbefoundlessthan
MaxKickDegree
thentriestofindan
anglebetweenmindegreeand
MaxKickDegree
andaforceless
than
MaxKickForce
.Iftheangleandaforcefoundwiththese
propertiestheDMreturns
true
,otherwiseitreturns
false
.
Figure3.Shootingtoapositionrequiresminimumangleátobelessthan(
MaxKick
Angle
)andminimumdistance
d
lessthan
(MaxKickDistance)

∙
 DMforShoottogoal

First,theagentquantizesthegoal, to
n
discrete positions.
Foreachpositionfirstchecksthe
Shoottoposition
conditions,
iftheresultistruethenchecksthefollowingcondition:
Let Tbbe the time takes ball to meett he target with the
maximum speed, an dT r be the rotati on time for the ball
controllertoadjustit’spositionbesidetheball.Tgrepresents
thetimetakesgoalietocatchtheball(Figure4).Havingcalculated
theabovethreeparameterswedefine
Dt
asfollowing[7]:
)(
 TrTbTgt
+ - = D(1)
Figure4.Thetimeintervalsneededforpredictioncalculationappliedinshoot
togoaldecisionmaker.
Thesignof
Dt
showsiftheagentcanshoottothatposition
ornot.

DMforpa ss

ThisDMgetstheballpositionandthepassreceiverasinput
parameters.toknowwheretousewhichtypeweimplemented
afuzzyalgorithm.Thisfuzzyalgorithmaccordingtotheball
positionandtheplayersituationtellsus which typeis more
properandalsogivesthe
MaxAllowedPassError
:
Figure5.Fuzzyinputmemberfunctions;a.ballpositioninputvariable
b.passreceiversituationinputvariable
a
b
Fuzzyrulebaseforpassisasfollows:
1.If(positionisDanger)then(passtypeisSecure).
2.If(positionisSafe)and(situationisBad)then(passtypeis
Secure).
3. If(position is Safe) and (situation is Normal) then (pass
typeisNormal).
4.If(positionisSafe)and(situationisGood)then(passtype
isRisky).
5.If(positionisRisk)and(situationisBad)then(passtypeis
Secure).
6.If(positionisRisk)and (situationisNormal) then(pass
typeisNormal).
7.If(positionisRisk)and(situationisGood)then(passtype
isRisky).

Asimplemethod fordecision making inRoboCup soccersimulation 3D environment  –  Malekiet al113
Figure6.Therelationshipbetweenballpositionpassreceiversituationand

MaxPassError
accordingtothefuzzyalgorithm.
Afterthepasstypeand
MaxPassError
becamedetermined,
agentcheckshisabilityforperformingthistypeofpass.
Firsttheagentchecksifhe
canshootto
thepasspositionor
notthentheamountofriskwillbecalculated.Thefollowing
factorshavetobecheckedinordertocalculatetheriskofpass:
1. Opponentsaroundtheball
2. Therelativeagentpositionwithball.
3. Themaxerrorforgettingpositionbehindtheball
4. The position of opponents around the player we want to
passto,andtheirdistanceswithhim.
5. Theballpath
Let
t1
bethetimetakesplayertogetpositionbehindball.It
dependsonfactors1,2,and3whichvarywithpasstype,and

t2
bethetimetakesforopponentstoreachtheball,theplayer

canpass
when
t1
islessthan
t2
.(SeeFigure7.a)
Let
t(i)
bethetimetakesforopponent
i
tointercepttheball
consideringtheballheightfromthefieldatposition
p(i)
with
error
E1
,and
t1(i)
isthetimetakesforballtoreach
p(i)
with
error
E2
,where
E1
and
E2
varywithpasstype.If
 t1(i)<t(i)

forall
 i
opponentsthenplayer
canpass
.Andthelastfactoris
whenthetargetplayer,getstheball.Hemusthave
SecureTime

(t5inFigure7.a)tocontroltheball,which varieswithpass
type.Nowwiththeamountofrisk,theplayercandetermine
whetherhecanpasswithrequestedtypeornot.
a
Figure7.a. Player1 wants topass toplayer2, b. d1isthe Secure
DribbleDistance and á is the Secure dribble angle
b
∙
 DMforDribble

Todetermineifitissafeenoughforplayertodribble(run
withball)
First,theplayerdrawsaconewith
SecureDribbleAngle
in
frontofhimself,ThenhechecksIfthereisnoopponentwith
distancelessthanorequalwith
SecureDribbleDistance
inside
theconesubsequentlyhepredictstheballpositionafterkick
whentheballspeedislessthan
MaxBallControllableSpeed

letittaket1,nexthepredictstheopponent playersplusthe
agenthimselfpositionsaftert1,assumingthattheymoveto
theballpredictedpositionwiththeir
MaxSpeed
.Ifthenearest
playertotheballaftert1isthe agenthimselfthen the DM
returnstrue,otherwisereturnsfalse(Figure7.b).
∙
 FindOpponenttomark(DMformarking)

Theplayersearcheshisstrategicarea(seesectionVformore
details)for
 not marked
 opponents, if there werem oret han
oneopponent,andthenchecksthepossibilityofmarkingeach
opponent. Thi s possibility ma y have th e same factors we
representedin
canpass
function.Atlast,playercandetermine
thebestplayertomark.
IV.DEC I SIO NM AKING PHASE 2
Themajorissueswehaveaddressedinthis phase are the
static assignment of roles and dynami c team strategy. We
adopteda formation/rolesystemsimilar toone describedin
[13,14,and15]eachformationcontains:
•
Formation name
 : Like real soccer team formations (e.g.
4_4_2)
•
Str ategic  area
: Th e area in  which the pla yer is mostly
supposedtobe.
•
Centerofstrategicarea
:alsoknownasthehomeposition
•
Playerrole
:weintroduced4applicablerolesforagents:
goalkeeper,defender,halfbackerandattacker.

RevistaAvancesenSistemaseInformática,Vol.5No.3,Diciembrede2008,Medellín,ISSN16577663
114
Inthismethodologytheplayerroleisstaticallyassignedto
theplayeraccordingtohisplayernumber,buttheteamstrategy
varieswithdifferentfactors.Themostimportantfactoristhe
ballposition.Accordingtothischangesinstrategythestrategic
areaoftheplayer,changes.Toselecttheappropriatestrategy
wedevelopeda fuzzyalgorithm (Figure8). Inadditionthis
algorithmhelpsanagenttofindouttheproperdistancewith
ballaccordingtohisstrategicarea(Figure9.).
Figure8.Fuzzymemberfunctionforselectingstrategy;a.outputvariable

stra tegy
b.inputvariable
ballPosition

Fuzzyrulebaseforstrategyisasfollows:
1.If(BallPositionisSec1)then(strategyisDanger)
2.If(BallPositionisSec2)then(strategyisCareful)
3.If(BallPositionisSec3)then(strategyisAttack)
4.If(BallPositionisSec4)then(strategyisGoodToGoal)
Figure9.Therelationshipbetweenballpositionanda
halfback
player
strategicpositioninfuzzyalgorithm.
a
b
Dependingontheagent’steamortheopponents’teamowns
theball(seesec. IV.
B
.)the output variable strategy(Figure
8. a) of fu zzy fun ction m ay chan ge. T he la st r emain ing
conditioniswhentheagentownstheball(seesec.IV.
A
.),in
this state agent usesthephase1decisionmaking results to
performanappropriateoperation(Figure10).
Figure10.Thediagramforactionselectionwhenplayersowntheball

A.Whoisthenearestplayertoball?

Afunctionisimplementedthatpredictstheballpositionafter
itsspeedislessthanorequalwith
MaxBallControllableSpeed
,
Thenreturnstheplayerwhoistheclosesttothatposition.

B.Whichteamownstheball?

Anotherfunctionisimplementedtocheckiftheballspeedis
lessthan
MaxBallControllableSpeed
 then callsthe
Nearest
Pla yer t o ball
 function, if th e player team side which th is
function r eturns, was t he same of ours, then t his function
returns 1 otherwisereturns0,Butiftheball speedismore

Asimplemethod fordecision making inRoboCup soccersimulation 3D environment  –  Malekiet al115
thanthevaluementionedabove,thenthisfunctionreturns1,
thatmeanswecannotdeterminewhetherballisoursornot.
V. RE SULT S
Weimplemented this meth odology on Scorpius Soccer
SimulationTeam.Astherearenotmanysourcecodesand/or
binariesof3Dteamsadoptivewithlatestchangesofserver
6
,
wedecidedtocompare thisteamwithitspreviousversion
7
.
Theresultsshowedthesuccessofthismethodology;theteam
performance in coordination an d collaboration h ighly
improved, in fact the players switched th eir strategic area
smoothlyastheteamstrategychangedinareasonablemanner,
the agents ca rried out th e high level ski lls much more
efficientlyandatlastthefinalresultsenhancedsignificantly
(Table1.Showsthefinalmatchresults).
VI.C ONC L USION
Inthispaper,anewmethodfordecisionmakinginRoboCup
soccer 3Dsimulation environmenthas beenproposed.First
an id ent ifica tion ap pli ed t o the Robocup Soccer 3D
environment whichledtomechanicalformulas,thensoccer
skillswereintroducedandclassifiedindifferentlayersthena
twophase mechanism for decision making is presented, in
this mechan ism both fuzzy and nonfuzzy algor ithms are
appliedand finallytheproposedmethodologyimplemented
onaRobocupsoccersimulationteamandtheresultsshowed
theefficiencyofthismethodology.
ACK NOWLE DG EM ENT
AuthorsexpressthankstoMr.S.MahdiHosseinifromthe
ECEdepartment,UniversityofTheran,fortheideas,which
oneswereusedatwritingarticle.
RE FE RENC ES
[1]StoneP.andMcAllester,D.,2001.AnArchitectureforActionSelectionin
RoboticSoccer.
AGENTS’01,
Montr´eal,Quebec,Canada.
[2]Veloso,M.,Pagello,E.andKitano,H.,editors,2000.RoboCup99:Robot
SoccerWorldCupIII,SpringerVerlag,Brelin.
[3]Noda,I.,Matsubara,H.,Hiraki,K.andFrank,I.,1998.Soccerserver:Atool
forresearchonmultiAgentsystems,AppliedArtificialIntelligence,12:pp.
233250.
6.The simu lation env ironme nt used for experimental results is
:rcsoccersim3D.0.5.5
7. This version wa sso similar to which placed 6t h  in Iran Open
2006 In ternationalRobocup Competitions.
[4]  Alireza F. Naeeni, 2004 .Advanced MultiAgent Fuzzy Reinforcement
Lear ning. D EGR EE PROJ ECT  Compu ter Eng ineeri ng, Hogs kola n
Dal ama u niversi ty Pre ss.
[5]  Dezfou lian, M. H. ,Ka viani, N. , Nikanja m,A. a nd RafaieJok andan,
M., 2 005 . Trai ning a Si mula ted Socce rA gent how to  Shoot u sing
Artifi cial  Neura l Net works. P roceedi ng of the I rania n Resea rchers
Confe rence  in Eu rop e.
[6]   Mal ec, J.  and Go rbet,  R., 2 003 . Ath esis for  Maste r of Sci ence in
Engin eering Physi cs. Lund u niversit y Press, Sweden .
[7]  Mozafa ri, M., Fard, A. M., Salma ni,V. and Naghibzadeh, M ., 2006.
An Improved Fuzzy Mechanism for 3D Soccer Player Agent’sShoot
Skill. IE EE INDICON , New Delhi, India.
[8]  Kok, J. R., Vlassis, N. and Groen, F., 2003. UvATrilearn 200 3Team
Desc ript ion.  Robot  Socce r World Cup V, LN AI, S prin ger Verla g,
Berlin.
[9]  Guestrin, C., Venkatara man, S. and Koller,D., 2 002. Context speci¯c
mu ltia gent  coor dina tion  and  plan ning  with  fact ored  MDPs. I n
Pro ceedi ngs  of th e Ei ghtee nth  Na tiona l C onfer ence  on Ar tific ial
Int ellig ence, Edmon ton,  Cana da.
[10] RoboCu pS occerSer ver3 D maintenance group, 200 3. The RoboCup
Socce r Simu lator  Manu al. h ttp:/ /sserver. source forge.ne t.
[11] Reis,L. P.and Lau, N., 2003 .RoboCup 3D Simulator   FCPortu gal
user ’s guide,  http: //www.ieeta .pt/r obocu p/archi ve.htm .
[12]  RoboCu p Soccer Server 3 D maint enance grou p, 200 3. Text instea d
of M anua l, h ttp:/ /sser ver.sou rcef orge.ne t.
[13 ] Cast elpietr a, C.,  locchi, L ., Nard i, D., P iaggio , M., Sca lzo, A. and
sgor bissa , A., 2 00 0. C oordi nat ion a mong  Hete rogen eous Rob otic
Soccer  Playe rs, In Pr oc. of Int ernat ional C onference o n Intel ligent
Robots a nd Systems (IR OS 20 00), Vol. 2 pp. 13 8513 90.
[14 ] Stone P., and Veloso M., 1 99 9. Task  decomposi tion,  dynami c role 
assignment and lowb andwidthcommu nication forr eal time strategic
team wo rk.  An Artifi cial I ntelli gence 11 0(2) , pp. 2 1127 5.
[15] Stone, P.a ndVeloso, M., 1998. Individua la ndcolla borativebehavior
in a tea m ofh omogeneous robot ic soccer agent. I n Proc. Of the 3
rd
int erna tiona l con feren ce on mu lt i agen t syst em.
K hash aya r N iki M aleki is B.Sc. student in Co mputer
Hardwa re Engineering in Depa rtment of Electrical an d
Compu ter E ngine ering, S hahi d Raj ae e Univer sity,
Tehran , Iran . He is a mem ber of Scor pius Simu latio n
Tea m, whi ch ha s particip at ed in ma ny Robocup
compet ition s. He  is IEE E/ACM  stud ent memb er. His
inter ested  resea rch area s are  mu lt i agent sy stems,
rein forcemen t lear ning a nd Syst em Ident ificat ion a nd
prog rammin g for emb edded syst ems.
M oha m ma d H a di Valip ou r  rec eived the B.Sc . degree
in I nformatio n Technology  Engin eering from
Depa rt ment o f Elec trica l a nd Co mpu ter E ngine erin g,
Shahid Rajaee University,Tehran, Iran in 20 08. He is a
member  of Scorpiu s S imul at io n Te am , wh ich h as
parti cipated in ma ny Robocup co mpetitions. Also he is
IEEE/ACMstudent member.His interestedresearchareas
are soft ware archi tecture,  agent ori ented progra mming,
communi cation syst ems, and mul ti agent sy stems.
Sad egh  M oka r i is B.Sc. student i nCo mputer Har dware
Engin eering i n Depar tment o f Electri cal an d Compu ter
Engineering, Sha hid Rajaee Univ ersity,Tehran, Ir an. He
is a mem ber of Scor pius Si mulat ion Team, whi ch have
partici pated in many Robocup  competitions. He is ACM
student  member. His inter ested resea rch area s are mul ti
agent  systems and  artifici al neu ral net works.

RevistaAvancesenSistemaseInformática,Vol.5No.3,Diciembrede2008,Medellín,ISSN16577663
116
Roo h ollah  Yey la g h i Ash r af i is B. Sc. student  in
Compu te r H ardw ar e E ngine ering  in  Departmen t of
El ectr ica l and  Comp uter  Eng ineer ing,  Shah id R aja ee
Uni versi ty, Teh ran , Ir an.  He i s a m ember  of S corp ius
Simulation Te am, wh ich ha ve partici pa te d in ma ny
Robo cup c ompetit ions.  He is  inter ested i n open  sour ce
proje cts and m ulti  agent sy stems.
M oh am m ad  Re za  J am a li re ceived t he B.Sc . degr ee
in C ompu ter Ha rdw are En gine ering  from Ami rka bir
Univ ersit y of Tec hnolo gy in 2 00 1. H e got  the M .Sc.
degree fro mS hiraz Uni versity in t he field of Artificia l
In tell igenc e in 2 003 . He i s a P h.D. c andi dat e sinc e
2004 under the supervisionof ProfessorLuca sin Centre
of Excell ence for Contr ol and Intel ligent Proc essing at
the De pa rt ment o f El ect ri ca l a nd  Compu ter
Engineer ing, Universi ty of Tehran, I ran. He i si nterested i n evoluti onary
algor ithms, multi  agent systems and design  methodologies in con trol and
aut omati on syst ems. Also he i s the le ader o f Scorp ius Si mula tion t eam
since  2005 .
C ar o  L u ca s rec eived  the M. Sc. degr ee from the
Uni ver sity  of Tehr an,  Ira n, i n 19 73 , an d the  Ph.D .
degree fr om the Uni versity of Cal ifornia , Berk eley, in
19 76 . He  is a  Pr ofesso r of C ent re of  Exce llen ce fo r
Cont rol and I ntel ligen t Pro cessin g at t he De par tment
of Electr ical a nd Comput er Engin eering, Uni versity of
Tehran,  Iran,  as well as a R esearch er at the S chool of
Intel li gent S ystem s (SIS), Insti tu te for St udie s in
Theor etical Physics and M athematics (I PM), Tehran, Ira n. He has served
as the Director of SIS (1 99319 97), Cha irman of the ECE Depart menta t
the University of Tehran (1 9861 988), M anaging Editor  of theM emories
of the Engin eering Facul ty,Un iversity of Tehran ( 1979 1991 ), Reviewer
of Mat hematica l Reviewers (since 1 987) ,Asso ciate Edi tor of J ournal o f
Intel ligent a nd Fuzzy sy stems (19 921 999 ), and  Chairma n of the IE EE,
Ira nS ection (1 990 1992 ). He was a lso a Visiting Associat eP rofessor at
the University of Toronto (summer, 198 91990 ), University of California ,
Ber keley  (19 88 198 9) , an Assi stant  Profess or a t Gar you nis Un iversi ty
(1 984 19 85 ), Un iver sity o f Ca lifor nia  at L os Angel es (1 97 51 976 ),  a
Sen ior R esea rch er a t the  Int erna tio nal  Cent re f or T heor etic al Phys ics
and the In ternati onal Centr e for Genetic En gineering a nd Biotechnol ogy,
bot h in T rie ste,  Ita ly, t he I nsti tut e of App lied  Mat hema tic s Ch ines e
Aca demy  of S cie nce s, H arb in In sti tu te o f El ect ric al T echn olo gy, a
Re sea rch A ssoc ia te a t t he M anu fa ctu ri ng R ese ar ch C orp ora ti on o f
Ontar io, and a Resea rchAssi stant at the Elec tronic Researc h Laborat ory,
University ofCa lifornia, Berkeley. He isth eholder of Patent on «Speaker
Independent Fa rsi Isolated Word Neurore cognizer». His resea rch interests
in clu de bi olog ica l com put ing , com put ati ona l in tel lige nce,  unc ert ain
system s, int ellig ent cont rol, n eura l netw orks,  mult iagen t syste ms, dat a
mi ning, b us in ess i ntel li ge nc e,  fi na ncia l mode li ng  and know le dg e
man agem ent.  Profe ssor L uca s has s erved  as th e Chai rma n of sev eral
In ter nat ion al C onf ere nces . He  was  the Fo und er o f th e SI S an d ha s
assist ed in foun ding severa l new resea rch org anizat ions and eng ineerin g
disci plines i n Ira n. He i s the re cipient  of sever al r esearch  gran ts at t he
Unive rsity o fTehr an and  SIS.