AmoiensisNQ-2D Soccer Simulation Team Description

Jianhuai Cai, Wuyi Yu, Wei Li, Zishen Lin, Qinquan Gao, Xiao Lin

Automation Department of Xiamen University

Abstract Completely understanding the running mechanism of Robocup simulation game model--SoccerServer and basing on the basic code of TsinghuaAeolus2002 team, we built ourselves simulation team AmoiensisNQ. Player agent adopts layer architecture, and has various individual skills, including interception, kick, block, mark, and follows a high level strategy which is composed of positioning, handling ball, vision and hearing, and has a good cooperation to realize the collective goal--beating the opponent.

Introduction

The AmoiensisNQ have been participating in RoboCup's soccer simulation tournaments since 2003 and ever achieved a number of successes. The team description paper at hand focuses on the AmoiensisNQ 2D, our team competing in soccer simulation's 2D league. To exploit AI and machine learning techniques wherever possible is the goal of our team, now we're working hard at this.

Apply BP Neural Network in Team Design

Agents' moving with ball's displacement is a much complicated nonlinear problem, and that the simulated platform provides information along with noise, thus it is rather difficult to describe the position of the 11 agents precisely with mathematical models. We employ the function of BP neural network which can realize the input and output nonlinear mapping of any precision, making use of the knowledge on soccer expert, to design and train the various basic formations of robot soccer team.

Design and Train Team Formation

Figure 1 suggests the result of network training with neural network toolbox in Matlab. After a 120-Epoche training, BP neural network's studying tend to converge quickly. Testing the trained network with 120 samples,the average error of abscissa X is 0.69m with the maximum 1.03m, while the average error of vertical coordinate is 0.42m with the maximum 0.75m. In a simulated platform with a length of 105m and a width of 68m, the precision of network's studying can meet the demand of agents' running oriented to the ball.

We use the method represented above to train the agents with various formations, such as 451, 433, 442, 352, 532, etc., and save the weight values to build up a formation database.

Fit offence and defence Sensitivity of Field

We use offence and defence sensitivity to describe the importance of position in the field, according to soccer expert, the closer the position to opponent's goal's center, the more benefit to offence, and the greater the offence and defence sensitivity is, the closer to self goal's center, the more unfavorable to defense, the smaller the offence and defence sensitivity is. Use the neural network toolbox in Matlab to train the BP network represented above with captured samples, and fit the offence and defence sensitivity of the field base on this network, the result presents as figure2 shows.

Describe Interception Success Rate

In order to make decision of interception, we have to describe the relationship of two different interception periods, so we use a BP neural network to describe this relationship. The network has two inputs: one is the difference of two periods, another is the period which is smaller, and one of outputs of the network is the interception success rate of the smaller period relative to the bigger one. Select one typical period pair, according to soccer expert, assign a success rate for the period pair, the bigger the difference of the periods is, the greater the success rate is, the smaller the smaller period is, the greater the success rate is. Use the neural network toolbox in Matlab to train the BP network constructed above with captured samples, utilize this network to calculate the success rate of 2500 period pairs, part of result represented as table1 shows.

Table 1. Interception Success Rate

Handle Ball Based on Fuzzy Inference System

Strategy for handling ball includes three parts: kick ball, pass ball and drive ball. To win the game, kicking is set the highest priority. The angles between player who controls ball and both gateposts of opponents are considered as optional kicking angle. Scanning from left gatepost and ranging every 3 degrees, we can get a sequence of data on the direction and velocity of kicking goal as supposing ball is moving at max velocity. Through calculating the minimum intercept cycles of all the opponent players and comparing the results to the cycles of ball moving from contemporary position to across opponent's goal line, we obtain the success rate of every kicking angle. If at least the success rate of one angle is above 0.5, kicking is the right decision. The player will choose the angle with highest success rate as kicking direction.

Passing and driving ball is the core of handling strategy, also the complex part. Based on the fuzzy inference system, we choose the path with highest priority as the most proper one for passing and driving ball. To reasonably remark the priority of passing and driving ball, we definite a estimating factor F. By multiply F by the max priority of passing ball, the standard for two priorities will be unified. The value of F for different player in different region is set according to the transcendental knowledge and modified according to the result of games. As the factor F is normalized, the robot player will carry out the motion with higher priority after comparing the highest priority of passing ball and driving ball.

Heterogeneous Players Strategy Based on Fuzzy Evaluation and Inference

Fuzzy Evaluation of Heterogeneous Players

SoccerServer introduced the conception of heterogeneous players from version 7. Each competition contains seven different kinds of players including isomorphic players, and each player has 11 parameters reflecting capabilities in different sides and only isomorphic players' parameters are changeless. The parameters of heterogeneous players random produce on certain range. Contacting the simulation motion model in the SoccerServer, we use fuzzy evaluation factor set U= {MaxSpeed, AccelTime, StamCos, StamRes, KickArea, TurnIner}, calculation formula of the six factors as flows:

$$MaxSpeed = Min(\frac{dash_power_rateeffort_maxstamina_max}{1-player_decay}, player_speed_max)$$

$$AccelTime = \frac{\ln(1 - 0.99 * MaxSpeed * (1 - player_decay) / (dash_power_rateeffort_maxstamina_max))}{\ln(player_decay)}$$

$$StamCos = (1 - player \ decay)/dash \ power \ rate*effort \ m$$

$$StamRes = stamina inc max$$

$$KickArea = kickable \ margin$$

$$TurnIner = 1/(1 + MaxSpeed * player_decay * inertia_moment)$$

A fuzzy membership function which builds its factor set on the judgement set is the most important section of the fuzzy evaluation. Connecting with football expert knowledge and making over through and through by observing actual competitions, we confirm the fuzzy membership function of the six factors. The judgement set V uses a three-grade evaluation set, the evaluation result is divided into "good, general, poor", that is to say V= {good, general, poor}. According to the parameters of each kind player produced randomly, we have a 6*3 fuzzy relation matrix from the factor set to the evaluation set of this kind player, and the subscript "id" shows the NO. of the player.

The weight of each index in the ascertain factor set is also important .The robot soccer team is made up of eleven agents. Each agent has different task. In the team we designed ,there are seven roles, including the goalie(GL),side back(SB).center back(MB), side attack midfield (SM),center attack midfield (MM),side vanguard(SF),center vanguard(MF).We just consider other six roles' isomeric choice for the goalie can only use isomorphic role. The weight of index in the ascertain factor set is different for each role. According to the soccer expert's knowledge and repeating test, we have the weight for each role as the expression shows:

$$W_{rd=SB} = (0.10, 0.30, 0.18, 0.27, 0.10, 0.05)$$

$$W_{rd=MB} = (0.10, 0.30, 0.18, 0.27, 0.05, 0.10)$$

$$W_{N_{rd-SM}} = (0.10, 0.25, 0.20, 0.30, 0.10, 0.05)$$

$$W_{r_{d-MM}} = (0.10, 0.25, 0.20, 0.30, 0.05, 0.10)$$

$$W_{rd=SF} = (0.10, 0.40, 0.15, 0.20, 0.10, 0.05)$$

$$W_{rd=MF} = (0.10, 0.40, 0.15, 0.20, 0.10, 0.05)$$

Thus we have the result of the No. id player matching the role rd is

$$B_{rd-id} = W_{rd} \circ R_{id} = (b_{rd-id-1}, b_{rd-id-2}, b_{rd-id-3})$$

(The symbol " ° " is weighted average $(\bullet, \oplus)$ ). For the roles rd, the membership degree is $b_{rd-id-1}$ when the player is remarked 'good'; the membership degree is $b_{rd-id-2}$ when the No. id player is remarked "general", the membership degree is $b_{rd-id-3}$ when the No. id player is remarked "poor". For all these seven kinds of players, we sort the $b_{rd-id-1}$ from big to small according to their value, making them the candidates of the role rd. Then we have the matrix $Candi_{6\times7}$ of all candidate players for all roles and the priority matrix $Pri_{6\times7}$ as follows:

$$Candi_{6\times7} = \begin{pmatrix} C_{11} & \dots & C_{17} \ \vdots & \ddots & \vdots \ C_{61} & \dots & C_{67} \end{pmatrix}$$

$$Pr_{i_{6\times7}} = \begin{pmatrix} P_{11} & \dots & P_{17} \ \vdots & \ddots & \vdots \ P_{61} & \dots & P_{67} \end{pmatrix}$$

In the matrix, $C_{ij}$ is the NO. of the candidate players, and $P_{ij}$ is the membership degree of the candidate player which is remarked "good".

Using Heterogeneous Players Strategy Based on Fuzzy Inference System

In order to solve the problem distribution of heterogeneous players among various roles of the same type, we use three global strategies, defense bias, and attack and defense deviation, attack bias. Different role has different priority of using the heterogeneous players in different global strategies, according to the football expert knowledge and tested and modified repeatedly.

We use formulas (15) to (17) to decide the priorities.

$$W_{\text{Stratesy=Defence}} = (0.215, 0.225, 0.165, 0.155, 0.125, 0.115)$$

$$W_{\text{Strategy=Balance}} = (0.165, 0.175, 0.165, 0.155, 0.175, 0.165)$$

$$W_{\text{Strategy=Offence}} = (0.115, 0.125, 0.165, 0.155, 0.225, 0.215)$$

We construct a fuzzy inference system to use different global strategies in the game. We choose three parameters, the match time, scoring gap and situation on stage, as the input of the system. The global strategy depends on the output of the system. The match time is the present period numbers since the starting of the game, divided into three fuzzy spaces (the beginning, the middle and the end). The scoring gap is the different of number of goals between us and opponent, divided into three fuzzy spaces (lagging, draw, leading), too. The situation on stage is to describe the situation of us on stage since the beginning of the game, using the average attackdefense sensitivity of ball point to represent the situation on stage, and the space is divided into five fuzzy spaces (disadvantage, less disadvantage, draw, advantage, more advantage). The global strategy is divided into three fuzzy spaces of the above mentioned (defense bias, attack and defense deviation, attack bias).

Fig 3 is the MATLAB simulation result of the input and output relation among match time, situation on stage and the global strategy

Strategy Application

Fig 4 is the MATLAB simulation result of the input and output relation among scoring gap, situation on stage and the global strategy.

Based on the above fuzzy inference system, we can use different global strategies dynamically during the game, and finally get a priority matrix for the using of the heterogeneous players , LPri i W 6 7 6 7 Strategy Pr (the operation is to multiply every row in Pr 6 7 ⁱ by the corresponding weight in ^WStrategy ) , considering the candidate player matrix of typesCandi6 7 traversaling the priority matrix LPri6 7 , to determine heterogeneous types of different roles by the weight. With the change of the global strategy, the corresponding priority matrix changes, and so does the heterogeneous type of different roles. Then we can adjust the heterogeneous types according to those changes.

Future directions

Coach is a new research direction of simulation competition, as a point which treats complete information centralized of the distributed system; it offers an intermediate zone between central study and distribution study. We will develop our research work in the fields of opponent modeling, planning recognition and mechanism of suggestions.

References

[1] A. Mackworth. On Seeing Robots[A]. In Computer Vision: Systems, Theory, and Applications[C], 1-13. World Scientic Press, Singapore, 1992. [2] The RoboCup Federation. What is RoboCup[DB/OL]. http://www.robocup.org, 1997-2005. [3] P. Stone. Layered Learning in Multi-agent System[D]. PHD thesis, Computer Science Department, Pittsburgh, Carnegie Mellon University, 1998. [4] P. Carpenter, P. Riley, M. Veloso, and G. Kaminka. AT-Humbolt Team Description[A]. In RoboCup-2000: Robot Soccer World Cup IV, Springer Verlag, Berlin, 2001. [5] M. Riedmiller, A. Merke, D. Meier, A. Hoffmann, A. Sinner, O. Thate, and C. Kill. Karlsruhe Brainstormers2000 Design Principles[A]. In RoboCup-2000: Robot Soccer World Cup IV, Springer Verlag, Berlin, 2001. [6] L. P. Reis, J. N. Lau, and L. S. Lopes. FCPortugal Team Description[A]. In RoboCup-2000: Robot Soccer World Cup IV, Springer Verlag, Berlin, 2001. [7] Jinyi Yao, Jiang Chen, and Zengqi Sun. An Application in RoboCup Combining Q-learning with Adversarial Planning[A]. The 4th World Congress on Intelligent Control and Automation[C]. WCICA, 2002. [8] Yunpeng Cai, Jiang Chen, Jinyi Yao, and Shi Li. Global Planning from Local Eyeshot: An Implementation of Observation-based Plan Coordination in RoboCup Simulation Games[A]. In RoboCup-2001: Robot Soccer World Cup V, Springer Verlag, Berlin, 2002. [9] Jinyi Yao, Jiang Chen, Yunpeng Cai and Shi Li. TsinghuAeolus2002 Basic Source Code[DB/OL]. [10] Mao Chen, Ehsan Foroughi and Fredrik Heintz. Users Manual of RoboCup Soccer Server[DB/OL].