ZJUBase 2004 Team Description (2D)

Guo Yejun, Guan Jun, Luo Dijun, Liang Daming

National Laboratory of Industrial Control Technology Zhejiang University, Hangzhou, 310027, P.R.China

Abstract This paper describes the main features of the ZJUBase soccer simulation team, which participated for the first time at the RoboCup-2003 competition. After the brief introduction of ZJUBase 2003, the characteristics of ZJUBase 2004 are represented. These include the hybrid agent architecture, heterogeneous agent, passing behavior and shooting behavior. Finally, we will describe our future research directions.

1. Introduction

The ZJUBase 2003 soccer simulation team has attended the RoboCup2003 (World Cup) held in Italy and the China Cup 2003 held in Beijing. Our team, which is based on the UvA Trilearn[1] 2002 source code, attained the Top24 in RoboCup2003, Padova, July 2003. And we achieved the runner-up in China Cup 2003, Beijing, August 2003, going beyond Everest and WrightEagle.

Last year, we named our team "ZJUBase", which means the team is the base agent for the future research on multi-agent systems in Zhejiang University, and it provides a clear interface with flexible extending ability for future development.

This year, we have made several extensions to our team. First of all, we apply the hybrid agent architecture and adapt the reaction modal online. Secondly we implement the assignment and recognition of heterogeneous agent by an online coach. Then, we inquire some approaches of interception skills. Finally, we improve passing behavior and shooting behavior.

2. Agent Architecture

The hybrid agent architecture is applied in ZJUBase. It consists of modeling, communication, actions, domain knowledge, deliberative reasoning and reactive module. The communication component between agents is used to share world model and some decision information. The perception component gathers information from the real world, and the choice deliberative reasoning module/reactive module handle it. The reasoning module is our main decision-making component. It depends on the world model that is updated by the modeling component, and integrates the domain knowledge and other agents' information gained by the communication component.

Our main improvement in the architecture is the reactive module. Suppose that the actions must be sent to the soccer server before m millisecond, and it needs n milliseconds to make decision. If the agent gets visual information before p=m+n (ms) at each simulate cycle, the action chosen by decision-making component will be more suited. So, when receiving the sense_body information, we send (turn_neck Angle) and (change_view Width Quality) immediately. The agent can see important objects with the Angle parameter and receive visual information before p millisecond with the Width and Quality parameters. Moreover, we can check whether the commands we have sent are executed successfully by the soccer server, comparing the perception component and action component. So, the parameter p can be adjusted online.

3. Heterogeneous Agent

We abstract 4 criterions from 11 heterogeneous parameters. First, we consider the player start-up cycle. Though player_speed_max=1.2, it does not mean that every heterogeneous agent can run with the speed player_speed_max. Then how many cycles will the heterogeneous agent reach his own max speed? So the first criterion we consider is the agent's own max speed. Second, how much stamina will the agent use when it reaches his max speed? The other two criterions are kick_rand and kickable_margin, representing the kick ability.

To combine the 4 criterions, we normalize them to [0.0 1.0]. Suppose there is a criterion 'm', the greater the better, and m_max/m_min is the max/min value of all the heterogeneous agents. So m is mapped with the formula m_nor=m/(m_max-m_min).

Finally, we give the total reward provided by heterogeneous agent. Reward = (P^start_cycle_nor)(Q^sta_used_nor) + (P^kick_rand_nor)(Q^kick_margin_nor) Here P and Q are prime integers.

The player_decay parameter is used to recognize the heterogeneous agent. This parameter is between 0.4 to 0.6, 0.002 in precision. It means that player_decay has 1001 possibilities. So, the probability that this parameter of different heterogeneous agents is the same is: Probability = 1 - P(1001,6)/(1001^6) = 0.0149.

4. Passing

The passing behavior is based on the interception skills. To search more passing selection, the time to calculate interception should be reduced as much as possible. The interception skill, using numerical algorithm which is complex, was predigested when the player starts to intercept the ball. The passing velocity of the ball is searched as shown in the following figure. Each point in the figure means a passing velocity vector. It takes about 2 milliseconds to search all the space with the simple/fast interception skills.

Now, we are trying to apply TP0T-RL[2] to our pass selection. The field is divided into 48 parts and each agent has 48 Q-tables, each table has 10 Q-values. All the Q-values are initialized with expert knowledge, given the positioning strategy.

5. Shooting

Shooting is another important behavior if we want more goals, to win the game. The shooter knows the ball and the opponent goalie accurately, because they are so near. So, we use the complex numerical algorithm to parse if the shooting will succeed. Furthermore, the random in ball movement is taken into account. We just consider the kick_rand, because the expectation of ball_rand is zero.

Suppose that the shooting velocity of ball is PD, it will be noised in rectangle D. So, the ball may move through (line r), departing from the goal line. Or, the random ball will be caught by the opponent goalie. Due to a step forward superiority over the player in adversarial environment, a shooter is developed, considering the ball random also, with the capability of more adaptability and more precise.

6. Conclusion and Future Directions

In this paper we have quickly addressed some improvements in our new soccer simulation team ZJUBase 2004. For future directions, we are interested in studying reinforcement-learning techniques and applying fuzzy control & expert control to our team strategy. Our current team strategy is mostly depending on the expert rules in each divided area. So, how to fuzzicate partitions and regulate rules using theory of expert control is our future work.

References

1. Remco de Boer, Jelle Kok. The Incremental Development of a Synthetic Multi-Agent System: The UvA Trilearn 2001 Robotic Soccer Simulation Team[Master Thesis]. Artificial Intelligence and Computer Science, University of Amsterdam. February 2002.
2. Peter Stone. Layered Learning in Multi-Agent Systems[PhD thesis]. Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA. December 1998.