Everest 2004 Team Description

Song Yifei, Zhao Kaiyong, Yu Zhongming, Zhangjia

Department of Automatic Control, Beijing Institute of Technology, 100081, P. R. China

Abstract This paper presents an overview of the architecture and scientific approach of the Everest 2004 simulation soccer team played in RoboCup environment. Our research is focused on how to apply fuzzy logic theory to multi-agent system, and now we have successfully realized basic skills like dribble, kick and high-level decision-making with fuzzy logic. Our setting goal is to take advantage of fuzzy logic in offline agent learning and combine that with online dynamic programming.

1 Introduction

Everest was initiated in June 2001 by 4 master students majored in Pattern Recognition and Intellectual System. Their names are Gu Yang, Liu Junfeng, Cui Lihui and Pan Feng. They had got the folow honours. Best Performance Award Winner of China RoboCup2001, Kunming - August, 2001. Champion of China RoboCup2002 Simulation League, League B (Server version 7), Shanghai -June, 2002. 3rd Prize Winner of China RoboCup2001, Kunming. 2nd Place Winner of RoboCup2002 Simulation League, Fukuoka, Japan - June, 2002.4th Place Winner of RoboCup2003 Simulation League, Padova, Italy - July, 2003 . There were five students including Song Yifei, Zhao Kaiyong, Yu Zhongming, Zhangjia and Malin continued this AI research,and we got 3rd and 4rh winners of China Robocup2003 Simulation League, Beijing –August,2003.This year there are also five students taking part in this research. They are Song Yifei, Zhao Kaiyong, Yu Zhongming Zhangjia and Zhang Xiaoyang. Our research will be based on Everest 2002 and continued on the area of fuzzy logic theory.

2 The research of Everest in agent's learning

We have made some experiments in the learning arithmetic of Multi-Agents, and we have put them into the basic skill layer. For example we used BP learning algorithms in the movement of intercepting, also reinforcement learning algorithms and fuzzy reasoning system have been used in kick action. We have used fuzzy control theory in dribbling movement. On the basis of this, we have also proposed a reinforcement learning model which is based on ANFIS.

Reinforcement learning is a focus question of present machine learning. Through the mutuality of environment,it adopts and try on the methods of trial and error from study and form a piece of optimum action tactics finally.

Our team adopted the new strategy of Fuzzy reinforcement-learning, which is based on ANFIS. This model adopts ANFIS as the approach fuction of reinforcement-learning. It's fit to the question of continuous input and output states.

The present reinforcement learning strategies are usually adapted to the matter of small dispersing state space. These strategies always implement the learning of best policy in every state by checking tables. And they have an obvious problem that it can't deal with the matter of multi-dimensions and continuous problems.

Import offline-learning into new network course, model which is based on the basis policy making can not have very good estimation publish among Multidimensional environment. The controls based on network are in the precondition of knowing the input and the output so we can simulate the middle course of information processing . It can be understand for the study of the setted fact. In this case, it is the simulation course from macroscopic to microcosmic. We try to receive some news mode of thinking from microcosmic to macroscopic. Thus we gained some breakthrough. If it is not anxious to constringent in the network dispersing, we introduce value function in the node of future of the network. Abandoning impossible values in order to determine the optimum value. The tactics can receive unexpected result in small range of cooperating, drawing virtual value in these results and reach the clear pass.

3 Formation

For the partition of the integer formation, we adopted fuzzy control, created world model, setted the action range for each player. We monitor the changing formation of players, statistic the player's position continuously in building the world model.After accounting we get the dangerous positions of the area,but not implementing the defined dangerous positions. The team of Tsinghua divide the field into sereval certain fields, but not define the dangerous positions by the changing formation. So, It 's misgivings to the various factors. In our policy, we create world model by players, but not the static scene.

4 Conclusion and Future Work

With the development of our team, we become to pay more attention to highlevel intelligent decision capabilities and cooperation. I Our next goal is to apply fuzzy logic to offline agent learning and combine that with online dynamic programming. We will do some work on world modle in future, and apply fuzzy logic to it.

We believe that our agents can achieve good performance after combining successful methods and new techniques.

Acknowledgements

We would like to own our thanks to RoboCup Committee for providing such a good platform for multi-agent system. We would like to thank CMU,FCPortugal,UVA and TsinghuAeolus for their published source code, which saved a mount of time at the beginning of our exploration.

References

[1] Yao Jinyi, Chen Jiang. Architecture of TsinghuAeolus. TsinghuAeolus 2001 Team Description Paper. ftp://166.111.68.41/robocup/Docs/TsinghuAeolus2001.pdf [2] Yao Jinyi, Chen Jiang. Q-Learning and Adversarial Planning with Application in RoboCup. ftp://166.111.68.41/robocup/Docs/TsinghuAeolus_kick.doc [3] Timothy J. Ross. Fuzzy Logic with Engineering Applications. 1995, McGraw-Hill Companies, Inc. [4] Gu Yang, Liu Junfeng, Cui Lihui, Pan Feng. Everest 2002 Team Description. In Proceedings of RoboCup-2002: Robot Soccer World Cup VI. [5] Cai Yunpeng, Chen Jiang, Yao jinyi and Li Shi. Global Planning from local Eyeshot: An Implementation of Observation-based Plan Coordination in RoboCup Simulation Games. Available at ftp:// 166.111.68.41/robocup/Docs/ TsinghuAeolus_defense.pdf