Team Description of TsinghuAeolus3D-2005

Yao Hengshuai, Liu Rujia, Li Yiming, Yang Fan, Sun Zengqi

State Key Lab of Intelligent Technology and System, Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, P.R.China

http://robocup.lits.tsinghua.edu.cn

Abstract Currently we are constructing basis of TsinghuAeolus3D, intended features and schemes such as the overall architecture of the team and skills of individual agent are discussed. In the end of the paper, the future work of our team is introduced.

1 Introduction

Our team, TsinghuAeolus3D aims at research on Machine Learning and Multi-Agent System. After the 3D Soccer Simulation Competition of RoboCup 2004 in Lisbon, we reconstructed our program, madeour agents acting more logically and sensibly. In the following parts, we'll introduce our agent architecture and the individual skills.

2 Architecture

Our 2D Team performed well in the past years, so the basic layered architecture of our 2D agent will be inherited. The most obvious limitation of our 2D architecture is that higher layer strategy and advices would be hardly taken by the lower layers, so we'll design a new interface in 3D architecture for decision-taking in order to make both layers work harmoniously.

We are considering a "full-duplex communication" agent architecture, that lower layers collect information for the higher layers and get the decisions. Both requests and responds transfer in both directions.

The overall structure of our agent is divided into several layers: The overall structure of our simulator is divided into several layers:

Data Layer: this layer is the basic of Aeolus3D, deals with the data received from server, maintains the local world model and constructs the recognition of simulating field.

Motion Layer: in this part, methods of machine learning are introduced to realize the process of decision making. It is responsible for generation, evaluation and mediation of actions to be chosen.

Strategy Layer: this part is for Long-term planning, such as multicooperation,global optimization and advice-taking from the tactic library.

Executor: this part is used to realize the decision.

3 Agent Skills

Agent Skills include interception, dribble, and kick, etc. However, currently most of the 3D teams do not have strong skills. We also focus our attention on this part as a start.

The process of the decision making of our agent is a two-stage solution. The first stage is off-line learning. In order to select sensible actions, Q-learning is applied to evaluate actions learned from the first stage. Based on the generated heuristic information gained from Q-learning, we apply heuristic searching on-line to obtain an optimal solution for the agent to choose.

Cooperation and advice-taking are also taken into consideration. In order to be a successful team, we also want to focus our research on enhancing the learning efficiency, tackling conflicts between short-term target and long-term one.

4 Conclusion

In this paper we described the overall architecture and individual skills of TsinghuAeolus3D. As 3D simulation is such a new field in RoboCup, we recently still focus on individual skills and method of action chosen, but sooner or later, we'll apply our experience of agents collaboration and tactic making in 2D to 3D.

References

1. Jinyi Yao, Ni Lao, Fan Yang, Yunpeng Cai and Zengqi Sun, Technical solutions of TsinghuAeolus, in RoboCup Symposium 2003, Padova, Italy
2. Marco Kogler and Oliver Obst, Simulation League: The Next Generation, in RoboCup Symposium 2003, Padova, Italy
3. Yang Fan, Li Xiang, Yao Henshuai, Zhang Zhuo, Team Description of TsinghuAeolus3D, in RoboCup Symposium 2004, Lisbon, Portugal