Team Description of TsinghuAeolus3D

Yang Fan, Li Xiang, Yao Hengshuai, Zhang Zhuo

State Key Lab of Intelligent Technology and System, Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, P.R.China

http://robocup.lits.tsinghua.edu.cn

Abstract In this paper we first introduced our team TsinghuAeolus3D. Then we described the multi-layer architecture of our simulator. In the third part, we described the two stage decision making of the agents. In the end of the paper we discuss the future work of our team.

Introduction

TsinghuAeolus3D is a new team developed for the 3D Soccer Simulation Competition of RoboCup 2004 in Lisbon. The team consists of two master students and three undergraduates. We aim at both research and education. TsinghuAeolus3D learns much from past successful experience of TsinghuAelous in 2D. However, for two reasons, we don't just translate the codes from 2D. Firstly, the environment of 3D is quite different from 2D. Secondly, our 2D simulator seems to be not so satisfying in advice-taking. Till now, we've read through the source code of rcssserver3D-0.2 and developed a good understanding of the structure of the server. In the following parts, we'll introduce our 3D simulator's architecture and the individual skills.

Architecture

The overall structure of our simulator is divided into several layers:

Data Layer: this layer deals with the data received from Server and maintains the local world model. It tries to construct our recognition of simulating field.

Motion Layer: in this part we adopt the methods of machine learning to realize the process of decision making. It is responsible for generation, evaluation and mediation of actions.

Strategy Layer: the layer does something like global optimization and takes cooperation into consideration.

Executor: this part is used to realize the decision.

Agent Skills

The process of the decision making of our agent is based on a two-stage solution. The first stage is off-line learning. We use Q-learning to evaluate actions in the discrete space. Use the result of this stage as a starting point, on-line programming is then used. We use the result of Q-learning as a sort of heuristic information to do heuristic searching to obtain an "optimal" solution.

Cooperation and advice-taking are also taking into consideration, but in near future they are not our focus.

Conclusion

In this paper we described the thinking of architecture and individual skills of TsinghuAeolus3D. Part of it is done; while part of it is sort of plan. As 3D simulation is such a new field, we believe in not a short time since now the main work lies in the low-level skills. Our most recent job will be to develop a adaptive and advanced lowlevel platform for further developing.

References

1. Jinyi Yao, Ni Lao, Fan Yang, Yunpeng Cai and Zengqi Sun, Technical solutions of TsinghuAeolus, in RoboCup Symposium 2003, Padova, Italy
2. Marco Kogler and Oliver Obst, Simulation League: The Next Generation, in RoboCup Symposium 2003, Padova, Italy