View Original PDF

TokyoTechSFC 2005 2D Team Description

Hidehisa Akiyama, Makinen Juho, Daisuke Katagami, Katsumi Nitta

Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology Nagatsuta-cho, Midori-ku, Yokohama-shi, Kanagawa 226-8502, Japan

http://www.ntt.dis.titech.ac.jp/~akiyama/robocup/

Abstract This paper describes the main features of the TokyoTechSFC 2D soccer simulation team that has participated in the RoboCup competition since 2000. We enhanced a player agent's basic skills using some learning methods and on-line search algorithms. And we introduced new agent architecture to realize a more realistic coaching system. Our system assumes that human works as a trainer. We are developing a GUI training tool that supports human trainer.

1 Introduction

The TokyoTechSFC is a 2D soccer simulation team that has been participating in the RoboCup competition since 2000. This team inherit all code from our former team HELIOS that ranked 7th in RoboCup 2002 and RoboCup 2003. Our code is written by C++ and implemented from scratch without the source code of any other simulated soccer teams. In this paper, we describe the outline of our agents abilities and our research topics.

Our main research purpose is to realize a more realistic adaptation mechanism using human interactive coaching system. We assume that our system accepts the human trainer's instruction and converts it to the format that a player agent can understand. In addtion to this, we have some research themes for the undergraduate or graduate students. Currently, we are trying to enhance player's basic skills using some learning methods.

2 Agent Abilities

2.1 World Model

Our localization method is based on the method of YowAI[9] and UvA Trilearn[4]. The idea of these methods are very effective for the self localization. In addition to these, we introduced a relative position management for the localization of other objects. This technique enables a player agent to recognize movable objects noiselessly even if the error of self localization is big. Especially, when a player agent controls the ball, this has much effect.

As a new feature, we introduced the direction accuracy information to the player agent's world model. This information helps a player agent to evaluate the action target point. For example, the direction accuracy gives an serious effect to the pass target selection and its result. And also, this information enables us to realize more strategic looking mechanism.

2.2 Basic Skills

Looking

In order to obtain the more exact world model, strategical/tactical looking mechanism is required. According to the current game situation, the accuracy of other objects and the accuracy of direction, player agents should determine the appropriate direction for their turn neck action. Player agents can calculate a utility for all possible neck angles and can select the best one.

Interception

Player agents can estimate their ball interception point using a relative ball position information. This method enables player agents to turn their neck to the other strategical points while they are trying to get ball.

Kicking

We introduced a new on-line search algorithm to improve the kicking skills. The basic concept is similar to the TsinghuAeolus's modified A* algorithm[3], but our method has a higher performance for the opponent avoidance. We made more effective heuristic by decision tree.

Kick target generator

When player agents perform the ball kicking action, they must predict a next ball owner and evaluate the result of that action. This requires much computation time. But our player agents can predict a ball owner using a modified Newton's method[8]. This method reduces much computation time and enables player agents to predict hundreds of kick target points.

Holding and Dribbling

A ball keeping skill is very important to avoid opponent defenders. When the ball owner player tries to keep the ball, he must estimate where is the best avoidance position. We are now applying the reinforcement learning to this problem as a undergraduate student's research theme.

3 Coaching System

3.1 Agent Architecture

We adopt the Priority/Confidence Model[5] combined with FC Portugal's SBSP methodology[7] to our agent architecture. This method parameterizes both of strategy and tactics. Each tactics is represented by the set of operating conditions and parameters and strategy consists of the set of tactics. Parameters in tactics define not only the team formation and role assignment, but also the priority list of player agents' action and coordination.

In our architecture, strategy includes one or more set of tactics. But, strategy must have one "basic tactics". Basic tactics do not have an operating condition. So, all player agents should always use this parameter set. Instead, we can not omit other parameters from this basic tactics. On the other hand, tactics other than basic tactics must include an operating condition. We call this type of tactics "additional tactics". We can omit a part of parameters in additional tactics. If some parameters in additional tactics is omitted, the parameters defined in basic tactics are derived and used. That is, additional tactics can become the differential parameters to basic tactics.

When player agents receive instruction information, they convert it to additional tactics. The additional tactics represents not only the feature of the individual behaviors by one player but also the feature of the cooperative behaviors by two or more players. So, this architecture has not only a strategical flexibility, but also a local tactical flexibility.

3.2 Training Process

In order to acquire the effective tactics, it is indispensable to verify the validity of tactics by the prior test. However, it is very difficult to acquire effective tactics automatically. For the complex tasks like a soccer, first of all we should analyze and imitate a human's decision-making process. So, we do not try the automatic tactics acquisition, but we try to acquire tactics by off-line training that human gives advice.

We assume that human works as a trainer to acquire the effective additional tactics in off-line games and a coach agent can use that data in on-line games as an advice. For this purpose, it is necessary to record the human's instruction information and convert it to the format that a player agent can understand. For the generality, we use the CLang[1] as a default information format. And, in our research, we are developing the tool that can convert the human trainer's instruction to CLang[2].

3.3 Training Tool

This tool helps us to instruct the player agent and has following features:

  • It can visualize all information that a coach agent can get. This tool can visualize not only all objects in the environment, but also communication messages between player agents. We can get all coach agent's sensory information visually.

  • It has the interface for the human trainer. This tool enables us to input the several instruction information using a mouse and a keyboard. This means that we can easily input the intuitive judgement and evaluation.

  • It can generate advice message represented by CLang. The input data is converted to the actual advice message and sent to player agents. And also, all of generated advice can be stored to the disk as additional tactics.

  • It can make the specific situation. This feature helps us to retry the same situation and to design a set-play.

  • It can visualize the player agents' internal information. This feature helps us to recognize a cross perceptual aliasing [6] between a player agent and a coach agent.

4 Conclusion and Future Works

We improved player agent's basic skills using some learning methods and on-line search algorithms. Based on these basic skills, we implemented a new agent architecture that can accept the external instruction information. We are developing the training tool that enables human to design team strategy.

However, it is important to consider about the problem of cross perceptual aliasing. Cross perceptual aliasing means a gap of recognition between a teacher and a learner. This problem is also concerned with the modeling capability of a coach agent. We have to consider about the method to model the other agents more strictly. And finally, we have to realize a coach agent that can use the stored tactics as an advice in a real game. On-line game analysis is our next research focus.

References

  1. Users Manual: RoboCup Soccer Server. http://sserver.sourceforge.net/.
  2. Hidehisa Akiyama, Daisuke Katagami, Yoshiaki Yasumura, and Katsumi Nitta. Adaptation to the Multi-Agent Environment by External Advice. In Proceedings of the SCIS and ISIS 2004, 2004.
  3. Yao Jinyu, Chen Jiang, and Sun Zengqi. An application in RoboCup combining Q-learning with Adversarial Planning. 2002.
  4. Jelle R. Kok, Remco de Boer, Nikos Vlassis, and F.C.A. Groen. UvA Trilearn 2002 team description. RoboCup 2002: Robot Soccer World Cup VI, 2002.
  5. Jan Lubbers and Rogier R. Spaans. The Priority/Confidence Model as a Framework for Soccer Agents. In Proceedings of the Second RoboCup Workshop, 1998.
  6. Chizuko Mishima and Minoru Asada. Active Learning from Cross Perceptual Aliasing Caused by Direct Teaching. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 1999.
  7. Luis Paulo Reis, Nuno Lau, and Eugenio Oliv´eira. Situation Based Strategic Positioning for Coordinating a Simulated RoboSoccer Team. Balancing Reactivity and Social Deliberation in MAS, pages 175–197, 2001.
  8. Peter Stone and David McAllester. An Architecture for Action Selection in Robotic Soccer. In Proceedings of the Fifth International Conference on Autonomous Agents, pages Pages: 316–323, 2001.
  9. Takashi Suzuki. Team YowAI Description. RoboCup 99: Robot Soccer World Cup III, 2000.