FUT-K 3D Team Description Paper 2013

Masato Ishitaka, Tatsuya Fujii, Minoru Shibata, Kazutomi Sugihara, Kazumasa Ohkuma, Teruya Yamanishi

Department of Management Information Science, Fukui University of Technology Gakuen, Fukui 910–8505, Japan

Abstract This paper describes concepts of movements for agents and unique tactics of team FUT-K 3D in the simulation league of RoboCup 3D Soccer. In addition, our future work is mentioned based on the simulation experiments.

Introduction

FUT-K 3D that is mainly composed of undergraduate students of Fukui University of Technology in Japan has been organized since fall 2007. The purposes of our team are to grow knowledge and experience of the computer language and the information science through applying themselves to RoboCup Soccer 3D Simulation. Though almost members of our team are unskilled at programming yet, we believe that now our team is developing with getting advice from other teams.

We had been participating in the world competition from RoboCup 2009 in Graz to RoboCup 2012 in Mexico City and could get to a lot of things about soccer strategies and techniques of the movements for humanoid robot as the 3D soccer agent from these competitions.

In this paper, our activities for developing the 3D soccer agent of this year are introduced as follows:

– Simultaneous updating of the probabilities on every "Probabilistic Behavior Selection" in the similar situations – Long-duration test with a view of verifying the potency of "Probabilistic Behavior Selection"

The details are explained in the following sections.

Our Previous Work: Probabilistic Behavior Selection

Two years ago, our agents were implemented the function called "Probabilistic Behavior Selection" that even if they encounter the identical situation, they stochastically select one action from multiple pre-defined behavior.

Definition of selectable behaviors

First, we defined the behavior to be selected. There are three types: "Passing", "Turing-aside" and "Breakthrough", and then the following eight options on these types.

– Passing: Direction(Right or Left) – Evasion: Direction(Right or Left) and Speed(Quick or Slow) – Breakthrough: Kicking or Dribbling

When a specific condition(e.g. one enemy's agent approaches our agent) was satisfied, the agents keeping a ball stochastically selects one from these options.

Judgment of game situation and Setting probabilities of each behavior

In order to implement the function of Probabilistic Behavior Selection in our agents, we made our agent discriminate the situations in a game. That is, situations are described by five factors:

– Area of our agent keeping a ball – Distance between our agent keeping a ball and the nearest enemy's agent – Distance to the nearest friend's agent – Elapsed Time – Point Spread(Goal Difference)

Our attempt made the distinction between 384 patterns in a game. The probabilities of each behavior under each situation were calculated by the following procedure.

– STEP1: We pick up some situations at random as samples. By Analytic Hierarchy Process(AHP), the probabilities to select each behavior are given in each sample situation. – STEP2: Using the three layer neural network, the probabilities in the remaining situations are estimated based on the probabilities obtained in STEP1.

Furthermore, we tried to update the probabilities during a game, based on the result of Probabilistic Behavior Selection. The results are expressed by three states, "Success", "Failure", and "Behavioral incapacitation". The conditions of three states on each behavior and the method of updating are described in the TDP of RoboCup 2010 [1].

Simultaneous updating of the probability in the similar situations

As a second step, we devised the updating method to raise the frequency during a game. Our new method was to update simultaneously the probabilities on every Probabilistic Selection Behavior in the similar situations. To introduce the simultaneous updating makes it possible to appropriately change agent's principle during a game, depending on opponent's tactics within a restricted time frame.

To raise the frequency of the updating, we introduce the similarity of situations in a soccer game. A set of situations is expressed by U and the probability of behavior b ($b \in B$) in a situation X ($X \in U$) is denoted as p (b|X). The similarity of situations X, X' is defined as follows:

$$sim(X, X') = \prod_{i=1}^{k} c(x_i, x'_i),$$

where $x_i(x_i')$ is a dummy variable of with respect to a factor i and $c(x_i, x_i')$, which is satisfied with $0 \le c(x_i, x_i') \le 1$, is a coefficient of a factor i.

On update of the probabilities, when an agent selects a behavior $b_0$ ($b_0 \in B$) under a situation X, the selection probability $p(b_0|X)$ is updated, based on the result of $b_0$, as follows:

$$p(b_0|X){new} = p(b_0|X){old} + v(r),$$

where v(r) is an increase or a decrease of probability assigned to a result r of $b_0$. At the same time, the probabilities of other behaviors $b(b'_0 \in {B \setminus {b_0}})$ are done by the following equations.

$$p(b_0'|X){new} = p(b_0'|X){old} - \frac{v(r)}{card(B) - 1}.$$

In addition, for the other situations $X'(X' \in {U \setminus {X}})$, the probabilities are updated with the similarity $\sin(X, X')$ as follows:

$$p(b_0|X'){new} = p(b_0|X'){old} + v(r)\sin(X, X'),$$

$$p(b_0'|X){new} = p(b_0'|X){old} - \frac{v(r)\sin(X, X')}{card(B) - 1}.$$

As the proposed updating method brings the possibility that the selection probabilities are negative, the threshold of probability is set to avoid a negative value.

Experiment

To verify the validity of our method, we attempted the following experiment. Our agents except "goalkeeper" and "sweeper" are implemented Probabilistic Behavior Selection. According to the format of RoboCup 2010, our team with the implemented agents played against another team, which was about as good as ours, over an extended time period. Fig.1 shows the process of "Point Spread" in 20 games against the team. Furthermore, our team with non-implemented agents played in much the same way and the result is illustrated in Fig.2. The result by the implemented agents is quite-variable compared to the result by non-implemented ones in the early stages and the process became stable as time advances. This result means that update of the probabilities effected the instability of the behavior selection in many games.

On the other hand, according to Table 1, there are differences among the first half and the latter in a game. Especially, the result by implemented agents reached statistical significance for "Goal Difference". In these experiments, as no regard is given to setting initial values of selection probabilities, the setting of proper ones should be sufficiently discussed.

Results of the experiment

( ) means the result of non-implemented Probabilistic Behavior Selection.

Conclusions and Future Works

For the autonomous decision of agents, we have implemented "Probabilistic Behavior Selection" and then updated the probabilities of each behavior during a game. Next, we devised the updating method to raise the frequency during a game. Our new method was to update simultaneously the probabilities on every Probabilistic Selection Behavior in the similar situations. Introducing the simultaneous updating is expected to appropriately change agent's principle during a game depending on opponent's tactics within a restricted time frame. In this paper, the results of two long-duration tests are reported with a view of verifying the potency of our approach. However, we could not find a significant outcome from the results of games.

As mentioned in the previous TDP[3], one of our future work is to add a new twist to our approach to affect the results in the simulation league of RoboCup 3D soccer. Moreover, another is to establish some approaches for prediction about the behavior of enemy's agent. The simplest approach is to estimate the destination of enemy's agent based on moving-average directional vectors. We have no doubt that advanced strategy can be formed by the union "Probabilistic Behavior Selection" and "Behavior Prediction".

Acknowledgements

This work is supported in part by research grants from Fukui University of Technology.

References

1. Yashiki, M., Miyajima, K., Sugihara, K., Ohkuma, K., Yamanishi, T. : FUT-K Team Description Paper 2010, Soccer 3D Simulation Team Description Paper for RoboCup2010 (2010).
2. Sugihara, K., Shibata, M., Kamada, Y., Ohkuma, K., Yamanishi, T., Uosaki, K. : Update of probabilistic behavior selection and its influence in the simulation league of RoboCup 3D soccer, The 74th National Convention of IPSJ, (2012).
3. Shibata, M., Ishitaka, M., Kamada, Y., K., Sugihara, K., Ohkuma, K., Yamanishi, T. : FUT-K 3D Team Description Paper 2012, Soccer 3D Simulation Team Description Paper for RoboCup2012 (2012).