View Original PDF

UTUtd 2004 Team Description

Mostafa Hadian Dehkordi, Peyman Zarrineh, Pedram Ghodsnia, Farid AmirGhiasvand, Hesamaddin Torabi Dashti

Computer Science and Mathematics Department, Faculty of Science, University of Tehran, Iran

http://www.fos.ut.ac.ir/~robocup

Abstract In the following, we briefly introduce the main ideas in UTUtd2004 development. The low-level abilities employed in designing our team are mainly based on Tsinghuaeolus[2] Team 2002. In this paper different parts of human brain have been described in terms of Balkenius model of learning [3] which has been presented as an emotional learning model. The model has been used for simulating a soccer player in which player can predict and decide his next move or action. The paper further presents an approach toward GCGA [1] and presents a brief solution. The player in this simulation is a trainable robot for a Robocup team.

Introduction

Biology has influenced computer science amazingly, therefore presenting an architecture which is using controls like "neural network", "genetic algorithms" and "hormonal system" (All biological based) for controlling agents described in computing terms can interesting.

What man do is some how related to his emotions. Emotion is essential for keeping human begin safety. Every human being emotion can be expressed his "Primary emotion" and "Secondary emotions". Primary emotion is unchanged and Secondary emotions are improved by emotional learning which in this paper is subject of our discussion. Fear, anger and aggressive are some most famous examples of the Secondary emotions. For every person a positive emotion predicts reward; on the contrary a negative one predicts punishment.

Because error-driven learning is too lazy and in many cases is impossible .bias learning must be used in many decision problems. For example fear from fire can not be learned. Before describing any emotional model the different part should be declared.

Problem statement

At the robocup contest each player must learn the way that he should do his task. He must decide when he can to shoot to goal. When he should pass or when he can through with posing the ball. The model is adopted for a Robocop player to learn what to do in a match or in other words provide the player to evaluate his secondary emotions by learning.

Methodology

How emotional learning works?

As shown in Fig.1, Thalamus gets the input from the environment and with projection derives the input in the form of vectors that can be proceed by the other parts of this model. And for a player (agent) some of the most famous parameters that can be used by other agents are pass, dribble and shoot .So a dynamic graph can be built and an optimization problem can be solved for each action or parameter. A vector is sent to the sensory cortex for primary processing then the output of the sensory cortex is used by the orbit frontal cortex and amygdale for constructing the output of the system.

Orbit frontal cortex decides with early driven learning. It can not be used alone for deciding because its process of learning takes a long time.

The emotional signals are built by amygdale. It is obvious if there is a good condition it can be good for all similar ones.

Emotional learning is done in amygdale and after the time that the output of the system is made and goodness of answer is derived the emotional part weight is tuned by reinforcement learning.

Figure 1: A graphical depiction of the Balkenius model.
Figure 1: A graphical depiction of the Balkenius model.

Agent Emotion

Stevo Bozinovski [4] has presented a working system which implements the concepts described above. We have used the working system for our model. The control architecture consists of four main parts:

  1. Crossbar associative memory
  2. Action computation module
  3. Emotion computation module
  4. Personality module

We applied this control architecture as a learning controller for a trainable goalkeeper robot in a RoboCup task. There are various roles in a robot football team; the basic ones are goalkeeper, defender and forward. In this paper we describe our work on developing a goalkeeper robot. The goalkeeper task is depicted in the below figure.

As shown in Fig.2, a simple strategy for a goalkeeper is to move along the goal line using some mechanism for line following and try to minimize the angle α (see Figure). We recognized the similarity between this control problem and the inverted pendulum control problem. So we applied the control architecture as a controller of the goalkeeper, as we did for the inverted pendulum problem. We divided the input space into 20 situations depending on the ball position and velocity

Figure 2: The goalkeeper task
Figure 2: The goalkeeper task

Geometric Areas

As we all know one of the most important challenges in Robocup is the decision that the players make. High level speaking this decision can be shooting, passing or dribbling. Shooting decision is made if it is understood that the goal probability is more than a specific range, else the decision will be limited to passing or dribbling.

Defining the dribble as a self-centered pass we can calculate the best pass using the below described model:

Passing is sending the ball to a point of the field under a known power and angle. Below you can find the description to some of the parameters which help our team to take the best decision:

− Global Position GP(T):

A model including:

  • (if t <= current time): The position of all moving components at a specific time (T)
  • (else): A prediction for The position of all moving components at a specific time (T)

− Virtual Global Position VGP(T,n,x,y):

A model similar to GP not including the position of the player number n which is in the (x, y) position and the ball which is with the same player.

− F_vgp (vgp1, n, x, y):

Imagine t_f is equal to the least time for sending the ball to the nth player at (x,y) from vgp1. F_vgp is the function which gives us the VGP (t_f, n, x, y).

− p(vgp1,vgp2):

The probability of changing the mood from vgp1 to vgp2. This is identified using the position of all the players in the field.

− g(vgp1):

Shows the strategic value of vgp1. This function returns a bigger value when we get nearer to our mission (scoring a goal). Due to the job this function does, there are parameters like security, getting nearer to the other team's goal and etc. used in it.

− f(vgp1,vgp2): f(vgp1,vgp2)=(g(vgp2)-g(vgp1))*p(vgp1,vgp2) Main function used in decision making.

− Action model:

As each action will lead to a move from vgp1 to vgp2, we can define all the actions like "action (vgp1, vgp2)"

− Priority(action(vgp1,vgp2)): The value of this function is equal to f(vgp1, vgp2).

− What is the decision made by the player who has the ball?

The player will choose the action which has the maximum result for Priority(action).

Conclusion

This trade off between bias learning and error driven learning is an important matter in neural network. In such problems where there is no time for long time learning, Emotional learning makes good answers, also many models are suggested for emotional learning. But none of them is complete. moren and balkanius is one of those models and has made a good solution at a robocup simulation

References

  1. Mostafa Hadian, Payam Mahmoudian, Farid AmirGhiasvand, Pedram Ghodsnia, HesamAddin Torabi Dashti, The UTUtd2003 Simulator Team: Globally Computable Geometric Area, 2003
  2. Yao Jinyi, Cai Yunpeng, Lao Ni and Yang Fan. Tsinghuaeolus source code, 2002. Accessible from http://robocup.lits.tsinghua.edu.cn.
  3. Jan Morén, Christian Balkenius, "A Computational Model of Emotional Learning in the Amygdala", Proceedings of the 6th International Conference on the Simulation of Adaptive Behaviour, Cambridge, Mass., 2000. The MIT Press.
  4. Stevo Bozinovski, Training a football playing robot using emotion based learning architecture, Proceedings of workshop Affect in Interaction, Siena, October 1999 / Pavia, 1999