View Original PDF

RoboSina 2006 Team Description

H. Bashiri, N. Farahmand, H. Ghiasi-Rad, N. Jafari, N. Kaviani, A. Mokhtari, A. Nouri, M. Rafaie-Jokandan, M. Taheri

RoboSina Simulation Research Group, Department of Computer Engineering, Bu-Ali Sina University, Hamedan, Iran

Abstract RoboSina 2006 soccer simulation team is the result of a research project in Bu-Ali Sina University which was started in June 2002 at the Department of Computer Engineering and funded by university. In its first attempt, RoboSina achieved 10th place in the RoboCup 2003 world cup. Also RoboSina 2004 and RoboSina 2005 achieved 5th place in the RoboCup 2004 and RoboCup 2005 world cup. Learning strategy is the most improvement which has been made on this team compared to its preceding edition, RoboSina 2005. This paper describes the main features of RoboSina 2006 as well as the learning improvements that have been made on this team.

Introduction

RoboSina 2006 is a 2D soccer simulation team that participates for the fourth time in the RoboCup world cup. In its first attempt, RoboSina ranked 10th place among 48 teams participating the simulation league of RoboCup 2003 world cup in Italy. Also RoboSina 2004 and RoboSina 2005 achieved 5th place in the RoboCup 2004 and RoboCup 2005 world cup competitions. RoboSina is the result of a research project funded by Bu-Ali Sina University in Iran. Some changes had been made in new team member and now the research group consists of six undergraduate students as well as an instructor who serves as the head of the research group. Two other members advise team in high strategic cases.

Totally in the most of lower level parts hence have been redesigned and reprogrammed by our team members.

Having studied the low-level methods used by some previous top teams such as CMUnited, UvA Trilearn, and TsinghuAeolus [2–4], we provided several essential ideas to improve the existing low-level methods and implement them in our team. Although our higher-level strategies were very simple in the first stage of developing the team, having powerful low-level skills such as dribbling and accurate localizing made RoboSina a successful team in RoboCup 2003. Since then, we have made several improvements in both low-level and strategy-level skills of our team. Using neural networks to achieve a more reliable scoring policy, employing reinforcement learning to obtain an enhanced blocking strategy and train team by last matches are some of our new improvements applied to RoboSina 2006.

Improved Localization Algorithm

We have an improvement in previous version but it was not perfect in RoboCup 2005. Our applied algorithm is so fast in comparison with same class algorithms but we need some amends to reach accurate place in world model. Because of existence of noise in our perception information, an agent must have ability to find its location in such an environment with the least possible error. We have made a substantial effort on providing a good algorithm for localizing a player agent in the soccer simulation environment, which leads to a fast O(n lg n) time algorithm, where n is the number of flags visible in the field. We have used some computational geometry algorithms for intersecting convex polygons and adopt them to reach a very fast and accurate localization algorithm.

Our algorithm constructs a convex polygon corresponding to visual information obtained from each flag in the field. The algorithm then employs a plane sweep method in which a sweep line is moved downward over the plane and maintains the convex polygons intersecting it. Using a divide and conquer method, we developed an O(nlgn) time algorithm to compute the intersection of n convex polygons, which is a good approximation for the exact position of the player. The error of our algorithm is considerably smaller than what have already reported by many other teams. Details of our method can be found in [5].

Reinforcement Learning

The idea of reinforcement learning is based on the principle of interaction with a system and learning from success and failure. In RoboCup, reinforcement learning can successfully applied: We applied reinforcement learning to learn skills like kicking and dribbling the ball, as well as multi-agent coordination and team-play.

Although reinforcement learning works fine in the adaptations have to be made to be able to use to restricted and imperfect knowledge of the environment. Reinforcement learning can be done on two layers for our agents in team: the layer of basic skills of a single agent like locate in a specified position, intercepting the ball or dribbling the ball, and the layer of team-play on which the agents need to learn cooperative behavior. We were able to present successfully learned applications on the layer of skills as well as on the layer of cooperation.

The complexity of the task arises from the complex dynamics of the system on the one hand and from the objective to learn a time-optimal policy, on the other hand. The main algorithmic problems which have to be tackled are the high dimensional, continuous state and action spaces that arise for problems on the real agents. We address these problems using neural and grid based function approximators, abstract states and memory based approaches that efficiently exploit the training trajectories. In a second experiment we let the agent learn to intercept a rolling ball. An ongoing

Conclusions

In this paper we have discussed the main features and improvements of RoboSina 2006 soccer simulation team. Our lower-levels are currently works reasonably and several improvements have been made on higher-level issues. Our future research will mainly focus on improving our higher-level strategies, using machine learning techniques and employing a coach to analyze the game and give advice on the best possible strategy.

References

  1. R. de Boer and J. Kok. The Incremental Development of a Synthetic Multi-Agent System: The UvA Trilearn 2001 Robotic Soccer Simulation Team. Master's thesis, University of Amsterdam, The Netherlands, Feb. 2002.
  2. R. de Boer, J. Kok, and F. Groen. UvA Trilearn 2001 Team Description. In Robocup-2001: Robot Soccer world cup V, Springer Verlag, Berlin, 2002.
  3. Y. Jinyi, C. Jiang, C. Yunpeng, and L. Shi, Architecture of TsinghuAeolus. In Robocup-2001: Robot Soccer world cup V, Springer Verlag, Berlin, 2002.
  4. P. Stone. Layered Learning in Multi-Agent Systems. PhD thesis, Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, 1998.
  5. H. Zarrabi-Zadeh, M. Rafaie, M. Shabani, and N. Kaviani. Robot Positioning Using Visual Information. In 2nd National Computer Conference (NCC'2003), Mashhad, Iran, Dec. 2003.