OXSY 2017 Team Description

Sebastian Marian, Dorin Luca, Bogdan Sarac, Ovidiu Cotarlea

OXygen-SYstems laboratory, Str. Constantin Noica, Bl.5, Sc.C, Ap.36, C.P. 550169, Sibiu, ROMANIA

http://oxsy.ro

Abstract Oxsy team was founded in July 2002 for a graduation project of one student, Sebastian Marian, in the field of Multi-Agent Systems, at the Department of Computer Science of Lucian Blaga University (Sibiu - Romania). After graduation he continued the work on this project and so was born Oxsy team. As we started from scratch, our ideas, concepts and beliefs, was implemented year by year and today, we are happy to see that we are on the right way, as our team was growing in these years, more than we expected from the beginning. If we'll qualify to the competition, this year we'll reach at the 15th consecutive participation, in RoboCup Soccer Simulation League.

Introduction

In July 2003 at RoboCup competition, which was held in Padua - Italy, we won the first round and for us it was a good surprise for first year of participation. Then, in the next year, we participated in Lisboa - Portugal for the second time, and again we obtained a good result (the 11th place). In 2005 in Osaka – Japan, we participated for the third time and finally we entered in the first 8 teams of soccer simulation league, as we won (the 8th place). In 2006 the competition was held in Bremen – Germany and we won (the 7th place). In 2007 we went to Atlanta – Georgia (U.S.A), where we obtained (the 5th place), the same result which we achieved in 2008 in Suzhou – China. Finally, in 2009 in Graz, we entered in the first 3 teams in the soccer simulation league, as we won (the 3rd place), the same result which we achieved in 2010 in Singapore. In 2011 we came back from Istanbul - Turkey with (4th place). In 2012 we were in Mexico City, where we had a bad experience as we made some major changes in our defensive system, and also many others overall our team strategy, changes which was not very well balanced at that time, with all others characteristics of our team, as we didn't qualified for finals, from the second round groups. In 2013 we came back in top, as we won (the 6th place), in Eindhoven – Netherlands. In 2014 the competition was held in Joao Pessoa – Brazil, and we entered on the stage for the third time in our participation history, as we won again (the 3rd place). In 2015 we won the 4th place as we played the semifinals in Hefei – China. Last year in Leipzig – Germany, we missed the semifinals and we came back with (5th place). This year the competition will be held in Nagoya - Japan. As we already have a very good experience in 2D Soccer Simulation league, we hope that our new ideas and improvements will be reflected in the competition where we will also test other tactical elements developed.

Adaptive route for achieving the goal

2010 was the first year when we have involved the coach in our team strategy. Beside of his classical attributions, of changing player types or recognizing opponent player's type, which already were implemented, we felt that we can use it more efficiently, in order to gives some tactical advices during the game. As the coach has the privilege to receive full visual information, without any kind of noise, we can use it to make an opponent modeling. In fact, we believe that it is more important to adapt the strategy during the game, instead before it. We also think that importance of the coach is not speculated very well right now, and maybe it will be a good point for research, not only for our team but for all the teams involved in soccer simulation. So, in one hand based on some typical neural networks that we developed to be used by the coach in some specific way and in the other hand based on the power of the coach, who has a full view of the whole field without any kind of noise, this year we extend the coach attribution with the following:

• Adaptive selection of the next action, in offensive phase, based on the starting cluster and on its route scoring.

Clustering the pitch

After we analyzed many games against different types of opponents we have decided to divide the pitch in 24 clusters (6 x 4). Each cluster will retain for every player from our team, all the routes and theirs scoring during many games against different opponents. Actually, each opponent team will have his own route profile for every cluster in the pitch. As the coach has the privilege to receive full visual information, this job will be in his charge. In fact his job is to analyze all the routes created by our players by passing the ball from one player to another and after this he must set the score of the effectiveness of all these routes. Actually, each pass will be done from one of these 24 clusters. All this chain of passes will define a route and each route could involve many clusters and of course many players. One route will be done when our team will lose the ball or the play mode will changing from "play_on" to any other one. If one route will go through many clusters, each cluster involved in this route will retain a sub-route of the whole route. For example, in (Fig. 1) if we assume that the ball is in possession of the central defender number 4, who will pass the ball to the left side defender number 3, who at his turn will pass the ball to the offender number 11, who will lose the ball trying to dribble an opponent, then we will have the following situation:

In the above scenario, the complete route will be 4 – 3 - 11. Now, the clusters involved in this route will be 10, 9 and 17. So, the coach will have to assign for each of these 3 clusters one route or one sub-route, as well as the score for each of them. In this way, the route 4 – 3 – 11 will be assigned to the cluster 10, then the route (subroute) 3 – 11 will be assigned to the cluster 9 and finally the route (sub-route) 11 (as the left side offender which have received the ball, didn't pass the ball to another teammate and he just tried to dribble one opponent without success and finally he lost the ball) will be assigned to the cluster 17, the cluster in which he touched the ball for the first time and where he also lost the ball.

Selecting the best route

Of course that the final goal of this approach is that the player who is in the possession of the ball to choose the best next action / route (pass or dribble the ball), based on the cluster from which he will kick the ball and on the scoring of all the available routes of this cluster. Routes will be created through many games played against different teams and each team will have his own profile of routes in each cluster. The responsibility of observing, creating, scoring or re-scoring, new or existing routes will be on the coach. Then, these useful information will be used by the whole team, just to achieve better results in the offensive phase. This approach will refer only to the high level of the action (to which teammate, the player in possession of the ball, should try to pass the ball if this teammate is available for passing right now) and not to the low level of it (with which power and direction the ball should be kicked). In fact, if the player in possession of the ball will have for example many alternative to continue the current offensive phase, with almost the same achievements calculated by our main kick action decision module, he can select the final decision after checking the best route proposal of this new module developed. This approach could be seen as an additional decision factor when the player in possession of the ball has to chose from two or many actions in the next cycle. In (Fig. 2) we show the most used routes during the games with different opponents, without taking in count one specific cluster. We observed that the side defenders number 3 or 2 never made a back pass to the goalie, as the central backs 4 and 5 did it in certain situations. Also, we have discovered that the central midfielder number 10 is the most involved player in our offensive phase. There are also some other routes which are not represented in (Fig. 2), as they have happened only in some certain situations when we have played against certain opponents.

We refer here to some depth passes from central midfielder number 10 to the side forwarders number 11 and 7, and also from the left midfielder number 6 to the right forwarder number 7, as well as from the right midfielder number 8 to the left forwarder number 11.

Scoring and re-scoring the route achievements

When the coach add a new route to certain cluster he also will associate to this route an achievement score. The score will be greater if the last player from the route will get closer to the opponent's goal. The responsibility of the coach is to observe not only the final result of the action but the whole action as well and to evaluate the score according to more factors. Then he will set the score to the route and to the all sub-routes in every cluster through which the ball has passed. One of the most important thing of this approach is happening when the current evaluated route already exists and in this case the coach should only re-score it. This is very important for the dynamic adaptability of our team, to the opponent's changes during the game. In this way, one good initial route could be replaced by another one, which will be better than the first one, because of some tactical changes made by the opponent during the game. Also the coach could change the balance factor between the importance of the main kick action decision module comparative with the best route decision module, when a decision regarding to the next cycle action should be taken. In this way the best route approach could be used in certain situations when apparently the decision selected by the main kick action module seems to be better.

For example in (Fig. 3) the central offender number 9 has to take a decision for the next cycle. Despite the fact that the main kick action decision module, decides that the best action for the next cycle will be a dribble, the best route decision module returns a better score for the route 9 – 10 – 11 – 6, which should be started from the current cluster number 19. Finally our offender number 9, decides to pass the ball to the central midfielder number 10 instead of making a dribble. As we played many games against different opponents, we clearly observed a good improvement by using the best route decision module in our kick decision, as the number of the goals scored and also the situations of scoring have increased, as well as the possession of the ball.

Future work

For the next future, we'll involve our coach in many others issues, where the team really needs his help. Even if the free form messages, are limited by count and periods of sending, the power of the coach remains very important, as he can receives free noises information. In this way, he can analyzes many important aspects of the games and if he'll deliberate based on these information, he can gives valuable advices to his own team. We must accept that right now, many teams involved in this competition, adapt theirs strategy before the start of the game instead of while it's running. A team will be more powerful, if it can adapt correctly its strategy depending of the opponent's behavior and not by the opponent's name, and also if it can do this during the game and not only before its start. In this way, we tried to adapt our team to some unexpected situations, which are generated by differently playing style of our opponents. In the real soccer, the role of the coach during the game is very important, and his importance is motivated not only because of the players that he's changing, but because of many good advices that he gives to his team during the game. In the same way, we must really think to the power of the coach and how can we involve him, more and more, in our simulator.

References

1. Peter Stone Layered Learning in Multi-agent System [D]. Pittsburgh: school of computer science, Carnegie Mellon University, 1998.
2. Jinyi Yao, Jiang Chen, Zengqi Sun. An application in RoboCup combining Q-learning with Adversarial Planning. Available in http://robocup.lits.tsinghua.edu.cn/download/document/tsinghuaeolus_kick.zip.
3. M. Riedmiller and Artur Merke, " Using machine learning techniques in complex multiagent domains," In I. Stamatescu, W. Menzel, M. Richter and U. Ratsch, editors, Perspectiveson Adaptivity and Learning, LNCS, Springer, 2002.
4. Itsuki Noda et al, Soccer Server Manual, RoboCupFederation. http://www.robocup.org.
5. Y. Jinyi, C. Jiang, and S. Zengqi. An application in RoboCup combining Q-learning with Adversarial Planning. In the 4th World Congress on Intelligent Control and Automation (WCICA'02), Shanghai, China, June 2002.
6. Peter Stone and David McAllester. An Architecture for Action Selection in Robotic Soccer. In Proceedings of the Fifth International Conference on Autonomous Agents, pages Pages: 316–323, 2001.
7. Takashi Suzuki. Team YowAI Description. RoboCup 99: Robot Soccer World Cup III, 2000.
8. Pressing, by Lucchesi Massimo, editing Bryan R. Beaver, printed by DATA REPRODUC-TION Auburn, Michigan.