OXSY 2022 Team Description
Sebastian Marian, Dorin Luca, Bogdan Sarac, Ovidiu Cotarlea
OXygen-SYstems laboratory, Str. Constantin Noica, Bl.5, Sc.C, Ap.36, C.P. 550169, Sibiu, ROMANIA
Abstract Oxsy team was founded in July 2002 for a graduation project of one student, Sebastian Marian, in the field of Multi-Agent Systems [1], at the Department of Computer Science of Lucian Blaga University (Sibiu - Romania). After graduation he continued the work on this project and so was born Oxsy team. As we started from scratch, our ideas, concepts and beliefs, was implemented year by year and today, we are happy to see that we are on the right way, as our team was growing in these years, more than we expected from the beginning. If we will qualify to the competition, this year we'll reach at the 18th consecutive participation, in RoboCup [4] Soccer Simulation League.
Introduction
In July 2003 at RoboCup [4] competition, which was held in Padua - Italy, we won the first round and for us it was a good surprise for first year of participation. Then, in the next year, we participated in Lisbon - Portugal for the second time, and again we obtained a good result (the 11th place). In 2005 in Osaka – Japan, we participated for the third time and finally we entered in the first 8 teams of soccer simulation league, as we won (the 8th place). In 2006 the competition was held in Bremen – Germany and we won (the 7th place). In 2007 we went to Atlanta – Georgia (U.S.A), where we obtained (the 5th place), the same result which we achieved in 2008 in Suzhou – China. Finally, in 2009 in Graz, we entered in the first 3 teams in the soccer simulation league, as we won (the 3rd place), the same result which we achieved in 2010 in Singapore. In 2011 we came back from Istanbul - Turkey with (4th place). In 2012 we were in Mexico City, where we had a bad experience as we made some major changes in our defensive system, and also many others overall our team strategy, changes which was not very well balanced at that time, with all others characteristics of our team, as we were not qualified for finals, from the second round groups. In 2013 we came back in top, as we won (the 6th place), in Eindhoven – Netherlands. In 2014 the competition was held in Joao Pessoa – Brazil, and we entered on the stage for the third time in our participation history, as we won again (the 3rd place). In 2015 we won the 4th place as we played the semifinals in Hefei – China. In 2016 the competition was held in Leipzig – Germany, we missed the semifinals and we came back with (5th place). In 2017 we came from Nagoya with (3rd place) for the fourth time in our participation history. In 2018 the competition was held in Montreal – Canada and we won the 4th place as we played in the semifinals. In 2019, unfortunately, our team missed the competition which was held in Sydney – Australia. In 2020 the competition has been postponed for 2021 and last year it was held online due to pandemic situation. Our team won the 6th place. This year the competition will be held in Bangkok - Thailand. As we already have a very good experience in 2D Soccer Simulation league, we hope that our new ideas and improvements will be reflected in the competition where we will also test other tactical elements developed.
This year we further developed and extended our architecture, based on a very powerful concept that we have used in our early times in 2D Soccer Simulation Competition, by using the concept of "Holon". Multi-agent (intelligent) systems (MAS) are software systems composed of several autonomous software agents running in a distributed environment. Besides the individual goal of each agent, global objectives are established committing all or some agent groups to their completion. This concept of "Holon" was proposed by the Hungarian philosopher A Koestler (1967,1978) and explains the importance of the hierarchy of a system. Each organ is an element of the organic system while the organ is itself a system composed of multiple tissues. This relationship appears at every level of the system. This means that a system element is located at a hierarchy node and has both characteristics as a whole and as a part. Koestler named the node of hierarchy "Holon". (Hino R. 1999)[6] based on the combination of the Greek word "holos" that means "whole", and the suffix "on" meaning particle or part. Accordingly a "holarchy" is a hierarchy of self- regulating control building blocks holon, which function:
- (a) As autonomous wholes in supra-ordination to their parts;
- (b) As dependent parts in subordination to controls on higher levels;
- (c) In coordination with their local environment;
Why holarchies are suitable for Soccer Simulation?
By definition, the holon is an excellent concept for modeling a soccer team because of its main characteristic: it can act as a part (modeling a player) or as a whole (modeling the team). One of the most important characteristics of the holarchies is the capacity to modify themselves, i.e. to create temporary hierarchies (Giebels, M. et. al. 1999) [7]. Like modern industry, the soccer is very dynamic, i.e. not only that each team comes with its own style and game strategy, but also each game phase has a dose of novelty. In this way, it isn't possible to create only one command scheme that will work correctly for each adverse team and each game phase. The holarchies aren't rigid, they change themselves according to necessities (strategic and tactic) and the structure of the decisional schema will dynamically change (by modifying momentary priorities). Another advantage of the use of holons is providing a balance between the two possible approaches of the leading process: the hierarchical control (fixed, static, pre-established) and the heterarchial (autonomous, decentralized, flexible but not very efficient), it can be seen in (Fig. 1).
Architecture
Our architecture has been extended based on the PROSA model, developed at PMA/KULeuven as a reference model for Holonic Manufacturing Systems (Van Brussel at al. 1998, Wyns 1999)[5]. The acronym PROSA came from Product-Resource-Order-Staff Architecture, the holons type used. The resource holon contains a physical part namely a prodution resource of the manufacturing system, and an information processing part that controls the resource. The product holon holds the process and product knowledge to assure the correct making of the product with sufficient quality. The order holon represents a task in the manufacturing system. It is responsible for performing the assigned work correctly and on time. The staff holon is implemented in the idea to assist the rest of three holons in performing their work. The concept and it's assignment can be seen in (Fig. 3).
Fig. 3 The PROSA model architecture
| Hr | Holon Resource |
| Ho | Holon Order |
| Hp | Holon Product |
| Hs | Holon Staff |
At the (level 0) we have the "RoboCup environment" containing the soccer server rules (Itsuki Noda et al. 1998)[3].
At the (level 1) we have holons who implements opponent strategy analyzing, opponent modeling without noise, make some player changes (heterogeneous). Communication with others holons from others players can be made through (Say) command.
At the (level 2) we have holons who implements some game schemes, strategy based on opponent modeling and also support for asking spares players (heterogeneous). Communication with others holons from others players can be made through (Say) command.
At the (level 3) we have holons who implements the skill (i.e. pass, dribble, shoot), the goal oriented behavior and the strategy.
At the (level 4) we have holons who implements effectors (like body components) that can do actions like (kick, dash, turn, etc.).
All these levels should be understood like a holarchy not as a hierarchcal organization. Please see the (Fig. 4).
Implementation
This year we extended our architecture based on the PROSA model where each holon has a well-defined functionality. We have added the functionality for the product and order holons at the team level. The holons that we are using have similar functionality with the original (Table 1).
Table 1
| PROSA | Coach Level | Team Level | Player Level | Player Components |
|---|---|---|---|---|
| Product Holon | - change players state - logs |
- strategy depends of the game state - players positioning and regioning +others advice(pressing, support, etc) - logs - spares players |
- strategy(demarcate, solve some situations) - skills(ball intercepting, ball handling, pass towards team mates, shoot towards goal, dribbling, etc.) - learned skills |
- kick ball - dash forward or backward - turn neck or body - view width narrow, normal or wide - speak (say) - listen (hear) - legs - neck - head - (eyes, ears, mouth) - execute base commands (Turn, Kick, Dash, etc.) |
| Resource Holon | - global and noisily free view - analyze opponent strategy |
- game schemes (learn and test schemes) - players - advice for wining game(load strategy files automatically depends of the game state) |
- wins: (ball, duel, energy) - takes roles - co-operation plan |
"unimplemented yet" |
| Order Holon | - watch some log games | - spares players | - co-operation plan distributed through all players - strategically positioning for players in different game phases using neural networks |
"unimplemented yet" |
| Staff Holon | - watch some previous opponent log games | - spares players | - co-operation plan distributed through all players - strategically positioning for players in different game phases using neural networks |
"unimplemented yet" |
Future work
In this paper we describe our new extended architecture that tries to combine in a synergistic manner the approaches applied in two, up to now insufficiently related domains: MAS and HMS. The preliminary results suggest that we are in a very promising starting point for future work. This validates the approach of adding functionality (first of all, flexibility) stepwise, i.e. improving the old architecture. The results show that at this stage the most useful is the product holon. For the game strategy the order holon is still insufficiently adapted. The staff holon will be useful when the evolution of the game rules will give an even more substantial role to the coach. Another problem for the future is improving negotiation between holons.
References
- Peter Stone Layered Learning in Multi-agent System [D]. Pittsburgh: school of computer science, Carnegie Mellon University, 1998.
- M. Riedmiller and Artur Merke, " Using machine learning techniques in complex multiagent domains," In I. Stamatescu, W. Menzel, M. Richter and U. Ratsch, editors, Perspectiveson Adaptivity and Learning, LNCS, Springer, 2002.
- Itsuki Noda et al, Soccer Server Manual, RoboCupFederation. http://www.robocup.org.
- Y. Jinyi, C. Jiang, and S. Zengqi. An application in RoboCup combining Q-learning with Adversarial Planning. In the 4th World Congress on Intelligent Control and Automation (WCICA'02), Shanghai, China, June 2002.
- Wyms, J. and Van Brussel, H. and Bogaerts, L. (1999) Design Pattern for Integrating centralised scheduling in distributed holonic manufacturing control systems. Proceedings of the second International Workshop on Intelligent Manufacturing Systems, Leuven, Belgium, pp.75-82
- Hino, R. and Moriwaki, T.(1999) Decentralized Scheduling in Holonic Manufacturing System. Proceedings of the second International Workshop on Intelligent Manufacturing Systems, Leuven, Belgium, pp.41-47
- Giebels, M. and Kals, H. and Zijm, H. (1999) Building Holarchies for Concurrent Manufacturing Planning and Control. Proceedings of the second International Workshop on Intelligent Manufacturing Systems, Leuven, Belgium, pp.49-56