BahiaRT 2020: Team Description Paper for RoboCup 3D Soccer Simulation League

Marco A C Simões, Gabriel Sousa, Jadson Nobre, Ana Patricia Magalhães, Jorge Campos, Josemar Rodrigues de Souza, Robson Marinho da Silva

Bahia State University (UNEB/ACSO), Salvador, BA, Brazil

http://www.acso.uneb.br/bahiart/

Abstract This paper describes the significant enhancements in team BahiaRT since RoboCup 2019. We describe our work about setplays development, including some enhancements on existing tools for setplays design and learning setplays from demonstration. The paper presents a list of minor improvements with a high impact on the team's overall performance. These modifications are related to path-planning, selfcollisions reducing, shots to goal, and defensive organization. We summarize the projects we still working on at the end of the paper. They are related to making decisions under imprecise conditions, kick optimization, and learning setplays.

1 Introduction

Bahia Robotics Team (BahiaRT) has experienced a continuous evolution in RoboCup competitions in the last years. In the world championship, BahiaRT ranked fifth in 2014, sixth in 2017 and fourth in 2015, 2016, 2018, and 2019. In the last five years, we were also Champions(in 2015) and runner-up(2016) of the Portuguese RoboCup Open competition. We won the previous seven editions of Brazilian Robotics Competition and the last six editions of Latin American Robotics Competition.

Since RoboCup 2019, we have worked in several minor enhancements in our path planning strategies. In particular, we have optimized the obstacle avoidance when players are performing backward walking behavior. We also have developed a better decision-making solution for a quick and fast shot to goal when the attacker is near the opponent's goal. We have adapted our existing defensive system[4] to the foul model used in the RoboCup 3D Simulator. Section 3 describes all these minor enhancements.

We have also continued our work with setplays. We created a dataset schema to represent setplays' demonstrations generated by users watching logs of 3D simulation matches[9]. This work is a refinement of our previous proposal to learn new setplays from demonstration[8]. To accomplish the goals described in these two works, we need to adapt RoboViz, Strategy Planner (SPlanner)[2], and FCPortugal Setplays Framework (FSF)[6]. These contributions are described in the Section 2.

Since 2013, BahiaRT uses a source base code, named FCPBase 2013, as part of an international scientific cooperation partnership with team FC Portugal from Universities of Aveiro and Porto in Portugal.

2 Setplays development

Due to the good results obtained in the application of static setplays, we have initiated a process of development and application of automatically generated setplays, allowing the design stage to become faster.

We decided to use the toolkit provided by team FCPortugal: the SPlanner[2] and the FSF[6]. We have first used SPlanner and FSF when we have designed our kickoff setplay[7]. We have developed both a hand-coded version and a SPlannergenerated version of the kickoff setplay. The preliminary results showed great potential for improvement. In this early stage, we have not modified SPlanner or FSF to adapt to all skills present in our team. We missed some important behaviors, such as marking and different kinds of passes. In the last year, we have been working on adaptations and updates in SPlanner and FSF to allow better use of these tools in 3D Soccer Simulation Competition. We will release a new version of SPlanner and FSF as soon as we finish all tests.

2.1 Enhancements in SPlanner and FSF

We have modified SPlanner to support defensive setplays. Although this option was present on the screen of the previous version, it was disabled. We have enabled this option and made changes to support defensive setplays:

• added a set of 11 opponent players which can be used as a reference when designing both defensive and offensive setplays;
• enabled different choices for ballHolder and setplayLeader players. In the previous version, The same player always fills both roles. We have changed to enable the assignment of different players to these roles. In defensive setplays the ballHolder can be an opponent while the setplayLeader is a teammate.

Opponents can not perform the same behaviors as teammates. The unique behavior available for them is move to enable the setplay designer to estimate the movements of the opponents from the current step to the following one. If the opponent is also the ball owner, the designer can also estimate a kick to change the ball possession and a shoot to the goal. Figure 1 shows an example of defensive setplay where the opponents are used as a reference for players to execute defensive behaviors such as marking and intercept the ball.

We have added two new options of behaviors to the teammates:

– Run - straight - a behavior to represent fast walking or running movements when players are not aware of collision avoidance;

– passToBestPlayer - a behavior where the ball holder can choose the best teammate to receive the pass.

Many teams have developed skills for running or walking faster without worrying about avoiding obstacles to enable their players to move faster to their target. When we offer the option Run - straight to the setplay designer, he can choose when this kind of movement is necessary to accomplish designed setplay.

The passtoBestPlayer is a behavior that transfers to the teams' agents the task of choosing the player who will receive a pass. When using this behavior, the setplay can be more flexible and does not need to define the exact player to receive the ball when performing a pass. The user needs to create one transition for each option of pass receiver he wants to take into account.

The changes in SPlanner required modifications in the setplay library [6] to extend the S-expression language defined in FSF to enable transfer of setplays from SPlanner to the FSF. The modified language enables teams who extend the FSF classes to take advantage of all enhancements described here.

3 Skills and Strategies enhancements

During the past year, we have made many fixes and small changes to the BahiaRT code. The original plan was to improve the agents' path-planning algorithm by studying and implementing new state of the art approaches found during research through highly regarded scientific repositories such as IEEE Xplore and SpringerLink. Though we did find some good examples and ideas to move from, such as a method for path planning using Zeros and Poles attraction-repulsion principle[5], a footstep planning approach with moving obstacles[1], and an algorithm that uses Markov decision processes combined with fuzzy logic[3], we had many small issues to adjust before moving on.

3.1 Reactivating the defense strategy

We have adjusted our team's defense strategy[4], where we have two leading roles: an offensive marker and a defensive marker. The defensive marker is split between the active marker and the passive marker. The offensive marker is used during offensive situations, and this role is responsible for keeping the closer enemy agents from taking the ball. The active marker is a defensive player that is in a better position to mark the opponent with ball possession than our active teammate. Furthermore, the passive marker is responsible for marking the opponents that might receive a pass or be useful in any case for the enemy team.

We have deactivated this system earlier since both offensive marker and passive marker were causing the players in the respective roles to be beamed out of the field for committing faults by illegally touching or pushing the opponent agents. Although, this new configuration for the defensive system led our team to concede more goals than in earlier years.

After re-evaluating the code and taking a look at the existing fault model in the 3D server, we came up with two solutions, one for each of the marker roles. For the offensive, we have calculated the distance between the ball and the opponents that our players were supposed to mark. The position for the marker is dynamically placed in between the ball and the opponent. We have set the opponent as an obstacle, so the closer the opponent gets to the ball, the closer the marking player would get without bumping into him. For the passive marker, we decided to place him directly in front of the opponent but keeping a certain distance to prevent the faults, that front being a position between the enemy agent and our goal. This way, the marker could intercept any attempt of a pass made by the adversary team besides being in position for any strategy in the middle of the game. No changes were necessary for the active marker.

These solutions enable us to reactivate our defensive system reducing the number of fouls charged to our markers. In the RoboCup Brazil Open 2019, our team conceded only one goal in eleven matches. This result is a clear indication that our defensive system is now effective again.

3.2 Backward Walking

Another example of adjustment was in the backward walking skill. The idea was that the agents could walk backward when their strategic position was set behind them, without having to take the time to turn around, allowing for a faster reposition in the field. The problem was that on many occasions, the agent would either miss the ball position or bump into another player while walking.

That was partially solved by forcing the agent never to use that skill if the ball is not on its sight, and making it chose a random direction, either left or right, to turn to if any obstacle crossed its path.

3.3 Kicking conditions

The direct kick was implemented during the Latin American Robotics Competition - LARC in 2018, and it was our team's test of a kick without any previous positioning. Since the player would simply try to kick the ball in front of him without having any plan of where it is supposed to end, it was quite a risky skill. We have decided not to use the direct kick until, in 2019, we had the idea to only try it when very close to the opponents' goal and at risk of losing the ball to the opponent team.

Though many goals were scored using that move, it still needs some tuning to be used in other less risky occasions.

3.4 Self Collisions

The Self Collisions fault was introduced during the Portugal Open and finally fully implemented as a new rule later on the RoboCup 2019. That rule defined as a fault any collision detected between an agent body parts. Legs and arms, for instance. Many teams, including our own, had to look for ways to solve that situation and even though that is still a work in progress, we managed to identify the most common cause for our team's self-collision to be one of the robot types ways of getting up, since the agents were putting their arms through their thighs during that action. We were able to significantly reduce the number of self collisions in our team at the moment by merely changing the robot type.

3.5 General Collisions

Following the subject of collisions, another significant improvement was in the path planning of non-active players (the ones that did not have the "active" role). We noticed that many agents were colliding with each other, including the ones with the "active" role, which could cause the team to frequently lose the ball or miss a chance of scoring a goal, among other situations.

After some tests, we have noticed that the radius sizes of the obstacles set on all agents were miscalculated, so we fixed that by simply defining static numbers for when the agents are fallen or standing. We also increased the radius of the obstacle set in the ball and added four new obstacles on the field, one for each goal post, since some players would ignore and frequently get trapped around them.

These minor improvements have enhanced the BahiaRT overall performance and prepared it for more significant improvements.

4 Work in Progress

At this time, we are working on three research projects that can benefit the BahiaRT global performance. We are considering kick optimization, decisionmaking under imprecise conditions, and learning setplays from demonstration as our main goals. In the following subsections, we shortly describe these initiatives.

4.1 Kick optimization

The optimization of the BahiaRT team kicking skills is one of the projects in development for 2020. There are currently two kick types, one static for long distances that takes a few seconds to prepare and one dynamic for medium distances, which is a little bit faster. The idea is to optimize the kicking parameters for both kicks in order to make them faster and more reliable to use. Since we are working in a stochastic environment, we plan on using machine learning techniques and algorithms for further improvement and expect to have both kicks better optimized by the RoboCup 2020 event.

4.2 Machine Learning application for decision-making under imprecise conditions

One of the biggest dead-locks on the BahiaRT team occurs during the assignment of the roles to be performed by the agents in the field, as in the choice of the most appropriate player for a certain action during the game, such as: the most apt player to drive the ball, the most likely player to intercept the ball, the player in the possession the ball, amongst others of this kind. These are the characteristics of the agent's indecisiveness problems for the choice of the ideal active player to perform a specific behavior at a particular moment of the game. These problems generate some situations in which there is a vacancy (none of the players becomes active) or ambiguity (more than one active player) in the role of active.

We are doing some research looking for the machine learning algorithm that best fits our demands to come to the ideal solution, completely free of ambiguity and vacancy, or the closest that we can get to make the selection of the player more optimized in the situations described.

Information is also being collected on the events where there is no vacancy or ambiguity during the game so that these data can become base parameters fed to the machine learning algorithm that will be implemented.

We are working so that, in RoboCup 2020, we can obtain an effective mechanism for the selection of the player to play the role of active without ambiguity or vacancy or the closest to the ideal, with this we also want to obtain a mechanism to determine whether the player is the more suitable to possess the ball or not. Through these improvements, in the subject of indecision of the player, we will seek a better overall performance of the BahiaRT team in the next games.

4.3 Learning Setplays from Demonstration

We are working on a reinforcement learning engine to use the dataset schema created to support learning setplays from demonstration[9]. We will use the modified tools described in Section 2 to populate the dataset to apply the clustering solution presented to keep similar setplays together in a group. As we use a fuzzy clustering model, a single setplay (dataset instance) can be a member of more than one group at the same time. Figure 2 shows the organization of our dataset schema.

First Level (identifying the set plays) Second Level (The stepswithineachset play) setplay #1 ourPlayersNumber theirPlayersNumber abortCondition Steps stepsList setplay #1 / step #0 ourPlayerInStep theirPlayersInStep waitTime ... ourPlayerInStep theirPlayersInStep waitTime ... ourPlayerInStep theirPlayersInStep waitTime ... ... behaviorsList setplay #1 / step #1 setplay #2 ourPlayersNumber theirPlayersNumber abortCondition Steps stepsList ... ourPlayersNumber theirPlayersNumber abortCondition Steps stepsList behaviorsList behaviorsList setplay #2 / step #0 ourPlayerInStep theirPlayersInStep waitTime ... ourPlayerInStep theirPlayersInStep waitTime ... ourPlayerInStep theirPlayersInStep waitTime ... ... setplay #2 / step #1 behaviorsList behaviorsList behaviorsList ... / ... ourPlayerInStep theirPlayersInStep waitTime ... behaviorsList

In the first level, only general features of the setplays are considered. In the second level, internal features in the steps of each setplay are used for clustering the instances in the dataset.

We will train our reinforcement learning engine to select an adequate group of setplays for each game situation. Then, one of the setplays in the selected group will be chosen using the case-based reasoning approach present in the FSF[2]. We expect to use this solution in RoboCup 2020 to check if the use of the setplays demonstrated by humans can enhance the overall performance of our team.

Acknowledgements

This project is partly funded by CNPq/PIBIC, FAPESB/IC, UNEB/PICIN. We thank team FCPortugal for their cooperation and partnership.

References

1. Ahn, J., Campbell, O., Kim, D., Sentis, L.: Fast kinodynamic bipedal locomotion planning with moving obstacles. CoRR abs/1807.03415 (2018), http://arxiv.org/ abs/1807.03415
2. Cravo, J., Almeida, F., Abreu, P.H., Reis, L.P.: Strategy planner: Graphical definition of soccer set-plays. Data & Knowledge Engineering 94, 110–131 (2014)
3. Fakoor, M., Kosari, A., Jafarzadeh, M.: Humanoid robot path planning with fuzzy markov decision processes. Journal of Applied Research and Technology 14(5), 300 – 310 (2016), http://www.sciencedirect.com/science/article/pii/ S1665642316300700
4. Laranjeira, C., Soares, A., Argollo, E., Frias, D., Simões, M.A.C., de Souza, J.R.: Uma abordagem multiagentes para sistema defensivo em um time de futebol de robôs bípedes. Revista de Sistemas e Computação - RSC 5(1), 38–49 (Jan–Jun 2015), http://www.revistas.unifacs.br/index.php/rsc/article/view/3644/2639
5. Martinez Santa, F., Rivera, S., Arbulu, M.: Global navigation approach for assistant robot. Tecnura 21, 105–117 (01 2017)
6. Mota, L., Fabro, J.A., Reis, L.P., Lau, N.: Collaborative behavior in soccer: The setplay free software framework. In: Robot Soccer World Cup. pp. 709–716. Springer (2014)
7. RAMOS, C.E.d.R.: Planejador Multiagentes para criação de jogadas ensaiadas em um time de futebol de robôs simulados. Bachelor thesis, Universidade do Estado do Bahia, Salvador, Bahia, Brazil (2017), in Portuguese.
8. Simões, M.A.C., Nogueira, T.: Towards Setplays Learning in a Multiagent Robotic Soccer Team. In: 2018 Latin American Robotic Symposium, 2018 Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education (WRE). pp. 277–282 (Nov 2018)
9. Simões, M.A.C., da Silva, R.M., Nogueira, T.: A Dataset Schema for Cooperative Learning from Demonstration in Multi-robot Systems. J Intell Robot Syst (Dec 2019), https://doi.org/10.1007/s10846-019-01123-w