BahiaRT 2017: Team Description Paper for RoboCup 3D Soccer Simulation League

Marco A C Simões, Claudia Elizabete Reis, Emmanuel Argollo, Mateus Nascimento, Sérgio Souza Jr., Wellington Correia, Josemar Rodrigues de Souza

State of Bahia University (UNEB/ACSO), Salvador, BA, Brazil

http://www.acso.uneb.br/bahiart/

Abstract For this year our main focus is the high-level cooperative behavior. We are running an important research project aiming the development and learning of an advanced setplays system. We also have enhanced some of our basic skills to achieve an overall performance gain to the team. This paper summarizes these developments and works in progress.

Introduction

Team BahiaRT has experienced a remarkable evolution in RoboCup competitions during the last three years. In the world championship, BahiaRT ranked fifth in 2014 and fourth in 2015 and 2016. In the last two years, we were also Champions(in 2015) and runner-up(2016) of the Portuguese RoboCup Open competition. We won the last four editions of Brazilian Robotics Competition and the last three editions of Latin American Robotics Competition.

Aiming to maintain this evolution, we have decided to enhance some of our high-level AI strategies. So we have worked with path planning, self-localization and setplays. We also have performed optimization on some of our basic movements like walking and kicking.

This TDP describes this work and also our work in progress. Since 2013, team BahiaRT is based on a source base code named FCPBase 2013, as part of an international scientific cooperation partnership with team FC Portugal from Universities of Aveiro, Minho and Porto in Portugal.

Path Planning

During the RoboCup 2016, we noticed that our team's path planning was making agents move in a trajectory that wasn't expected to be executed by the agent. That meant that our player was trying to move to a defined point in a way that the calculated path was longer than the expected path.

The high-level planning defines the final target for the agent to reach, in our previous code, this target would always be either the opponent goal or the ball, for the active agent, and it would try to reach in a straight line, using the methods defined by the mid-level to attempt to dodge possible obstacles and position itself in front of the ball. In search for an improved solution for the high-level, the agent will now analyze the best temporary target to reach. The whole field will be analyzed with the potential fields technique, with which all agents will receive proper charges, to guide the agent through the field. With the resulting path, we will determine the best target for the robot to move the ball, avoiding areas with too many opponents nearby.

With the mid-level, we will define targets to react to changes in the field, slightly modifying our defined path whenever needed. The method previously used was not efficient in defining this target, resulting in longer deviations from the path and in many cases, loss of ball possession. The new method, as with the high-level, will the potential fields though in a smaller scale. The repulsion charges will be smaller so the agent will end up making small adjustments to avoid a certain obstacle, rather than avoid groups of opponents, as is with the high level.

The low level must define the movement the agent must make to reach the targets in the shortest time. Using our Omnidirectional walking engine, we must provide the relative speeds in the X and Y axis, along with a theta value to determine its angular speed. As the speed performed by the movement in different axis is not the same, we decided to have the agent rotate around its own axis once the angle between its orientation and the vector to the target was above a certain threshold, in other cases, the robot will attempt to move towards the target in a straight line or with a small curve, if needed.

Movement optimization

In order to have a highly competitive team in the 3d simulation league a team must have two main things, efficient movements and good strategy. Our team has been able to perform well by using above average movements and good strategies. However, in the latest competitions it was seen that many teams have improved their strategies and have better movements than our own. So, in order to remain competitive and to improve the team's performance overall, we decided it was time to work better on the agents' movements, optimizing them.

Once we decided this route, we had to choose which movements to optimize first. As the building blocks of a soccer team, we decided to optimize the walking and the kicking actions first, making use of the fact that they are dynamic movements, easily parametrized for the optimization process.

Walking

As we performed various tests on our walking behaviours, we were able to confirm that we had a movement about 20% slower than most teams in the top of the RoboCup 3d simulation league. Being one of the fundamental behaviors in the game, responsible for both dribbling the ball and marking opponents, such disadvantage would definitely make a difference in our performance, making it a priority in our efforts to improve.

The walk behavior used by our team is based in a ZMP Controller using the cart-table model. As the model was already implemented, all we had to do is choose the parameters to be optimized. For the current walking model, we chose the 13 parameters described in the table 1. For the fitness value of our optimization, we decided to use the time it took the agent to carry the ball from point A to point B as our first minimization goal.

Table 1. Table of chosen variables for walking optimization

With the parameters and model to be optimized defined, we prepared the environment for optimization. We opted to use CMA-ES(Covariance Matrix Adaptation Evolution Strategy) as the optimization algorithm for this problem, as its favorable results had been reported by other teams with similar goals [1]. As for the optimization process, we modified the Magma Challenge Tool [2] to execute the main optimization loop, starting the server and agent, collecting the fitness and feeding it back to the CMA-ES process running in the background.

Kicking

In recent competitions, we noticed that our team's kicking wasn't giving enough strategic advantage over other teams because of its low range and long delay in preparing to kick. We decided to improve this skill to enable our pass and kick strategies to become more effective.

If we split the kicking action for humanoid agents (or bipeds in general), it is possible to perceive three main steps: (1) to hold the agent in one of his legs without losing balance, (2) put the leg behind the ball to increase the force applied to the ball (3) performing a controlled collision between one of the agent's legs and the ball.

Based on our static kick, which can reach 15 meters, we have decided to compare the kicking behaviors (static and dynamic) to understand their differences. We have observed the body parts movement during the kick and we have noticed that the center of mass position and the kick preparation phase has a major influence in the range and the preparation time to kick, giving the agent the force necessary to perform a good kick.

Using the knowledge that we got from this research and UT AustinVilla's kick optimization paper [3], we were able to develop a new dynamic kick that uses less preparation time, being faster, while also becoming able to reach the range of the static one. This optimization also used the CMA-ES and the modified Magma Challenge Tool to execute the main optimization loop, using the distance of the kick as the fitness value.

Localization and information fusion

The BahiaRT's agent information is derived from two sources: its vision system and communication with allies. The information coming from the vision is delivered every 60ms, while the communication information is 40ms, because of frequency of actualization the information derived from the vision is considered more reliable [4]. In this operating model, it is necessary to update all information to let the agent decide the best strategy to apply according to the current conditions of the game.

The agent localize itself through a set of eight landmarks distributed over the field. First the agent chooses a landmark to guide itself, each mark has a quality measure which determines how reliable is the calculated information based on this landmark. In case of the landmark's information isn't reliable, the agent uses an information originated from communication. All the mobile object information in the field is updated based on this principle, for example, the ball's position is determined based on agent's position, thus, if the agent see the ball its confidence will be the same as the agent's.

Work in Progress

Our work in progress comprises three main research initiatives that we are actively developing.

SetPlays

Setplays are predefined plans used by many teams in various sports, it defines all the sequence of actions that should be executed synchronously by a set of participating agents who cooperate to achieve a common goal [5]. Today our setplays are hard-coded, they aren't adaptable to different team's strategy, thus are used just in set pieces like corners or kick ins.

Our motivation with this work is lead with dynamic situations during the game, for example, if during a pass the player responsible to intercept the ball doesn't have time to get it, he can assumes a role of marker and another teammate closer to the point of receiving assumes the role of receiving the ball increasing the chance of that pass being successfully executed. For this purpose we intend to develop an approach that includes predefined plans and learning techniques to select the action most appropriate in the plan [6].

Humanoid Running

With the introduction of the 4th heterogeneous robot type in 2014 as a modification of the previous Nao-based model with the addition of an extra articulated joint in each foot, representing an artificial toe, it was thought that, considering the simulated environment with its looser restraints, the 3d simulation league could work towards the first running robots [7]. A definition of the running motion, in comparison to the fast walking currently used, is the presence of a flight phase in the movement gait, during which, neither of the agent's feet should touch the ground. We hope that by making use of this type of motion, we can gain a speed advantage in the competition.

Information fusion

To accomplish the objective of maintain a World State updated and reliable, it's necessary to create an efficient information fusion system that allows the merge between the information from the vision and the communication sent from ally agents. Even the vision information being trustworthy, it's important not to trust in only one source, because the information can lose its reliability. Currently our team only checks the information that is more reliable in the cycle and utilizes its data.

One of the techniques for fusion information is Kalman Filter, it's essentially a set of mathematical equations that implement a predictor-corrector type estimator that is optimal in the sense that it minimizes the estimated error covariance—when some presumed conditions are met [8]. For the ball position it can be used to detected hard deviations on the ball path. First based on the ball movement the noise can be statistically estimated by taking measurements of a static position at known distances [9]. Thus using the filter a priori estimation, a system to detect great differences between the expected and read positions can be implemented, allowing to detect the deviations on the path.

Conclusions

Considering the new improvements in team BahiaRT code for RoboCup 2017 and also the working in progress, we expect to perform well during the world championship. We will try to overcome our best historic placement and try to go up to the final match.

During this year we also plan to publish scientific papers describing in a more detailed way our developments and results.

Acknowledgements

This project is partly funded by CNPq/PIBIC, FAPESB/IC, UNEB/PICIN. We thank team FCPortugal for their cooperation and partnership.

References

1. Daniel Urieli, Patrick MacAlpine, Shivaram Kalyanakrishnan, Yinon Bentor, and Peter Stone. On optimizing interdependent skills: A case study in simulated 3d humanoid robot soccer. In The 10th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 769–776. International Foundation for Autonomous Agents and Multiagent Systems, 2011.
2. Klaus Dorer, Jens Fischer, Stefan Glaser, Duy Nguyen, Michael Obrecht, and David Weiler. The magmaoffenburg 2016 robocup 3d simulation team. In Proceedings of RoboCup 2016, 2016.
3. Mike Depinet, Patrick MacAlpine, and Peter Stone. Keyframe sampling, optimization, and behavior integration: Towards long-distance kicking in the robocup 3d simulation league. In Robot Soccer World Cup, pages 571–582. Springer, 2014.
4. Luís Paulo Reis and Nuno Lau. FC Portugal Team Description: RoboCup 2000 Simulation League Champion. In Proceedings of RoboCup 2000, 2000.
5. Luis Paulo Reis João Alberto Fabro, Vinícius Brenner and Nuno Lau. FCP GPR 2015 Team Description Paper: Using Setplays and Self Adjusted positioning in simulated soccer teams. In Proceedings of RoboCup 2015, 2015.
6. JA Fabro, LP Reis, and N Lau. Using Reinforcement Learning Techniques to Select the Best Action in Setplays with Multiple Possibilities in Robocup Soccer Simulation Teams. In Branco K.c.francelin Romero R.a.becker M.grassi V.osorio F.s.wolf D.f., editor, Proceedings - 2Nd Sbr Brazilian Robotics Symposium, 11Th Lars Latin American Robotics Symposium and 6Th Robocontrol Workshop on Applied Robotics and Automation, Sbr Lars Robocontrol 2014 - Part of the Joint Conference on Robotics and Intelligent Systems, Jcris 2014, pages 85–90, 2015. Citations: crossref, scopus.
7. Hidehisa Akiyama, Klaus Dorer, and Nuno Lau. On the progress of soccer simulation leagues. In Robot Soccer World Cup, pages 599–610. Springer, 2014.
8. Greg Welch and Gary Bishop. An introduction to the kalman filter. Technical report, Chapel Hill, NC, USA, 1995.
9. João Silva, Nuno Lau, João Rodrigues, José Luís Azevedo, and Antônio J. R. Neves. Sensor and Information Fusion Applied to a Robotic Soccer Team, pages 366–377. Springer Berlin Heidelberg, Berlin, Heidelberg, 2010.