View Original PDF

Borregos RoboCup 2D Simulation League 2012 Team Description Paper

Daniel Marquez, Johny Hayworth, Braulio Chavez, Eduardo Barrera, Alejandro Garza, Ivan Gonzalez, Leonardo Garrido

Department of Computer Science Tecnlogico de Monterrey, Campus Monterrey Monterrey, 64849, NL, Mexico

Abstract This paper shows the description of our team Borregos from the Tecnologico de Monterrey, Campus Monterrey. Borregos team has been participating in the RoboCup competitions since 2004, starting in the 2D Simulation League and moving forward to the 3D Simulation League with spheres in 2006, the 3D Simulation League with humanoids in 2007 and the RoboStadium Simulation League since 2008. In 2009 we started a new team "Borregos Humanoids" which participated in the Mexican RoboCup Open. In 2010 we also qualified in the Standard Platform League and 3D Simulation League in the RoboCup Singapore 2010 competitions. In 2010 and 2011 we won the 2nd place in the Robotstadium competition. In 2011 we rebuild the 2D Simulation League team and started a new code base for the team in the Java Platform.

1 Introduction

Soccer game is the central topic of research in RoboCup, a very interesting and exciting game that incite researchers o create innovative solutions that can be applied to socially significant problems. One of the soccer leagues is the RoboCup 2D Simulation League, in which all teams compete into a soccer simulator so they can focus only on the development of high-level techniques, representing a great area of opportunities for those who want to do research about intelligent agents who interacts with other agents in the pursuit of a common goal.

The goal of RoboCup is that by the year 2050 a team of fully autonomous humanoid robots can win against the human world soccer champion team. Fulfilling this goal is not an easy task. Several methodologies have to be proposed in areas such as robotics, electronics, computer science and artificial intelligence (IA). Specifically, AI is a branch of computer science which studies and design intelligent agents, where an intelligent agent is a system that perceives its environment and takes action in order to maximize its chances of success. The 2D Simulation League is an effort to create a realistic environment that provide to all teams of a platform where AI techniques and algorithms can be tested.

Borregos team is part of the Intelligent Autonomous Agents at Tecnlogico de Monterrey, Campus Monterrey, which objective is to develop innovative technology oriented towards distributed knowledge handling by researching about machine learning techniques and multiagent systems. In the RoboCup domain, the research interest is coordination and cooperation among autonomous agents.

For Borregos team, RoboCup 2D Simulation League represent an excellent opportunity area to do AI research, specially in multiagent systems and dynamic cooperation. The team is interested in research about intelligent agents with long life, autonomy, reaction capabilities, planning, learning and reasoning. The main research goal for Borregos team is obtain emergent team behaviors through multiagent planning, coordination and decision making research. The team is focused on reasearch about high-level abilities like team coordination, planning and learning.

This paper is distributed in sections. Section 2 describes the Borregos team constitution. Section 3 shows the Borregos team background.

2 Team Constitution

Currently, our team is mainly formed by students of Bachellor Degrees on engineering, a student in M. Sc. Degree in Intelligent System and a Ph. D. in Artificial Intelligence student at Tecnlogico de Monterrey, Campus Monterrey. Daniel Marquez is a student of Mechatronics Engineering in our campus and has experience in automation and programming. Johny Hayworth and Braulio Chvez are students in Computer Science Engineering and their expertise is in software architecture and programming in Java, C++ and C# languages. Eduardo Barrera is a stundent in Digital Systems and Robotics Engineering student and has experience with the C++ language. Alejandro Garza holds a B.S. in Industrial Physics and is a M. Sc. Student in Intelligent Systems with specialty in heuristics and algorithms. M. Sc. Ivn Gonzlez is a student of the Ph. D. program with special focus on Intelligent Systems and his expertise is in Java and C++ language programming and algorithms. Ph. D. Leonardo Garrido is the leader of our Borregos team and head of the Intelligent Autonomous Agents Research Group of the Computer Science Department at Tecnlogico de Monterrey, Campus Monterrey. He has several publications related to probabilistic learning, case based reasoning, multiagent systems, robotics and computer simulations.

3 Team Background

Our team Borregos was created back in 2004 and participated in the RoboCup Portugal competitions in that year in the Soccer Simulation 2D category with M. Sc. Emmanuel Martnez as the team leader, under the supervision of Ph. D. Ramon Brena [8]. Emmanuel, in his master's thesis, proposed a new technique using a search algorithm based in game theory for the best action in a certain time. His method consists of selecting the best action in step T of the simulation, based on the evaluation of the sequence of possible actions in step T + 1, creating a decision tree over which the search is performed.

In 2005, M. Sc. Carlos Bustamante started to develop a new team team for the simulation 3D league with spherical agents and was classified to participate in the RoboCup Germany international competitions in the city of Bremen, under the supervision of Ph. D. Leonardo Garrido. A fuzzy bayesian approach for decision making in RoboCup 3D was presented in the RoboCup Symposium [5]. Later, a comparison between fuzzy bayesian classifiers and gaussian bayes classifiers was published in [4]. Another student, Csar Flores, contributed to the develop of the physics model which were used in the ?go to? and ?dribble? behaviors of the agents. This approach was published in [1] and later in a book chapter [3].

Afterwards, a hybrid monte carlo localization with Kalman filter sensor fusion approach was used for diminishing the effects of noise and uncertainty in the agent self localization process, and was published in [2]. With this approach, Borregos 3D participated in the RoboCup Brazil Open 2006 competitions and won the third place.

In July 2007, we were TOP 16 in the first humanoid simulation competition in the RoboCup USA international competitions in the city of Atlanta, in which humanoid robot models (based on the Fujitsu HOAP-2 robot) were used for the first time. In November 2007, we were the organizers of the 3D simulation league in the 3rd RoboCup Latin American Open celebrated in Monterrey, Mexico.

In 2008, Borregos team participated in the RoboCup China international competitions in the city of Suzhou, being one of the TOP 8 teams of the simulation 3D league and joining the first RobotStadium online soccer simulation competition, a new simulation league using the Webots simulator by Cyberbotics/Gostai, and the URBI (Universal Real-time Behavior Interface) programming language for the scripting of controllers.

During the first half of 2009, Borregos team participated in the RobotStadium competition with a refactored controller in Java, remaining in the first place for much of the competition and finishing as fourth in the last round. During this competition a lot of work was done and the team become one of the strongest in the RobotStadium League. The main feauture of the team were: good basic behaviors (walk, turn, stand up, shoot), robot coordination through communication, landmark based auto-localization and a high quality vision module for detecting the ball and the goals. Later, Borregos team won the second place in both Robotstadium 2010 and Robotstadium 2011.

In 2009, the team participated in the 3D simulation league in RoboCup Austria competitions. The team finished at the middle of the ranking. Borregos team had good performance in terms of basic skills and its main characteristic for this competition was the adaptation of agents to the new restricted vision rule in the league.

Also in 2009, in order to participate in the 2nd Mexican RoboCup Open 2009, within the category of humanoid robots, we started to develop our Borregos Humanoids team using the RoboNova robot. The Borregos Humanoids team was developed with basic behaviors like walking, turning, shooting, standing up as well as searching and tracking the ball.

In 2010, we were qualified to participate in the RoboCup 2010 Singapore in the Standard Platform Category. And we also participated in the Robotstadium 2010 competition where we won the 2nd place. In 2011 we could not participate in Istanbul due to some financial problems. However we remotely participated in Robotstadium 2011 and we won the 2nd place again.

We published three papers about the tecniques implemented in our Borregos team in Robotstadium: about ball chasing coordination [6], CPG based locomotion for robot Nao [9] and multiagent reinforcement learning for motion policies [7].

In 2011, the 2D Simulation League team was rebuilded and new students were recruited to implement new techniques and to develop new methodologies in the AI area of multiagent systems.

In the following sections, we present some of the current features of our Borregos 2D team for RoboCup 2012 and some future work to be implemented in these months before the competition.

4 Software Architecture

The programming language chosen was Java because of the fast implementation of String manipulation methods. Our current bot is divided in three principal layer: Communication layer, Information layer and Intelligent layer. This architecture is showed in the figure 1. The figure 1 shows how the information is passed from one layer to other and what type of data is transmitted between two layers. The bottom layer (Communication layer) has as principal role the communication with rcssserver through UDP packets. It receives commands from the information layer and sends this commands to the server. On the other side, the Communication layer receives data from the server and passes this data to the Information layer as a String object.

The information layer is responsible for the conversion of data to information. It receives String objects from the Communication layer and extracts the useful information and converts it to Java objects. Then the Information layer sends this objects to the Intelligent layer. Some of the objects the Information layer sends are:

  • server and player parameters, also the body state of the player
  • absolutes x and y coordinates of the player
  • absolute facing angle of the player
  • players in the vision range (both colleagues and opponents)
  • messages sent by the coach, referee and players

The Information layer also receives data from the Intelligent layers through methods. The arguments of this methods indicate the parameters of each commands that is sent to the server. The Information layer then converts this arguments to a String object and sends it to the Communication layer.

Fig. 1. Software architecture
Fig. 1. Software architecture

5 The Information Layer

As mentioned earlier, the Information layer is responsible for the conversion of data to information. This task is accomplished through the parsing of the String objects sent by the Communication layer and map this information in a hash map Java object. This String manipulation is done by a mapping of String objects to a container java object (usually Vector or HashMap). So the Intelligent Layer can access to the information through the name of the parameter.

One of the crucial task of this layer is localization. In a first attempt, this localization is done by a triangulation with two flags of the field. The triangulation took the distance of the player to each flag with known coordinates and compute all the angles of the triangle. Then the method computes the vector components and with this information the approximate coordinates of the player can be known. From the coordinates and the direction of one flag the absolute facing direction can be computed. However, because of the noise introduced by the model the coordinates and direction of the player are not accurate.

So, we can implement a filter, so we can take many measurement of all the information of the field and filter all the extraneous data. Is intended to maintain a internal physical model of the player, so we can countercheck our measurement with the internal model and make better approximations to the coordinates and direction of the player.

6 The Intelligent Layer

The main feature of the Intelligent layer is the use of the Contributed Distributed Problem Solving, for the decision making among the defensive tasks. CDPS consists of distributing tasks among agents to achieved a global goal, when an independent agent can?t do it by itself. This is applied when the team is under attack and the main goal is to avoid a score. Task are distributed to the defensive players. The tasks consists of avoiding that the opponent make a good pass to theirs attackers, trying to get the ball from the opponent, and reducing the area for shooting to the goal. To achieve a good coordination of the team Contract Net Protocol is used. A defensive leader, announces the tasks that need to be performed, so that every attacker is covered by a defender and one try to get the ball. Also the goalie can establish a contract with a player so it can form kind of a ?wall? on a side of the goal, so that the opponent shooting angle is reduced, and the goalie has better chance of stopping the ball.

Internal heuristics are calculated for all the task that a player is qualified to make. The task are distributed in defensive, used for the CDPS, offensive with ball and offensive without ball. This heuristics has the same base for all players, but at the end a weight, depending on the role of the player, is considered. For example, a left midfielder will have a low weight in the move with ball to the left heuristic, since to the left, the field ends.

When the player has determined which action to take based on the heuristic, it determines if the action itself needs from other player to be realized in order to place a contract. Then the Contract Net Protocol is in charge to select from a stack of contracts which one has a bigger priority to publish it and be ready to take bids.

The bids are placed basically the same way that an action is selected, the agent receive the contract specifications and determines if it is suitable to perform the action and if it does then it places a bid, the winner agent or actuator is then selected based on the weight of the bids placed.

7 Strategies

The first basic strategy consists on each agent simply searching for the ball and going towards the ball until it is close enough to shoot in the direction of the opponent goal. After going and kicking the ball we decided it was time for defense strategies and end up with another basic strategy that consists on covering certain area and make sure the ball is nowhere near your position on the field and if it does move the ball towards the opponent goal.

Then some other strategies were created such as "Dribble with Ball", "Goalie Catch Ball", "Kick and then Delay" among others needed for the team to play in a competitive way. By combining these strategies we were able to develop more complex strategies such as "Shoot to Goal", "Do Pass" , etc.

After the actions were created and tested, an heuristic was needed to make an agent different from the others in a way that affects only his personal decisions and then he can do actions corresponding on the flow of the game , to do that a method to calculate the optimal heuristic was created in order to have control over the decisions on the player.

First a calculation in the probability that a pass, goal kick, intercept ball, defend left , defend right, catch ball or any other action that can be performed by the player. after that an array is created with those values and then it is multiplied by the real value heuristics matrix which is defined for each player as the list of "can do" and "can not do". This algorithm is shown in algorithm 1.

To calculate a real value of an heuristic considering the formation and the team?s strategies a weights matrix was implemented. The content consists of a value between zero and one, that represents how much we want for that action to happen. Different matrices are made, to change strategies in-game, for a better performance, depending on the score. Also, the approaches to score can differ from one matrix to another, so we can try different strategies in the game and adjust. This can be simple like making more passes to the sides, shooting from mid range, only making very accurate passes, among others.

Algorithm 1 Calculate heuristics

pass ← calculatePass(worldModel)
shootToGoal ← calculateShootToGoal(worldModel)
defendRight ← calculateDefendRight(worldModel)
...
array ← [pass,shootToGoal,defendRight,..]
heuristics ← array ∗ weightMatrix
action ← getMaxValue(heuristics)

8 Conclusions

Borregos team has been actively participating in the RoboCup competitions since 2004. We have several publications related to RoboCup simulation and we have experience working with the rcsssoccer server for RoboCup 2D League. Our goal is to develop a very competitive team for the RoboCup 2D League. We also want to include probabilistic localization and fuzzy bayesian decision making to our bot. Moreover, we want to try reinforcement learning methods in continuos environments and efficient parser and localization algorithms with good accuracy.

References

  1. Bustamante, C., Flores, C., Garrido, L.: A physics model for the robocup 3d soccer simulation. In proceedings of Agents-Directed Simulation Symposium (ADS 07) (2007)
  2. Bustamante, C., Garrido, L.: Monte-calor localization with kalman filter sensor fusion. In proceedings of the 4th IEEE Latin American Robotic Symposium (2007)
  3. Bustamante, C., Garrido, L.: Soccer Robotics, chap. Probabilistic and Statistical Approach for High-Level Decision Making in Soccer Simulation Robotics. I-Tech Education and Publishing (2007)
  4. Bustamante, C., Garrido, L., Soto, R.: Comparing fuzzy naive bayes and gaussian naive bayes for decision making in robosoccer 3d. In proceedings of the 5th Mexican International Conference on Artificial Intelligence (MICAI 06) (2006)
  5. Bustamante, C., Garrido, L., Soto, R.: Fuzzy naive bayesian classification in robosoccer 3d: A hybrid approach to decision making. In proceedings of the RoboCup International Symposium, Bremen, Germany (2006)
  6. Carbajal, E., Garrido, L.: Ball chasing coordination in robotic soccer using a response threshold model with multiple stimulus. Lecture Notes in Artificial Intelligence Series: Lecture Notes in Computer Science (December 2011)
  7. Garcia, D., Garrido, L.: Introduction to Modern Robotics II, chap. Generation of Motion Policies Applying Multiagent Reinforcement Learning is Simulated Robotic Soccer. iConcept Press Australia (2011)
  8. Martinez, E., Brena, R.: Maximizing future options: An online real-time planning method. In proceedings of the 4th Mexican International Conference on Artificial Intelligence (MICAI 05) (2005)
  9. Torres, E., Garrido, L.: Automated generation of cpg-based locomotion for robot nao. Lecture Notes in Artificial Intelligence Series: Lecture Notes in Computer Science (December 2011)