CAMBADA'2014: Team Description Paper

R. Dias, F. Amaral, J. L. Azevedo, R. Castro, B. Cunha, J. Cunha, P. Dias, N. Lau, C. Magalhães, A. J. R. Neves, A. Nunes, E. Pedrosa, A. Pereira, J. Santos, J. Silva, A. Trifan

Transverse Activity on Intelligent Robotics IEETA/DETI – University of Aveiro, Portugal

Abstract This paper describes the CAMBADA Middle Size robotic soccer team for the purpose of qualification to RoboCup'2014. During the last year, improvements have been made in a significant number of components of the robot. Most important changes include the ongoing implementation of a new platform, implementation of a new modular vision system and the use of new digital cameras, ball detection in 3D space, improvements in the world modeling and sensor fusion, development of a model for ball control using the robot's body and several improvements in the high-level coordination and control, namely a new model for the software agent based on utilities, that includes the use of setplays, adaptive strategic positioning, passes and learning for behaviors development.

1 Introduction

CAMBADA is the RoboCup Middle Size League (MSL) soccer team of the University of Aveiro, Portugal. The project involves people working on several areas contributing for the development of all the components of the robot, from the hardware to the software.

The development of the team started in 2003 and a steady progress was observed since then. CAMBADA has participated in several national and international competitions, including RoboCup world championships (5th place in 2007, 1 st in 2008 and 3rd in 2009, 2010, 2011 and 2013), the European RoboLudens, German Open (2nd in 2010), Dutch Open (3rd place in 2012) and the annual Portuguese Open Robotics Festival (3rd place in 2006, 1st in 2007, 2008, 2009, 2010, 2011, 2012 and 2nd in 2013). Moreover, the CAMBADA team achieved excellent results in the technical challenge of the RoboCup MSL: 2nd place in 2008, 1st place in 2009, 4th place in 2010 and 1st place in 2012 and 2013. A 1st place in 2011, 2nd place in 2012 and 3rd in 2013 in the RoboCup Free Challenge was also achieved.

The general architecture of the CAMBADA robots has been described in [1, 2]. Basically, the robots follow a biomorphic paradigm, each being centered on a main processing unit (a laptop), which is responsible for the high-level behavior coordination, i.e. the coordination layer. This main processing unit handles external communication with the other robots and has high bandwidth sensors, typically vision, directly attached to it. Finally, this unit receives low bandwidth sensing information and sends actuating commands to control the robot attitude by means of a distributed low-level sensing/actuating system.

This paper describes the current development stage of the team and is organized as follows: Section 2 describes the recent improvements of the hardware. Section 3 presents the current version of the vision system. Section 4 addresses the world modeling and the control of the robots. Section 5 describes the highlevel coordination and control framework and, finally, Section 6 concludes the paper.

2 New Platform

During the last year, the design and construction of a new platform was finished, which reused the model and functionalities that have proven to be efficient in the previous platform and introduces new changes in some aspects that require a new approach, namely the ability to move faster than 3 m/s top speed and the ability to actively control the ball in a more efficient way.

The main issues that are addressed in the new platform can be summarized as follows: new, custom made, omni-directional wheels based on an aluminum 3 piece sandwich structure (see details in the mechanical drawings) in which 2 sets of 12 off-phase free rollers are supported.

New geometric solution with an asymmetrical hexagon shape to exploit side dribbling possibilities.

A new power transmission system, based on synchronous belts and sprockets. This allows the team to re-use current Maxon 150W DC motors providing power transmission to the wheels by a synchronous belt system instead of the "old" direct drive approach. New motor control boards were also developed in order to improve the control of the motors in this new configuration.

A new ball handling mechanism. This mechanism is based on a double active handler similar to some of the solutions already presented by other teams. Direction and speed of the ball interface rollers is closed loop controlled in order to ensure full compliance with ball handling current rules.

A new kicker device with improved efficiency and better force and aim control over the ball.

A new vision support system. The previous solution used to support the catadioptric mirror/camera solution proved to be mechanically weak. The new solution adopts a much stronger structure and resorts to titanium bars to interconnect the catadioptric set.

3 Vision System

The current vision system of the CAMBADA robots is based on an omnidirectional setup described in [3]. Additionally, the goal-keeper also has a Kinect camera to help in the detection of balls in a 3D space, as described in the next section.

The general architecture of the vision system of the CAMBADA team is presented in Fig. 2. Based on the previous software pipeline of the RoboCup robotic projects from UA [3, 4], a computer vision library has been developed. The library, called UAVision provides all the necessary software for the implementation of a vision system and is designed following the factory design pattern [5]. At this moment, it supports the usage of Firewire, USB and Ethernet cameras and it has a modular structure that allows it to be easily exported and manipulated.

Several algorithms for camera calibration have been developed in the last years and are described in [6–8].

3.1 3D ball detection using Kinect

In the last years the CAMBADA team tried to develop algorithms to detect balls in the air. The first approach was based on the use of a single perspective camera pointing forward [9]. The detection of the ball in the image was made by using a hybrid color/shape approach, which uses color detection to extract regions of the image where the ball can be and then analyses these small images to assert, by circularity detection, if a ball is really present in that region of the image.

In a new approach, using a Kinect camera, we try to use geometry to detect balls in a cloud of points, namely fitting half spheres to the data and try to find areas of interest (see Fig. 3), this fitting approach provides promising results within the typical Kinect working range. The first step of the algorithm is to voxelize the cloud of points space for a given grid size. For each voxel, the number of points inside is computed. A flying object is defined as an object that occupies a given number of voxels with a minimum number of points and whose surrounding voxels are empty. It can be thought as a 3D mask with the inside voxels non empty (containing a minimum number of points) and all outside voxels empty. In Fig. 3 we present three correctly detected balls in three consecutive points of cloud corresponding to a kick away from the sensor.

4 World modeling and robot control

Some improvements to the world model has been or are being made. The main changes aim to improve the precision of the ball and obstacles perception, obstacles avoidance, motion model, kicking calibration and the adaptation of the basic behaviors in order to comply with the novel architecture of the high level software agent based on utilities and priorities, as will be described later.

New behaviors for ball handling with the robot's body are also being developed. These new behaviors have the goal to control the ball using the robot's body, so that it can push it to the desired direction, which may be the opposing goal or simply prevent the ball to go out of bounds. To do so, some auxiliary points are calculated so that the robot passes through these points, adjusting his position and orientation to reach the desired destination with the desired velocity.

In terms of obstacles perception [10], we are developing methodologies for obstacle tracking for persistent representation in the worldstate. This model will represent the global information of the obstacles on the field, rather than an individual perspective of each robot. This representation will be used by the utility map, as described later.

The reactive component of obstacle avoidance algorithm continued to be improved in order to try to ensure that the probability of robot to robot or robot to obstacle crash, or even touch, is reduced to a minimum. The system relies on a set or fully configurable virtual sonars, based on a set of parametric values, and is supported on the vision subsystem. This allows the use of different sonar configurations according to each particular game situation.

We are improving the self-calibration process of the kicking device using two robots communicating with each other [11]. New algorithms are being developed for 3D ball detection using high-speed cameras and 3D cameras. Moreover, we are complementing this process with the study of the real ball's trajectory that will allow the robots to have a more precise kick.

5 High-level coordination

With respect to the software architecture, the fast evolution of the code development over the last years led to a lot of outdated modules and unused portions of code. Therefore, we decided that this was the perfect time to rethink the high-level software architecture. Most of the code was adapted and some weak points were addressed in this new approach, such as the lack of Behavior history, non-smooth transitions and decisions merely based on the current agent cycle.

In the context of MSL, with such a dynamic environment, there is a growing need of predicting the near future, because making decisions based on the current information is not very effective, either due to the delays in communications or the fast moving opponents. So as to overcome this problem, we are evolving to an hybrid agent, which makes decisions based on priorities and a set of utilities (each one testing the expected success a different option) but also on simple conditions. This will allow us to ease the algorithm development of the various roles, by providing the agent an array of different choices in advance, each with some prior conditions and a given priority.

In order to train some behaviors, there was an effort to build a set of Reinforcement Learning tools. These will be used to primarily train the dribbling and pass receiving behaviors.

5.1 Adaptive Strategic positioning

In order to improve how agents decide the best positions to occupy on the field, depending the game state, the CAMBADA agent is being changed to support an utility map. This leads to more dynamic positions in relation to the opponents, and not only to the ball. To do that, the agent is being adapted to support height maps. These maps takes into consideration the opponents and the ball positions as well as other restrictions, namely the field of view. From them it is extracted the most advantageous position, closest to the strategic position defined by SBSP or DT (as presented in the last years), for the robot in a certain moment. After analyzing these maps the agent will choose the position to be occupied.

The Fig. 4 depicts an evaluation of an utility map in two different game situations.

5.2 Reinforcement Learning for behaviors

The MSL provides a interesting environment and a rigorous testbed for the application of Reinforcement Learning methods for robotic behavior generation. The research goals of the CAMBADA team in this field cover not only the application of state-of-the-art methods, but also a more theoretical and fundamental research to develop efficient learning methods for robotic applications.

In the continuation of the research carried out, over the last year the CAMBADA team has developed learning tasks that aim to learn efficient controllers for the dribbling and passing behaviors. With the construction of the new hardware platform, we are also exploring the possibilities of learning how to control the new ball handling device applying RL methods directly in a micro-controller. Additionally, the team has developed a new RL update-rule [12] and is applying new function approximators that should improve the performance and stability of the learning methods used.

5.3 Coach

In the scientific challenge of the RoboCup 2013, the CAMBADA team presented a coach for the MSL that allows in real time the choice of the best formation for the robots based on a set of rules that evaluates several game statistics and the game state. A screeenshot of the the coach application and the corresponding configuration application is presented in Fig. 5.

The team continued the development of the referred coach during the last year, including new features to be used in the next RoboCup competitions.

6 Conclusions

This paper described the current development stage of the CAMBADA robots. Since the last submission of qualification materials, in January 2013, several major improvements have been carried out, namely the ongoing implementation of a new platform, implementation of a new modular vision system and the use of new digital cameras, ball detection in 3D space, improvements in the world modeling and sensor fusion, development of a model for ball control using the robot's body and several improvements in the high-level coordination and control, namely a new model for the software agent based on utilities, that includes the use of setplays, adaptive strategic positioning, passes and learning for behaviors development. Moreover, four Ph.D and six MSc students are doing work in the project, as well as several students are also collaborating in the project as volunteers.

References

1. Neves, A., Azevedo, J., B. Cunha, N.L., Silva, J., Santos, F., Corrente, G., Martins, D.A., Figueiredo, N., Pereira, A., Almeida, L., Lopes, L.S., Pedreiras, P.: 2. In: CAMBADA soccer team: from robot architecture to multiagent coordination. I-Tech Education and Publishing, Vienna, Austria (In Vladan Papic (Ed.), Robot Soccer, 2010)
2. Azevedo, J.L., Cunha, B., Almeida, L.: Hierarchical distributed architectures for autonomous mobile robots: a case study. In: Proc. of the 12th IEEE Conference on Emerging Technologies and Factory Automation, ETFA2007, Greece (2007) 973–980
3. Neves, A.J.R., Pinho, A.J., Martins, D.A., Cunha, B.: An efficient omnidirectional vision system for soccer robots: from calibration to object detection. Mechatronics 21(2) (mar 2011) 399–410
4. Trifan, A., Neves, A.J.R., Cunha, B., Lau, N.: A modular real-time vision system for humanoid robots. In: Proceedings of SPIE IS&T Electronic Imaging 2012. (January 2012)
5. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-oriented Software. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA (1995)
6. Cunha, B., Azevedo, J.L., Lau, N., Almeida, L.: Obtaining the inverse distance map from a non-SVP hyperbolic catadioptric robotic vision system. In: Proc. of the RoboCup 2007. Volume 5001 of Lecture Notes in Computer Science., Atlanta, USA, Springer (2007) 417–424
7. Trifan, A., Neves, A.J.R., Cunha, B.: Evaluation of color spaces for user-supervised color classification in robotic vision. In: Proc. of the 17th International Conference on Image Processing, Computer Vision, & Pattern Recognition, Las Vegas, Nevada, USA (July 2013)
8. Neves, A.J.R., Trifan, A., Cunha, B.: Self-calibration of colormetric parameters in vision systems for autonomous soccer robots. In: RoboCup 2013: Robot Soccer World Cup XVII. Lecture Notes in Computer Science, Springer (2013)
9. Silva, J., Antunes, M., Lau, N., Neves, A.J.R., Lopes, L.S.: Aerial ball perception based on the use of a single perspective camera. In: Proc. of the 16th Portuguese Conference on Artificial Intelligence. Volume 8154 of LNAI., Angra do Heroísmo, Azores, Portugal, Springer (September 2013) 235–249
10. Silva, J., Lau, N., Neves, A.J.R. In: Cooperative detection and identification of obstacles in a robotic soccer team. iConcept Press (February 2013) 219–235
11. Dias, R., Neves, A.J.R., Azevedo, J.L.: Autonomous calibration for the kicking device of a soccer robot. In: Proc. of the 5th International Workshop on Intelligent Robotics, IROBOT 2011, Lisbon, Portugal (Oct. 2011)
12. Cunha, J., Lau, N., Neves, A.J.R.: Q-Batch: initial results with a novel update rule for Batch Reinforcement Learning. In: Advances in Artificial Intelligence - Local Proceedings, EPIA 2013 - XVI Portuguese Conference on Artificial Intelligence, Angra do Heroismo, Azores, Portugal, 9-12 September. (September 2013) 240–251