CAMBADA'2013: Team Description Paper

R. Dias, A. J. R. Neves, J. L. Azevedo, B. Cunha, J. Cunha, P. Dias, A. Domingos, L. Ferreira, P. Fonseca, N. Lau, E. Pedrosa, A. Pereira, R. Serra, J. Silva, P. Soares, A. Trifan

Transverse Activity on Intelligent Robotics IEETA/DETI – University of Aveiro, Portugal

Abstract This paper describes the CAMBADA Middle Size robotic soccer team for the purpose of qualification to RoboCup'2013. During the last year, improvements have been made in a significant number of components of the robot. Most important changes include the ongoing implementation of a new platform, implementation of a new modular vision system and the use of new digital cameras, ball detection in 3D space, improvements in the world modeling and sensor fusion, development of a model for trajectory generation and several improvements in the high-level coordination and control, namely a new model for the software agent based on utilities, that includes the use of setplays, strategic positioning, passes and learning for behaviors development.

1 Introduction

CAMBADA is the RoboCup Middle Size League (MSL) soccer team of the University of Aveiro, Portugal. The project involves people working on several areas contributing for the development of all the components of the robot, from the hardware to the software.

The development of the team started in 2003 and a steady progress was observed since then. CAMBADA has participated in several national and international competitions, including RoboCup world championships (5th place in 2007, 1st in 2008 and 3rd in 2009, 2010 and 2011), the European RoboLudens, German Open (2nd in 2010), Dutch Open (3rd place in 2012) and the annual Portuguese Open Robotics Festival (3rd place in 2006, 1st in 2007, 2008, 2009, 2010, 2011 and 2012). Moreover, the CAMBADA team achieved excellent results in the mandatory technical challenge of the RoboCup MSL: 2nd place in 2008, 1st place in 2009, 4th place in 2010 and 1st place in 2012. A 1st place in 2011 and a 2nd place in 2012 in the RoboCup Free Challenge was also achieved.

The general architecture of the CAMBADA robots has been described in [1,2]. Basically, the robots follow a biomorphic paradigm, each being centered on a main processing unit (a laptop), which is responsible for the high-level behavior coordination, i.e. the coordination layer. This main processing unit handles external communication with the other robots and has high bandwidth sensors, typically vision, directly attached to it. Finally, this unit receives low bandwidth sensing information and sends actuating commands to control the robot attitude by means of a distributed low-level sensing/actuating system.

This paper describes the current development stage of the team and is organized as follows: Section 2 describes the recent improvements of the hardware. Section 3 presents the current version of the vision system. Section 4 addresses the world modeling and the control of the robots. Section 5 describes the highlevel coordination and control framework and, finally, Section 6 concludes the paper.

2 New Platform

Although the current CAMBADA platform has proved to be stable, robust and reliable, a set of specific drawbacks have been identified in view of the growing challenges in the league. This is the case, namely, of the ability to move faster than the current 3 m/s top speed and the ability to actively control the ball in a more efficient way. These drawbacks have triggered the design of a new platform, currently under construction, which tries to reuse the model and functionalities which have proved to be efficient and introduces changes in those aspects that require a new approach.

The main issues that are addressed in the new platform can be summarized as follows: new, custom made, omni-directional wheels based on an aluminum 3 piece sandwich structure (see details in the mechanical drawings) in which 2 sets of 12 off-phase free rollers are supported.

New geometric solution with an asymmetrical hexagon shape to exploit side dribbling possibilities.

A new power transmission system, based on synchronous belts and sprockets. This allows the team to re-use current Maxon 150W DC motors providing power transmission to the wheels by a synchronous belt system instead of the current direct drive approach. New motor control boards were also developed in order to improve the control of the motors in this new configuration.

A new ball handling mechanism. This mechanism is based on a double active handler similar to some of the solutions already presented by other teams. Direction and speed of the ball interface rollers is closed loop controlled in order to ensure full compliance with ball handling current rules.

A new kicker device with improved efficiency and better force and aim control over the ball.

A new vision support system. The previous solution used to support the catadioptric mirror/camera solution proved to be mechanically week. The new solution adopts a much stronger structure and resorts to titanium bars to interconnect the catadioptric set.

A new mirror design. Spatial resolution of current mirrors is clearly in excess for short distances, while decreasing for long distances. This also reduces the ability to precisely track obstacles and ball at long distances (> 4m). The new design compensates for this effect and results in a more balanced distribution of the spatial resolution.

Finally, an electronic system for integral in board battery charging including the laptop computer battery has been design minimizing the need for battery replacement either during setups or even during the games.

3 Vision System

The current vision system of the CAMBADA robots is based on an omnidirectional setup described in [3]. Additionally, the goal-keeper also has a Kinect camera to help in the detection of balls in a 3D space, as described in the next section.

The general architecture of the vision system of the CAMBADA team is presented in Fig. 2. Based on the previous software pipeline of the RoboCup robotic projects from UA [3, 4], a computer vision library has been developed. The library, called UAVision is shared by all the robotic projects from University of Aveiro and it provides all the necessary software for implementing a robotic vision system from scratch. From modules for image acquisition for several types of digital cameras, to camera calibration algorithms and algorithms for colorbased and generic object detection, the library is suitable for the implementation of any time constrained vision system.

3.1 3D ball detection using Kinect

To detect balls in the air, in the last year we tried an approach based on the use of a single perspective camera pointing forward. The detection of the ball in the image was made by using a hybrid color/shape approach, which uses color detection to extract regions of the image where the ball can be and then analyses these small images to assert, by circularity detection, if a ball is really present in that region of the image [5].

As ongoing work, some preliminary tests have been done to allow 3D detection of flying balls in the field using a Kinect camera. The idea is to overcome the actual limitation of most of the MSL teams that can only detect precisely the position of the ball on the field when it lies in the ground.

In a first approach we try to use geometry to detect balls in a Kinect cloud of points, namely fitting half spheres to the data and try to find areas of interest. However given the discretization of the Kinect for longer distance (see Fig. 3), this fitting approach only appeared to be interesting when close to the sensor (within the typical Kinect working range - less than 4m). For longer distances, the spherical shape of the ball becomes difficult to detect.

The approach we use is to detect flying object within the Kinect field of view. The first step of the algorithm is to voxelize the cloud of points space for a given grid size. For each voxel, the number of points inside is computed. A flying object is defined as an object that occupies a given number of voxels with a minimum number of points and whose surrounding voxels are empty. It can be thought as a 3D mask with the inside voxels non empty (containing a minimum number of points) and all outside voxels empty. In Fig. 3 we present three correctly detected balls in three consecutive points of cloud corresponding to a kick away from the sensor.

4 World modeling and robot control

Some improvements to the world model has been or are being made. The main changes aim to improve the precision of the ball and obstacles perception, obstacles avoidance, motion model, kicking calibration and the adaptation of the basic behaviors in order to comply with the novel architecture of the high level software agent based on utilities, as will be described later.

In terms of obstacles perception, we are developing methodologies for obstacle tracking for persistent representation in the worldstate. This model will represent the global information of the obstacles on the field, rather than an individual perspective of each robot. This representation will be used by the utility map, as described later.

The reactive component of obstacle avoidance algorithm continued to be improved in order to try to ensure that the probability of robot to robot or robot to obstacle crash, or even touch, is reduced to a minimum. The system relies on a set or fully configurable virtual sonars, based on a set of parametric values, and is supported on the vision subsystem. This allows the use of different sonar configurations according to each particular game situation.

We are improving the motion model based on trajectory generation [6] which uses an heuristic to compute the points in a trajectory in a 2D space based on the trapezoidal velocity profile, where the movement occurs in three phases: initial acceleration from initial velocity to a plateau speed (phase 1), travelling in uniform motion at plateau velocity (phase 2) and acceleration from plateau velocity to final velocity (phase 3). The computed trajectory is subject to the constraints of initial position and velocity, final position and velocity and the values for acceleration and plateau speed.

We are improving the self-calibration process of the kicking device using two robots communicating with each other [7]. New algorithms are being developed for 3D ball detection using high-speed cameras and 3D cameras. Moreover, we are complementing this process with the study of the real ball's trajectory that will allow the robots to have a more precise kick.

5 High-level coordination

With respect to the software architecture, the fast evolution of the code development over the last years led to a lot of outdated modules and unused portions of code. Therefore, we decided that this was the perfect time to rethink the high-level software architecture. Most of the code was adapted and some weak points were addressed in this new approach, such as the lack of Behaviour history, non-smooth transitions and decisions merely based on the current agent cycle.

In the context of MSL, with such a dynamic environment, there is a growing need of predicting the near future, because making decisions based on the current information is not very effective, either due to the delays in communications or the fast moving opponents. So as to overcome this handycap, we are evolving to an hybrid agent, which makes decisions based on a set of utilities (each one testing the expected success a different option) but also on simple conditions. This will allow us to ease the algorithm development of the various roles, by providing the agent an array of different choices in advance, each with some prior conditions and a given priority.

In order to train some behaviours, there was an effort to build a set of Reinforcement Learning tools. These will be used to primarily train the dribbling and pass receiving behaviours.

5.1 Setplays

The setplay concept is very frequently used in several collective sports to improve the effectiveness of the teams strategy. A general framework for setplay execution was developed that can be used to specify, select and aid execution of setplays in several different RoboCup leagues [8].

Setplays can be defined using an interactive graphical tool, which outputs a textual description of the setplay in a predefined language. During normal operation setplay selection is achieved using a Case-Based Reasoning approach that learns from previous executions in similar contexts. The execution of the setplay is aided by an execution engine that can be easily integrated with the robot agents by providing implementations for several Conditions and Actions needed by the setplay. The execution of seplays in CAMBADA, using this setplay framework, has already been performed in simulation and its adaptation to the real robots is currently under development.

This was firstly presented in MSL at RoboCup 2012 Free Challenge by the CAMBADA team.

5.2 Strategic positioning

The formation model used by CAMBADA since 2006 is an adaptation of Situation Based Strategic Positioning (SBSP) developed in the Simulation League. After RoboCup 2010, the CAMBADA agent has been enhanced with a new formation model (called DT), also originally developed in the Simulation League, where the positioning of each player results from defining a set of key ball positions for which the player target position is defined and then interpolating these positionings using a Delaunay Triangulation of the key ball positions.

The CAMBADA formation can now use SBSP formations, simpler but less flexible, or DT formations, that allow a more precise positioning of the formation over the entire field. This was presented in MSL at RoboCup 2011 free challenge by the CAMBADA team, achieving the 1st place.

5.3 Utility Based Agent

In order to improve how agents decide the best way to reach their goal, the CAMBADA agent code is being changed to support an utility map. This utility map evaluates behaviors according to the current world state and the future consequences that each behavior could cause. This evaluating allows utility based agents to choose at a certain moment the best behavior and the best position in the field for him to reach his goal. The Fig. 4 depicts an evaluation of an utility map.

5.4 Reinforcement Learning for behaviors

Many problems in Robotics can be formulated as sequential decision problems. In this context the field of Reinforcement Learning (RL) offers a valuable set of tools to develop efficient robotic behaviors. This is highly desirable in a competitive environment such as the Middle Size League. The success of the application of Reinforcement Learning approaches in the Middle Size League was previously demonstrated by the Brainstormers Tribots [9] which contributed to the victory of two consecutive RoboCup World Championship titles.

The CAMBADA team is is planning to apply Reinforcement Learning in order to increase the performance of existing behavior as well as develop new behaviors through learning. Of particular focus are the dribbling and pass receiving behaviours, essential in a robotic soccer game. Moreover, we plan to develop a learning task to obtain a highly accurate kicking behaviour. Simultaneously, the CAMBADA team is tackling more fundamental RL problems, developing faster learning methods, improving on the Neural Fitted Q Iteration success.

6 Conclusions

This paper described the current development stage of the CAMBADA robots. Since the last submission of qualification materials, in January 2012, several major improvements have been carried out, namely the ongoing implementation of a new platform, implementation of a new modular vision system and the use of new digital cameras, ball detection in 3D space, improvements in the world modeling and sensor fusion, development of a model for trajectory generation and several improvements in the high-level coordination and control, namely a new model for the software agent based on utilities, that includes the use of setplays, strategic positioning, passes and learning for behaviors development. Moreover, four Ph.D and several MSc students are doing work in the project, as well as several students are also collaborating in the project as volunteers.

References

1. Neves, A., Azevedo, J., B. Cunha, N.L., Silva, J., Santos, F., Corrente, G., Martins, D.A., Figueiredo, N., Pereira, A., Almeida, L., Lopes, L.S., Pedreiras, P.: 2. In: CAMBADA soccer team: from robot architecture to multiagent coordination. I-Tech Education and Publishing, Vienna, Austria (In Vladan Papic (Ed.), Robot Soccer, 2010)
2. Azevedo, J.L., Cunha, B., Almeida, L.: Hierarchical distributed architectures for autonomous mobile robots: a case study. In: Proc. of the 12th IEEE Conference on Emerging Technologies and Factory Automation, ETFA2007, Greece (2007) 973–980
3. Neves, A.J.R., Pinho, A.J., Martins, D.A., Cunha, B.: An efficient omnidirectional vision system for soccer robots: from calibration to object detection. Mechatronics 21(2) (mar 2011) 399–410
4. Trifan, A., Neves, A.J.R., Cunha, B., Lau, N.: A modular real-time vision system for humanoid robots. In: Proceedings of SPIE IS&T Electronic Imaging 2012. (January 2012)
5. Silva, J., Antunes, M., Lau, N., Neves, A.J.R., Lopes, L.S.: Aerial ball detection in RoboCup Middle Size League using polar histograms. In: Proc. of the 17th Portuguese Conf. on Pattern Recognition, Recpad-2011, Porto, Portugal (October 2011)
6. Fonseca, P., Neves, A.J.R., Azevedo, J.L., Silva, J.: An heuristic for trajectory generation in mobile robotics. In: Proceedings of IEEE 16th Conference on Emerging Technologies and Factory Automation (ETFA). (September 2011)
7. Dias, R., Neves, A.J.R., Azevedo, J.L.: Autonomous calibration for the kicking device of a soccer robot. In: Proc. of the 5th International Workshop on Intelligent Robotics, IROBOT 2011, Lisbon, Portugal (Oct. 2011)
8. Mota, L., Reis, L.P., Lau, N.: Multi-robot coordination using setplays in the middlesize and simulation leagues. Mechatronics 21(2) (mar 2011) 434–444
9. Riedmiller, M., Gabel, T., Hafner, R., Lange, S.: Reinforcement learning for robot soccer. Auton. Robots 27(1) (2009) 55–73