ISePorto Robotic Soccer Team for Robocup 2012: Improving team perception and robot calibration

Jose Almeida, Alfredo Martins, André Dias, Hugo Silva, Carlos Almeida, Nuno Dias, Luís Lima, Tiago Santos, Ivo Costa, Guilherme Silva, João Ribeiro, Rui Serra, Nuno Nunes, Eduardo Silva

LSA - Autonomous System Laboratory Institute of Engineering of Porto Rua Dr Antonio Bernardino de Almeida 431 4200-072 Porto, Portugal

http://www.lsa.isep.ipp.pt

Abstract This paper presents the ISePorto Middle Size League Team's main goals for Robocup 2012, and describes the major research efforts and achievements attained. It presents current work for improving team play's capabilities and the setup times of the robots. In the first line of work, topics required for better dynamic pass capabilities and global awareness were addressed, such as cooperative perception, clock synchronization and ball control and perception at kicker level. In the second line of work, an automatic calibration method for robots setup was developed. This significantly reduces the time to find all the parameter necessary to define the relation between all the sensors frames and robot frame, taking in count with all rotating parts of the robot.

1 Introduction

The ISEP Autonomous Systems Lab. (LSA) robotic football team provides an excellent tool to develop and demonstrate the research in the areas of interest associated with autonomous systems, like: sensor fusion, mobile robotics navigation, nonlinear hybrid feedback control, system architectures, coordination of teams of robots in dynamic environments, vision systems, and embedded systems.

The main goal for this year is keeping improving team play capabilities such as formation maneuvers and dynamic pass. In addition high dynamic air ball events are becoming relevant in game playing. Improving ball perception thus addressed with emphasis on multi robot 3D ball tracking.

In terms of game play functionalities some coordination requirements need to be addressed. Partial mechanic redesign with emphasis on the kicker and ball control guides, are under development. This entails a kicker with ball force control, ball stopping mechanism and retractable ball guides. The novelty of this device is the capability to sense the ball by the opposing torque at the kicker device in few milliseconds, allowing to perform a kick without stopping or grabbing the ball. One issue that had been constrained the development of the team is the setup time necessary to calibrate all the huge number of robot parameters, due to usage of directional cameras mounted in rotating parts. The manual procedure to estimate those parameters is very time consumption, taking much of setup the time in competitions that could be better used in testing and validation of developed systems and algorithms.

2 3D Ball Trajectory Estimation

ISePorto Middle Size League (MSL) team was one of the first teams to introduce mechanical systems and visual 3D ball estimation in Robocup MSL League. One of the first major accomplish was achieved in Bremen 2006 Robocup Competition, went the ISePorto Goalkeeper made the first aerial defense against the World Champion 2006 Tribots Team[7].

The MSL Team ISePorto robots, have as primary mean of perception, three perspective cameras. However, due to a restrictive number of factors, mainly related to the lack of camera synchronization together with available cameras field of view makes stereo vision not our most suitable option. Considering previous restrictions, 3D ball estimation by a single monocular camera is a demanding task[5], since current MSL robots are able to kick a soccer ball at very high velocities (more than 10 m/s). In particular when the ball is kicked into the air at high velocity and describes a 3D trajectory.

2.1 Ball Estimation Architecture

The 3D ball estimation is achieved using a ISePorto robot that has a differential traction wheelbase with an additional rotating upper body housing the on-board computer, wireless communications module, inertial and magnetic sensors and the kicking device. This section has one fixed camera used mainly for close range ball and target tracking. On top of the upper body the robot has rotating head with a camera for long range target tracking. Overall system architecture is described in figure 2. The vision system starts by acquiring images at a pre-determinate frame rate (in this case 30 fps 640x480 image resolution). Afterwards, image segmentation is conducted and relevant object (ball) information is retrieved from the image, according to the work described in detail by [8]. Basically, image ball information is retrieved using image blob extraction techniques together with edge detection that uses weighted least squares circle estimation, in order to obtain a more robust ball estimation.

2.2 Ground-Truth 3D Ball Tracking

In order to evaluate the quality of the 3D ball estimation from the ISePorto robot, an exogenous system of cameras in a stereo baseline was introduced as a ground-truth, figure 3. Cameras were positioned with a baseline of 13 meters and connected via Gig-Ethernet to a Intel Pentium Dual Core PC 2 GHz running a Linux operating system. The cameras used are Jai CB080GE working at 20 FPS with 1024x768 resolution with external sync trigger. The first step in ball detection by ground-truth cameras is to extract moving objects by a background frame subtraction operation. The pixel-wise subtraction extracts all the moving objects in the frame obtaining not only a ball but also other type of objects. We eliminate all objects that are not possible ball candidates by performing colour threshold segmentation in HSV space.

3 Clock Synchronization

Centralized data fusion is a very well know topic in robotics. However, applying data fusion techniques to team of robots (like MSL) still a subject of continuous development, specially if one considers decentralized data fusion techniques to enhance autonomous cooperation between a team of robots.

Multi robot scenarios impose several sources of complexity specially when the main goal is to fuse proprioceptive and exteroceptive information acquired by each teammate, in order to achieve identification from correlated input items. Those items must be acquired, referenced to a global timestamp imposing clock synchronization between the team of robots.

In order to do so, several constraints and requirements are defined by the Middle Size League, namely: wireless link quality, network infrastructure, reboot robot frequency during the game and robot self-synchronized (<1 ms).

Another view for clock synchronization requirements, is dynamic ball passing between the teammates. In order to perform dynamic passing, the robot must be able to accuracy estimate the ball trajectory. If one considers has an example: a passing where the ball travels at 3m/s with 20ms offset will give 6cm ball trajectory error.

Considering the present constraints, three clock synchronization protocols were evaluated for the MSL scenario: NTP (Network Time Protocol)[2], PTP (Precision Time Protocol)[1] and Chrony. The experimental tests were conducted with 3 MSL robots and a server connect to a wireless network with similar hardware.

3.1 Remarks

NTP - Network Time Protocol

• When the system is in a non-converged state, such as after diverging on boot, the time it takes for the system to converge is unacceptably long, becoming a huge drawback for application were is needed to reboot several times.

  • Takes 10 hours of sync to have low offset and a good stability
  • Applying NTPDATE we can synchronize the robot from the source with 1 ms offset, but the clocks start to diverge and often take hours to converge to a steady state.

• When the operating conditions suddenly change, the system diverges dramatically, and sometimes becomes unstable/divergent.

  • In particular, a pathological case we have seen is when the wireless link is near saturation for an extended period of time. After the wireless link become stable, the offset from the source is almost 30ms.

• Steady State with low offset, 1ms (best result for 10 hours of sync)

PTP - Precision Time Protocol

• The Precision Time Protocol (PTP, defined in IEEE 1588) is a standard protocol which can replace NTP within a local area network. It is based on measuring the send and receive timestamps of regular multicast messages sent from a master to all slaves (Sync) and with low frequent messages sent from each slave to the master (Delay Req). The Sync and Delay Req values will be used to infer correction in the clock. The correction is done by a PI controller and a IIR Filter.
  • The PTP can be very unstable (presents overshoot in the offset).
  • Requires a previous analyze (for each hardware board + CPU) of the PI and IIR Filter parameters (Tuning).
• Steady State with low offset, <1ms (result after 10 min of sync).

Chrony

• Chrony is compatible with the NTP, we can use a NTP server to the sync process or we can configure the chrony has a server.
  • Takes 20/30 min of sync to obtain the correct values for the next reboot.
  • Receives a sync message with a max periodicity of 16s.
• Maintain the time across reboots, by working out the offset and drift rate of the computer's real-time clock (RTC) and using this information to set the system clock correctly at boot up.
  • After reboot, takes 0.2s to stable the offset.
  • The drift rate, the offset and the RTC are stored in a text file and used for the next boot.
• When the operating conditions suddenly change (wireless link), the system keeps stable
• Steady State with low offset, <2.6us

Considering the results and the MSL constraints, the clock synchronization protocol that we propose to use during the Robocup 2011 is the Chrony protocol.

4 Kicking Device

The kicking device not only was developed with the sole purpose of improving robot shooting but specially to improve their passing ability. Furthermore, a ball sensing mechanism was developed in 2011 to receive and pass the ball without visual perception due to the latency drawback in our USB camera device driver. The sensing capability was significantly improved in 2012. With the field size increase, the cooperative and behavior algorithms implementation in the game the robots are now capable to receive and pass the ball between them. To sustain this it was development a shooting device with force control and retractable ball guides.

5 Robots auto calibration tool

We developed an auto-calibration tool based in an Extended Kalman Filter (EKF), that with the intrinsic parameters of all cameras previously estimated, and with the a set of visual measurements from a chessboard acquired synchronized with encoder information from all rotating axes, allow to fully calibrate all rotating axis, angles and offsets distances of an ISePorto Robot's. So the calibration procedure is reduced to the placing the robot near a chessboard target and moving it around the target while running the Extended Kalman Filter to estimate the set of the required 15 parameters. This reduced the calibration time from more than a hour to a few minutes.

6 Conclusions and Future Work

Current developments in the design and implementation of 3D Ball trajectory estimation, clock synchronization to support data fusion and a kicking device with ball sensing are presented. Two main lines of research are under attention for this Robocup edition, one concerning to multi-robot coordination and dynamic pass under severe dynamical environments and the other on ball perception in 3D with efficient real-time target tracking under restrictive conditions such as monocular 3D tracking and unavailability of stereo solutions. The team has participated in several competitions namely in the Robocup 2003, 2004, 2006, 2009, 2010 and 2011, GermanOpen 2002, 2003, 2005 and 2008, Robotica 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 and 2011 in Portugal. The test of the new integrated features needs to be validated in game conditions in Robocup 2012 competition.

7 Acknowledgement

The authors would like to thank the LSA/ISEP team for their efforts and dedication in developing the robots and support on the participation of competitions and events. This work is partially sponsored by INESC TEC and ISEP/IPP.

References

1. Vallat, A. and Schneuwly, D. Clock Synchronization in Telecommunications via PTP (IEEE 1588) Frequency Control Symposium, 2007 Joint with the 21st European Frequency and Time Forum. IEEE International, 2007.
2. Cristina D. Murta , Pedro R. Torres Characterizing quality of time and topology in a time synchronization network 49th IEEE Global Telecommunications Conference, IEEE GLOBE-COM, 2006
3. Nestares, O.; Fleet, D.J. Error-in-variables likelihood functions for motion estimation Image Processing, 2003. ICIP 2003, 2003
4. Farina, A. and Di Lallo, A. and Timmoneri, L. and Volpi, T. and Ristic, B. CRLB and ML for parametric estimate: new results Signal Process, 2006
5. Farina, A. and Di Lallo, A. and Timmoneri, L. and Volpi, T. and Ristic, B. Estimating 3D Positions and Velocities of Projectiles from Monocular Views IEEE Trans. Pattern Anal. Mach. Intell., 2009
6. Oliver Birbach and Jorg Kurlbaum and Tim Laue and Udo Frese Tracking of Ball Trajectories with a Free Moving Camera-Inertial Sensor RoboCup 2008: Robot Soccer World Cup XII Lecture Notes in Computer Science, 2009
7. First aerial ball defense in the world against Tribots, available at http://www.lsa.isep.ipp.pt/movies/iseporto_tdp11d.mp4.
8. Silva,H and Almeida,J and Lima,L and Martins,A and Silva,E.P A Real Time Vision System for Autonomous Systems: Characterization during a Middle Size Match RoboCup 2007 Robot Soccer World Cup XI Lecture Notes in Computer Science, 2008