ISePorto Robotic Soccer Team for Robocup 2013: Improving Team Perception
Jose Almeida, Alfredo Martins, André Dias, Hugo Silva, Carlos Almeida, Nuno Dias, Luís Lima, Guilherme Silva, Eduardo Silva
LSA - Autonomous System Laboratory Institute of Engineering of Porto Rua Dr Antonio Bernardino de Almeida 431 4200-072 Porto, Portugal
Abstract This paper presents the ISePorto Middle Size League Team's main goals for Robocup 2013, and describes the major research efforts and achievements attained. It presents current work for improving team play's capabilities and the setup times of the robots. In the first line of work, topics required for better team global awareness were addressed, such as cooperative perception, clock synchronization and ball control and perception at kicker level. In the second line of work, an automatic calibration method for robots setup was developed. This significantly reduces the time to find all the parameter necessary to define the relation between all the sensors frames and robot frame, taking in count with all rotating parts of the robot.
Introduction
The ISEP Autonomous Systems Lab. (LSA) robotic football team provides an excellent tool to develop and demonstrate the research in the areas of interest associated with autonomous systems, like: sensor fusion, mobile robotics navigation, nonlinear hybrid feedback control, system architectures, coordination of teams of robots in dynamic environments, vision systems, and embedded systems.
The main goal for this year is keeping improving team play capabilities such as formation maneuvers and dynamic pass. In addition high dynamic air ball events are becoming relevant in game playing. Improving ball perception thus addressed with emphasis on multi robot 3D ball tracking.
In terms of game play functionalities some coordination requirements need to be addressed. Partial mechanic redesign with emphasis on the kicker and ball control guides, are under development. This entails a kicker with ball force control, ball stopping mechanism and retractable ball guides. The novelty of this device is the capability to sense the ball by the opposing torque at the kicker device in few milliseconds, allowing to perform a kick without stopping or grabbing the ball. One issue that had been constrained the development of the team is the setup time necessary to calibrate all the huge number of robot parameters, due to usage of directional cameras mounted in rotating parts. The manual procedure to estimate those parameters is very time consumption, taking much of setup the time in competitions that could be better used in testing and validation of developed systems and algorithms.
Real-Time Visual Ground-Truth System for the MSL Scenario
The visual ground truth evaluation scenario will be contextualized to the Robocup Middle Size League (MSL) due to be an important testbed for robotic applications where a team of five robots are playing together by sharing and fusing information in order to present cooperative behaviors and achieve a final goal: play soccer. In this specific and complex scenario there are two works[1][10] in the state of art that were considered as references to characterize what should be the requirements for the proposed system applicable to the envisioned Robocup scenario. These references were chosen due to the fact (apart of being visual ground truth systems) of being visual tools for the MSL developed for monitoring data logs providing state estimation (localization and target estimation) and path planning of each robot over the video recording of the game.
The proposed architecture is organized in three major layers: Data Acquisition, Image Processing and 3D Stereo Triangulation (see figure 1). The layers were implemented under the ROS (Robotic Operating System) framework due to being able to support multiple camera drivers, its large impact on the robotics community, providing a middleware for inter-process communication and being an open and modular framework suitable for further feature development and integration.
Although the architecture supports multi-cameras, we will detail the three layers contextualized to a visual ground truth system composed by two gigabit ethernet cameras in a stereo baseline (see figure 2) and applied to the Middle Size League(MSL) scenario.
3D Ball Trajectory Estimation
ISePorto Middle Size League (MSL) team was one of the first teams to introduce mechanical systems and visual 3D ball estimation in Robocup MSL League. One of the first major accomplish was achieved in Bremen 2006 Robocup Competition, went the ISePorto Goalkeeper made the first aerial defense against the World Champion 2006 Tribots Team[8].
The MSL Team ISePorto robots, have as primary mean of perception, three perspective cameras. However, due to a restrictive number of factors, mainly related to the lack of camera synchronization together with available cameras field of view makes stereo vision not our most suitable option. Considering previous restrictions, 3D ball estimation by a single monocular camera is a demanding task[6], since current MSL robots are able to kick a soccer ball at very high velocities (more than 10 m/s). In particular when the ball is kicked into the air at high velocity and describes a 3D trajectory.
Considered the fact that the estimate must be computed in real time we combine two methods: the Maximum Likelihood method (MLM)[4][5] which is able to provide an initial estimate of the trajectory and the Extended Kalman filter (EKF)[7] which is able to recursively update the trajectory using new information.
Ball Estimation Architecture
The 3D ball estimation is achieved using a ISePorto robot that has a differential traction wheelbase with an additional rotating upper body housing the on-board computer, wireless communications module, inertial and magnetic sensors and the kicking device. This section has one fixed camera used mainly for close range ball and target tracking. On top of the upper body the robot has rotating head with a camera for long range target tracking. Overall system architecture is described in figure 4. The vision system starts by acquiring images at a pre-determinate frame rate (in this case 30 fps 640x480 image resolution). Afterwards, image segmentation is conducted and relevant object (ball) information is retrieved from the image, according to the work described in detail by [9]. Basically, image ball information is retrieved using image blob extraction techniques together with edge detection that uses weighted least squares circle estimation, in order to obtain a more robust ball estimation.
Clock Synchronization
Centralized data fusion is a very well know topic in robotics. However, applying data fusion techniques to team of robots (like MSL) still a subject of continuous development, specially if one considers decentralized data fusion techniques to enhance autonomous cooperation between a team of robots.
Multi robot scenarios impose several sources of complexity specially when the main goal is to fuse proprioceptive and exteroceptive information acquired by each teammate, in order to achieve identification from correlated input items. Those items must be acquired, referenced to a global timestamp imposing clock synchronization between the team of robots.
In order to do so, several constraints and requirements are defined by the Middle Size League, namely: wireless link quality, network infrastructure, reboot robot frequency during the game and robot self-synchronized (<1 ms).
Another view for clock synchronization requirements, is dynamic ball passing between the teammates. In order to perform dynamic passing, the robot must be able to accuracy estimate the ball trajectory. If one considers has an example: a passing where the ball travels at 3m/s with 20ms offset will give 6cm ball trajectory error.
Considering the present constraints, three clock synchronization protocols were evaluated for the MSL scenario: NTP (Network Time Protocol)[3], PTP (Precision Time Protocol)[2] and Chrony. The experimental tests were conducted with 3 MSL robots and a server connect to a wireless network with similar hardware.
Remarks
When the system is in a non-converged state, such as after diverging on boot, the time it takes for the system to converge is unacceptably long, becoming a huge drawback for application were is needed to reboot several times.
- Takes 10 hours of sync to have low offset and a good stability
- Applying NTPDATE we can synchronize the robot from the source with 1 ms offset, but the clocks start to diverge and often take hours to converge to a steady state.
When the operating conditions suddenly change, the system diverges dramatically, and sometimes becomes unstable/divergent.
- In particular, a pathological case we have seen is when the wireless link is near saturation for an extended period of time. After the wireless link become stable, the offset from the source is almost 30ms.
Steady State with low offset, 1ms (best result for 10 hours of sync)
NTP - Network Time Protocol
When the system is in a non-converged state, such as after diverging on boot, the time it takes for the system to converge is unacceptably long, becoming a huge drawback for application were is needed to reboot several times.
- Takes 10 hours of sync to have low offset and a good stability
- Applying NTPDATE we can synchronize the robot from the source with 1 ms offset, but the clocks start to diverge and often take hours to converge to a steady state.
When the operating conditions suddenly change, the system diverges dramatically, and sometimes becomes unstable/divergent.
- In particular, a pathological case we have seen is when the wireless link is near saturation for an extended period of time. After the wireless link become stable, the offset from the source is almost 30ms.
Steady State with low offset, 1ms (best result for 10 hours of sync)
PTP - Precision Time Protocol
The Precision Time Protocol (PTP, defined in IEEE 1588) is a standard protocol which can replace NTP within a local area network. It is based on measuring the send and receive timestamps of regular multicast messages sent from a master to all slaves (Sync) and with low frequent messages sent from each slave to the master (Delay Req). The Sync and Delay Req values will be used to infer correction in the clock. The correction is done by a PI controller and a IIR Filter.
- The PTP can be very unstable (presents overshoot in the offset).
- Requires a previous analyze (for each hardware board + CPU) of the PI and IIR Filter parameters (Tuning).
Steady State with low offset, <1ms (result after 10 min of sync).
Chrony
Chrony is compatible with the NTP, we can use a NTP server to the sync process or we can configure the chrony has a server.
- Takes 20/30 min of sync to obtain the correct values for the next reboot.
- Receives a sync message with a max periodicity of 16s.
Maintain the time across reboots, by working out the offset and drift rate of the computer's real-time clock (RTC) and using this information to set the system clock correctly at boot up.
- After reboot, takes 0.2s to stable the offset.
- The drift rate, the offset and the RTC are stored in a text file and used for the next boot.
When the operating conditions suddenly change (wireless link), the system keeps stable
Steady State with low offset, <2.6us
Considering the results and the MSL constraints, the clock synchronization protocol that we propose to use during the Robocup 2011 is the Chrony protocol.
Kicking Device
The kicking device not only was developed with the sole purpose of improving robot shooting but specially to improve their passing ability. Furthermore, a ball sensing mechanism was developed in 2011 to receive and pass the ball without visual perception due to the latency drawback in our USB camera device driver. The sensing capability was significantly improved in 2012. With the field size increase, the cooperative and behavior algorithms implementation in the game the robots are now capable to receive and pass the ball between them. To sustain this it was development a shooting device with force control and retractable ball guides.
The kicking device has two loops of control: position and force control. The position control is responsible to ability the kicker to receive the ball, stop and prepare to pass, the force control loop will be applied in the kicking and the pass between teammates.
Robots auto calibration tool
We developed an auto-calibration tool based in an Extended Kalman Filter (EKF), that with the intrinsic parameters of all cameras previously estimated, and with the a set of visual measurements from a chessboard acquired synchronized with encoder information from all rotating axes, allow to fully calibrate all rotating axis, angles and offsets distances of an ISePorto Robot's. So the calibration procedure is reduced to the placing the robot near a chessboard target and moving it around the target while running the Extended Kalman Filter to estimate the set of the required 15 parameters. This reduced the calibration time from more than a hour to a few minutes.
Conclusions and Future Work
Current developments in the design and implementation of 3D Ball trajectory estimation, clock synchronization to support data fusion and a kicking device with ball sensing are presented. Two main lines of research are under attention for this Robocup edition, one concerning to multi-robot coordination and dynamic pass under severe dynamical environments and the other on ball perception in 3D with efficient real-time target tracking under restrictive conditions such as monocular 3D tracking and unavailability of stereo solutions. The team has participated in several competitions namely in the Robocup 2003, 2004, 2006, 2009, 2010 and 2011, GermanOpen 2002, 2003, 2005 and 2008, Robotica 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 and 2011 in Portugal. The test of the new integrated features needs to be validated in game conditions in Robocup 2012 competition.
Acknowledgement
The authors would like to thank the LSA/ISEP team for their efforts and dedication in developing the robots and support on the participation of competitions and events. This work is partially sponsored by INESC TEC and ISEP/IPP.
References
- TechUnited Middle Size League Team. Greenfield augmented reality. http://www.techunited.nl/wiki/index.php?title=GreenfieldAugmentedReality, 2010.
- Vallat, A. and Schneuwly, D. Clock Synchronization in Telecommunications via PTP (IEEE 1588) Frequency Control Symposium, 2007 Joint with the 21st European Frequency and Time Forum. IEEE International, 2007.
- Cristina D. Murta , Pedro R. Torres Characterizing quality of time and topology in a time synchronization network 49th IEEE Global Telecommunications Conference, IEEE GLOBE-COM, 2006
- Nestares, O.; Fleet, D.J. Error-in-variables likelihood functions for motion estimation Image Processing, 2003. ICIP 2003, 2003
- Farina, A. and Di Lallo, A. and Timmoneri, L. and Volpi, T. and Ristic, B. CRLB and ML for parametric estimate: new results Signal Process, 2006
- Farina, A. and Di Lallo, A. and Timmoneri, L. and Volpi, T. and Ristic, B. Estimating 3D Positions and Velocities of Projectiles from Monocular Views IEEE Trans. Pattern Anal. Mach. Intell., 2009
- Oliver Birbach and Jorg Kurlbaum and Tim Laue and Udo Frese Tracking of Ball Trajectories with a Free Moving Camera-Inertial Sensor RoboCup 2008: Robot Soccer World Cup XII Lecture Notes in Computer Science, 2009
- First aerial ball defense in the world against Tribots, available at http://www.lsa.isep.ipp.pt/movies/iseporto_tdp11d.mp4.
- Silva,H and Almeida,J and Lima,L and Martins,A and Silva,E.P A Real Time Vision System for Autonomous Systems: Characterization during a Middle Size Match RoboCup 2007 Robot Soccer World Cup XI Lecture Notes in Computer Science, 2008
- Andreas Koch, Adam Berthelot, Bernd Eckstein, Oliver Zweigle, Kai Haussermann, Uwe-Philipp Kappeler, Andreas Tamke, Hamid Rajaie, and Paul Levi. Advanced data logging in robocup. In Dillmann, Beyerer, Stiller, and Zollner, editors, Proceedings of the AMS 2009, Autonome Mobile Systeme, Karlsruhe, Germany, December 3-4, 2009, Informatik Aktuell, pages 1 – 8. Springer Verlag, December 2009.