NuBot Team Description Paper 2010

Wentao Yu, Huimin Lu, Shengcai Lu, Shaowu Yang, Peng Dong, Shaoke Qian, Dengke Zhu, Shuai Tang, Zhiwen Zeng, Hui Zhang, Xun Li, Zhiqiang Zheng

College of Mechatronics Engineering and Automation, National University of Defense Technology, China, 410073

Abstract This paper mainly presents the developments of our middle-size league robot team "NuBot" for RoboCup 2010. A new robot platform, active ball handing system, electromagnetic shooting system, and the electromagnetic shielding method are developed in hardware; novel ball velocity estimation, camera parameter auto-adjustment, and reinforcement learning in behavior control, and flexible obstacle avoiding method based on blank regions are proposed in software algorithms.

1 Introduction

The middle-size league competition of RoboCup which obtains more and more attention provides a standard test-bed where many technologies of robotics and artificial intelligence can be examined and integrated.

NuBot (Fig.1) is the RoboCup Middle Size League team of National University of Defense Technology. This team was founded in 2004. Since 2006 we have continuously participated in four World RoboCup competitions and entered into the top 8 in recent three years. We have also participated in RoboCup China Open and won the 1st-place from 2006 to 2008 and the 3rd-place in 2009. Now our research focuses are on multi-robot cooperation, robust robot vision, robot control, etc.

This paper describes the recent developments of NuBot comparing to those presented in the former TDP [1, 2, 3]. The developments involve new robot platform, new active ball handling system, new electromagnetic shooting system and electromagnetic shielding method, ball velocity estimation, camera parameter autoadjustment, reinforcement learning in behavior control, and flexible obstacle avoiding based on blank regions.

2 New Robot Platform and Active Ball Handling System

After RoboCup 2009 Graz, we have developed a totally new robot platform, as shown in Fig.1 and Fig.3. The motion ability especially the acceleration can be improved greatly comparing to our former vehicles. We also design a new omnidirectional wheel, as shown in Fig.2. This wheel can provide more friction than our former wheels. More details can be found in our mechanical and electrical description materials.

The active ball handling system, which is designed for dribbling the ball, is made up of the active ball handling mechanism and its control system. The ball handling mechanism is shown in Fig.3.

The main parts of our active ball handling mechanism are wheels (one at each side), DC motors, axis, spring and the parts which used to limit the rotation angle of the axis. When the robot is dribbling, the spring keeps the pressure between the wheels and the ball, and the wheels driven by the DC motors provide frictions to make the wheels keep in touch with the ball. The control architecture has two levels. On the lower level, the DSP controls the velocity of the wheels. On the high level, the PC decides when the active ball handling system works. Comparing with the passive ball handing mechanism used before, the active ball handling mechanism has three advantages: firstly, it makes the opportunity to drive the ball not only forward, but also at any direction, which makes the path planning easier. Secondly, the robot could stop the ball at any place, which is useful when the robot is dribbling near the corner and sideline. Thirdly, the robot is capable to grab the ball from opponent robot with the mechanism. We will further our research on the following items: ameliorating the mechanism, improving its dependability, enhancing the adaptability to the dynamic environment, and optimizing path planning based on active ball handing system.

3 Electromagnet Shooting System and Electromagnet Shielding

The electromagnet shooting system (Fig.4) consists of four parts: the electromagnet, the electromagnet core, the capacitor and the shooting bat. Initially, the electromagnet core is rearward located within electromagnet. When the electromagnet is electrified, electromagnet will produce relatively strong electromagnetic field under which the electromagnet core will be pushed forward and then strike the shooting bat to crash the ball. After shooting, the core is pulled backward to its original position by the elastic stripe.

It is necessary to shield the electromagnetic field outside the electromagnet which can affect the communication between the computer and the DSP. An electromagnetic shielding shell is an ideal structure to prevent the electromagnetic field diffusing around the electromagnet. The shell must be made by the kind of material which is of high magnetic conductivity and does not tend to be magnetized. Our experiments prove that the electromagnet shielding is very effective. The electromagnetic field intensity around the shell is less than 10% of the level when there is not the electromagnet shell and the interruption caused by the electromagnetic field has been eliminated.

4 Ball Velocity Estimation Algorithm

We develop a novel method to estimate the velocity of the ball. The method is based on Kalman filter and RANSAC algorithm. Firstly we restore several cycles of the ball position and use Kalman filter to optimize the positions. Then we calculate all the possible velocity between every two cycles. After that we randomly choose several possible velocities and calculate the average velocity as a possible velocity model. We build hundreds of models and finally apply RANSAC algorithm to calculate the best velocity model as the final ball velocity. RANSAC algorithm can effectively eliminate the outliers, so when the ball position information is inaccurate, our method still can estimate the velocity accurately and detects the variation of the velocity faster. The Fig.5 shows the ball velocity estimation results by Least Squares Method (LSM) [4] and RANSAC algorithm.

5 Auto-Adjustment Algorithm of Camera Parameters

The final goal of RoboCup is that robot soccer team defeats human champion, so robots will have to be able to play competition in highly dynamic light conditions even in outdoor environment sooner or later. So designing robust vision system to recognize color-coded objects is a research focus in RoboCup community. Besides adaptive color segmentation methods [5], color online learning algorithms [6], and object recognition methods independent on color information [7], several researchers also have tried to adjust the camera's image acquisition parameters [8, 9, 10, 11] to make the outputting image describe the real scene as consistently as possible, which makes vision system work robustly under dynamic light conditions.

We develop a novel camera parameters auto-adjusting method based on image entropy [12]. Firstly we present the definition of image entropy, and use image entropy as optimizing goal for the optimization problem of camera parameters after verifying that image entropy can indicate whether the camera parameters are properly set by experiments. Then a searching path is defined in the parameter space to search the maximal image entropy, and the camera parameters corresponding to the maximal image entropy are the best ones for the vision system in current environment and current light condition. This method makes robot vision system adaptive to different light condition. The algorithm is tested using our omnidirectional vision system [13] in indoor MSL environment and outdoor RoboCup-like environment, and the results show that this algorithm is effective and color constancy to some extent can be achieved. The results of an experiment in indoor environment are shown in Fig. 6 and Fig. 7. The outputting image and the processing result with un-optimized camera parameters of changed light condition are shown in Fig. 6. It is obvious that the image is over-exposed and the processing result is unacceptable for robot vision. After the parameters have been optimized, as shown in Fig. 7, the image is well-exposed and the processing result is also good. Our algorithm has been applied in perspective vision and extended to adjust more parameters like iris etc [14].

6 Obstacle Avoiding Method Based on Blank Regions

Avoiding opponents' block and reaching the target point quickly is one of the basic behaviors of the robot. Global optimization method, tangent method, potential field methods are the most common obstacle avoiding methods. The MSL robots have to avoid obstacles in the highly dynamic and noisy environment, but global optimization method requires more time for optimization, while tangent method generally applies to the static environment. So various kinds of improved potential field methods are used to avoid obstacles and generate robot's trajectory, so the vibration of potential field methods can be avoided and the impact of the noise can be reduced.

While dribbling the ball, the main purpose of obstacle avoiding is to utilize flexibility to get rid of the opponents quickly. NuBot brings up a flexible obstacle avoiding method based on the blank regions. This method utilizes the information of the blank regions detected by vision algorithms to calculate the potential function, and the robots can move to the most suitable blank area by searching the direction with the least potential. This method is less sensitive to sensor noise, and is suitable for flexible obstacle avoiding. Further more, it can take full advantage of robots' motion property to get rid of the opponent by accelerating. The sketch of obstacle avoiding is shown in Fig. 8.

7 Application of Reinforcement Learning in Behavior Control

Nowadays different reinforcement learning techniques have been applied successfully in RoboCup simulation league. However it is difficult to apply it to real robots such as Middle Size League robots. Because of real robots' execution delay, perception noise and stability and so on, it is a big challenge to apply the reinforcement learning to them. By now, the applications of reinforcement learning in Middle Size League are chiefly in three levels: motor control, robot basic behavior control and learning the multi-robots' strategy.

NuBot primarily studies the application of reinforcement learning in the behavior control level, and proposes a reinforcement learning method that introduces linear interpolation based on Kuhn triangulation to reduce the state space in the CMAC network and the heuristic function is also used to accelerate the convergence of algorithm [15]. The simulation results show that reinforcement learning algorithm based on the improved CMAC neural network can significantly reduce storage expenses and improve the performance of the algorithm in the case of the same storage capacity. In the physical experiment, learning efficiency is notably increased with the combination of heuristic information. After dozens of learning, the robots can find the approximate optimal strategy and the rate of successful ball intercepting can exceed 70%.

8 Current Research Focus

Our current main research focuses are listed as follows:

Robust robot vision: The final goal of RoboCup is that the soccer robot team defeats human champion, so robots will have to be able to play outdoors and get rid of the color-coded environment sooner or later. We will go on developing our robot vision system to make that the robot can work well in the environment with highly dynamic illumination and even in totally new field without any off-line calibration. We are also doing further research on the arbitrary FIFA ball recognition method by our omni-directional vision system based on the developments we have achieved [13, 16]. We are also interested in recognize and distinguish the robots belonging to different teams and other generic objects by using the advanced pattern recognition techniques.

Multi-robot cooperation: Multi-robot cooperation holds an important place in distributed AI and robotics field. We have designed a good multi-robot cooperation mechanism and also realized several two-robot cooperative behaviors. Now we have to do deeper research to develop our robot's cooperation ability by involving more robots and more complex cooperative behaviors in this mechanism.

Multi-robot cooperative perception: Multi-robot cooperative perception is another important research focus in multi-robot system. As the MSL environment becoming more and more complex, and the field larger and larger, for the field of view of robot is limited, the occlusions of important objects such as ball is very common in dynamic competition, and the inconsistent world model of every robot brought by perception noise will also affect the team's strategy. So we are interested in building the accurate and consistent world model of multi-robot team by cooperative perception such as cooperative object localization and robot's cooperative localization.

9 Conclusion

This paper describes the current developments of the NuBot team. Comparing to what presented in the former TDP, several major improvements have been achieved as follows: new robot platform, new active ball handling system, new electromagnetic shooting system and electromagnetic shielding method in robot hardware; novel ball velocity estimation algorithm, camera parameters auto-adjustment algorithm, reinforcement learning in behavior control, and flexible obstacle avoiding based on blank regions in robot software. Our current research focuses are also presented finally.

Acknowledgement

We would like to thank Dan Hai, Xiangke Wang, Junhao Xiao, Yong Li, Ying Qu, Lin Liu, Fei Liu, Xiucai Ji, Yupeng Liu and Wei Liu for their cooperation to establish and develop our RoboCup MSL soccer robot team-NuBot.

References

1. Zhang, H., Lu, H., Ji, X., et al.: NuBot Team Description Paper 2007. RoboCup 2007 Atlanta, CD-ROM (2007).
2. Zhang, H., Lu, H., Wang, X., et al.: NuBot Team Description Paper 2008. RoboCup 2008 Suzhou, CD-ROM (2008).
3. Zhang, H., Wang, X., Lu, H., et al.: NuBot Team Description Paper 2009. RoboCup 2009 Graz, CD-ROM (2009).
4. Lauer, M., Lange, S., and Riedmiller, M.: Modeling moving objects in a dynamically changing robot application. KI 2005: Advances in Artificial Intelligence, pp. 291-303, 2005.
5. Gönner, C., Rous, M., and Kraiss, K.: Real-Time Adaptive Colour Segmentation for the RoboCup Middle Size League. RoboCup 2004: Robot Soccer World Cup VIII, pp. 402-409 (2005).
6. Anzani, F., Bosisio, D., et al.: On-Line Color Calibration in Non-Stationary Environments. RoboCup 2005: Robot Soccer World Cup IX, pp. 396-407 (2006).
7. Treptow, A., Zell, A.: Real-time object tracking for soccer-robots without color information. Robotics and Autonomous Systems 48, pp. 41-48 (2004).
8. Grillo, E., Matteucci, M., and Sorrenti, D.G.: Getting the most from your color camera in a color-coded world. RoboCup 2004: Robot Soccer World Cup VIII, pp. 221-235 (2005).
9. Takahashi, Y., Nowak, W., and Wisspeintner, T.: Adaptive Recognition of Color-Coded Objects in Indoor and Outdoor Environments. RoboCup 2007: Robot Soccer World Cup XI, pp. 65-76 (2008).
10. Lunenburg, J.J.M., Ven, G.V.D.: Tech United Team Description. In: RoboCup 2008 Suzhou, CD-ROM (2008).
11. Neves, A.J.R., Cunha, B., et al., Aotonomous configuration of parameters in robotic digital cameras, In: the 4th Iberian Conf. on Pattern Recognition and Image Analysis, ibPRIA 2009 (2009).
12. Lu, H., Zhang, H., et al.: A Novel Camera Parameters Auto-Adjusting Method Based on Image Entropy. In: Proceedings of RoboCup 2009 Symposium (2009).
13. Lu, H., Zhang, H., et al.: Arbitrary Ball Recognition Based on Omni-directional Vision for Soccer Robots. RoboCup 2008: Robot Soccer World Cup XII, pp. 133-144 (2009).
14. Lu, H., Zhang, H., et al.: Camera Parameters Auto-Adjusting Technique for Robust Robot Vision. Accepted by 2010 IEEE International Conference on Robotics and Automation, Anchorage, Alaska, USA (2010).
15. Sun, F., Zhang, H., et al. Efficient Learning on the Real Robot with Heuristic information. In: Proc. of 3rd International Workshop on Intelligent Robotics (IROBOT 2008) (2008).
16. Lu, H., Zhang, H., et al.: Vision-Based Ball Recognition for Soccer Robots without Color Classification. In: Proceedings of the 2009 IEEE International Conference on Information and Automation, pp. 916-921 (2009).