NuBot Team Description 2021

Huimin Lu, Junhao Xiao, Zhiwen Zeng, Qinghua Yu, Kaihong Huang, Zhiqian Zhou, Pengming Zhu, Ce Guo, Haoran Ren, Daoxun Zhang, Zhiqiang Zheng, Yibo Xu, Di Zhu, Xiao Li, Anqi Zhu, Sichao Lin, Jiayang Liu, Junnian Chen

College of Intelligence Science and Technology, National University of Defense Technology, China, 410073

http://nubot.trustie.net

Abstract This paper presents the developments of our middle-size league robot team "NuBot" for RoboCup 2021. In this paper, we will briefly introduce the mechanical platform, electrical system, software of the robots, and the simulation system.

1 Introduction

The middle-size league competition of RoboCup provides a standard test-bed where many technologies of robotics and artificial intelligence can be examined and integrated, especially in a highly competitive and dynamic environment. NuBot (Fig. 1) is the RoboCup Middle Size League team of National University of Defense Technology. Our team was founded in 2004 and has participated in nine World RoboCup competitions since 2006. Throughout this period, we entered into the top 8 in 2007, 2008, 2009 and 2013, and the top 6 in 2010, 2014, and 2015. And the top 4 in 2016, 2017, 2018 and 2019. Especially, during RoboCup 2015 in China, we won the 2nd place in the technique challenge and the 3rd place in the scientific challenge. Furthermore, in 2016, we achieved the 3rd place in the scientific challenge. And both the 3rd place in the scientific challenge and technique challenge in 2017 and 2018. Last year, we won the 1st place in the technique challenge in Montreal. We have also participated in RoboCup China Open and won the 1st place from 2006 to 2008, the 3rd place in 2009, 2014, 2016 and 2017, and the 2nd place in 2010 and 2018. For the time being, our research focuses are on multi-robot cooperation, multi-robot path planning robust robot vision, robot control, multi-robot cooperative perception, etc.

2 The Mechanical System

After RoboCup 2017 in Japan, we developed a new generation of robot platform as shown in Fig. 1. We re-designed the external frame of it to better adapt to the competition environment with huge impact. And it is proved that the generation of robot has a more reasonable layout and more stable performance. The rest of this part only details the active ball handling system and shooting system.

2.1 The Active Ball Handling System

The active ball handling system, which is designed for dribbling the ball, is made up of the active ball handling mechanism and its close-loop control system.

As illustrated in Fig. 2, there are two symmetrical assemblies, and each contains a wheel, a DC motor with a right angle reducer, a linear displacement transducer and a support mechanism. The wheels are driven by the DC motor and are always pressed by the ball, therefore they can generate various frictional force to the ball, making it rotate in desired directions and speeds together with the soccer robot. During dribbling, the robot will constantly adjust the speed of the wheels to maintain a proper distance between the ball and the robot using a closed-loop control system. This control system takes the actual ball distance as the feedback signal, which is measured indirectly by the linear displacement transducers attached to the supporting mechanism. As the ball moves closer to the robot, the supporting mechanism will raise, and then stretch the transducer; otherwise, the support mechanism will fall and compress the transducer. The information obtained from two transducers can be used to calculate the actual ball distance based on a given detailed geometry model and careful calibrations. This system effectively solves the ball handling control problem.

Fig. 2. Our active ball handling system.

2.2 The Electromagnet shooting system

After 2019 RoboCup Sydney, we improve the shooting system of NuBot. The shooting system is still based on an electromagnet with a high impulsive force. As depicted in Fig. 4, it consists of a solenoid, an electromagnet core, a shooting rod, a capacitor, and a DC motor. The shooting rod can be adjusted in height to allow for two different shooting modes, namely lob shot and ground pass. In order to control the angle of the shooting more accurately, we use the screw rod mechanism to replace the previous design. Through a screw rod mechanism and belt driving mechanism, two modes can be realized by using the DC motor to pull the hinge of the shooting rod to different positions. Compared with the former system, the ground pass mode has a higher shooting rod position, resulting in a faster ground pass without jumps. The shooting procedure remains the same. Firstly, the electromagnet core is rearward located within the solenoid and the capacitor is charged. When the shooting action is activated, the rod will be adjusted according to the selected mode. Then the control circuit board will switch on the solenoid by discharging the capacitor, thus producing a strong electromagnetic force to push forward the rod. The rod then strikes the ball and delivers momentum to it. After the shooting is finished, the core will slide back to its initial position, and the capacitor will be recharged again for the next shooting action.

Fig. 2. The former(a) and current(b) electromagnet shooting system of the NuBot soccer robot.

3 The Software Based on ROS

Robot Operating System (ROS) provides a set of software libraries and tools for building robot applications across multiple computing platforms, which has many advantages: ease of use, high-efficiency, cross-platform, supporting multiple programming languages and distributed computing, code reusability. In addition, it is open source under BSD license.

We have built our software based on ROS for our new robots. The operating system is Ubuntu 16.04, and the version of ROS is kinetic. The software framework, as shown in Fig. 4, is divided into 4 main parts: the Prosilica Camera node, the OmniVision node and the Kinect node; the NuBot Control node; the NuBot HWControl node; the RTDB and the WorldModel node. As for the goalie, the software framework is the same except that there are two Kinect nodes. These nodes will be described in the following subsections.

Fig. 4. The software framework based on ROS.

3.1 The OmniVision Node

The perception is the basis to realize the autonomous ability such as motion planning, control decision and multi-robot cooperation. Omnidirectional vision is one of the most important sensors for RoboCup MSL soccer robots. The image is captured and published by the Prosilica Camera node [1]. The OmniVision node realizes color segmentation, white line-points detection, robot self-localization, obstacle and ball detection, see [2] for algorithm details.

3.2 The Kinect Node

The 3D information of the ball is of great significance, especially for the goalie robot to intercept the lob ball. However, the omnidirectional vision system cannot obtain depth information directly. Therefore, we make use of two Kinect sensors to recognize and localize the ball and estimate its moving trace in 3D space. A color segmentation algorithm is employed to obtain some candidate ball regions. Then the random sample consensus algorithm (RANSAC) [3] is used to fit the spherical model using the 3D information of these candidate ball regions. The fitted ball is shown in Fig. 5. With the proposed method, only small amounts of candidate ball regions need to be fitted. Lastly, to intercept the ball for the goalie, the 3D trajectory of the ball regarded as the parabola is estimated and the touchdown-point in 3D space is also predicted [4].

Besides, real-time and accurate robot detection and localization is also important for soccer robots. In the typical method, the black-color-based information is used to distinguish robots from the environment, which is of a low accuracy and robustness. Therefore, we propose a novel approach for robot detection and localization based on Convolutional Neural Networks. The approach is composed of two stages: robot detection using the RGB image, and robot localization using the depth point cloud. The high accuracy and mean average precision (mAP) verify that the proposed method is suitable for robot detection in MSL[5]. Then, based on GPU parallel computing employing Kinect v2 and Jetson TX1 as the hardware platform, the algorithm also satisfies the realtime requirements[6]. Besides, we build a dataset for robot detection, containing fully annotated images acquired from MSL[7].

Fig. 5. The fitted ball by using the RANSAC algorithm.

Fig. 6. Detection examples on the MSL competition.

3.3 The NuBot Control Node

On the top level of the controllers, the NuBot soccer robots typically adopt the threelayer hierarchical structure. To be specific, the NuBot Control node basically contains strategy, path planning and trajectory tacking functions.

Considering the highly dynamic competition environment, path planning and obstacle avoidance is still quite a challenge. To deal with it, an online path planning method based on the subtargets method [8] and B-spline curve is proposed. Benefiting from the proposed method, we can obtain a smooth path and realize real-time obstacle avoidance with a high speed. The method can be summarized as follows:

Generating some via-points employing the subtargets method iteratively;
Obtaining a smooth path by using B-spline curve method between via-points;
Optimizing the planning path under some actual constraints.

In fact, this method is simple yet effective. Besides, we also notice that, for the original subtargets method, the local minima problem cannot be avoided. For example, while the destination is blocked by some obstacles, the robot oscillates back and forth and cannot find a path to the destination. Our method can identify this situation accurately, deal with it by exchanging the destination and the robot's position, and obtain a smooth path to the destination, see [9] for detail.

3.4 The NuBot HWControl Node

On bottom level of the controllers, the NuBot HWControl node performs four main tasks: controlling the three motors of the base frame, obtaining odometry information, controlling ball handling system and shooting system.

The ROS EtherCAT library for our robots is developed to exchange information between the industrial PC and some actuators and sensors (e.g. AD module, I/O module, Elmo, motors, linear displacement sensors). According to the action command from NuBot Control node, the NuBot HWControl node calculates the speed control commands and then sends them to four Elmo motor controllers of the base frame. Meanwhile, the motor encoder data are used to calculate odometry information, which are then subscribed by the OmniVision node. For the third task, high control accuracy and high-stability performance are achieved by feedback plus feedforward PD control for the active ball handling system, where the relative distance between the robot and the ball measured with two linear displacement sensors is regarded as feedback signal and the robot velocity is used as the feedforward signal. The shooting system firstly needs to be calibrated off-line. During competitions, the node adjusts the hinge of the shooting rod to different heights according to the received commands: ground pass or lob shot from the NuBot Control node. Then it receives the shooting commands, selects the shooting strength according to the calibration results and then kicks the ball out.

3.5 The WorldModel Node

The real-time database tool (RTDB) developed by the CAMBADA team [10] has been employed to realize robot-to-robot communication. The information of the ball, the obstacles and the robot itself provided by the OmniVision node and the Kinect node is combined with the data communicated among teammates to acquire a unified world representation in the WorldModel node. The information from its own sensors and other robots is of great significance for single-robot motion and multi-robot cooperation. For examples, every robot fuses all obtained ball information, and only the robot with the shortest distance to the ball should catch it and others should move to appropriate positions; each robot obtains accurate positions of the obstacles and of its teammates by communication, thus it can realize accurate teammate and opponent identification, which is important for our robots to perform the close-marking defense.

4 The Simulation System

The simulation system is developed based on ROS and the open source simulator Gazebo. A brief introduction is given in this article but readers are referred to [11,12]for more details.

To integrate the real robot code with the simulation model, those nodes related directly to hardware should be replaced with model plugins. This replacement requires an appropriate communication interface. For this reason, ROS messages-passing and services-calling mechanisms are employed. Finally, the data flow of the integration of the real robot code and the model plugin is shown in Fig. 7.

The simulation system can be used for designing single robot behaviors and multirobot cooperation strategies. Furthermore, equipped with an automatic referee, it can be employed to simulate a match between two teams, as shown in Fig. 8.

Fig. 7. The data flow graph of the integration of the real robot code and model plugin.

Fig. 8. Two sides of robot models compete in a soccer game.

5 Human-Computer Interaction

Our team's recent search is focus on the human-computer interaction between our human coach and robot players. We propose a human-robot interaction framework based on BML (Battle Management Language) for multi-robot system., which can convert human language into machine language and help robot players to understand the match and make better decisions. Now, the whole procedure has been tested on our simulation system and is able to execute action-level instructions. In the next two years, we will enhance the human-robot interaction to execute strategy -level instructions, to realize more complex coach instructions and try to introduce it into a real match.

Fig. 9. (a) Tactical vocabulary - free kick. (b) Coach after deleting the No.2 robot.

6 Research Focuses

Our current main research focuses are listed as follows:

Robust robot vision: Soccer robots will have to be able to play games in outdoor environments and get rid of the color-coded environment sooner or later. We will continue to improve the robustness of omni-vision, reduce the probability of misidentification, such as identifying shadows as obstacles and improve adaptability of highly dynamic lighting conditions.
Multi-robot cooperation: Multi-robot cooperation holds an important place in distributed AI and robotics field. We are always focus on the multi-robot task allocation, we hope that robots can develop better strategies in more complex environments and more complex multi-robot collaboration situations.
Multi-robot path planning: Multi-robot path planning is an crucial ability in the highly dynamic and aggressive environment. Though we have designed a basic online path planning method, we would like to provide a more practical, effective and general solution to realize a higher level of distributed AI.

References

http://wiki.ros.org/prosilica camera.
Lu, H., Yang, S., Zhang, H., et al. A robust omnidirectional vision sensor for soccer robots. Mechatronics 21, 373-389 (2011).
Schnabel R, Wahl R, Klein R, Efficient ransac for point-cloud shape detection. In: Computer graphics forum, Wiley Online Library, vol 26, pp 214-226 (2007).
Lu H, Yu Q, Xiong D, Xiao J, Zheng Z, Object motion estimation based on hybrid vision for soccer robots in 3d space. In: RoboCup 2014 International Symposium (2014).
Luo, Sha, Lu, H., et al. Robot detection and localization based on deep learning. 2017 Chinese Automation Congress, Jinan, China(2017).
S. Luo, W. Yao, et al. Object detection based on GPU parallel computing for RoboCup Middle Size League, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), Macau, 2017, pp. 86-91.
https://github.com/Abbyls/robocup-MSL-dataset.
Bruijnen D, van Helvoort J, Van de Molengraft R, Realtime motion path generation using subtargets in a rapidly changing environment. Robotics and Autonomous Systems 55(6):470-479 (2007).
Cheng S, Xiao J, Lu H, Real-time obstacle avoidance using subtargets and cubic b-spline for mobile robots. In: 2014 IEEE International Conference on Information and Automation (ICIA), IEEE, pp 634-639 (2014).
Almeida L, Santos F, et al. Coordinating distributed autonomous agents with a real-time database: The cambada project. In: Computer and Information Sciences-ISCIS 2004, Springer, pp 876-886 (2004).
Yao, W., Dai, W., et al. A Simulation System Based on ROS and Gazebo for RoboCup Middle Size League, IEEE Conference on Robotics and Biomimetics, Zhuhai, China (2015).
Z. Zhou, W. Yao, et al. Simatch: A Simulation System for Highly Dynamic Confrontations Between Multi-Robot Systems. 2018 Chinese Automation Congress, Xi'an, China(2018).