View Original PDF

Team-KMUTT: Team Description Paper

Kongkiet Rothomphiwat, Chattarin Rodpon, Nasrun Hayeeyama, Thirawat Chuthong, Wisanu Jutharee, Thavida Maneewarn

King Mongkut's University of Technology Thonburi Institute of Field Robotics (FIBO)

Abstract This paper describes the design and development of the kidsized humanoid robots of the Team-KMUTT team for RoboCup 2019 Humanoid League (Sydney, Australia). This paper will include system overview, vision module and next goal for development. We also discuss about our improvement in color segmentation, ball detection and introducing landmark detection for self-localization.

Introduction

Institute of Field Robotics(FIBO) at King Mongkut's University of Technology Thonburi (KMUTT) has developed the humanoid robot for paticipate humanoid kid-sized league under the name 'Team KMUTT'. In 2012, Our team under name 'Hanuman-FC' won the Thailand Humanoid Soccer Robot Championship. We participated World-Robocup 2013 (Eindhoven, Netherlands), and 2014 (Joo Pessoa, Brazil). At World-Robocup 2013, we passed into quarter-final. Our humanoid team development has been suspended since Thailand had no longer host the local league. In 2017, Thailand hosted Robocup Asia-Pacific 2017. It's our chance to participate humanoid league after 3 years break. Many new undergraduate member alongside some former members managed to win the 3rd place at the Robocup Asia-Pacific 2017. Our major development this year focuses on the new software framework and the new vision system. We expect to improve on the localization capability which our team did not have much success in the past. This year, we are looking forward to participate in Robocup 2019 (Sydney, Australia).

System overview

Our robots hardware does not change much since Robocup Asia-Pacific in 2017 which consist of 5 robots comprised of 2 sizes; 3 with 57 cm tall, and 2 with 47 cm as shown in Fig 1. Each robot have mechanical hardware, sensors, computing hardware and 20 Dynamixel servo-motor. The structure of all robots is made of aluminum alloy sheet metal(details are provided in the robot specification sheet).

All robots use 6 DOF IMU sensor (MPU- 6500) which contains a 3-axis gyroscope and a 3-axis accelerometer. The combination of IMU sensor used to adapt walking stability and detect falling state of robot. The Logitech camera installed. The computing system consist of 2 separate systems; high-level and low-level as we described in [4]. High-level computation used DROID-C1+ with ARM Cortex-A5 (1.5Ghz quad core CPUs) for computing image processing, decision making and navigation. Low-level computation used STM32F411RE micro-controller for operating locomotion system.

Fig. 1: Robots of Hanuman-KMUTT Team
Fig. 1: Robots of Hanuman-KMUTT Team
Fig. 2: System diagram of the computing system in the Hanuman-KMUTT Team
Fig. 2: System diagram of the computing system in the Hanuman-KMUTT Team

Vision based navigation system

Since camera is the only sensor our robot have for detecting object on the field, various image processing algorithm will be used to identify many type of objects on the field. Currently balls, field lines and field boundary can be detected as describe below. The position of objects in three dimensional coordinate can be determined from an image according to a principle of a pinhole camera model. Moreover, information outside of the soccer field can be used as addition landmarks for determining the direction toward the opponent goal. We found a method which has a potential for detecting objects around the soccer field which can be used as a landmark. However, this method is still at the experimental stage at the moment.

Position Determination

Three dimensional position of an object ($x, y, z$) can be estimated when the object is identified in an image. The necessary information consists of a selected pixel on image coordinate ($u, v$) that belong to the object, forward kinematic from a robot base to a camera and camera's properties. Any objects on the world can be projected on image plane via perspective transform as described in equation 1. We can also project object on image plane into another plane (in our case, ground is the projected plane) which gives us a three dimensional coordinate respect to camera. Coordinate of objects in robot frame can be obtained by transforming camera coordinate to robot coordinate using forward kinematic.

$$\frac{1}{w} \begin{bmatrix} u \ v \ 1 \end{bmatrix} = \begin{bmatrix} f_x & 0 & 0 \ 0 & f_y & 0 \ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x \ y \ z \end{bmatrix}$$

Color Segmentation

Each pixels in an image will be classified into eight colors (green, black, orange, blue, yellow, magenta, cyan and white). We use HSV color space for classifying color of each pixel based on their value. Then, the watershed segmentation [2] is applied to get a better result on color segmentation.

Fig. 3: color segmentation (black color represent unknown region, gray color represent black region)
Fig. 3: color segmentation (black color represent unknown region, gray color represent black region)

Field Boundary Detection

An information about field boundary is important for our soccer robot. This information can be used later on other image processing algorithm. We detect a field boundary by scanning an image from top to bottom and find a first green pixel for each columns of an image. Then, apply median filter to filter out some noise. We also detect an outliers by cut out points which has the position on y axis higher than 1.5 of standard deviation.

Fig. 4: Example of field boundary detection
Fig. 4: Example of field boundary detection

Ball Detection

From the competition in Robocup asia pacific 2017 we have tried using HAAR feature [5] for ball detection. A lot of false positive detections have occurred, so we have to remove the un-related background by ignoring all objects outside a field boundary. The approach for ball detection has been slightly changed as followed.

    1. As the ball contains white component mostly, the white color segmentation from 3.2 has been used. Ramer—Douglas—Peucker [3] algorithm was used to classify how well the circle shape the object is. And then the region of the interests (ROI) of the detected objects would be used for consideration.
    1. The ROI(s) from previous step were checked with well-trained HAAR cascade classifier (using adaboost). Because we have limited the ROI(s), the scanning is faster than before.
Fig. 5: Example of ball detection
Fig. 5: Example of ball detection

Line Detection

Line information should be useful for localization in future work. To detecting a line, we scan an image inside a field boundary from top to bottom every 40 image columns and find points where color is changes from green to white. Three dimensional coordinate of those point can be obtained as we mention in 3.1. Those points on a field line can be treated like a point cloud from laser scanner and be used by a particle filter.

Fig. 6: Example of line detection
Fig. 6: Example of line detection

Landmark Detection

From a new rule of Robocup humanoid soccer robot, there is no explicit landmark on the field to indicate direction toward our own goal or the opponent goal. We wonder if any landmark outside a field can be detected by a robot inside the field. These landmarks will be an important information for a robot to recognize a direction of itself.

Fully-Convolution Siamese Network is a kind of neural network which capable to track an arbitrary object. This algorithm was proposed in [1]. This network will look at example image of an object once and capable of tracking that object for an entire image sequence. We can tweak tracking algorithm and also some network architecture to make it more suitable for our problem. At this state, this concept of our landmark detection is still an experimental. Further work will be done to prove that this algorithm can be available for our robot.

Fig. 7: Demo of landmark tracking with fully-convolution siamese network
Fig. 7: Demo of landmark tracking with fully-convolution siamese network

Future Work

(no text)

Localization

Localization is important process in humanoid soccer. Robot cannot make decision correctly without knowing where it is. Hence, our goal of development for the next 6 months is to make localization system available for our robots. Object detection such as field boundary detection, lines detection and a new concept about landmark detection will be combined which will allow us to be closer to our goal of vision based localization.

Particle filter is one of interesting algorithm for localization in humanoid soccer robot since map of a field is known before hand. Our information from line detection will probably enough for this algorithm. A problem that we expect is symmetry of field which can cause robot to be confused about it heading direction. This problem will probably be handled by giving a robot an initial state or landmark detection if it's proven to be suitable for our problem.

References

  1. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fullyconvolutional siamese networks for object tracking. CoRR abs/1606.09549 (2016), http://arxiv.org/abs/1606.09549
  2. Beucher, S.: Image segmentation and mathematical morphology. http://cmm.ensmp.fr/ beucher/wtshed.html, accessed: 2018-12-02
  3. Ramer, U.: An iterative procedure for the polygonal approximation of plane curves. Computer Graphics and Image Processing 1(3), 244 – 256 (1972). https://doi.org/https://doi.org/10.1016/S0146-664X(72)80017-0, http://www.sciencedirect.com/science/article/pii/S0146664X72800170
  4. Suppakun, N., Hayeeyama, N., Rodpon, C., Jutharee, W., Maneewan, T.: Hanumankmutt: Team description paper. Robocup Asia Pacific (2017)
  5. Viola, P., Jones, M.: Robust real-time object detection. In: International Journal of Computer Vision (2001)