RoboDragons 2024 Extended Team Description

Shosei Fujita, Masahide Ito

School of Information Science and Technology, Aichi Prefectural University 1522-3 Ibaragabasama, Nagakute, Aichi 480-1198, JAPAN

https://robodragons.github.io/

Abstract RoboDragons are a team of the RoboCupSoccer Small Size League from Aichi Prefectural University, Japan. This paper shares two technical topics with respect to our system updates between 2023 and 2024: one is brief introduction of our new robots and another is experimental evaluation of a depth estimator based on on-board camera images.

1 Introduction

RoboDragons are a team of Aichi Prefectural University (APU) participating in the Small Size League (SSL) of RoboCupSoccer. This team originated from Owaribito—a joint team between APU and Chubu University—which was founded in 1997. In 2002, since two universities have been ready to manage each individual team, APU built a new team, RoboDragons. After that, RoboDragons have been participating in the SSL for more than 19 years including activities as CMRoboDragons—a joint team with Carnegie Mellon University in 2004 and 2005. Our best record was the second place in 2009. We also finished thrice in the third place (2007, 2014, and 2022) and four times in the fourth place (2004, 2005, 2013, and 2016). In RoboCup 2023 (Bordeaux, France), we placed fifth out of seven teams in Division A.

This paper provides the technical information about system updates that RoboDragons have tried between 2023 and 2024. Section 2 briefly introduces our new robots; Section 3 discusses whether or not a depth estimator based on on-board camera images can be used in the SSL through experimental evaluation.

2 Eighth-generation RoboDragons Robot

Since RoboCup 2017 (Nagoya, Japan), RoboDragons had used the seventh-generation (7G) robots [1] for twelve domestic and international competitions while adding some modification on the hardware [2–5]. For RoboCup 2024 (Eindhoven, The Netherlands), the eighth-generation (8G) robots (Fig. 1) will be newly introduced. Those robots, however, are still in development, which will finish at the end of April 2024. So, the initial version of this ETDP provides only the main changes between the 7G and 8G robots as shown in Table 1. The details will be revealed at the venue of RoboCup 2024 and in our ETDP of next year.

Table 1: Main changes between the 7G and 8G robots

3 Experimental Evaluation of Depth Observer

The SSL-Vision system is one of the unique characteristics in the SSL. This plays a vital role in the sense of providing global vision to detect the positions of the game objects—a ball and robots—on the field. However, the system is frequently affected by optical noises. Also, it occasionally fails to track when, for example, the objects are hidden from the field-of-view of the global cameras, which may lead the robots to behave unpredictably. Therefore, alternatives/supplimentaries based on local sensors are highly demanded. Although position estimating methods based on LiDAR or depth camera are well-known, those would not be suitable for the SSL robots due to limitations of the onboard computer and the fast-paced game. On the other hand, from the perspective of control theory, dynamical problems to estimate/reconstruct three-dimensional positions from the image sequences have been considered. In particular, an essential problem is depth estimation, and De Luca, et al. [6] propose a depth observer as a kind of soft sensor. To discuss the effectiveness of such a soft sensor in the SSL, this section evaluates the depth observer proposed by [6] through experimental results.

3.1 Brief Summary of Depth Observer [6]

Figure 2 shows the geometric relation between the camera frame $\Sigma_c$ and its image plane. Similarly to [6], assume that a target object does not move. Then, the motion of its coordinates in a moving frame associated with a pin-hole camera and the dynamics of its projected point on the image plane are described as follows:

$$\begin{bmatrix} \dot{X} \ \dot{Y} \ \dot{Z} \end{bmatrix} = \begin{bmatrix} -1 & 0 & 0 & 0 & -Z & Y \ 0 & -1 & 0 & Z & 0 & -X \ 0 & 0 & -1 & -Y & X & 0 \end{bmatrix} \begin{bmatrix} \boldsymbol{v}_c \ \boldsymbol{\omega}_c \end{bmatrix}, \tag{1}$$

$$\begin{bmatrix} \dot{f}{u} \ \dot{f}{v} \end{bmatrix} = \begin{bmatrix} -\frac{\lambda_{u}}{Z} & 0 & \frac{f_{u}}{Z} & \frac{f_{u}f_{v}}{\lambda_{v}} & -\left(\lambda_{u} + \frac{f_{u}^{2}}{\lambda_{u}}\right) & \frac{\lambda_{u}}{\lambda_{v}} f_{v} \ 0 & -\frac{\lambda_{v}}{Z} & \frac{f_{v}}{Z} & \lambda_{v} + \frac{f_{v}^{2}}{\lambda_{v}} & -\frac{f_{u}f_{v}}{\lambda_{u}} & -\frac{\lambda_{v}}{\lambda_{u}} f_{u} \end{bmatrix} \begin{bmatrix} \boldsymbol{v}{c} \ \boldsymbol{\omega}{c} \end{bmatrix},$$ (2)

where X, Y, Z represent the target position, $v_c := [v_x, v_y, v_z]^\top$, $\omega_c$ are the camera linear and angular velocities in the moving frame, $f_u$, $f_v$ means the coordinates of the projected point in the image plane, and $\lambda_u$, $\lambda_v$ are focal length in pixel, respectively.

Letting $\tilde{\boldsymbol{x}} = [f_u, f_v, 1/Z]^{\top}$ be the state vector and $\boldsymbol{u} = [\boldsymbol{v}_c^{\top}, \boldsymbol{\omega}_c^{\top}]^{\top}$ be the input vector, Eqs. (1) and (2) can be reduced to the following state-space representation:

$$\dot{\tilde{\boldsymbol{x}}} = \begin{bmatrix} -\lambda_u \tilde{x}_3 & 0 & \tilde{x}_1 \tilde{x}_3 & \frac{\tilde{x}_1 \tilde{x}_2}{\lambda_v} & -(\lambda_u + \frac{\tilde{x}_1^2}{\lambda_u}) & \frac{\lambda_u}{\lambda_v} \tilde{x}_2 \ 0 & -\lambda_v \tilde{x}_3 & \tilde{x}_2 \tilde{x}_3 & \lambda_v + \frac{\tilde{x}_2^2}{\lambda_v} & -\frac{\tilde{x}_1 \tilde{x}_2}{\lambda_u} & -\frac{\lambda_v}{\lambda_u} \tilde{x}_1 \ 0 & 0 & \tilde{x}_3^2 & \frac{\tilde{x}_2 \tilde{x}_3}{\lambda} & -\frac{\tilde{x}_1 \tilde{x}_3}{\lambda_u} & 0 \end{bmatrix} \boldsymbol{u}, \tag{3a}$$

$$\boldsymbol{y} = \begin{bmatrix} \tilde{x}_1 \ \tilde{x}_2 \end{bmatrix}. \tag{3b}$$

Note that the tracking target is supposed to be always in the field-of-view of the camera, so that $\tilde{x}_3$ can be defined (otherwise, the camera failed to track).

For the problem to estimates the state $\tilde{x}$, especially $\tilde{x}_3$ (= 1/Z), from the system (3), De Luca, et al. have proposed the following nonlinear observer:

$$\dot{\hat{x}} = \alpha(\hat{x}, y)u + \beta(\hat{x}, y, u) \tag{4}$$

with

$$\boldsymbol{\alpha}(\hat{\boldsymbol{x}}, \boldsymbol{y}) = \begin{bmatrix} -\lambda_{u}\hat{x}{3} & 0 & y{1}\hat{x}{3} & \frac{y{1}y_{2}}{\lambda_{v}} - \left(\lambda_{u} + \frac{y_{1}^{2}}{\lambda_{u}}\right) & \frac{\lambda_{u}}{\lambda_{v}}y_{2} \ 0 & -\lambda_{v}\hat{x}{3} & y{2}\hat{x}{3} & \lambda{v} + \frac{y_{2}^{2}}{\lambda_{v}} & -\frac{y_{1}y_{2}}{\lambda_{u}} & -\frac{\lambda_{v}}{\lambda_{u}}y_{1} \ 0 & 0 & \hat{x}{3}^{2} & \frac{y{2}\hat{x}{3}}{\lambda{v}} & -\frac{y_{1}x_{3}}{\lambda_{u}} & 0 \end{bmatrix},$$ (5)

$$\boldsymbol{\beta}(\hat{\boldsymbol{x}}, \boldsymbol{y}, \boldsymbol{u}) = \begin{bmatrix} k_{1}(y_{1} - \hat{x}{1}) \ k{2}(y_{2} - \hat{x}{2}) \ k{3}{(-\lambda_{u}u_{1} + y_{1}u_{3})(y_{1} - \hat{x}{1}) + (-\lambda{v}u_{2} + y_{2}u_{3})(y_{2} - \hat{x}_{2})} \end{bmatrix},$$ (6)

where $\hat{x}$ is the estimation of the state $\tilde{x}$, and $k_i$, i = 1, 2, 3 is the observer gain, respectively. Letting $e = \tilde{x} - \hat{x}$ be the estimation error vector, Reference [6] has proved a proposition that $\lim_{t\to\infty} |e| = 0$ holds if $|u_1(t)|, |u_2(t)|$ are bounded and $u_3 \equiv u_4 \equiv u_5 \equiv 0$, from the properties of persistency of excitation.

Experimental Evaluation

Experimental Setup In the experimental setup, a local vision system is constructed by using NVIDIA Jetson Nano Developer Kit and Intel RealSense D415 Camera. The main specification of each item is as follows:

NVIDIA Jetson Nano Developer Kit

CPU: Quad-core ARM A57 @ 1.43 GHz

GPU: 128-core Maxwell

RAM: 4 GB 64-bit LPDDR4 25.6 GB/s

Intel Realsense D415 (RGB color sensor)

Maximum resolution: $1920 \times 1080$ pixels

Maximum frame rate: 30 fps

Field of view: $69 \times 42^{\circ}$

In the experiment, the local vision system works at the following procedure:

• 1. The color sensor of Intel Realsense D415 camera captures images;
• 2. Jetson nano extracts the coordinates of the target feature point from the images; and
• 3. Jetson nano sends the extracted information to the host PC.

4 Concluding Remarks

This paper has shared the technical information of RoboDragons 2023–2024. The contents are divided into two parts: the first is a brief introduction of the 8G robots and the second is experimental evaluation of a depth estimator based on on-board camera images.

Acknowledgement

This work was partially supported by JSPS KAKENHI Grant Number 23K11338 and Aichi Prefectural University Alumni Association. The authors would like to thank S. Yamashita, H. Matsubara, and R. Shigaki (e-Valley, Co. Ltd., Japan) for their technical support. The authors also would like to thank the other active RoboDragons 2023–2024 members, Agatsuma, S., Ban, K., Shibata, M., Sugiura, K., Suzuki, T., Shimizu, T., Honobu, K., Miyazaki, Y., Mori, K., Iguchi, K., Kato, A., Nakayama, R., Nomura, H., Fujita, H., Yuge, H., Asano, T., Kusakabe, M., Saijo, N., Suzuki, M., Takemi, S., Tachi, K., Tanaka, K., Terao, R., and Wakayama, T. for their support.

References

1. Adachi, Y., Kusakabe, H., Suzuki, R., Du, J., Ito, M., and Naruse, T.: "RoboDragons 2017 extended team description," RoboCup Soccer Small Size League, 2017. Available online: https://ssl.robocup.org/wp-content/uploads/2019/01/2017_ETDP_RoboDragons.pdf (accessed on 13 Feb 2024).
2. Ito, M., Kusakabe, H., Adachi, Y., Suzuki, R., Du, J., Ando, Y., Izawa, Y., Isokawa, S., Kato, T., and Naruse, T.: "RoboDragons 2018 extended team description," RoboCup Soccer Small Size League, 2018. Available online: https://ssl.robocup.org/wp-content/uploads/2019/01/2018_ETDP_RoboDragons.pdf (accessed on 13 Feb 2024).
3. Ito, M., Suzuki, R., Isokawa, S., Du, J., Suzuki, R., Nakayama, M., Ando, Y., Umeda, Y., Ono, Y., Kashiwamori, F., Kishi, F., Ban, K., Yamada, T., Adachi, Y., and Naruse, T.: "RoboDragons 2019 extended team Description," RoboCup Soccer Small Size League, 2019. Available online: https://ssl.robocup.org/wp-content/uploads/2019/03/2019_ETDP_RoboDragons.pdf (accessed on 13 Feb 2024).
4. Ito, M., Nakayama, M., Ando, Y., Adachi, Y., Du, J., Suzuki, R., Ono, Y., Kashiwamori, F., Ban, K., Isokawa, S., Yamada, T., and Naruse, T.: "RoboDragons 2020 extended team Description," RoboCup Soccer Small Size League, 2020. Available online: https://ssl.robocup.org/wp-content/uploads/2020/03/2020_ETDP_RoboDragons.pdf (accessed on 13 Feb 2024).
5. Ito, M., Shibata, M., Agatsuma, S., Ando, Y., and Fujita, S.: "RoboDragons 2023 extended team Description," RoboCup Soccer Small Size League, 2023. Available online: https://ssl.robocup.org/wp-content/uploads/2024/02/rd-etdp23fin-v2-5.pdf (accessed on 1 Apr 2024).
6. De Luca, A., Oriolo, G., and Giordano, P.: "Feature Depth Observation for Image-based Visual Servoing: Theory and Experiments," The International Journal of Robotics Research, Vol. 27, No. 10, pp. 1093–1116, 2008. https://doi.org/10.1177/0278364908096706