View Original PDF

LAR@MSL Team Description Paper 2024

António Ribeiro, Carolina Lopes, Cláudio Ribeiro, Joel Costa, José Martins Paulo Oliveira, Rúben Silva, Rui Lima, Rui Martins, Rui Pereira, Tiago Oliveira, Gil Lopes, A. Fernando Ribeiro

Laboratório de Automação e Robótica (LAR) DEI - Universidade do Minho Campus de Azurém, 4800-058 Guimarães, Portugal

https://lar.dei.uminho.pt

Abstract The LAR@MSL team has been participating in MSL for many years, from 1999-2007, 2011, 2016 and again in 2023. This Team Description Paper intends to briefly explain the robot's structure, hardware and software and some of the most important changes implemented by this new team. It is important to point out that new students joined the team and are new to the league. Most of the changes, so far, have been regarding fixing the mechanical problems, some changes in the electronic system and some upgrades on the architecture and code organisation and control systems. A new vision system with a multi-camera setup has been developed, new stronger wheels and the addition of debugging LEDs to help in the development process. Regarding the software, a ground-up rebuild was carried out as the game strategy's new attempt was very different from the previous team. This paper describes a bottom-up view of the robots, the hardware used, the vision head, the low-level software and strategy and finishes with some conclusions. The whole code and the hardware description are available on a public GitHub repository.

1 Introduction

The Laboratory of Automation and Robotics belongs to the School of Engineering of the University of Minho, Guimarães, Portugal, and his first participation on RoboCup MSL was in 1999. Previously known as MinhoTeam, the LAR@MSL new name contains the research Laboratory LAR name (Laboratory of Automation and Robotics) and addresses the MSL league, as the LAR also participates on other RoboCup leagues (like @Home). This new generation of students working on the MSL team just started this academic year (back in September 2023) and intends to gather new students for the coming years to continue participating in the MSL league. The robots were completely built in the LAR along the years (mechanics, electronics, and software) and all 5 robot's code is the same, with some parameters to distinguish the goalkeeper.

2 New Omni-Wheels

The previous wheel hardware encountered several problems, making the team design and create new wheels. The new wheels are built in aluminium, featuring a two-axis omni-wheel design with a total of 22 rollers with a larger diameter of 125 mm. Additionally an external bearing system was incorporated to ensure the wheels remain parallel to the ground during movement. Using a laser cutting machine, the wheels were assembled by the team. These were firstly designed in 3D software and then manufactured. Its assembly on the robot is visible in the Figure 1. It also presents the first version of the outside bearing to support the robot's weight.

Fig. 1: Omni-Wheels: (a) One wheel assembled; (b) Wheel mounted on the robot; (c) Wheel with the outside bearing.
Fig. 1: Omni-Wheels: (a) One wheel assembled; (b) Wheel mounted on the robot; (c) Wheel with the outside bearing.

3 Debugging Tools - LED

Throughout the development process, the LAR@MSL team frequently encountered challenges related to code debugging. To address this issue, the use of LEDs was adopted as a diagnostic tool. The LEDs play a crucial role as a visualisation tool, facilitating not only the validation of the implemented strategy but also the verification of information perceived by the robot, as well as the data sent by other teammates. This visualisation mechanism through LEDs proves to be instrumental in real-time analysis and monitoring, providing an effective approach to assess the efficiency of the implemented code. Its use aims to enhance understanding of the robot's behaviour and optimise collaboration among team members, thereby contributing to the continuous improvement of the robotic system performance.

4 Movement Algorithm

The previous control system movements were not very flexible, and it did not allow path planning. In order to improve it was implemented a new movement algorithm that generates path planning with or without obstacle avoidance.

First, the field element's positions are received, and then the predicted displacement of obstacles is checked to see if there will be any possible collision. If it does not exist, the path planning is a straight line between the robot and the target. Figure 2a) represents a path without collisions.

In case of possible collisions, intermediate points are placed in relation to the intersection between the robot and obstacle trajectories. As demonstrated in Figure 2b), intermediate points are offset towards intersection points, based on predicted time movements between the robot, obstacle, and intersection points.

Through the position of the robot and target, intermediate points are made with polynomial interpolation. This polynomial function described the path planning. The final path that the robot should take is illustrated in Figure 2c).

Fig. 2: Path planning (a) With no collisions; (b) Intermediate points; (c) Final path.
Fig. 2: Path planning (a) With no collisions; (b) Intermediate points; (c) Final path.

5 Vision-System

One of the new updates for this year's version of the LAR@MSL robots is the new vision system. The entire structure was reviewed so that three depth cameras could be installed at the top, where the catadioptric system was previously installed. The cameras are positioned 120 degrees apart and tilted down 30 degrees. Figure 3 illustrates the new hardware design for the vision system.

Fig. 3: New Vision System: (a) 3D CAD; (b) 3D printed part, with all 3 cameras installed.
Fig. 3: New Vision System: (a) 3D CAD; (b) 3D printed part, with all 3 cameras installed.

Figure 4 provides a visual representation of the field of view that the robot has access to. This perspective is achieved through the merging and processing of three separate camera images, which ultimately allows the robot to more accurately perceive its surroundings. The team is currently working on algorithms to calculate an object's distance, speed, and trajectory.

Fig. 4: Concatenated image of the robot's field of view.
Fig. 4: Concatenated image of the robot's field of view.

This decision was made based on some limitations that the catadioptric system has proven to have. The main advantages of the new vision system are the distance at which objects can be seen with less distortion compared to the previous vision system and the number of available frames that can be processed to get information from the game's environment. This can be especially important for a quicker reaction time from the goalkeeper.

5.1 Distortion

The introduction of new cameras into the system also brings with it distortion in the images collected; in this case, it is a fisheye distortion, which highlights the crucial importance of implementing effective camera calibration methods.

Fig. 5: Fisheye distortion: (a) Without correction; (b) With correction.
Fig. 5: Fisheye distortion: (a) Without correction; (b) With correction.

6 Self-Localisation Method

Localisation systems on soccer robots usually rely on either a single camera or a combination of sensors, such as LiDAR (Light Detection And Ranging) or an IMU (Inertial Measurement Unit). However, these methods may not be entirely reliable as they can be affected by environmental factors like shadows, vibrations or even electromagnetic interference. Neural networks have emerged as a promising approach for localisation problems as they can learn to associate features present in camera images with the robot's position and orientation. This allows neural networks to be more robust to errors in the environment than traditional methods.

6.1 Simulator

LAR@MSL does not have access to a complete MSL field so a simulator was developed using Webots to create a realistic environment. The simulator represented in Figure 6 consists of a field, two goals, and a robot equipped with three cameras.

Fig. 6: Simulator environment and simulation elements.
Fig. 6: Simulator environment and simulation elements.

To ensure that the simulated environment is as close as possible to reality, it was necessary to determine the camera lens parameters. The intrinsic parameters include the polynomial mapping coefficients of the projection function, and these parameters were calculated through a calibration process that used a 2D chequerboard pattern. The cameras are positioned in a way that accurately reflects their positioning in the real world. The multi-camera configuration represents the extrinsic parameters, which refer to the position and orientation of the cameras concerning the robot's coordinate systems. Figure 7 presents a comparative visualisation illustrating the camera intrinsic parameter discovery process. The upper images depict camera images generated by a simulated environment, while the lower images showcase real-world images captured using an actual camera at varying distances (45 cm, 73 cm, 184 cm and 215 cm, respectively). The intrinsic parameters include the radial distortion coefficients (k1, k2, k3, k4), focal length (f) and the principal point coordinates (cx, cy).

Fig. 7: Camera intrinsic parameter discovery process. Upper images: Simulated camera images. Lower images: Real-world images captured at different distances. The developed code facilitated accurate alignment between simulated and realworld perspectives.
Fig. 7: Camera intrinsic parameter discovery process. Upper images: Simulated camera images. Lower images: Real-world images captured at different distances. The developed code facilitated accurate alignment between simulated and realworld perspectives.

After identifying the lens parameters, images from the simulator were extracted and used to create a comprehensive dataset to train the neural network. The images from the three cameras were concatenated and subsequently subjected to a binary filter, which helped get rid of any extraneous information present in the images. Figure 8 demonstrates the image processing pipeline, which involves correcting lens distortions, concatenating multiple images for panoramic view generation, and applying filtering techniques to enhance feature detection and segmentation and reduce the size, using only one colour channel instead of three.

Fig. 8: Image processing pipeline. Top row: Three raw camera images captured without any filter or lens deformation. Middle row: Lens deformation applied to the images, followed by concatenation to form a panoramic view. Bottom row: Binary filter applied to the concatenated image to enhance feature detection and segmentation.
Fig. 8: Image processing pipeline. Top row: Three raw camera images captured without any filter or lens deformation. Middle row: Lens deformation applied to the images, followed by concatenation to form a panoramic view. Bottom row: Binary filter applied to the concatenated image to enhance feature detection and segmentation.

6.2 Neural Network

It is being implemented a custom neural network architecture using the Keras framework to achieve precise robot self-localisation. The architecture, as illustrated in Figure 9, integrates convolutional and recurrent layers to extract spatial features from input images and sequential dependencies from sensor data, respectively. To ensure an accurate localisation, the network must be able to handle different types of data classes. This is necessary to achieve robustness in the system. Even though the localisation accuracy is not yet maximised, the team aims to achieve this through rigorous experiment and optimisation, without disregarding computational efficiency and generalisation capability across environments.

Fig. 9: Neural network architecture for robot self-localisation. Input data, including camera images and sensor readings, are processed through hidden layers to predict the (x, y) position of the robot within the environment.
Fig. 9: Neural network architecture for robot self-localisation. Input data, including camera images and sensor readings, are processed through hidden layers to predict the (x, y) position of the robot within the environment.

7 Conclusions

In conclusion, this paper has provided an overview of the LAR@MSL team's efforts in developing and improving their robotic system. Addressing hardware issues, such as wheel problems, was a crucial step in enhancing the robot's performance. Additionally, the team created various tools to streamline the debugging process, ensuring efficient troubleshooting. A new movement algorithm was implemented to enable the robot to navigate its environment while avoiding obstacles effectively. Furthermore, a new vision system was introduced, incorporating three cameras with a 360-degree field of view, significantly improving the robot's perception capabilities. Finally, a novel localisation method using neural networks is being developed, enhancing the robot's ability to determine its position. These advancements collectively demonstrate the team's dedication to pushing the boundaries of robotic technology and overcoming challenges through innovative solutions. The strategy being used is the Probability-Based Strategy presented in the 1. The LAR@MSL public repository is available on GitHub (https://github.com/LARobotics).

References

  1. Ribeiro, A.F.A.; Lopes, A.C.C.; Ribeiro, T.A.; Pereira, N.S.S.M.; Lopes, G.T.; Ribeiro, A.F.M. "Probability-Based Strategy for a Football Multi-Agent Autonomous Robot System". Robotics 2024, 13, 5. Available online: https://doi.org/10.3390/robotics13010005.
  2. A. Ribeiro, J. Costa, J. Martins, R. Silva, R. Lima, C. Lopes, G. Lopes, A. Fernando Ribeiro "LAR@MSL Description Paper 2023 ". Available online: https://lar.dei.uminho.pt/images/downloads/LAR@MSL_TDP%202023.pdf.
  3. António Fernando Alcântara Ribeiro, "Probability-Based Strategy for Football Cooperative Multi-Agent Autonomous Robots". Available online: https://lar.dei.uminho.pt/images/downloads/Antonio_Ribeiro.pdf.
  4. Joel Alberto Abreu Sousa da Costa, "Desenvolvimento de Robô Guarda-Redes Para a Equipa MinhoTeam" Available online: https://lar.dei.uminho.pt/images/downloads/Joel_Costa.pdf.
  5. P. Lima, A. Bonarini, C. Machado, F. M. Marchese, C. Marques, F. Ribeiro, D. G. Sorrenti, Title: "Omni-Directional Catadioptric Vision for Soccer Robots", Special Issue of the Robotics and Autonomous Systems Journal, Elsevier.
  6. Daniele Nardi, Itsuki Noda, Fernando Ribeiro, Peter Stone, Oskar von Stryk, Manuela Veloso, "RoboCup Soccer Leagues", AI MAgazine, Vol. 35, No. 3, pag.77- 85, DOI: http://dx.doi.org/10.1609/aimag.v35i3.2549, Fall Issue, 2014.
  7. Fernando Ribeiro, Ivo Moutinho, Nino Pereira, Fernando Oliveira, José Fernandes, Nuno Peixoto, Antero Salgado, "High accuracy navigation in unknown environment using adaptive control", RoboCup 2007: Robot Soccer World Cup XI", Lecture Notes on Artificial Intelligence series, volume 5001, ISSN: 0302-9743, página 312-319, Atlanta, USA, July 2007. http://dx.doi.org/10.1007/978-3-540-68847-1_30.
  8. Nino Pereira, António Fernando Ribeiro, Gil Lopes, Daniel Whitney, Jorge Lino, (2016),"Path planning towards non-compulsory multiple targets using TWIN-RRT*", Industrial Robot: An International Journal, Vol. 43 Iss 4 pp. 370-379.
  9. Fernando Ribeiro, Ivo Moutinho, Pedro Silva, Carlos Fraga, Nino Pereira, "Vision, Kinematics and Game strategy in Multi-Robot Systems like MSL RoboCup", RoboCup'2004 – Team Description Paper, RoboCup'2004.
  10. Fernando Ribeiro, Gil Lopes, João Costa, João Pedro Rodrigues, Bruno Pereira, João Silva, Sérgio Silva, Paulo Ribeiro, Paulo Trigueiros, "Minho MSL - A New Generation of soccer robots", ", RoboCup'2011 - Istambul, Team Description Paper, Julho 2011.
  11. F. Ribeiro, G. Lopes, H. Ribeiro, P. Silva, T. Maia, R. Roriz, A. Gomes and N. Ferreira, "MinhoTeam '2016: Team Description Paper", RoboCup'2016 - Leipzig, Team Description Paper, Janeiro 2016.