View Original PDF

Rhoban Football Club – Extended abstract Humanoid Kid-Size League, Robocup 2023 France

J. Allali, M. Daniel, K. Deflesselle, W. Dequiret, C. Dobigeon, M. Duclusaud, C. Gaspard, L. Gondry, O. Ly, S. N'Guyen, G. Passault, A. Pirrone

CNRS, LaBRI, University of Bordeaux and Bordeaux INP, 33405 Talence, FRANCE

Abstract This extended abstract presents some current ongoing work of the Rhoban team for the RoboCup 2025.

1 Footsteps planning improvement

Our footsteps are currently planned using a custom Reinforcement Learning (RL) formulation. We published this work in [3] and made the code of this planner open-source and independent from our main code to make it available as a separate software component usable by other teams.

This environment is also currently being improved to better take in account near obstacles such as opponent robots, and to embrace a more accurate specification of the possible footsteps that the robot is able to take.

2 Actuators modelisation

We developed a new modelisation of the actuators of the robot, that better takes into account the friction of the motors. This modelisation is described in [1]. This modelisation allows a better simulation of the robot, which is particularly useful when using RL. Indeed, the simulation is used to train the RL agent, and a better simulation can help to minimize the sim-to-real gap, and thus improve the transferability of the RL agent on the real robot.

3 Stand up strategy

The previous stand up strategy was based on a key-frame based approach, where the robot was supposed to reach a sequence of key poses to stand up. This approach was not robust enough, and we developed a new strategy based on an RL approach. This approach, called FRASA, is described in [2].

We are currently working on the integration of this stand up strategy to the playing pipeline of the robots.

4 Adding RL in the walk

We aim to improve the robustness of the walk by using RL. The walk is currently planned using a custom optimization algorithm, but we believe that RL can provide a more robust walk, especially in the presence of perturbations.

One approach we are currently exploring is to use a RL agent to modify the position and duration of the footsteps in order to make the walk more robust. This approach is still in its early stages, but it has the potential to add robustness without requiring changes to the entire walk planning pipeline.

A second approach is to use RL to learn an end-to-end walk. This approach is more ambitious, but could provide a more robust walk, as the whole walk is learned in a single step.

5 End-to-end localization

The current localization approach relies on features detection (using YOLOv8[4] network) that are fed to a custom particle filter. The annotation process is however tedious, and the features we annotate are mere proxies to the real quantity that we want to estimate: the pose of the camera on the field.

Thanks to laboratory ground truth (in our case, HTC Vive trackers), we are able to auto-label images and efficiently build datasets where the camera pose is known. We created a first dataset containing 15k such images, and are exploring the possibility of training a end-to-end neural network for camera pose estimation for the case of RoboCup soccer fields.

6 Experimenting new robot design

We are currently working on a new robot design with torque controllable motors. The main idea is to use a mix of harmonic drive and quasi-direct-drive motors to increase the torque and speed of the actuators. We also plan to use new transmission systems such as timing belts and linkages to reduce the inertia of the robot's end effectors.

References

  1. Marc Duclusaud, Grégoire Passault, Vincent Padois, and Olivier Ly. Extended friction models for the physics simulation of servo actuators, 2024.
  2. Clément Gaspard, Marc Duclusaud, Grégoire Passault, Mélodie Daniel, and Olivier Ly. Frasa: An end-to-end reinforcement learning agent for fall recovery and stand up of humanoid robots, 2024.
  3. Clément Gaspard, Grégoire Passault, Mélodie Daniel, and Olivier Ly. Footstepnet: an efficient actor-critic method for fast on-line bipedal footstep planning and forecasting, 2024.
  4. Ultralytics. Github repository : Ultralytics/ultralytics: Yolov8, December 2023.