View Original PDF

Tsinghua Hephaestus 2025 AdultSize Extended Abstract

Xinyu Han, Qingkai Li, Yushi Wang, Penghui Chen, Hao Dong, Mingguo Zhao

Tsinghua University, Beijing 100084, China

Abstract Tsinghua Hephaestus is applying to participate in RoboCup 2025 with their Booster T1 robots for soccer competitions, building on the lessons learned from RoboCup 2024. We have successfully implemented a reinforcement learning framework to achieve fast walking with push recovery. The robots will be equipped to autonomously get up, dribble, pass, and save balls, all of which will enhance their soccer skills.

1 Lessons

Tsinghua Hephaestus returned to RoboCup2024 with Booster T1[1]. We learned three lessons during the competition.

The robot should be durable against falls. First, it is important to maintain balance in a collision during the game, and staying on the field rather than being picked up is an important factor in winning the game. The reinforcementlearning-based locomotion is the solution. Second, advantages can be gained, such as the robot not getting damaged even if it falls during the competition. The time it takes for the picked-up robot to return to the field will be drastically reduced. We do many falling tests, making it more robust in mechanical, electrical, joint, and software.

Making fast decision helps to gain an advantage in the game. Making fast decision can lead to kicking the ball earlier than your opponents, with more chances to shoot and clear the ball, without bringing about more collisions.

The robot should be able to play the game without handlers. For the ultimate goal of the RoboCup initiative, which is to achieve a team of fully autonomous humanoid robot soccer players that can win a soccer match under official FIFA rules against the WorldCup champions[2], the handler needs to be removed to take this goal a step further.

2 Major Problems

Walking with Push Recovery It is very hard to maintain balance in a collision with an adult-sized robot. The limitation of model-based walking is the insufficiently accurate modeling of impacts. Reinforcement-learning-based walking shows great potential to improve walking stability.

Autonomous Fall Recovery Robots may experience control failures or unintended falls during the competition, making autonomous fall recovery appealing. The fall detection can be realized by state estimation from IMU sensors and joint feedback. Getting up motion from any fall position can be introduced by reinforcement learning.

Offensive and defensive strategies Robots showing human-like offensive and defensive strategies in games will make for a more exciting game. On offense, the robot should always be at the optimal shooting angle. And in defense, the robot should always be in the optimal defense zone. By collecting data from humans operating robots playing soccer, human-like strategies can be trained by reinforcement learning.

3 Plans for RoboCup 2025

Changes will occur in

Gait: We will introduce reinforcement-learning-based walking to improve walking speed and enhance walking stability after a collision.

Basic Skills: The autonomous getting-up motion will be introduced in the game. And our robot can play the game without handlers.

Ball handling Skills: The striker can dribble and pass the ball in the game, while the goalkeeper can make a save in the game.

4 Current Status

We developed a reinforcement learning (RL) pipeline[3]. This pipeline features a flexible RL training toolkit based on PyTorch and Isaac Gym and a C++ deployment of the policy using Eigen. This enables us to train controllers for locomotion skills and achieve zero-shot sim-to-real transfer. We used the threephase training procedure to train a locomotion policy that achieves a walking speed of 0.8 m/s. In the final phase, we integrated the World Model Denoiser structure, adding a privileged information decoder to enhance training efficiency. Based on the framework, we realized the robot's getting-up motion from both face up and down. Moreover, we have trained a policy that robots can dribble the ball from any position on the pitch to score a goal.

References

  1. Wenhan Cai, Songrui Huang, Kang Li, Mingwei Zhang, Yuanye Wu, Cheng Cheng, Binzhi Chen, Yuesong Wang, and Mingguo Zhao, "Tsinghua Hephaestus Extended Abstract for Robocup 2024 Humanoid League". In: Humanoid League Team Descriptions, RoboCup 2024, Eindhoven (July 2024)
  2. H. Kitano and M. Asada, "The RoboCup humanoid challenge as the millennium challenge for advanced robotics," Adv. Robot., vol. 13, no. 8, pp. 723-737, 2000.
  3. Mingguo Zhao, Hao Dong, Xinyu Han and Qi Li, "Towards the ultimate goal of RoboCup with a standard humanoid robot platform," In: Workshop on Humanoid Soccer Robots, IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids), Nancy, France, November 2024.