NomadZ Call For Participation 2023

Giuliano Albanese, Filippo Spinelli, Arka Mitra, Zichong Li

ETH Zurich, Rämistrasse 101, 8092 Zürich, Switzerland; Automatic Control Laboratory, Physikstrasse 3, 8092 Zürich, Switzerland; Computer Vision Lab, Sternwartstrasse 7, 8092 Zürich, Switzerland; Center for Project-Based Learning D-ITET, Gloriastrasse 35, 8092 Zürich, Switzerland

https://robocup.ethz.ch/ · https://github.com/nomadz-ethz/nomadz-code-release · https://github.com/nomadz-ethz/spl-player-tracking-release · https://robocup.ethz.ch/projects/reports/ · https://swissroboticsday.ch/previous-editions/previous-editions-2022/

Abstract NomadZ is a team in the RoboCup Standard Platform League affiliated with ETH Zürich. For RoboCup 2023, we intend to participate in the regular team competition, Champions Cup division, and some of the technical challenges. Thanks to our performance at RoboCup 2022, we pre-qualified for this year. Our robot soccer software is based on the 2013 B-Human code release, which we extended with our own perception, behavior and motion control modules. We are currently transitioning to a new framework based on ROS2 and developed completely by us. This response to the call for participation includes a team presentation, previous results, a summary of the team's impact, and a description of its current research activities.

1 Team information

We are team NomadZ from ETH Zürich. The team was founded in 2012 by the Computer Vision Lab (CVL) and the Automatic Control Laboratory (IfA) of the Department for Information Technology and Electrical Engineering. In 2021, the Center for Project-Based Learning (PBL) also became an official partner of the project. Our team is composed of four student assistants who lead ten Master's students. Prof. Luc van Gool (CVL), Prof. John Lygeros (IfA) and Dr. Michele Magno (PBL) officially head the project. Academic and research projects are managed by scientific supervisors: Jan-Nico Zäch and Dr. Ajad Chhatkuli from CVL, Alexandros Tanzanakis and Dr. Raffaele Soloperto from IfA, Dr. Seonyeong Heo and Dr. Luca Pascarella from PBL.

In the academic years 2020-2023, the team had 5 master students who carried out their semester projects (280 hours of research work) on RoboCup related topics, and 3 bachelor students who completed their thesis (up to 400 hours of individual work). The rest of the team has been working on several improvements to the NomadZ robot soccer framework. In addition, a few alumni members of the team are still participating in development and tutoring activities. Our team currently has 11 NAO V6, one more will be purchased by RC23. The robots are used during competitions as well as for public events.

At Robocup 2023, we plan to participate in the soccer tournament, Champions Cup, and some of the technical challenges.

2 Code Usage

Our robot soccer software is based on B-Human's 2013 code release. We have then integrated BHuman 2018's implementation of the Walk2014 by rUNSWift. In the last years we have been developing new functionalities that we plan to use in the competition. Specifically, a ROS2 framework that will lead to partial/complete substitution of the current modules and a walking engine based on a more dynamic gait planner. Our code is publicly available on GitHub.

3 Own Contributions

We have extended the 2013 B-Human code release with our own implementation of several behavior, motion control and perception modules. For example:

• Behavior | Implemented a variety of state machines to improve defensive and offensive gameplay, field positioning, ball assignment and ball search by leveraging shared information. Investigated the use of non-linear optimization techniques and data-driven methods for optimal strategies.
• Perception | Investigated classical computer vision techniques and more recent ones (such as random forests and CNN) for the detection of objects and field features. Developed algorithms for long-term player tracking in video recordings of SPL matches as part of the ORC of RoboCup 2022.
• Control | New walk and kick controllers, state estimation via Extended Kalman Filters and Particle Filters.

In the last 2 years we have been developing a ROS2-based framework, aiming to simplify the development process by leveraging upon the available literature and software. We are implementing a new simulation based on RaiSim to increase the physical reliability. Our most recent contributions:

– ROS2-based robot soccer framework | We decided to restructure our robot soccer framework using ROS, a widely popular collection of open source packages for robot software development actively maintained by a large community. Specifically, we are rewriting our existing modules on top of a new core framework based on the latest version of ROS 2, which unlike ROS 1 has been designed with real-time performance constraints in mind. We believe that this transition will significantly increase our development speed, as ROS 2 encourages modularity and isolation between components by design, and makes it simpler for newcomers to contribute.

• Walking Engine | Our old walking engine makes the implementation of new in-walk features a tedious and time-consuming task. For example, it does not allow the behavior to command a local step pattern, making it impossible to control each foot directly. Therefore, the walking module has been expanded with a Pose2D step planner which allows the execution of various ball-handling motions and dribble. We can command both the straight inwalk kick as well as a diagonal in-walk kick which utilizes the inner side of the feet, more accurately due to the flat geometric design. These will be the foundation of cooperative gameplay. The module also allows for more walk configurations, step height, step width or the generation of intra-walk phase trajectories. Besides the previous existing normal modes, the step planner provides the walking engine with the optimal step patterns for the ballapproaching maneuver, which replaces the previous heuristics.
• Kick Engine | We have developed a novel omnidirectional kick engine that generates a kick trajectory that can propel the ball in any intended direction at a commanded speed. The trajectories are generated using Bézier curves. The robot should remain stable during the kick and be able to compensate the external disturbance forces; this is achieved by using a ZMP based preview controller in combination with the proportional arms controller. We have provided validation results in simulation on a variety of possible scenarios. We are now focusing on improving its stability.
• Object detection with deep neural networks | Instead of using two separate models to detect balls and robots, we are replacing our object detection module with a single neural network which can jointly detect instances of both classes. Our current model is based on the SSD meta-architecture and is already capable of robustly detecting robots under different lighting conditions. We are currently working on enhancing its detection performance and robustness to motion blur for the ball class. We have also developed a separate model for semantic line detection and we are working on integrating it as a separate head of the above mentioned detection network. Nevertheless, this feature is still in its experimental phase and might not make into the version of our framework that we will deploy at RoboCup 2023.
• Activity Recognition | For the Visual Referee challenge, we developed a pipeline able to detect human pose and motion. The current implementation uses 3D CNN with Convolution LSTM, which is inspired by the paper for gesture recognition and has a validation accuracy of 96% on our custom dataset. We created a video dataset using gestures from different members of the team and the university. In order to make the model background-independent and to make it more robust to visual disturbances, we used a green screen and changed the backgrounds in post-processing. The final dataset consists of 1409 samples, with each sample being a sequence of 15 frames. The resulting model is quite accurate and robust to people in the background, it does not require the referee to wear specific close and to be standing at the T junction. Our method ranked second among all the teams in the competition. For this year's challenge, we plan to expand the dataset so that the robot can detect the referee from any position in the field.
• Open Research Challenge | Our goal for the Open Research Challenge at RoboCup 2022 was to develop a pipeline able to calculate camera parameters from the camera feed and to track and identify all the players on the field. This information could then be used to compute game statistics. To achieve this goal we adopted a sensor fusion approach based on non-linear optimization. Player tracklets are first generated using Tracktor. The assignment of these short-term tracklets to players is done by solving a non-linear integer program. The cost terms are computed based on the jersey color of each player tracklet, which is predicted by an ad-hoc CNN, and the player self-localization and game event data from the TeamCom and GameController logs. Our method is able to correctly assign the bounding box to the correct robot 98% of the time and is robust to occlusions. We plan to improve the assignment technique by using graph neural networks. The source code for this project is available on GitHub.
• Whistle Detection | Previously, the whistle detection module used thresholding on the volume of sound to identify when the whistle has been blown, feasible approach only in a very limited setting. To tackle additional complexity, we trained a neural network model to identify when the whistle has been blown. The method takes the Short Term Fourier Transform of the sounds which it hears and converts it to a spectogram. The spectogram is then passed into a custom model which identifies whether the sound is a whistle or not. The present version is much more robust and less prone to false positives.
• Distributed Robotic Swarms Coordination | The transition from 5vs5 to 7vs7, looking towards the 11vs11, has pointed out the fallacies of our current behavior framework, which heavily depends on hard-coded information. We have therefore started the transition from a state-machine based framework to a distributed optimization one. The first step is to provide an algorithm able to jointly optimize over the position of our robots accounting for shared and local information flawlessly. Getting inspiration from gametheoretic self-organization theory and leveraging upon the novel Feedback Equilibrium Seeking algorithm, we aim to provide a global position planner to correctly steer our robots during different match conditions. This is achieved by defining generalized equations which can model a broad spectrum of useful objectives, accounting for the local information each robot can extract from the environment.

4 Past History

In 2014 we participated at our first events, the German Open in Magdeburg, the Night of Science in Frankfurt and the RoHOW in Hamburg. We continued in 2015 with the Iran and German Open, and the drop-in player competition and technical challenges at RoboCup 2015. In 2016 the European Open in Eindhoven and RoboCup 2016 were attended, followed by the German Open in 2017, the RoboCup event and the Phoenix Contact Robotics Cup. In 2018, we attended the German Open and the RoboCup. In 2019, we attended the German Open, the RoboCup, the Night of Science in Frankfurt and the Makerfaire in Rome. We qualified for Robocup 2020 in Bordeaux, but due to the Covid pandemic it wasn't held. In 2021, due to persistent Covid restrictions, we only participated in the remote Robocup 2021. In 2022 we joined the RoboCup 2022 in Bangkok, also competing in 2 of the 4 technical challenges.

Results history

Results from RoboCup competitions.

2022 Results

– 2022

We weren't able to joint the GORE and we directly participated in the RoboCup22 in Bangkok. We joined the soccer competition and 2 challenges: the Visual Referee Challenge and the Open-Research Challenge. We achieved the 2 nd position in the Visual Referee Challenge collecting 22.5 points, the 3 rd position in the Open Research Challenge for 21.9 points, overall ending as 3 rd team. Our results in the soccer competition, despite some initial problems, allowed us to pass the Seeding round as 8 th team overall. We then lost the quarter-finals match againts B-Human.

– 2021

We participated in the 1vs1 challenge, starting from a Round Robin group stage with HTWK Robots and Berlin United - NaoTH. HTWK got a 1.5 multiplier due to the autonomous calibration, process that we weren't able to implement. We played the Play-ins as 2 nd team in the group, then we reached the Quarter Finals against B-Human (also endorsed with the multiplier) where we ended our adventure, as 5 th team. We also participated in the Obstacle Avoidance challenge, ending in position 10. Overall, we were the 10th classified team in the Robocup 2021.

2021 Results

– 2020

This competition was cancelled due to Covid pandemic spreading, but we would have participated in the Challenge Shield where we qualified.

– 2019

We attended the German Open in Magdeburg and RoboCup in Sydney. The German Open is the yearly test run for us to try out new modules and algorithms and the usage of the new V6 robots before attending RoboCup. Our results:

2019 German Open Results

2019 RoboCup Results

At the RoboCup 2019 in July, we started in the Challenge Shield and were able to rise up to the Champions Cup Competition! There, we lost both round robin rounds and lost against the HULKs in the Champions Cup Playoffs. Nevertheless, we were satisfied with our results and it showed us that the time and effort invested by our students were worth it and allowed us to compete with the world's best teams. Overall, we were placed 9 th for the main team competition.

5 Impact

Over the years the student members of our team have worked on several research projects within the scope of RoboCup whose outcomes have been documented and made available on our website. Table 1 shows a list of the most recent ones. We believe that these might help other SPL teams with their research. In addition, we plan to release our ROS2-based robot soccer framework. This would allow an easy integration of many functionalities developed by the robotics community, as well as an easier collaboration among the SPL teams and beyond.

5.1 League

Over the years the student members of our team have worked on several research projects within the scope of RoboCup whose outcomes have been documented and made available on our website. Table 1 shows a list of the most recent ones. We believe that these might help other SPL teams with their research. In addition, we plan to release our ROS2-based robot soccer framework. This would allow an easy integration of many functionalities developed by the robotics community, as well as an easier collaboration among the SPL teams and beyond.

Table 1. Most recent projects supervised by our group

5.2 University

Over the recent years, we continuously contributed to the university's PR activities. Our most important event in the past year has been the Swiss Robotics Day. It is the most important robotics conference in Switzerland, one of the most important in Europe, and attracts companies and visitors from all over the World. There, we promoted SPL among both industry and private visitors. We are regularly approached by student associations to present our activities at open lab days and similar events. We are able to tell students about our exciting work and fascinate them with robotics demonstrations. We also played games and presented the idea behind RoboCup at the FIFA Museum Zurich Nachtaktiv 2021 event, whose subject was football and technology.

Team NomadZ provides an important platform for the student of ETH to put the theory learnt in class into practice. For the students who want to get more experience with a physical robotic platform, the team provides a great environment to experiment and to learn. Furthermore, we organize 2 classes for ETH Bachelor students. From 2020 to 2022, we have provided the lab course "Vision and Control in RoboCup". It covers all theoretical fundamentals required to successfully play and provides the students with first hands on experience using NAO. Since 2023 the class has been renamed to Robocup: Learning and Control, focusing more on the modeling and control areas. Since 2021 we also provide the course Introduction to Program Nao Robots for Robocup Competition, a more practical class where Bachelor students can code and test basic functionalities on NAO V6 robots.

6 Conclusion

We had great experiences at the RoboCup tournaments in Hefei, Leipzig, Nagoya, Montreal, Sydney, the remote tournament in 2021 and in Bangkok. We are looking to share again with the RoboCup community this exceptional event. We are hungry for more wins, and the road is still long, as we are approaching other teams' levels. Let's continue the journey with the RoboCup 2023 in Bordeaux!

Thank you for your work and for keeping Robocup the amazing competition it is. We know from our own experiences that Robocup is able to fascinate many (prospective) students to work in robotics by showing how much fun it can be!

References

1. Belgioioso, G., Liao-McPherson, D., de Badyn, M.H., Bolognani, S., Lygeros, J., Dorfler, F.: Sampled-data online feedback equilibrium seeking: Stability and tracking. In: 2021 60th IEEE Conference on Decision and Control (CDC). IEEE (dec 2021). https://doi.org/10.1109/cdc45484.2021.9683614
2. Bergmann, P., Meinhardt, T., Leal-Taixé, L.: Tracking without bells and whistles. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 941–951 (2019)
3. Hengst, B.: Robocup standard platform league (2014)
4. Hwangbo, J., Lee, J., Hutter, M.: Per-contact iteration method for solving contact dynamics. IEEE Robotics and Automation Letters 3(2), 895–902 (2018), www.raisim.com
5. Jaleel, H., Shamma, J.S.: Distributed optimization for robot networks: From realtime convex optimization to game-theoretic self-organization. Proceedings of the IEEE 108(11), 1953–1967 (2020). https://doi.org/10.1109/JPROC.2020.3028295
6. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S.E., Fu, C., Berg, A.C.: SSD: single shot multibox detector. CoRR abs/1512.02325 (2015), http://arxiv.org/abs/1512.02325
7. Röfer, T., Laue, T., Müller, J., Bartsch, M., Batram, M.J., Böckmann, A., Böschen, M., Kroker, M., Maaß, F., Münder, T., Steinbeck, M., Stolpmann, A., Taddiken, S., Tsogias, A., Wenk, F.: B-human team report and code release 2013 (2013), only available online: http://www.b-human.de/downloads/publications/2013/CodeRelease2013.pdf
8. Zhang, L., Zhu, G., Mei, L., Shen, P., Shah, S.A.A., Bennamoun: Attention in convolutional lstm for gesture recognition. In: Neural Information Processing Systems (2018)