SPQR SPL Team

Prof. Daniele Nardi, Prof. Luca Iocchi, Vincenzo Suriani, Emanuele Musumeci, Graziano Specchi, Francesco Petri, Daniele Affinita, Flavio Maiorana, Elisa Santini, Valerio Spagnoli, Flavio Volpi, Prof. Domenico D. Bloisi, Prof. Fabrizio Caccavale, Prof. Francesco Pierri, Monica Sileo, Francesco Laus, Michele Brienza, Rocchina Romano

Department of Computer, Control, and Management Engineering "Antonio Ruberti", Sapienza University of Rome, Italy; University of Basilicata, Italy

http://spqr.diag.uniroma1.it · https://sites.google.com/unibas.it/wolves · github.com/SPQRTeam/SPQRBallPerceptor · https://sites.google.com/diag.uniroma1.it/robocupcoach · https://github.com/unibas-wolves/MARIO · https://www.youtube.com/channel/UCRboLHM75uGB4TQH7s1APUg · https://it-it.facebook.com/SPQRTeam · https://www.instagram.com/spqrteam/ · http://www.dis.uniroma1.it/~labrococo/?q=node/459 · https://sites.google.com/unibas.it/crowdsounddataset · https://drive.google.com/drive/folders/1wY9Xsz3O_gYc4BbGb4p_gALotynjcH-E

Abstract SPQR is the group of the Department of Computer, Control, and Management Engineering "Antonio Ruberti" at Sapienza University of Rome (Italy), participating in the RoboCup Standard Platform League since 2008. Since RoboCup 2022, the robot soccer team of the University of Basilicata (named UniBas Wolves) joined SPQR Team to participate in official competitions. For RoboCup 2023, SPQR is adopting the B-Human 2021 framework with modifications in perception, coordination, and decision making. Key contributions include a machine learning-based Ball Perceptor for robust outdoor play, a unified YOLO-based vision recognition system, planning and learning-based behaviors with reinforcement learning, and team behavior conditioning through natural language commands.

Team Description Paper 2023

1 Team Information

Team name: SPQR Team

Website: http://spqr.diag.uniroma1.it https://sites.google.com/unibas.it/wolves

SPQR is the group of the Department of Computer, Control, and Management Engineering "Antonio Ruberti" at Sapienza University of Rome (Italy). The SPQR Team has been involved in RoboCup competitions since 1998 in different leagues: Middle-size 1998-2002, Four-legged 2000-2007, Real rescue robots 2003-2006, @Home in 2006, Virtual-rescue since 2006 and Standard Platform League (SPL) since 2008. SPQR team members have served the RoboCup organization in several ways: Prof. Daniele Nardi served as Exec, Trustee, President of RoboCup Federation from 2012 to 2014 and was RoboCup Symposium co-chair in 2004; Prof. Luca Iocchi is Exec member of RoboCup@Home, Trustee and was RoboCup Symposium co-chair in 2008. SPQR team members published a total of 22 papers in RoboCup Symposia (including best paper awards in 2006, 2015 [2], and 2021 [11]), in addition to other RoboCup-related publications in International Journals and Conferences in AI and Robotics (including IROS RoboCup Best Paper Award in 2016 [4]).

Since RoboCup 2022, the robot soccer team of the University of Basilicata (named UniBas Wolves) joined SPQR Team to participate in official competitions.

1.1 Joint Team Members

Sapienza University of Rome:

• Faculty: Prof. Daniele Nardi (Team Leader), Prof. Luca Iocchi
• Members: Vincenzo Suriani (Team Leader), Emanuele Musumeci, Graziano Specchi, Francesco Petri, Daniele Affinita, Flavio Maiorana, Elisa Santini, Valerio Spagnoli, Flavio Volpi.

UniBas:

• Faculty: Prof. Domenico D. Bloisi, Prof. Fabrizio Caccavale, Prof. Francesco Pierri
• Members: Monica Sileo, Francesco Laus, Michele Brienza, Rocchina Romano.

Currently, Sapienza owns six NAO V6 robots and UniBas ordered two additional NAO V6 robots to complete the team.

2 Code Usage

Before 2013, SPQR used its own framework, called OpenRDK[1]. From 2013 onward, SPQR Team has chosen the B-Human Team framework as base for developing its code. We acknowledge the B-Human team members for their great contribution and work in the SPL league. Since RoboCup 2019 up to RoboCup 2022, SPQR Team adopted the B-Human 2018 framework, widely modified to be adapted for the V6 NAO robot hardware and low level software. For RoboCup 2023, SPQR is going to adopt the B-Human 2021 framework, modified in the following areas: perception, coordination, and decision making.

3 Own Contributions

We provide our own contributions related to the following critical areas:

3.1 Robot vision

Our goal is to create a perception tier separated from the B-Human frameworks. To this end, from 2017, SPQR Team is using its own Ball Perceptor, a newly developed machine learning approach inspired by our deep learning work on NAOs [1]. This allows us to play outdoors without color and camera setting calibrations [5, 7]. We forked the B-Human 2016 repository adding our Ball Perceptor code (github.com/SPQRTeam/SPQRBallPerceptor). Two teams (from China and Brazil) asked us to use our Ball Perceptor and at least another team (from Holland) tested our approach. In RoboCup 2019, we presented a supervised approach to detect also robots and gestures [10] by using the NAO V6 Hardware. Such an approach has been presented during the Open Research Challenge 2019 by deploying the gesture communication to a corner kick situation. A further step in creating our own perception tier is going to be part of our codebase in RoboCup 2023, since we are incorporating a pose-detection system for NAO robots and human referees.

Unified Yolo-Based Vision Recognition System

The images taken from the cameras are the main source of information for computing the world's state. The frames are processed in order to detect field lines, opponents, teammates, generic obstacles, and the ball. At the time being, the images are processed through different pipelines that rely also on different technologies like neural networks or classic computer vision approaches. We are currently exploring the possibility of a single unified recognition system, based on one of the most recent versions of YOLO[13], capable of recognizing all the entities in which we are interested in a single forward pass in the network. The main challenge is to prune the Yolo architecture in order to make it lighter and usable in the real-time environment, considering the computational limits of the NAO's hardware.

3.2 Behavior Architecture

Coordination

To create effective modeling of the world, we developed an algorithm that has been presented at the 2016 RoboCup Symposium being awarded as IROS RoboCup Best Paper 2016 [4]. By exploitation of the high-level information about game-related situations, we enforce specific behaviors as responses to environmental stimuli. To this end, distributed task assignment and distributed world modeling are combined ensuring higher robustness.

In 2016, we presented a method based on a combination of Monte Carlo search and data aggregation (MCSDA) [3] that adapts discrete action soccer policies of our robots to the opponents' strategy. A supervised learning algorithm is trained over an initial collection of data consisting of several simulations of human expert policies. Then, Monte Carlo policy rollouts are generated and aggregated to previous data, improving the learned policy over multiple epochs and games.

Planning and Learning based behaviors

Since 2017, we decided to adapt our framework also to allow the possibility of having some behaviors completely based on a planning system. In fact, having several agents acting in a real and dynamic environment requires multiple behavior configurations that are difficult to encode as fixed behaviors. Thus, we choose to have a Monitoring Replanning algorithm to evaluate the best next action by predicting the one (or the list of) with the best outcome and replanning when the environmental state changes. It is possible to define different utilities, heuristics and goals, thus to use the presented planning system over a heterogeneous set of agents. Since 2019, this approach has been extended to be used as a base for efficient Reinforcement Learning procedures for soccer robots. The Monitor Replanning algorithm has been used to lead the exploration during the training of a Deep Neural Networks for RL. This kind of method is suitable both for Actor-Critic algorithm and for Temporal Difference systems. Actually, the solutions applied so far are based on Deep Q Networks. These networks have been used for several behavior applications such as soccer contrasts or shooting decisions. More details about the Planning and Learning integration for RoboCup behaviors can be found in [12]

Team Behavior Conditioning

Although it has not been used in competition yet, in the last RoboCup Symposium, we laid the foundation for working on a higher level of abstraction in the decision-making process that can condition the strategies of a robot team through the use of intelligible commands [15]. It uses a modular architecture that is easy to adapt to different purposes and teams. Furthermore, the use of hard and soft constraints also allows for adapting the commands given to different areas, such as robot security, allowing to model strategies that can ensure the safety of both the robot and any human operators working in contact with the robot itself. In the future, it would be interesting to extend this work to create a system capable of automatically learning a domain from natural language, for example, dynamically modifying the behaviors of robots from the RoboCup regulation of the current year. The work has been publicly released at https://sites.google.com/diag.uniroma1.it/robocupcoach

In conclusion, this work is a first step towards using and learning new forms of interaction and conditioning between natural language and robot behavior. This allows the creation of new strategies to generalize and deal dynamically with unexpected and complex situations such as those that the RoboCup environment can create.

Network

Our current network management approach is a hybrid between a slow periodic update and an event-driven system that immediately notifies the team under specific circumstances, such as seeing the ball after it had been lost. The periodic update may have a different sending rate per robot depending on their conditions. We are also working to further refine our networking modules in order to reduce the information exchange between robots. For RoboCup 2023, we are developing an approach to evaluate the novelty of the information sent.

3.3 SPQR Team Research Roadmap

SPQR Team is interested in detaching the robot perception system from the RoboCup field peculiarities. To this end, we started with a ball perceptor that does not rely on ball and field colors and we will keep removing the constraints of the software.

4 Past History

SPQR Team joined the RoboCup competitions in 1998. The following tables contain our results in RoboCup competitions from 2019 onward.

SPQR Results at RoboCup 2019, (Main competition)

*SPQR defeat Bembelbots in a shoot out to determine the winner of the pool

SPQR Results at RoboCup 2019 (Main competition) - Champions Play-in and Champions second RR

SPQR Results at RoboCup 2021 (Worldwide) challenges

*SPQR got to the 7th place in the overall ranking.

SPQR Results at RoboCup 2021 (Worldwide) "1vs1" Challenge

SPQR Results at RoboCup 2022

SPQR Results at RoboCup 2022 challenges

*SPQR achieved 1st place (with B-Human) in the Open Research Challenge and got to the 4th place in the overall ranking.

5 Impact

Impact in SPL/RoboCup Community

The Ro.Co.Co. (Cognitive Cooperating Robots)[2] laboratory has been participating in the RoboCup since the beginning of the SPL. The aim is to transfer our research in machine learning, behavior formalization and coordination in the RoboCup competition and to contribute in the develop of a more reliable soccer team in the pursuing of the goals of the league. In 2016, we have created the SPQR Team NAO Image Data set for helping other groups in creating deep learning based perception methods. In 2017, we proposed a supervised method for detecting the realistic black and white ball in images captured by a NAO robot. The approach proved to be robust to changes in the lighting conditions. In 2017, two teams (from China and Brazil) asked us to use the SPQR Ball Perceptor. In 2019, with the adoption of the new robotic platform, i.e. the V6 NAO robot, Starkit team from Russia have been involved in the competition by using the Code Base released by SPQR. In 2021 we introduced the concept of audio exploitation for capturing the crowd sentiment. Our work has been awarded as Best paper in 2021 RoboCup Symposium.

In 2022, we presented MARIO[14] a fully-automatic system specifically designed for analysing NAO soccer robot matches. MARIO ranked first, ex-aequo with the B-Human Team's system, in the Open Research Challenge at RoboCup 2022. Robot and ball tracking in MARIO are done automatically. Game analysis can extract trajectories, passes made, and heatmaps through graphs and tables containing both traditional statistics and more advanced statistics within the field, such as falls and foul actions made by the robots. In particular, we describe two supervised algorithms included in MARIO. The first is devoted to the calculation of the poses of the NAO robots, while the second aims at detecting falling down events. The code is publicly released at https://github.com/unibas-wolves/MARIO.

Impact in University/Community

Our University strongly supports our work in RoboCup competitions, which are an excellent testbed for validating our research results. The Petri Net Plans (PNP) framework has become, in our laboratory, the standard tool for robot behavior design and formalization, after to the work done in RoboCup experience. In the last years, we started exploiting our knowledge on vision and dynamic walking engine to better govern the NAO platform and deploy NAO robots in other applications.

We are promoting research in AI and Robotics through several types of media channels to disseminate our research results. In order to pursue this goal, we have a YouTube Channel[3], a Facebook page[4], and an Instagram profile[5] rich in contents about RoboCup. This effort is pursued also by participating in Italian Tv shows ("I Fatti Vostri", "Laudato sii'", "Tg2 insieme") and in relevant exhibitions that take place in Italy (IAB Forum, Wired Next Fest, Blue Fest, Unirete, RomeCup, MakerFaire). In one of these exhibitions, the Maker Faire 2019 European Edition, we hosted a friendly tournament in Rome involving other two SPL teams (i.e., HTWK and NomadZ).

In another edition of the same exhibition, the Maker Faire 2021 European Edition, we hosted another friendly tournament in Rome involving a team in person, the HULKs from Hamburg. Moreover, we hosted remotely NaoDevils and B-Human teams. In 2022, during Maker Faire 2022, we had, in person, the NaoDevils. A video of the event on our Instagram page reached 30K views.

6 Other

SPQR Nao Image Dataset

As mentioned in Section 2, SPQR Team has adopted its own ball perceptor. The detector is based on a supervised approach and consists of an LBP based cascade classifier trained on real images from the SPQR Team NAO Image Data set. In Robocup Symposium 2016, we presented a novel approach for object detection and classification based on Convolutional Neural Networks (CNN)[1]. Quantitative experiments have been conducted on a data set of annotated images captured in real conditions from NAO robots in action. In Symposium 2019, we presented a hand-to-hand gesture signal communication [10] trained by creating another outdoor data set. Part of all the used data sets was made available for the community. They can be downloaded at http://www.dis.uniroma1.it/~labrococo/?q=node/459.

SPQR RoboCup@Soccer Sound Dataset

In 2019 SPQRTeam used online available videos of the RoboCup finals to gather the audio from the crowd. To this end, we collected the video of the SPL finals of RoboCup competitions from 2016 to 2019 and GermanOpen 2019. These videos have been analyzed to find out intervals of interest. In this analysis, considering the moves of the robots in the videos, we labeled the intervals with three values, i.e., Highlights, Potential goal moves, and Goal moves. All these labeled situations have been publicly released and can be found at: https://sites.google.com/unibas.it/crowdsounddataset

The inspection of the audio files allowed us to recognize the waveform pattern in a goal situation and then we used this signal as a reward for a reinforcement learning agent that had improved a previous policy as illustrated in [11].

UNIBAS NAO Pose Dataset

NAO are humanoid robots, however there are some differences between the NAO structure and the human one. Thus, we have created a specific dataset for detecting the pose of a NAO robot. In particular, we collected video frames from the RoboCup Standard Platform League (SPL) teams. Using the COCO Annotator tool [11], we labeled 451 frames containing about 3,000 NAO robot instances in the well-known COCO format. All annotations share the following data structure. The pose is represented by up to 18 key points describing ears, eyes, nose, neck, shoulders, elbows, wrists, hips, knees, and ankles. The annotations are stored using a JSON structure. UNIBAS NAO Pose Dataset is available at https://drive.google.com/drive/folders/1wY9Xsz3O_gYc4BbGb4p_gALotynjcH-E.

References

[1] D. Albani, A. Youssef, V. Suriani, D. Nardi, D. D. Bloisi. A Deep Learning Approach for Object Recognition with NAO Soccer Robots. RoboCup 2016: Robot World Cup XX pages 392-403 [2] G. Gemignani, M. Veloso, D. Nardi. Language-based sensing descriptors for robot object grounding. 19th Annual RoboCup International Symposium, 2015. Science Best Paper Award. [3] F. Riccio, R. Capobianco, D. Nardi. Using Monte Carlo Search With Data Aggregation to Improve Robot Soccer Policies. Proceedings of the 20th International RoboCup Symposium, 2016. [4] F. Riccio, E. Borzi, G. Gemignani, and D. Nardi. Multi-Robot Search for a Moving Target: Integrating World Modeling, Task Assignment and Context, 2016. International Conference on Intelligent Robots and Systems (IROS'16). IROS-RoboCup Best Paper Award. [5] D. D. Bloisi, F. Del Duchetto, T. Manoni, V. Suriani. Machine Learning for RealisticBall Detection in RoboCup SPL. arXiv:1707.03628 [6] F. Previtali, D. D. Bloisi, L. Iocchi. A distributed approach for real-time multi-camera multiple object tracking - Machine Vision and Applications, pages 1-10, 2017. [7] F. Previtali and G. Gemignani and L. Iocchi and D. Nardi. Disambiguating Localization Symmetry through a Multi-Clustered Particle Filtering, 2015, IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pages 283-288. [8] E. Borzi. Multi-robot coordination under unreliable network conditions. 2015, Master thesis. [9] A. Pennisi, D. D. Bloisi, L. Iocchi and D. Nardi Ground Truth Acquisition of Humanoid Soccer Robot Behaviour, RoboCup 2013: Robot World Cup XVII pp 560-567 [10] V. Di Giambattista, M. Fawakherji, V. Suriani, D. D. Bloisi, and D. Nardi On Field Gesture-Based Robot-to-Robot Communication with NAO Soccer Players Robot World Cup, 2019 [11] E. Antonioni, V. Suriani, and F. Solimando, D.D. Bloisi, D. Nardi Learning from the crowd: Improving the decision-making process in robot soccer using the audience noise, Robot World Cup, 2021 [12] Antonioni Emanuele, Riccio Francesco, and Nardi Daniele Improving Sample Efficiency in Behavior Learning by Using Sub-optimal Planners for Robots. [13] Wang, Chien-Yao and Bochkovskiy, Alexey and Liao, Hong-Yuan Mark Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. [14] D. D. Bloisi, A. Pennisi, C. Zampino, F. Biancospino, F. Laus, G. Di Stefano, M. Brienza, R. Romano Mario: Modular and extensible architecture for computing visual statistics in RoboCup SPL, 2022. URL: https://arxiv.org/abs/2209.09987. doi:10.48550/ARXIV.2209.09987. [15] E. Musumeci, V. Suriani, E. Antonioni, D. Nardi, and D. D. Bloisi Adaptive Team Behavior Planning using Human Coach Commands. Robot World Cup, 2022