Team Qualification Document for RoboCup 2023 Bordeaux, France - Dutch Nao Team

Wike Duivenvoorden, Lex Bolt, Derck Prinzhorn, Fyor Klein Gunnewiek, Lasse van Iterson, Hidde Lekanne gezegd Deprez, Thomas Brouwers, Gijs de Jong, Harold Ruiter, Arnoud Visser

University of Amsterdam, The Netherlands

http://www.dutchnaoteam.nl

Abstract This document serves as the team qualification document for the Dutch Nao Team participating in RoboCup 2023 in Bordeaux, France. The team consists of twenty-three bachelor students, two alumni and one staff member from the University of Amsterdam. The document details the team's technical contributions, including perception improvements through camera calibration and object modeling using Kalman filters, as well as gamecontroller updates for 7v7 play. It also covers the team's history of participation in RoboCup competitions since 2010, their various academic contributions, and their broader impact on robotics education and research.

Team Information

This is the qualification document for the Dutch Nao Team with Wike Duivenvoorden as its team leader. The team consists of twenty-three bachelor students, two alumni and one staff member from the University of Amsterdam, The Netherlands. In the last twelve years the team has bought 25 NAO robots, although not all of them are operational anymore. The team currently has six NAO V6 robots and is planning to buy one more robot, so will have seven NAO V6 in time for the robocup. The qualification video is available on our YouTube channel. A research report describing the technical details of the team's work for RoboCup 2022, has previously been published on our website.

Code Usage

From April 2017 onward, the team has been using its own framework in C++. Since the Dutch Nao Team has become substantially larger in member count, the skills of the team grew as well. Probably the most important example are the new software engineering skills in the team. We now have four experienced software engineers who all have a preference for programming in Rust instead of C++. This is mostly due to Rust being a very stable programming language, especially for a larger team, because writing and mostly maintaining the code is easier. In Rust, you have to write code explicitly rather than implicitly. In addition to this the project structure in Rust makes more sense and the package manager is also better.

So far, the team has noticed that despite the obvious drawbacks of having to recreate basic functionality, the educational value of our new framework has increased the motivation of (newer) team members and has had a positive impact on the overall productivity.

We know the new framework won't be ready until after the Robocup, so at the Robocup we're going to play with an adapted version of the 2021 HULKs code release. We would like to add communication and fall prevention to this release before the Robocup starts.

Own Contribution

This section describes the technical contributions made by the Dutch Nao Team for RoboCup 2023.

Perception

Camera Calibration In order to get the best performance out of our line and object recognition implementations, it is vital to have constant lighting conditions. However, this is not always possible to keep constant. Factors such as: time of day, different sizes of rooms and how close the robot is to a window have a significant impact to how much light reaches the robot's cameras. To help with these variations in lighting conditions the Dutch Nao Team implemented an automatic camera exposure module.

The robot's camera previously calculated the camera settings based on the entire image, which resulted in poor performance when one part of the image was over- or under exposed. Using the green field detection, ball detection and robot detection modules it is possible to weigh parts of the image differently. This gives the option to choose which parts of the image are more important to have proper lighting conditions. Instead of including areas outside of the field in the calculation, the new approach would exclude parts of the image that didn't have the field in them. It also placed a higher importance on parts of the field that had a ball or a robot on them.

Object moddeling Before this feature was added, our framework had object permanence only by remembering the last detection for a few seconds. However, with the combination of slow and fast object detection there was a need to be able to combine and denoise object detections. Specifically our YOLO object detector works on cycle of 700 milliseconds and only on the top camera, while our Haar classifier works on a cycle of 17 milliseconds and in both cameras. Because of these differences the module has two main aims, firstly; give greater importance to the YOLO detection whenever it finishes a cycle while still using the Haar classifier for movement tracking, secondly; to account for the time difference between input received and detection verified for object location. Additionally, the topic of detected robot permanence also needed to be tackled.

The goal was achieved by the use of a Kalman filter. The ball is assumed to be stationary, and observations after a certain amount of time are removed from the data. This makes sure the ball position can still change even though we do not do any velocity estimations. Additionally, the YOLO observations get a much higher importance. Since we found out that the YOLO model almost never produces a false positive, we decided to invalidate any Haar detections if they deviate too far from the YOLO detection. Hence the Haar classifier is used to update the ball position slightly. In the bottom camera there is no YOLO detection so no filtering is done for those observations.

The second goal was achieved by keeping location updates of the last second in memory. Then when the robot receives a YOLO detection, the location is recalculated 700 milliseconds into the past according to the odometry information. This recalculation is taking the current location and updates it inversely with the past odometry updates. Important to note is that this excludes any correction mechanisms such as re-localisation and the filtering of particles. Finally, robot detections are assigned to the nearest modelled robot, or if there are none close by, a new robot is created. This solution only works for general robot locations as the margins for a 700 millisecond cycle are too wide for detecting robots that are right next to each other. This requires future work.

In conclusion, an object modelling module is implemented which effectively combines object detections from two sources with different time delays and accuracies. This posed challenges but was solved with the use of a Kalman filter, custom detection filtering and time aware algorithms.

Gamecontroller

For the 2022 Robocup in Bangkok, a new technical challenge was added that introduced 7v7 play. In order for this to work, the Dutch Nao Team and all other teams had to update their framework in order to support 7v7 matches. Furthermore, a new rule was added that limits the total number of messages sent on the network to 1200 packets, and for every minute of irregular extra time, the limit is increased with another 60 packets per minute. In addition to the new rule, there were also some minor changes to the data that has to be sent to the game controller. The following new fields were added to the packets:

• uint8 t fallen indicates if a robot has fallen
• float pose pose information containing x, y and theta
• float ballAge indicates the time since the ball was last seen
• float ball contains the relative position of the ball to the robot

It is vital for the Dutch Nao Team to stay up to date with every new gamecontroller update. If the robots fail to connect to the game controller it is not possible to participate in the matches.

Past History

The predecessor of the Dutch Nao Team was the Dutch Aibo Team. The Dutch Nao Team debuted in the Standard Platform League (SPL) competition at the German Open 2010. Since their founding, the Dutch Nao Team has been qualified for the world cup competitions in Istanbul, Mexico City, Eindhoven, João Pessoa, Leipzig, Nagoya, Montreal, Sydney, Worldwide and Bangkok.

Besides the major RoboCup events, we have attended multiple GermanOpens, IranOpens, the Humanoid Soccer School 2013, the Mediterranean Open 2011, the Colombia Robotics week, Tech-Fest 2015, the European Open 2016, Rodeo 2019 and every Robotic Hamburg Open Workshop between 2016 and 2022. At the Benelux Conference on Artificial Intelligence 2016 the team received the award for best demonstration, at the Iran Open 2017 the team received the Award in the Open Challenge with a presentation on our behaviour engine.

The results from 2019 onward in major RoboCup competitions are presented in Table 1. In Sydney we were able to score twice in-game and promoted to the champions cup second round robin by beating Camellia Dragons in a penalty shootout. And after a long break due to COVID we managed to score in our first match in Bangkok.

Game scores for RoboCup 2019, 2021 and 2022.

Although not visible in the scores, the field play has improved a lot, resulting in games with a lot of ball possession. The Dutch Nao Team will come well prepared to the competition in Bordeaux: in December 2022 the Dutch Nao Team attended the RoHOW and we plan to go to the German Open Replacement Event in April 2023.

Impact

During the participation in the RoboCup, the Dutch Nao Team has provided its support or resources in a number of bachelor & master theses and projects that lead to publications on a large variety of topics. At the Maastricht University, a PhD thesis is finished based on e.g. a paper on learning a more stable gait, compared to the energy efficient gait from earlier work. Additionally side projects were done regarding ball-detection. The Dutch Nao Team extended the application of the Nao robot to the @Home league of the RoboCup: the Nao robot was used to help in a kitchen environment by finding a tomato and grabbing it from a table. Finally, the Dutch Nao Team has made the penalty shootout situation into a standalone demonstration which it premiered at the Benelux Conference on Artificial Intelligence 2016 and won the first prize for best demonstration.

Earlier the Dutch Nao Team has published papers in the International Conference on Advanced Robotics, the Performance Metrics for Intelligent Systems Workshop, the RoboCup IranOpen Symposium, the RoboCup Symposium and the international conferences as International Conference on Autonomous Robot Systems and Competitions. The Dutch Nao Team also proposed and supervised RoboCup related projects as part of a compulsory course in the Artificial Intelligence bachelor at the University of Amsterdam.

Other

For the broader community, the Dutch Nao Team continues to provide many lectures about robotics and AI, and demonstrations of autonomous football at companies and schools throughout the year. This spreads knowledge about robotics and AI, and is a way for the Dutch Nao Team to fund the trip to the RoboCup. After RoboCup 2016 a foundation was started to allow for transparent financial communication, solely for the benefit of AI and robotics research.

References

1. Lex Bolt, Fyor Klein Gunnewiek, H.L.g.D.L.v.I.D.P.: Dutch nao team - technical report. Technical report (2022)
2. Kaizer, J., gezegd Deprez, H.L., Duivenvoorden, W., Heeman, P., van der Weerd, R., Wiggers, T.: Dutch nao team - technical report. Technical report, University of Amsterdam (2022)
3. Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME, Journal of Basic Engineering 82 (1960) 35–45
4. Oomes, S., Jonker, P., Poel, M., Visser, A., Wiering, M.: Dutch aibo team at robocup 2004. In: Proceedings CD of the 8th RoboCup International Symposiums. (July 2004)
5. Visser, A., Iepsma, R., van Bellen, M., Gupta, R.K., Khalesi, B.: Dutch Nao Team - Team Description Paper - Standard Platform League - German Open 2010 (January 2010)
6. Verschoor, C., ten Velthuis, D., Wiggers, A., Cabot, M., Keune, A., Nugteren, S., van Egmond, H., van der Molen, H., Rozeboom, R., Becht, I., de Jonge, M., Pronk, R., Kooijman, C., Visser, A.: Dutch Nao Team – Team Description for RoboCup 2011 - Istanbul. In: Proceedings CD of the 15th RoboCup Symposium. (January 2011)
7. Verschoor, C., ten Velthuis, D., Wiggers, A., Cabot, M., Keune, A., Nugteren, S., van Egmond, H., van der Molen, H., Rozeboom, R., Becht, I., de Jonge, M., Pronk, R., Kooijman, C., Visser, A.: Dutch Nao Team – Team Description for RoboCup 2012 - Mexico City. In: Proceedings CD of the 16th RoboCup Symposium. (June 2012)
8. de Kok, P., Girardi, N., Gudi, A., Kooijman, C., Methenitis, G., Negrijn, S., Steenbergen, N., ten Velthuis, D., Verschoor, C., Wiggers, A., Visser, A.: Team Description for RoboCup 2013 in Eindhoven, the Netherlands. Proceedings of the 17th RoboCup International Symposium (May 2013)
9. de Kok, P., ten Velthuis, D., Backer, N., van Eck, J., Voorter, F., Visser, A., Thomas, J., Delgado Lopes, G., Ras, G., Roos, N.: Dutch Nao Team team description for RoboCup 2014 - João Pessoa, Brasil (June 2014)
10. de Kok, P., Negrijn, S., Karaalioğlu, M., Lagrand, C., van der Meer, M., Gerbscheid, J., Groot, T., Visser, A.: Dutch Nao Team Team Qualification Document for RoboCup 2016 - Leipzig, Germany (November 2014)
11. Lagrand, C., Negrijn, S., de Kok, P., van der Meer, M., van der Wal, D., Kronemeijer, P., Visser, A.: Team Qualification Document for RoboCup 2017, Nagoya, Japan. Technical report, University of Amsterdam, Science Park 904, Amsterdam, The Netherlands (November 2016)
12. Lagrand, C., Negrijn, S., van der Meer, M., van der Wal, D., Petrini, L., Deprez, H.L., Kronemeijer, P., Rajamanickam, S.K., Nzuanzu, J., Jelinek, L., Visser, A.: Team Qualification Document for RoboCup 2018, Montreal, Canada. Technical report, University of Amsterdam, Science Park 904, Amsterdam, The Netherlands (January 2018)
13. Kronemeijer, P., Lagrand, C., Negrijn, S., van der Meer, M., van der Wal, D., Nzuanzu, J., Hesselink, R., Zwerink, W., Gupta, A.R.K.N., Visser, A.: Team qualification document for robocup 2019, sydney, australia. Technical report, University of Amsterdam, Science Park 904, Amsterdam, The Netherlands (2019)
14. gezegd Deprez, H.L., Kronemeijer, P., Caitlin Lagrand, T.W., van der Wal, D., Coltof, Q., Zwerink, W., Visser, A.: Team qualification document for robocup 2020 bordeaux, france. Technical report, University of Amsterdam, Science Park 904, Amsterdam, The Netherlands (2020)
15. Wike Duivenvoorden, Hidde Lekanne gezegd Deprez, T.W.J.K.R.v.d.W.P.H.A.V.: Team qualification document for robocup 2022 bangkok, thailand. Technical report (2022)
16. Lagrand, C., de Kok, P., Negrijn, S., van der Meer, M., Visser, A.: Autonomous robot soccer matches. In: BNAIC2016 Proceedings. (2016) 237–238
17. Lagrand, C.G.: Learning a robot to score a penalty – minimal reward reinforcement learning. Bachelor's thesis, Universiteit van Amsterdam (June 2017)
18. Negrijn, S.: Exploiting symmetries to relocalise in robocup soccer. Master's thesis, Universiteit van Amsterdam (December 2017)
19. van Heusden, R.: Making a robot stop a penalty – using q learning and transfer learning. Bachelor's thesis, Universiteit van Amsterdam (June 2018)
20. Garritsen, T.: Using the extended information filter for localization of humanoid robots on a soccer field. Bachelor's thesis, Universiteit van Amsterdam (June 2018)
21. gezegd Deprez, H.L.: Enhancing simulation images with gans. Bachelor thesis, Universiteit van Amsterdam (2020)
22. Montè, X.: Neural factorization of shape and reflectance of a football under an unknown illumination. Bachelor thesis, Universiteit van Amsterdam (2023)
23. Lagrand, C., van der Meer, M.: The roasted tomato challenge. Amsterdam Science 05 (April 2017) 4
24. van Heusden, R.: Making a robot stop a penalty. In: Proceedings of the 30th Belgian-Netherlands Conference on Artificial Intelligence (BNAIC). (November 2018) 89–90
25. Sun, Z.: An Energy Efficient Gait for Humanoid Robots Walking on Even and Uneven Terrains. PhD thesis, Universiteit Maastricht (March 2019)
26. Sun, Z., Roos, N.: A controller for improving lateral stability in a dynamically stable gait. In Iliadis, L., Maglogiannis, I., eds.: Artificial Intelligence Applications and Innovations, Cham, Springer International Publishing (2016) 371–383
27. Sun, Z., Roos, N.: An energy efficient dynamic gait for a nao robot. In: 2014 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC). (May 2014) 267–272
28. Lagrand, C., van der Wal, D., Kronemeijer, P.: Detecting a checkered black and white football. honour's project report, Universiteit van Amsterdam (February 2017)
29. van der Weerd, R.: Project ai: Real-time object detection and avoidance for autonomous nao robots performing in the standard platform league. Technical report, University of Amsterdam (2021)
30. Lagrand, C., van der Meer, M., Visser, A.: The roasted tomato challenge for a humanoid robot. In: Proceedings of the IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Bragança, Portugal (2016)
31. Wiggers, A., Visser, A.: Discovering reoccurring motifs to predict opponent behavior. In: Proceedings of the 16th International Conference on Advanced Robotics, Montevideo, Uruguay (2013)
32. van Noort, S., Visser, A.: Validation of the dynamics of an humanoid robot in usarsim. In: Proceedings of the Performance Metrics for Intelligent Systems Workshop (PerMIS'12). (2012)
33. Visser, A., de Bos, D., van der Molen, H.: An experimental comparison of mapping methods, the gutmann dataset. In: Proceedings of the RoboCup IranOpen 2011 Symposium (RIOS11). (2011)
34. van Noort, S., Visser, A.: Extending virtual robots towards robocup soccer simulation and @home. In: RoboCup 2012: Robot Soccer World Cup XVI. Springer (2013) 332–343